Do not use criminals to colonize new planets unless you have a prison improvement to use because you'll just create new criminals on a new world. Getting the Orbital Prison solves most crime problems.
I definitely use them on constructors, no down side to that that I've seen.
Got the Outland reference Colonel. That's not one of his best movies, but I love Sean Connery.