Select the colony you're considering upgrading. You'll see something like this in the lower left corner:
Click on the planet icon on the lower right and it will bring up a list of possible upgrades:
Mouse over them to see what they do, and if you want one, click on it to select it and then click on upgrade.
You're limited to one upgrade per colony.
Note that if your colony is defended, the colony might look like this:
In which case you'll have to click on the second icon (looks like people) to show the upgrade icon.
