There are some buildings that are not supposed to be built on planets with a civ capital. A supply depot is one.
Edit:
At start of a new game, I can build a capital mainframe and industrial center. If I destroy the capital city, I can build build the capital city, colonial generator, and supply depot. If I build the capital city, I lose the ability to build colonial generator, and supply depot. If I build either the colonial generator or supply depot, I lose the ability to build the capital city.
Personally, I rather have the capital city. It has many bonuses, some of which are rare. The game should mention that these buildings can't be built on a planet with a capital city.
Edit 2:
I should also note that other core worlds don't have a capital city, they have core world capitals. They're not as good as capital cities. Though you can get more capital cities by taking the home world of other civs. By the way, if you destroy a capital city when you have 2 or more, you only get core world capitals back.