It comes down to what you like to play. As you noticed, population (race) density is what matters. If you want a deep-space resource-grab game your optimal would be different than if you like long development and late wars. You don't need to invade planets to have a war over usable resources.

Currently I'm using a Large Galaxy with eight races including my own. I think the set up I used for this game is a little too under populated: Abundant Stars and planets with Habitable Planets on Common. It gave me long development times for the core planets, research and building a small fleet or two for pirate hunting, but the contact with a second major was at the first IP meeting.

Then you get into the galactic architecture... Loose clusters; then the details... number of stars, of planets, Even a densely packed medium galaxy with everything about planets on "Abundant" may not be very crowded.Galaxy size relates to the amount of time it takes to get any where. The rest is how often you run into things to slow you down before you get there.

If you want to get a feel for how dense the map will be packed grab a race with Knowledgeable and test out some different numbers. You'll be able to see where everyone spawns from the start which could help you dial in the settings you prefer. I personally prefer mine to be more densely packed affairs that push diplomacy issues. I use abundant settings with 26 major races (myself and 25 AI) and abundant minors on a gigantic map. It's a little emptier than I'd prefer but that just means I have some room to add a few more races in when I get the inspiration to create something new.