OK, maybe my refit idea wasn't the best.
One thing that would add a lot of realism would be a more sophisticated demographic system. At the moment, all that matters is the raw number of people. Here's what I propose:
Instead of simple population, there are three different population numbers: children, adults, and elderly. Agriculture and services percentages are based off of the total population. Children count as .5, adults and elderly as 1. Only adults can work. Each build cycle, a certain percentage of adults have children, a certain percentage of children become adults, a certain percentage of adults become elderly, and a certain percentage of elderly die. The percentages can be modified by biotech technologies.
This change would also probably change the way population limits work. Each world would have a "carrying capacity" that defines how many people it can support. This is based on hydrosphere, tectonics, land area, etc. Birth rate is partially based on percent of carrying capacity. Infrastructure-supported colonies work the same way, and the capacity can be raised with infrastructure.
I hope all of the above makes sense. It might be a good idea to set it up as an option.