(civilians will generate more infrastructure down the line wont they?)
It really depends on where you are with your game. If you have only a single planet with reasonable population (your homeworld) then you can colonize a planet and leave it, assuming you have sufficiently large civilian line. In such a scenario your colony can grow to ridiculous size without any effort on your part, all the while generating free infrastructure and wealth.
The problem is that if you have two or more well developed planets, that are close together your shipping lines will just move goods between them and will not support new colonies. As such, unless you are roleplaying, colonization of planets occurs because of one of the following reasons:
1. Mining (or other similar reason, like alien ruins or anomalies).
2. Wealth generation issues. Smaller colonies have larger population growth (in terms of percentages) so numerous small colonies mean the fastest population growth which means the fastest growth of wealth from taxation.
3. Positioning (the colony basically becomes a military base).
4. If you're left with spare terraformers you may as well use them on a random planet which you can later colonize.