STO's are limited due to range, but I imagine having enough of them on the ground would make ground invasion impossible since transports would just get blasted out of the sky by un-interceptable beam weapons. Pretty sure you can make STO's into PD weapons as well to prevent being bombarded from long range.
Aside from cost/tonnage efficiency, STOs en masse basically force the enemy to bombard the planet from orbit to eliminate them (unless the enemy has a large number of heavily-armored large dropships). This slows down their offensive, and reduces the usability of the colony after they take it over due to dust levels. Ideally you don't lose the colony in the first place, but if you do then you can do a bit of a scorched earth tactic on the way down.
For orbital defenses, I think making fighter-sized missile launchers supported by fighter-sized active sensor craft (both with no engines) would be best. You want them to be fighter sized so they can be easily built and transported to frontier worlds.
While system defense fighters are a viable plan, it's worth noting that if you have a fleet of tugs larger stations are logistically easier to manage - transport isn't a problem, and you get a lot more use out of limited commanders than if you have a dozen patrol fighters/FACs all sucking up commanders. Orbital stations can also mount heavier armor and additional command modules (most importantly CIC for that bonus %CTH), plus usually are much more effective PD platforms if you use AMMs or turrets.
But what about FACs and other small ships, is it worth stationing them at colonies? To this day, I haven't been able to build a FAC-sized combat ship I thought was worthwhile.
The main benefit of small ships is that they're logistically easy - you can dump a dozen maintenance facilities and a refueling center on any hunk of rock and put some FACs there armed with missiles or PD turrets, and they're quick to build rapidly if you find a hunk of rock that needs them in a hurry. FACs are never as capable as large ships but they're very flexible and less of an investment when you don't need a lot to start with.
To build capable FACs don't try to get fancy. Put a boosted engine on them and fill them up with box launchers or PD weapons (if your FACs are fast, you don't even need turrets). Probably need a separate sensor ship as well as active sensors take up valuable tonnage space.
The main limitation of FACs is that without a real fleet or orbital/surface defense backing them up they have very low endurance and won't do much more than slightly delay a large attacking fleet, but they will be sufficient to force an enemy to attack with a large fleet and not just send a lone transport shuttle with three marines and a picnic basket to take control of your outposts.
And I suppose you could always cheese things by camping heavy beam ships on jump points to immediately shoot the enemy to bits. They'd have to rotate in and out as their deployment dictates, but I imagine if engines and range aren't a concern you can build some pretty beefy monitor ships.
This is a pretty standard strategy once you have enough shipyards to support dedicated monitor fleets.
Hangar outposts on remote rocks seem interesting, but anything that needs missiles would be a pain in the ass logistically as you'd want to update the stockpile whenever you get a new model. And God help you if you decide to change missile sizes...
The main issue with hangar outposts is that they can be avoided by just sailing around them, at which point your ships are out of position. Thus they're best in mostly empty systems that just have a bunch of comets or something, unless you're very confident in your sensor net.
