If you want to make it so two forces both set on defense wont fight, the easy way to do that is to just make it so units on defense only fire on units on attack (units on attack would engage both units on defense and attack). Then eliminate/massively reduce the supply penalty for attack, but give attackers a chance to take territory and damage rear echelon units, and it should be decently balanced. No worries about different formation sizes (however the attacker would still want either every front line formation on attack or none of them, lest they be defeated in detail).
This throws a wrench into the battle, though, in that attackers could land on a planet and just begin to fortify. If attackers permanently lose their fortification bonus, this wouldn't gain them much (they'd still have to attack eventually), but if attackers retain their fortification bonus but just don't use it on the attack, it means the attacker would want to wait to fortify before attacking, so they could retreat to the fortifications if the battle doesn't go well.
You could add another layer to that by allowing bombardment units to still hit units on defense, so that if both sides choose defend then their artillery will still pound positions, but that wouldn't work with the support approach you were going for before where support formations can be assigned a front line formation and will fire on anyone that engages them. However, this risks making defense too hard since it would allow the enemy to land ground forces, not attack, and just keep bombarding the defending troops (including from orbit) until they gave up and launched their own assault.
Overall, it's a bit of a more complex scenario but one with potential balance issues.
On the other hand, I was thinking of attack as more like a fast assault, and therefor two sides both on defense would work out more like trench warfare in WWI, with constant skirmishing. This would probably result in a simpler, more attrition based fight, but that isn't a downside to me (to me the strategy and complexity is more in the strategic layer and space battles).
An invader could set their troops on defense and settle in for the long haul; they wouldn't have any fortification at first, but would slowly build some. Or they could go for an quick assault, hoping to disable the enemy STO weapons quickly as well as take some of the territory which is the whole point of invading in the first place.