I think a "combat width" system would have some problems, frankly a much smaller force should be surrounded and annihilated not allowed to hole up and take out the enemy peacemeal. It would create a metagame where heavy armor (either vehicle or IFN power armor) is dominant against anything lighter - if a formation of 12-armor tanks is limited to only fight up to 3x its size you can see the problem readily.
I don't agree that it needs to be a problem, first of all it would be more realistic... it certainly is not realistic that 100.000 soldiers can engage 1000 at the same time. It just would mean that defending can delay a bit more... but it would also be combined with the concept of defending the colony or the entire planet.
While maybe not realistic, I don't think making the 100,000 vs 1,000 matchup more fair is terribly important for good gameplay. It reminds me, for example, of playing multiplayer in old RTSs like Age of Empires where your opponent would hide his last villager in a map corner and force you to spend 30 minutes hunting him down to win the game. At some point, mechanically it makes sense to allow a bit of unrealism for the sake of wrapping up the battle.
There would be a slight advantage to high quality troops and technology, but I would not call that an issue... just a change from how it is now. You also could tailor your shock forces, artillery would probably be more important as it would be granted special powers, such count for less size on their attacks etc... air combat (especially if fixed) would also have a bigger impact.
The advantage would be more than slight. With unit costs being balanced as they are now, a 12-armor UHV formation which costs 12x as much as 1-armor infantry is expected to be roughly equivalent to 12 of those infantry formations of the same size, assuming some reasonable distribution of weapon types of course. So if the combat width is, for sake of example, 3x the targeted formation size, only three of those 1-armor infantry formations can engage the 12-armor UHV formation at once. While one might think at first glance that, okay fine, so we send 12 infantry regiments and they just take longer to fight to mutual destruction, but in fact due to the way combat works the UHV will be able to sit and tank the fire of far more than 12 infantry regiments with this mechanic (if you expect the first 3 INF regiments to do 25% damage to the UHV, in fact they will do more like 6% before being wiped out due to square law scaling of combat power with numbers). Therefore, you create a system by game mechanics alone which incentivizes heavy armor and/or HP upgrades as the exclusive optimal point by a huge degree.
While the current ground combat is imperfect and has a fairly obvious metagame (CAP spam), this is not enforced by the mechanics, rather by a combination of NPR army design being infantry-heavy and the values in the DB which are readily tweaked even if finding a good rebalance is not straightforward. I think this is strongly preferable to a state where the game mechanics force an optimal strategy.
It's always tricky to discuss "balance" in Aurora, but I think most can agree that we do not demand perfect metagame balance but simply that many options are viable and even if some are stronger than others on average there are interesting gameplay decisions to be made. Often the tactically-optimal solution is not the best in grand strategic terms for example. The implications of a size-based stacking width system would violate this concept of balance, IMO - and it is not so easy to find a better basis for stacking width, for example a cost basis heavily penalizes special forces types of units, which are IMO already a bit less than optimal on average and so are not in need of such a harsh nerf. Similarly, what factor should the combat width be? If we say, okay, let's avoid the armor problem by making it 12x size, then 120,000 soldiers can assault 10,000 with no hindrance, so have we really made any change to the game on a practical level?
I think you miss the whole point of why you would like to have this... it would be to simulate the possibilities to hold territories for reinforcement to arrive, the fact that high quality become slightly better is not really that important... you still would want cheap forces to garrison planets as they are not suppose to "win".
Currently high quality troops is not really much worth it anyway... over time cheap troops is more effective as troop ships are not terribly expensive and will not cost you any maintenance once built. As you use commercial engines on them anyway they also will be very cheap to upgrade. I have done the numbers on this... thus making high quality troops have a niche role can be important. The attacker can tailor its forces to attack whatever enemy they are facing anyway, so this is a moot point.
This way there would be more choices in the type of troops you would like to deploy rather than the cheapest troops available.
When you play in a multi-faction environment you will learn that more cheaper troops is almost always better than quality troops, at least if you don't care about losses for winning. The only reason I still build high quality troops is role-play... more or less, to preserve soldier life, even if less efficient investment of time and resources.
The current way it works is just too simple to math, there is very little choice in actual troops types, anything bigger than medium vehicles is pretty much pointless, more or less... you might go heavy vehicles if you have technologic superiority. Genetic enhancement is also to some degree worth it if you are on par or better technology, mainly due to weapon layouts. Armoured infantry is not really worth it, most of the time, too expensive. Genetically enhance PWL infantry specifically suited for their environment can be very effective, especially if you have superiority in weapon tech, which is relatively common if you use plasma weapons, at least in my campaigns. It is very common in my campaigns that weapons tech is one or two level above armour tech for ground forces for this very reason, plasma weapons also is a very good early to mid game technology... which is where most of my campaigns tend to gravitate around.
Even investing in heavier armour for vehicles and statics is a luxury in most of my campaigns only relegated for the factions with lots of research capacity.
With the current rules, protecting planets with small number of garrisons is a role-play decision only, more or less... I would like for it to be more worth it mechanically also.
I also would not implement the combat width system in a linear scale... I would make it so that you can overwhelm a very small force more than a larger one... but I would tie this to if you defend the colony or the planet as well as the planets terrain etc... this would make combat more dynamic and harder to predict with math, I see that as a good thing not a bad one.