After playing around with Ground Combat it feels like something that I'm missing is that feeling that veteran units with alot of combat experience should perform significantly better than a freshly built formation.

The current ground combat model favors heavily armored units mainly to reduce casualties, but the IMO main purpose of reducing casualties should not be to save BP in less replacements needed, it should be to not lose the valuable experience your units gain from battles.

I agree, there's not much benefit to having any veteran units other than those who've been sitting around under a commander with GCT for years, never seeing any combat, which is the opposite of how things should actually work. I suppose this is due to the unit Morale also representing their training level, which is awkward, as you would expect Training and Morale to be separate stats, with Training increasing slowly while idling and quickly when participating in combat, and Morale dropping rapidly while taking losses and increasing while inflicting casualties, performing breakthroughs or idling. The ground unit system is already complex as it is, but I think such a change would be more than justified.

The weapons do fire independently but as you observed will usually hit infantry, so 4 SHAV is overkill as you will largely be shooting at infantry and if only tanks remain 1 SHAV will still kill a tank per attack.
In practice any SHAV is pretty much overkill as running into something heavier than Medium vehicles is currently almost impossible, I still make super heavy tanks with a SHAV as an RP decision but I do make sure I have a lot of Autocannon and HCAP weapons as they kill infantry. My superheavy tanks carry a SHAV and HAC's which also kill a lot of medium tanks , for pure efficiency HCAPS are better but I overdo it for RP reasons

If you really want to mop up infantry, equip your SHV with as many HCAP as you can fit on it. I usually call this an "IFV". Infantry shots will almost certainly bounce right off your SHVA. Their anti-vehicle units will have to shoot your SHVs to really have any chance at destroying them, but they too will have a lower chance of destroying your IFVs (assuming you have equal armor and weapons tech to each other) because SHVA is literally "super heavy". It's very hard to break.

I also make a "tank" that includes 1 HCAP and the rest are SHACs. So, these too will mow through infantry while also possessing firepower to deal with heavy armored.

And finally, I make a "tank destroyer" which includes SHACs and 1 SHAV, just in case I need that extra firepower to take out something heavier than I can field due to tech differential or after their infantry have been wrecked by my IFVs and I want to mop up the remaining heavies.

Losing one of these hurts a bit since they take a lot of BP to make. You can either deal with it by having a lot of extra production for ground units, or you can do what other players do and be a bit monstrous by having infantry soak up shots. I prefer to have the extra production.

Don't forget HB artillery in the rear echelon supporting your assault forces.

Question for players: how to ensure that shots will hit enemy heavily armoured targets? Not be wasted on infantry?

Deploy infantry-killer units first, kill all the enemy infantry, then deploy the SHAV units in a second wave to kill the enemy armor.

No way to do it.
Targetting is random