Fair warning, I am prejudiced against size 1 missiles swarms. Just against the aesthetic of building attacking missiles so cheaply that it costs the defender too much to try to shoot them down.
But how do they really compare?
At high levels of boost, the fuel efficiency of large missiles means they can be much longer ranged. As agility increases accuracy per msp, larger missiles can afford more agility. And sensors as well. And even armor may be a competitive counter to AMMs at higher levels, compared to simply having lots of small missiles.
Larger missiles take longer to load, but for fighter loadouts that is less of a consideration. Sure, a size 12 missile takes an hour and a half to load in a hangar, but the fighter is likely flying out and back for several hours.
Size 12 is about the largest missile I would expect to see, as beyond that are no further gains in fuel efficiency and therefore range.
But large missiles have another drawback, especially early in a campaign. It costs a lot more research points to keep large missile designs up to date. It is almost trivial to keep the AMM designs up to date.
The thing is, switching one's standard missile size is expensive. You basically have to scrap or drastically refit your entire missile fleet, plus missile stores. So if you pick size 4 launchers in the beginning, you are likely stuck with that size for most of the campaign.
In favor of large missiles, especially late in the campaign, the ability to deal shock damage. A size 36 warhead is very likely indeed to inflict shock damage, even if it won't take down the shields. This is especially important for fighter missions, as the huge amount of time between strikes means that shields would likely have been completely regenerated between strikes. And the chance of a Golden BB hit, that causes secondary explosions (say of boosted engines) is a worthy consideration.
So, many sizes of missiles do you have in your fleet? size 1 for AMMs, size 2 or 3 for anti-fighter, anti-LAC, and size 4-6 for long ranged missiles? What is the largest sized missile you have developed and actually used and been happy with?