Honestly all you'd need would be to make it be % chance to ignore armor as opposed to % chance to ignore each layer of armor. Right now at max meson retardation (7% chance to be absorbed per layer), ships of armor level 10+ become virtually immune to mesons.
I think mesons were meant to be an anti-small weapon and not be very powerful against large ships, but they get completely superseded by AMMs in the anti-small role.
Actually for a 7% retardation chance, 10 layers of armor would block about 50% of meson shots, as (1-0.07)^10 = 0.484. So mesons are a bit less bad than they seem but still pretty bad given that this is the max tech and at the same tech level ten layers of armor is really a bit light if anything.
In my view a big problem with mesons is that they are RP-intensive (not ideal for a specialized weapon with limited applications), and two of the techs are both purely +range effects and are quite redundant as a result. That said, I don't think having a flat %chance to ignore all armor is a good buff, as it is too extreme and basically brings back the VB6 mesons (very problematic) with a side helping of RNG which can only add frustration. Frankly the current state feels like a harsh nerf intended to remove mesons from the game without actually removing mesons from the game since people like the flavor but the actual concept is problematic.
I think any solution is going to have to come from giving the caliber/focal size tech line a better effect, which is tricky as any new effect needs to scale so as not to become a perfect anti-armor weapon at high tech levels while being passably useful at lower tech levels. One thought would be to have the caliber techs specify the number of armor layers which can be passed through by (1-x)% chance where x is the retardation tech level. For balance this may need to be something like sqrt(caliber/10) with a minimum of 1 (10 cm) and maximum of 2.83 (80 cm).
Then the formula for the %chance a meson shot penetrates changes from (1-x)^n, where n is the number of armor layers, to (1-x)^(n/c) where c=sqrt(caliber/10) is the number of armor layers bypassed by each check and fractional values are preserved. In this case at max tech level a meson can penetrate about 28 layers of armor about 50% of the time, rather than 10, which is I think reasonable given the cost, size, and ROF for such a large weapon. At base tech level nothing has changed, so much like shields a few levels of research are needed to produce a useful weapon. A mid-tech meson cannon with 25 cm caliber and 24% retardation will have a 50% chance to penetrate 4 layers of armor, which sounds maybe a bit frightening but this is already possible with a 9-damage laser that also does armor damage so I think it is fine.