Are you doing geologic scans of the systems as well?
I find that my grav ships, if kept svelte, repeatedly are able to do a survey of a system without ever being detected by swarm or precursor. One engine, one scanner, one small jump drive and the size is small enough to go under the radar. Geo ships, on the other hand, fly right up to the precursor ships and poke them in the eye and get themselves blown to bits on a regular basis. The grav survey only occasionally comes close enough to bodies to be detected on occasion.