I think the biggest thing with Raiders is that they are just randomized Invaders. The way the build up and interact with the player and npr races are near identical except Raiders are more likely to run if they are outnumbered and can spawn anywhere and everywhere rather than be limited to Aether Rifts. They are just as capable of wiping out the player as the Invaders, Swarm, or a game start NPR. The Raiders imo should be closer in capabilities and function as Precursors and Rahkas, more of a road block rather than an existential threat.
I would suggest that they don't build up over time at all, but rather every build cycle have a % chance of spawning a ship, few ships, fleet, or multiple fleets. These ships would focus on destroying or capturing (they really should have boarding craft imo) civilian shipping or attacking undefended colonies to steal population or mineral stockpiles. They should avoid contact with warships unless they feel they are significantly stronger than the player's ship(s) or planetary defenses (they really need a speed bump to run more effectively). But most importantly I think that once their ships run out of fuel they move to their initial jump point and get out, closing the door behind them. Whatever they steal in BP value could then be added to their next incursion.
This I think would better represent them as pirates and raiders and keep them in that roadblock level of difficulty as Precursors and Rahkas