I agree wholeheartedly about the AI being one of the weakest parts of this amazing game, and while I understand that creating a competent artificial intelligence is a civilization-scale task, not a single-developer one, there's plenty of small changes that can be made to turn NPRs into more formidable foes.
On the lowest level, in ship-to-ship combat, I believe the solution is to make the AI utilize its intelligence of enemy classes to determine a so-called Goldilocks zone where the ratio of its ships' DPS to the known enemy ships' DPS is the highest. This way, if there's one player fleet in the system and the AI has a fleet with at least one ship whose weapons outrange those of all known player classes, it will attempt to maintain the minimum distance from which it can fire unopposed. If the player's anti-ship weapons have longer range but the AI fleet is faster, it will attempt to close distance, potentially aiming to sit just outside the enemy PD range instead. If the player has two such fleets in close formation, the AI will attempt to close in with one such fleet while keeping as far away as possible from the second one, to minimize its effective DPS, and if the NPR detects two different fleets on the opposite sides of the system, it will attempt to engage the weakest one since that's where its ratio would be the highest.
In the latter case, two distance checks (one based on time for the NPR fleet to intercept either of the fleets, another based on time for either of the fleets to intercept anything the NPR assigns value to, such as itself, other fleets or colonies) could be added to assign more value to closer or more threatening fleets. As such, a closer, but more dangerous fleet would be a higher priority target for an NPR fleet than a distant, weaker one, unless the weaker one is able to intercept another fleet or colony, and the first NPR fleet can intercept it before these things happen, while the stronger fleet has no chance of intercepting anything during that time. I understand this wouldn't be as simple as I described and probably opens up additional cheese possibilities, but even if this DPS ratio evaluation behavior were to apply for close threats (within beam weapon range of a fleet) only, this would still boost AI combat efficiency considerably.
Additionally, NPR AI only considers
(missile range*((missile speed - target speed)/missile speed)) when determining the distance at which it can begin firing its missiles at the enemy. I understand this is added to prevent cheese, but it also leads to NPR ships with 100mkm missile range potentially being outranged by player ships with 70mkm missile range, depending on the approach vector. I propose the AI should take the approach vector into account, potentially firing at (or close to) max range if it's being chased, but only until a salvo fired thusly runs out of fuel, at which point the AI will "realize" it's getting cheesed and have the fleet in question revert to its current conservative behavior. The fleet can switch back to long-range fire after a sufficiently long time elapses to make cheesing the AI in such a fashion unattractive to the player (hours? days?) or after it detects inbound missile contacts and assumes its being outranged again (with a random delay up to several minutes to make it unclear to the player when exactly he should begin retreating).
In a similar vein, the AI will be more than happy to fire missiles at an enemy who can reach a jump point before the missiles can impact. I'm not sure how to handle this, banning the AI from firing missiles before the missile time to impact is shorter than the enemy time to JP (or assumed enemy jump shock duration) seems a bit too radical, but maybe the AI could at least wait to reload its missile launchers before transiting a JP through which a hostile contact fled seconds or minutes ago.
On a higher level, I made some observations and possible solutions
here, and I see homeworld defense in particular being discussed in this thread. There should be no reason whatsoever for NPR fleets to attempt to escape the home system if a stronger enemy fleet is present, they should rally at their homeworld and go down fighting. As far as fleets in distant systems are concerned, I understand that feigning attacks on a home system to draw the NPR from its colonies could be considered cheesy, however it would make sense for the NPR to at least commit all forces from the neighbouring systems into a threatened system with higher System Value, doubly so if both significant enemy contacts and own ship losses have been detected recently. This, however, brings me to:
The reason that NPRs may commit forces piecemeal is that they will react only to detected enemies.
This is the biggest problem with the AI thus far and should be addressed before everything else. The fact that NPRs have no concept of a "Lost Contact" like we do makes them incredibly weak and predictable. Using Precursors as an example, they can be lured piecemeal from the safety of their orbital bases and STOs by enabling transponders or sensors on a single ship, lose their forces to an overwhelming player fleet they couldn't see, then fall for the same bait again and again. I understand that a proper threat evaluation system for lost contacts would be tricky to implement, NPRs can't simply behave like every enemy fleet ever detected remains there forever until detected again, but in case of losing some ships to a superior enemy force they should definitely hunker down for a significant amount of time and cautiously use their scout ships (and scout fighters, maybe?

) to re-establish contact or verify its absence before they can resume normal operations. I'd suggest waiting double or triple the time it would take for known enemy contacts to approach any NPR colony or fleet, or leave the system (whichever is longer) with no other contacts or ship losses in-system during that time, before the AI can "forget" about the contacts, assume they're gone and regain its normal threat response.