I think 3) is fine as a 'good enough' solution. I have another proposal, however.
Just have the NPR believe the player. NPRs have complex logic behind which systems they will claim vs which they will not. Players get to make a choice. When a player claims a system, he's telling his NPR contactees, "Don't touch this system; I'm willing to go to war or provoke a diplomatic incident if you do." The why doesn't matter; if a player wants to use strong-arm diplomacy to try to control NPRs while eating the relationship penalties, that's his risk.
I understand the question isn't, "Why do players make claims?" but, "How do NPRs evaluate a response?" I suggest just taking the question of whether a player population already exists out of the calculus of how an NPR responds or not and just using the calculation you posted that involves relative military strength, etc.
Where existing populations vs not might come in is in the diplomatic impact of the claim. If an NPR doesn't know you have a population in-system already the relationship hit for claiming a system should be substantially greater, like twice as great. For that, you could easily just use the population as it actually exists.