So in theory a small marine scout team could cause a whole sector fleet to surrender if it goes badly. This could probably be changed to only allow the handover mechanic if there is actual population or industry at the colony surrendering. Or even better some sort of weighting formula like %-surrendering = colony-pop% of empire-pop, so it also avoids cases where you come to orbit-assist a small outpost who has only a tiny garrison.