It turns out that you were only out by a factor of two:
Cost = max(5, power_mod < 1.0 ? hs*tech*power_mod^2/2 : hs*tech*power_mod/2)
or in spreadsheet terms:
=MAX(5, IF(ChosenPowerMod<1, ChosenEngineSize*ChosenPowerPerHS*ChosenPowerMod^2/2, ChosenEngineSize*ChosenPowerPerHS*ChosenPowerMod/2))