Those major penalties include massive negative modifiers to production, mining, fuel refining, ship maintaining, etc. Once you start getting a major deficit, it is very difficult to recover from then on. Once rebellions are coded in at some point in the future, it may also spark them on colonies.