Colonization in 6.21: Where did it go wrong?

[telegraph (OP)]

for 30 years I was only colonizing solar system. It was more or less ok, since my civilian companies were only starting out. After I ran out of "perfect" worlds(Earth, Luna, Mars, Ganymede, Io, Europa, Callisto) I started to look for acceptable extrasolar planets. While I was surveying and building jumpgates my "7 colonies" were all set to "Stable".
No civilian passanger transportation were observed during this period. Luxury Liners were idling at spaceports. This is confirmed by absence of any tax income on Passanger Liners.

So my first question:
I thought that Luxury Liners would ferry rich citizens back and forth regardless of the immigration settings. Is it not true? Are they just mini-colonizers now? Or have I messed something up? Is it possible to get them working between established colonies?
========================================================
Once jumpgates were complete I contracted civilians to ship 100 Infrastructure from Mars to my first extrasolar colony, Cluntius Placidus-A II. I also set Mars to be "Source of Colonists" for a fairly short period.
Civ. freighters took the required ammount of Infrastructure and went for jumpgate. All civ. colonyships AND luxury liners took full load of colonists and were just sitting at Mars heading nowhere. My civilian lines are pretty rich. We are talking about 260 ships, of total tonnage: 10M tonns. A lot of them are colony ships, so...
After a while, once initial 100 Infrastructure was unloaded, ALL of civ. colonyships AND luxury liners decided to move people to their new home. And so they did: now I have 7.5M population on a planet with only 100 Infrastructure (supports 0.5M population). I can accept a "slight" overachievement, like shipping maybe 10% more colonists then required, but to miss 1500% is quite unnerving. What will happen after 30 more years, when civilian corporations will flourish from interstellar trade?

Hence my second question:
How to colonize right? Is there any way to ban some of the colony ships?  Should I have contracted 10k Infrastructure in the initial shipment? How to create tiny colonies then?

[Nightstar]

Odd. This update should have fixed that: http://aurora2.pentarch.org/index.php/topic,3372.msg36089.html#msg36089

I suggest filing a bug report.

For you current problem, you can probably set your new small colonies as stable and ship over colonists with government transports. (Assuming you have any, that is.)

You might be able to use civilian colony ships by somehow occupying most of them when you set up the orders. Seems even less convenient though.

[James Patten]

Strange.  I've had few problems with having my Civilians do all of the colonization.  I've got several out-system colonies.  In fact I have to manage when I build jump gates to open up new worlds, since I want them to concentrate on certain colonies first.

My biggest problem seems to be that the Civilians only like to transport Human colonists.  I've conquered a couple of pre-TransNewtonian worlds, and the civs don't seem to want to transport the aliens.

[TheDeadlyShoe]

Civilian colonisation is very aggressive in the current version, you pretty much have to roll with it.  I'm pretty sure Civilians only look at trades when they start the trip, as checking orders every time interval would be an unnecessary slowdown.  But they are supposed to coordinate with eachother so if that is turned off it is a bug.

There is a 'ban body' function in the f9 screen. it will ban a destination for civilian transports and as a default destination for military fleets. IIRC, it has a 5%(?) chance of not working for civilians for any given trip.  The original purpose of Ban Body was to restrict civilians from warzones, and I believe Steve thought it would be interesting to have an Idiot Quotient.

Another way to control civilian movement is with jumpgates, or rather a lack thereof. Civilians don't have jumpdrives and they won't use yours, only jumpgates.  So any system without jumpgate access won't have civilians mucking it up. You can delete existing jumpgates in the f9 screen if you have SM mode enabled before you open the screen.

Luxury liners should be working regardless *shrug*

[telegraph (OP)]

I have figured out Luxury Liners part. They only work "as intended" between world set as "Source of Colonists".
They do not work if all planets set to Stable, Destination or combination of these. They transport colonists between "sources" or from "source" to "destination" pretty well though.

Regarding overachieving colonizers:
Ban body and lack of jumpgates do not work for me because the main reason I am establishing extrasolar colonies is to have a nice economy running with imports and exports and trade routes and such.
Pre-Ocupying colonizers sounds a little bit too excessive. The only thing I can think of to ocupy them would be reshuffling colonists from Earth to Mars and to Earth again. That would pose a problem because I have highly distributed industry(research is going on Luna, Production on Mars, shipbuilding on Earth, and I am going to relocate some functions to Jupiter moons. My population is too thin to have it all in one place. Having distributed economy helps to get more workers), so any large colonists relocation would hurt production.

How do you guys colonize? What is your migration setup? How many civilian ships do you have? How are they performing?

I would like to hear your stories, because something is definitely fishy here. Maybe I have an ungodly number of civilian ships?

[metalax]

Odd. This update should have fixed that: http://aurora2.pentarch.org/index.php/topic,3372.msg36089.html#msg36089

I suspect that this is an edge case. While it checks that it's own colonists and any already under-way won't put the colony over the limit, if multiple colony ships are looking at the exact same time they are likely not considering each other as being under-way.

For you current problem, you can probably set your new small colonies as stable and ship over colonists with government transports.

You can't set populations less than 25 million to stable, they will simply reset themselves to destination.

I have figured out Luxury Liners part. They only work "as intended" between world set as "Source of Colonists".
They do not work if all planets set to Stable, Destination or combination of these. They transport colonists between "sources" or from "source" to "destination" pretty well though.

That is certainly a bug if they are not transporting between stable colonies. It may be one of the bugs introduced with the overhaul of civilians in 6.0 that hasn't come up yet as it definitely used to work.

How do you guys colonize? What is your migration setup? How many civilian ships do you have? How are they performing?

I would like to hear your stories, because something is definitely fishy here. Maybe I have an ungodly number of civilian ships?

I tend to drop off a seed population with my single colony ship along with enough infrastructure to support them then leave it to the civilians to build them up in jumpgate connected systems. Shipping lines do seem to tend to build far more colony ships than are needed and not enough freighters.

[Paul M]

This is present in 6.1 and frankly the only way to stop it is to not build jump gates then at least the madness is confined to one system.

The trouble is that if there is an opening on a world every colony ship sees it and then they all go there.  There is no solution to the problem as it is a programming issue that needs to be sorted out, and the sorting out unfortunately is likely to become rather complex.  Approaching the problem blind I see no simple solution.  One that might work is that each shipping firm is polled in order (with a flag to keep it from all going to the first firm), and then if there is an opening its colony ships are polled in order with flags set to stop the polling once the number of onboard colonists equals the current infrastructure limit.  The next opening would go to the next firm and so on.

With the current system you just get huge "dumps."  Worse the infrastructure demand of the colonies is not sensibly addressed either.  Basically it is an issue Steve has to fix in the game code, beyond not using jump gates you as the player can't do much about it.

[davidb86]

This is present in 6.1 and frankly the only way to stop it is to not build jump gates then at least the madness is confined to one system.

The trouble is that if there is an opening on a world every colony ship sees it and then they all go there.  There is no solution to the problem as it is a programming issue that needs to be sorted out, and the sorting out unfortunately is likely to become rather complex.  Approaching the problem blind I see no simple solution.  One that might work is that each shipping firm is polled in order (with a flag to keep it from all going to the first firm), and then if there is an opening its colony ships are polled in order with flags set to stop the polling once the number of onboard colonists equals the current infrastructure limit.  The next opening would go to the next firm and so on.

With the current system you just get huge "dumps."  Worse the infrastructure demand of the colonies is not sensibly addressed either.  Basically it is an issue Steve has to fix in the game code, beyond not using jump gates you as the player can't do much about it.

I am playing 6.21 and do not have this problem.  My civilian shipping is doing a good job of transporting colonists and infrastructure to my fledgling colonies.  One difference is that I am leaving some of my smaller established colonies as destinations, so that the civilians have a variety of locations to choose from.  This results in just a few trips to each colony.  I start a new colony by sending 10 infrastructure to it on imperial (not civilian) freighters.  The civilians then send colonists, and as the colony approaches the population limit they start sending infrastructure.  The only way to get huge dumps is to build up a large number of civilian colony ships and then only have one open world at a time, leaving established colonies open spreads the shipping out.  Forgoing any colonization for 5 to 10 years will thin out the number of civilian colony ships as they are retired.  The civilians

for 30 years I was only colonizing solar system. It was more or less ok, since my civilian companies were only starting out. After I ran out of "perfect" worlds(Earth, Luna, Mars, Ganymede, Io, Europa, Callisto) I started to look for acceptable extrasolar planets. While I was surveying and building jumpgates my "7 colonies" were all set to "Stable".
No civilian passanger transportation were observed during this period. Luxury Liners were idling at spaceports. This is confirmed by absence of any tax income on Passanger Liners.

So my first question:
I thought that Luxury Liners would ferry rich citizens back and forth regardless of the immigration settings. Is it not true? Are they just mini-colonizers now? Or have I messed something up? Is it possible to get them working between established colonies?
========================================================
Once jumpgates were complete I contracted civilians to ship 100 Infrastructure from Mars to my first extrasolar colony, Cluntius Placidus-A II. I also set Mars to be "Source of Colonists" for a fairly short period.
Civ. freighters took the required ammount of Infrastructure and went for jumpgate. All civ. colonyships AND luxury liners took full load of colonists and were just sitting at Mars heading nowhere. My civilian lines are pretty rich. We are talking about 260 ships, of total tonnage: 10M tonns. A lot of them are colony ships, so...
After a while, once initial 100 Infrastructure was unloaded, ALL of civ. colonyships AND luxury liners decided to move people to their new home. And so they did: now I have 7.5M population on a planet with only 100 Infrastructure (supports 0.5M population). I can accept a "slight" overachievement, like shipping maybe 10% more colonists then required, but to miss 1500% is quite unnerving. What will happen after 30 more years, when civilian corporations will flourish from interstellar trade?

Hence my second question:
How to colonize right? Is there any way to ban some of the colony ships?  Should I have contracted 10k Infrastructure in the initial shipment? How to create tiny colonies then?

The scenario described is exactly what I would expect to happen if you had a large number of civilian colony ships and only one destination.  It is not a bug, just the way the world works.  Once the 7.5 million colonists are landed, you should see the civilian freighters loading up on infrastructure.  It might take several months to get straightened out, but it is due to the constraints you placed in the first place, not a bug in the program.  I would suggest that you send a brigade or two of garrison troops to quell the riots. 

A solution to your problem (prior to them landing) is to clear the civilian orders and reopen some of your smaller Sol colonies as destinations, this will spread the colonists around and slow the growth of Cluntius Placidus-A II.  Civilians will not send colonists to a colony that is over the population limit.  You can check the wealth screen to see how much infrastructure a colony is demanding.  The higher the demand the higher priority it will get from the civilian freighters.

The only way I know to reliably ban a solar system is to change the empire possession (on the galatic map) to an empire that you are currently at war with.  I have seen civilians violate a body ban, they are just sure you are lying to them to protect it for your self.  But I have gone 20 years with a 0 cost habitable world in a system that I have told the civilians is possessed by the "Ancient Guardians" and never had a ship stray there even with jump gates on all of the jump points.

[Paul M]

I have 3 destinations for colonists in sol.  The problem is very simple.  I have >1 million colonists in orbit around earth at any time.  ~20xcolony ships each with 50,000 colonists on board.  When I opened up Io to colonization, I made the mistake of sending in a seed colony on my government transport and once I did that, and well before I could transport the infrastructure I had prepared so the colony would survive the first dump...every colony ship left Earth orbit and dumped 1 m people on the place.  It only checks: Is their space?  If yes then all firms send colony ships with people on them for that world.  If 50,000 people worth of space is available 250,000 colonists show up (5 shipping firms x 50,000 per firm)...seems (as I haven't rigourously checked) to be what will happen.

The AI is doing what it is programed to do but not what was intended.  It is absolutely reproducable.  Since I opened Io up for colonization it has been the only place in the system to recieve shipments of civillian infrastructure outside of the odd bit going to luna or mars.  And anytime I see it get ahead of the game along comes another major colonist dump.  The population is now 6.5 million.  In the same way they pushed the population of Venus up to 11 million (the colony has an ifrastructure demand of 4x the production of earth).  I have seen growth rates get as bad as -50%.  I sent Ganymede some extra infrastructure to give them some growing space and the next thing I know they are seriously over their support limit.  For that reason I refuse except for role playing purposes to do any more of that thing.  I will send them only enough infrastructure to bring them to just below their current population...but not create the problem over again by giving them space to expand.

[davidb86]

I have 3 destinations for colonists in sol.  The problem is very simple.  I have >1 million colonists in orbit around earth at any time.  ~20xcolony ships each with 50,000 colonists on board.  When I opened up Io to colonization, I made the mistake of sending in a seed colony on my government transport and once I did that, and well before I could transport the infrastructure I had prepared so the colony would survive the first dump...every colony ship left Earth orbit and dumped 1 m people on the place.  It only checks: Is their space?  If yes then all firms send colony ships with people on them for that world.  If 50,000 people worth of space is available 250,000 colonists show up (5 shipping firms x 50,000 per firm)...seems (as I haven't rigourously checked) to be what will happen.

The AI is doing what it is programed to do but not what was intended.  It is absolutely reproducable.  Since I opened Io up for colonization it has been the only place in the system to recieve shipments of civillian infrastructure outside of the odd bit going to luna or mars.  And anytime I see it get ahead of the game along comes another major colonist dump.  The population is now 6.5 million.  In the same way they pushed the population of Venus up to 11 million (the colony has an ifrastructure demand of 4x the production of earth).  I have seen growth rates get as bad as -50%.  I sent Ganymede some extra infrastructure to give them some growing space and the next thing I know they are seriously over their support limit.  For that reason I refuse except for role playing purposes to do any more of that thing.  I will send them only enough infrastructure to bring them to just below their current population...but not create the problem over again by giving them space to expand.

Paul,
you are absolutely right, the civilian lines evaluate independently and you make the problem worse by sending your own ships with infrastructure.  The civilians evaluate where to send colonists by where there is room, but each line makes the decision independently.  Once you open up some worlds outside of Sol system, especially worlds with low infrastructure cost, the combination of long travel times and multiple destinations will help you avoid the current problem.  Once you have a colony, there is no way to stop it from growing to 25 million, at which point you can stop colonization, but will still have internal growth.  Ask China and India about the effectiveness of asking people not to reproduce.  I solved the problem by not colonizing high cost worlds, that is what automated mines are for.

How did you get to 5 lines?  I am over 70 years in with 12 colonies in 8 systems and only have two lines with about 20 ships each. (approximately 3 harvesters, 2 liners, 5 colony and 10 freighters for each line).   The way to rein in civilian shipping is to keep a large number of targets (I have 7 worlds open for colonization right now, 5 of which are 0 cost).  I have 48 Terraforming bases (3 modules each) which can turn a 200 cost world into a 0 cost world in a couple of years.  My biggest problem is getting colony ships to grow these worlds.  With ten civilian colony ships (8 small and 2 large) and 5 large colony ships of my own, I have more available sites than I can ship colonists to. 

If you have too many civilian ships then do a great consolidation.  Shut down colonization for 5 years and you will see the number of colony ships drop as they are retired for age and replaced by something that will make money (freighters, harvesters, liners)

[Paul M]

There are 5 lines...two I subsidized early on and three that didn't get funding.  They vary from about 28 some ships to 2 ships (that one isn't making much money).

Burns Freight and Logistics has 28 ships (2 liners, 4 harvesters, 5 colony ships, and 17 freighters)
Willamson Logistics has 17 ships (1 harvester, 4 colony ships, and 12 freighters)
Hvitkarson Carrier Ltd has 17 ships (1 liner, 3 colony ships, and 13 freighters)
Dillahunt Contanier Line has 6 ships (1 colony ship, and 5 freighters)
Mahmoot Transport Group has 2 ships (1 colony ship and 1 freighter)

I don't have jump gates so outsystem colonization is handled by government freighters.  I don't have colonization going on, Io was the first new colonization effort in Sol in decades.  The fleets are getting smaller as Burns had 32 or 35 ships at its largest.  But their capacity is largely the same.  I make good use of the freighters moving stuff around and there is a brisk if budding internal economy.  I like seeing more harvesters as I need that fuel something fierce.

But the problem is fixable with a few programing flags I'd think.

[telegraph (OP)]

davidb86, I can see now why you have do not see this behavior as a problem. 40 ships total, out of which only 10 are colonizers will definetely not deplete your homeworld(s). In my case though we are talking about 300 ships at the moment, 256 of which belong to one company.

That world, Cluntius Placidus-A II, is now successfully colonized, it has 32M population. Guess why. Well, I'll tell you anyway:
every colonists shipment was about 7.5M. so after first shipment it was hugely overpopulated, and civilians started to bring in infrastructure. At the moment they brought enough of it colonizers decided to do another shipment. Again 7.5M. Again much over the limit. Rinse and Repeat.
Ofcourse people died from malnutrition and oxygen deprivation. We can now calculate the total cost of this colonization: There were 6 shipments in total, 7.5M each, totalling in 35M population. since we have only 32M - 3M perished.

Also note that I MUST have some worlds set as Source of colonists, as only such worlds can be used by luxury liners.

and there is a problem with leaving colonized worlds as destination: my colonizers are so eager that they will gladly deplete all my population if they are allowed to. total population of humans now is 1500M that is about 200 runs for my colonizers. And most of those people are busy working(my industry is decentralized, in order to get more workers), so moving them around just to get colonizers busy would hurt my governmental activities immensely.

EDIT: looks like I am indeed having a HUGE civilian fleet. Please give me more info about the fleets your civilians have.
For me it is : 300 ships on year 33. 256 of those belong to one company.

[davidb86]

davidb86, I can see now why you have do not see this behavior as a problem. 40 ships total, out of which only 10 are colonizers will definetely not deplete your homeworld(s). In my case though we are talking about 300 ships at the moment, 256 of which belong to one company.

That world, Cluntius Placidus-A II, is now successfully colonized, it has 32M population. Guess why. Well, I'll tell you anyway:
every colonists shipment was about 7.5M. so after first shipment it was hugely overpopulated, and civilians started to bring in infrastructure. At the moment they brought enough of it colonizers decided to do another shipment. Again 7.5M. Again much over the limit. Rinse and Repeat.
Ofcourse people died from malnutrition and oxygen deprivation. We can now calculate the total cost of this colonization: There were 6 shipments in total, 7.5M each, totalling in 35M population. since we have only 32M - 3M perished.

Also note that I MUST have some worlds set as Source of colonists, as only such worlds can be used by luxury liners.

and there is a problem with leaving colonized worlds as destination: my colonizers are so eager that they will gladly deplete all my population if they are allowed to. total population of humans now is 1500M that is about 200 runs for my colonizers. And most of those people are busy working(my industry is decentralized, in order to get more workers), so moving them around just to get colonizers busy would hurt my governmental activities immensely.

EDIT: looks like I am indeed having a HUGE civilian fleet. Please give me more info about the fleets your civilians have.
For me it is : 300 ships on year 33. 256 of those belong to one company.

Are you playing 6.1 or 6.21?  It sounds like 6.1 since Steve added a check in 6.2 or 6.21 that scraps commercial ships after 10-15 years.  In my own 6.21 game I have noticed that the composition of the commercial fleet changes based on what is generating income for the line.  Once I started buying fuel from the commercial lines, they began building more harvesters. Once I had more than one world over 25 million they began to add liners.

I am not suggesting limiting the sources of colonization, I am saying that you need to manage the destinations. If you only have one destination then all of the ships will try to go there (think shopping on the Friday after Thanksgiving).  If you have multiple sources and multiple destinations, then 6.21 does a pretty good job of spreading the colonists around.  Another way to choke off civilian shipping is to use 30 day turns, since the civilians only check for new destinations once per turn.  I do not think there is a way to forcibly retire civilian ships,  I think I tried targeting one once and couldn't do it.  You could SM in a player controlled race with a single battle cruiser and use it to thin the herd if necessary .  It is easier than abandoning a game and starting over.

In-system colonization is short distance, allowing the civilians to run wild since they can make a trip every 5 day or every hour for the Earth to Luna run.  Out system colonization is slowww.  I have colonies that are 2.8 billion KM to 9.8 billion KM from Earth.  These take an average of 30 days each way (current Small C4 is approximately 2500 km/s) which spreads the impact out.  I have tracked individual civilian ships and they appear to select a new target every time they get back to Earth. If you are in 6.21 and your out system world is colonized, do not colonize any more worlds for 5 years and you will see the number of colony ships drop dramatically.  Then send a single freighter with 10 infrastructure to several different worlds using the move to command.  Once everyone is there you can have them all unload simultaneously and see that the civilians will spread the wealth.  Another method is to use terraforming ships or bases to prep worlds so that the colonization cost is not so high, which decreases the impact of moving 7.5 million people at a time.

[telegraph (OP)]

I do use 6.21. companies do retire their ships, but they tend to build another one instead, replacing the old one.

I cannot do 30-days intervals because somewhere in space precursors, swarm and an NPR are heaving a party for the last 20 years, so whatever interval I choose - it all comes to either 5sec or 1day because of imminent fleet interception.

Just stopping colonization will not help because of the following problems:
1. Steave code only considers EMPTY ships for scrapping. (I guess it is good, but risky design decision)
2. Passanger ships load colonists at any "Source" planet FIRST and look where to dump colonists LATER. (This is either a bug or an oversight. ) This CAN lead to 7.5M just sitting in orbit, waiting for the next destination to arrive. None of these colonizers will be scrapped ever.
3. If I will remove ALL "Source" planets - luxury liners WILL STOP working.(This is a bug. Can anyone confirm that it is so in 6.21, or  only my game is bugged?)
4. If luxury liners will STOP working - my economy WILL suffer. (Works fine. I have a large military economy, so I need to have different kinds of income and keep colonizing to stimulate civilians.)

[Konisforce]

You said you've got 7 colonies in Sol or so.  Is there any one that could take a big load of colonists?  I mean, this is a micromanagey solution, but you could:

Turn on 1 colony (I'll say Mars, is it pretty well terraformed?) to be a destination again.  Ships all issue orders to head there and dump the population of Honduras (I looked, it's 7.5 million).  While they're headed there, you switch all your other colonies to destinations as well.  When they arrive and dump on Mars, you switch Mars back to source.  They now have 7 (6 + extra solar colony) different options for dropping off, and they'll end up pretty well staggered.

I know you'd already solved it (or, rather, it solved itself) by this post, but that'd be one way to do it.  Heck, you could even set Earth to be the Destination and get them to dump all 7.5, then switch it back, just to get things spread out.

It sounds like the only solution is to always have 2 or more destinations, at least with the size civ fleet you have.  I've run into the same, but not with the population of Honduras on ice around Earth.