Aurora 4x

VB6 Aurora => Aurora Suggestions => Topic started by: Steve Walmsley on September 08, 2014, 05:45:47 AM

Title: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 08, 2014, 05:45:47 AM
Probably the most common complaint about Aurora (besides the UI and the install :)) is the speed at which the game runs. This is particularly an issue for larger long-term games. Another element of this is the 5-second slowdown which can occur during NPR combat.

Two limitations on any attempt to increase execution speed are the age of the technology involved and the sheer number of things that happen every turn. I can't do anything about the former without rewriting Aurora so I've been looking more closely at the latter. The slowest parts of the turn are:

1) NPR Movement orders
2) Sensor Phase
3) Setting up Trade Runs for Civilian Shipping.

The worst culprit is the sensor phase, because it happens once per sub-pulse while the other two are once per increment. It suddenly occurred to me that to speed things up I could adopt the "If a tree falls in the forest..." philosophy. As an option, I could run the current detection code only for systems in which a player unit or population is present. This is less realistic than the current system but might be acceptable as a trade-off vs performance. There are two options for the situation where only NPRs are present in a system:

1) No detection
2) Automatic detection

Both of these would significantly reduce execution time, especially for campaigns with several NPRs. The former would also completely eliminate the 5-second problem because there would never be combat without a player in the same system. That is also a downside on the realism side as NPRs would be ignoring each other unless a player was watching - although that would also lead to more situations where a player sees NPR vs NPR combat as it wouldn't start until the player forces were in the system. The latter (auto-detection) would actually increase combat between NPRs so while perhaps more realistic than no detection it might increase the 5-second problem. Therefore I am leaning toward giving players the option of enabling no detection in systems without a player presence - or perhaps make that the default and have the option to enable detection in all systems.

No detection would also reduce the NPR Movement orders as well because there would be less calculation of which target to chase.

To reduce trade run calculations, I could change NPR new trade runs to be calculated during the 5-day increment. Currently both player and NPR trade runs are calculated during any increment longer than one hour.

Interested to hear comments and feedback
Title: Re: Idea for Speeding Up Aurora
Post by: alex_brunius on September 08, 2014, 06:23:23 AM
I am personally MUCH more exited in solutions that attack the root cause of the problem ( exponential expansion of NPRs triggering new NPRs and spoilers far far away ), instead of simply trying to mitigate the cause ( increase performance in any isolated system ).

We had a great discussion in this thread about this previously:

http://aurora2.pentarch.org/index.php/topic,7305.0.html

Other options to put limits on NPR expansion might be some way to pseudo limit their range based on technology or fuel/engine design values ( IIRC they run with infinite fuel currently ).


For the no/automatic detection solutions I would preffer if it was possible for NPRs to fight still in places that I am likley to visit. So my vote there would go for no detection unless in a system where players are present or an adjacent system.


To reduce trade run calculations, I could change NPR new trade runs to be calculated during the 5-day increment. Currently both player and NPR trade runs are calculated during any increment longer than one hour.

This would reduce civilian trade significantly, primary because it would limit trades between very close bodies ( like Earth - Luna ) that provide extreme ROIs for civilian lines and allow them to explode their operations.

Another way to regulate their expansion ( which might be a bit more boring though ) would be to have civilian ships earn income per time spent doing active jobs instead of deliveries.
Title: Re: Idea for Speeding Up Aurora
Post by: sloanjh on September 08, 2014, 06:34:44 AM
This would reduce civilian trade significantly, primary because it would limit trades between very close bodies ( like Earth - Luna ) that provide extreme ROIs for civilian lines and allow them to explode their operations.

Why do you say this?  My reading of what Steve said is that the change here would only affect NPR, not player races.

John
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 08, 2014, 06:37:20 AM
I am personally MUCH more exited in solutions that attack the root cause of the problem ( exponential expansion of NPRs triggering new NPRs and spoilers far far away ), instead of simply trying to mitigate the cause ( increase performance in any isolated system ).

We had a great discussion in this thread about this previously:

http://aurora2.pentarch.org/index.php/topic,7305.0.html

Yes, I know. I implemented several of those ideas, so reduced numbers of survey ships, NPRs don't generate precursors and I think the chance of NPRs activating new NPRs is reduced. I'll have to check the details once I get home.
Title: Re: Idea for Speeding Up Aurora
Post by: sloanjh on September 08, 2014, 06:42:40 AM
1) No detection
2) Automatic detection

My vote would be for auto-detection for two reasons:

1)  At least for NPR vs. NPR engagements, it makes up somewhat for the lack of "I" in "AI".  I assume that at present, if another race falls off an NPR's sensors in a system, the NPR completely forgets there was a ship there leading to e.g. "yo-yo" situations.  Perfect knowledge would solve this.

2)  With no detection, you could get weird situations where two NPR empires become intermingled until a player shows up, at which point they start shooting at each other at point blank.  I suppose this could be RP'd as war breaking out right when the player shows up, but still....

On the 5 second problem: How much of the 5 second problem is due to needing to check for changes to detection status?  My vague recollection is "a lot".  If you know that everything auto-detects, then you can change the code determining the next interrupt to take big timesteps when the missiles are far away from the targets.

John
Title: Re: Idea for Speeding Up Aurora
Post by: alex_brunius on September 08, 2014, 06:42:54 AM
Why do you say this?  My reading of what Steve said is that the change here would only affect NPR, not player races.

Ah I missed that part! But it would still prevent explosions of NPR trade operations then.

Yes, I know. I implemented several of those ideas, so reduced numbers of survey ships, NPRs don't generate precursors and I think the chance of NPRs activating new NPRs is reduced. I'll have to check the details once I get home.
Niiiiice  ;D

My response is what you get for not updating the Changelog thread! (http://aurora2.pentarch.org/index.php/topic,7255.0.html)
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 08, 2014, 06:52:59 AM
Ah I missed that part! But it would still prevent explosions of NPR trade operations then.
Niiiiice  ;D

My response is what you get for not updating the Changelog thread! (http://aurora2.pentarch.org/index.php/topic,7255.0.html)

I thought it was in the released version but after checking the dates that doesn't seem to be the case :). I'll look at what I actually changed and update the change long.

Interesting point about the trade runs. Earth - Luna run is a mini-exploit but moving all trade checks, including for players, to 5-day increment would reduce its effectiveness and improve performance. The downside is generally reducing income from shipping lines overall (even for longer runs). I'll give that some thought.



Title: Re: Idea for Speeding Up Aurora
Post by: Whitecold on September 08, 2014, 07:18:36 AM
Just a thought from my side, automatic detection could also be a good alternative to the sensors disabled options for multi- faction starts.

My vote would go to automatic detection, because I fear weird situations of intermingled NPRs that turn hostile as soon as you show up, and even worse, take breaks if you retreat.
Title: Re: Idea for Speeding Up Aurora
Post by: Beersatron on September 08, 2014, 08:59:26 AM
Auto Detection when NPRs are in the same system, then normal detection when a player enters would be my vote.

I like exploring and finding wrecks at battle sites :)
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 08, 2014, 09:45:08 AM
1)  At least for NPR vs. NPR engagements, it makes up somewhat for the lack of "I" in "AI".  I assume that at present, if another race falls off an NPR's sensors in a system, the NPR completely forgets there was a ship there leading to e.g. "yo-yo" situations.  Perfect knowledge would solve this.

NPRs create 'Points of Interest' (POI) when they detect something. There are different types of POI depending on the contact type (neutral, hostile, transiting, etc.). NPRs assign these POIs a priority and when they have nothing more important to do (chasing a hostile for example or checking out a current contact), they will send ships to visit the POIs in priority order. Less urgent POI disappear after a while if they are not investigated.

Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 08, 2014, 10:44:06 AM
1) No detection
2) Automatic detection
would it be possible to set minimum intervalls for detection, too? like one detection check every day or every 5 days? and have no detection inbetween. as far as i understand it, this would reduce detection checks by alot, still allow NPRs to fight each other but not make it automatically.

otherwise my vote is auotdetection.

also support reducing calculation time for NPR civilian shipping lines. they recently got a wealth bonus anyways (removed costs for research), if that hurts their wealth a little, take that as compensation.
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 08, 2014, 11:35:59 AM
would it be possible to set minimum intervalls for detection, too? like one detection check every day or every 5 days? and have no detection inbetween. as far as i understand it, this would reduce detection checks by alot, still allow NPRs to fight each other but not make it automatically.

You need to maintain contact in order to fight so occasional detection isn't really possible. It has to be all or nothing.

Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 08, 2014, 12:11:52 PM
the intervalls could be shorter after a successful detection to allow combat.

i assumed the sensor phase takes so long altough usually nothing happens.
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 08, 2014, 01:34:16 PM
the intervalls could be shorter after a successful detection to allow combat.

i assumed the sensor phase takes so long altough usually nothing happens.

It's not quite as simple as that. Contact would have to be maintained on every sub-pulse of every increment or missiles would lose lock, for example. Also, how do you decide when to stop using the constant sensor phase and go back to occasional?

A lot happens during the sensor phase but you are only notified when the status of a contact changes. Of course, you have to check every sub-pulse or you wouldn't know when there is a status change.

Title: Re: Idea for Speeding Up Aurora
Post by: Haji on September 08, 2014, 01:51:23 PM
Interesting point about the trade runs. Earth - Luna run is a mini-exploit but moving all trade checks, including for players, to 5-day increment would reduce its effectiveness and improve performance. The downside is generally reducing income from shipping lines overall (even for longer runs). I'll give that some thought.

For my games the shipping lines income is generally relatively low so I wouldn't mind that. However I would prefer this to be an option rather than a set value. Alternatively you could just have a main menu option where a player can specify at what intervals should the trade runs be calculated.

As for the main topic on detection, I'll just repeat what others have said for statistical purposes - no detection means too much weird stuff will happen with NPRs, like two races establishing a colony on the same planet (maybe even on each other's homeworld) and then suddenly nuking all of them at once when the player arrives. Not exactly fun. As such I'd like this to be no more than an option for players to increase their performance at the price of the realism. I'd never use it though.
Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 09, 2014, 02:10:19 AM
It's not quite as simple as that. Contact would have to be maintained on every sub-pulse of every increment or missiles would lose lock, for example. Also, how do you decide when to stop using the constant sensor phase and go back to occasional?
it's hard to make senseful suggestions in this regard without knowing the code =)

if possible i would add a flag (bool) to each system. if the flag is TRUE, sensor phase is done in short intervals (like it's done now), if it's FALSE, sensor phase is only done every 5 days (or any other interval that makes sense). the flag ist set to TRUE if any NPR requests it. the request would be a decision to attack, and maintaining it during the fight or engagement. when the combat is over or all involved AIs decide to stop engagement, the flag can be set to FALSE again. any time the use of a jump point results in a human player being in the same system as anything else sets the flag TRUE as well, until this changes.

this assumes that the AI at some point decides to attack, and also to stop attacking, and recognises that a combat is over. at these events the flag can be set/unset. there could be other events as well, depending on the set POIs or related actions. it also assumes that we can turn sensor phase on/off for each system separately (which seems to be the case from your suggestion). when implementing, care would need to be taken that no TRUE request can be canceled by any FALSE request.

this would make short interval sensor phases happen only where and when it matters, hopefully giving us the benefits of both solutions (NPR combat can still happen almost like now, but less calculations where it does not matter). it makes only sense if currently the sensor phase takes significant time even if it produces no results. otherwise this suggestion wouldn't do much.
Title: Re: Idea for Speeding Up Aurora
Post by: alex_brunius on September 09, 2014, 02:29:56 AM
As for the main topic on detection, I'll just repeat what others have said for statistical purposes - no detection means too much weird stuff will happen with NPRs, like two races establishing a colony on the same planet (maybe even on each other's homeworld) and then suddenly nuking all of them at once when the player arrives. Not exactly fun. As such I'd like this to be no more than an option for players to increase their performance at the price of the realism. I'd never use it though.

That is true, and another reason it might be good to at least simulate adjacent systems sensors fully too ( so that once a player get close systems start to behave more normally and the "dust" has time to settle, litteraly ).

This brings up another AI problem though, it's inability ( or reluctance? ) to use ground forces instead of overwhealming nukes even against bodies it has a large population of it's own on. If the AI would instead have ground wars and/or occupied populations as a result of the situation you bring up that would only be cool to watch and add a great sense of history and flavor to the game IMO.
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 09, 2014, 03:46:09 AM
this assumes that the AI at some point decides to attack, and also to stop attacking, and recognises that a combat is over.

How would you define "deciding to attack" and "recognising a combat is over".

Consider for example, a survey ship in the outer system detected at several billion kilometres. How about an intruding ship that is faster than the defenders? What about enemy warships that have launched a missile attack and are retreating toward a jump point?
Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 09, 2014, 04:11:20 AM
How would you define "deciding to attack" and "recognising a combat is over".

Consider for example, a survey ship in the outer system detected at several billion kilometres. How about an intruding ship that is faster than the defenders? What about enemy warships that have launched a missile attack and are retreating toward a jump point?
i don't know how the AI handles such things. but surely, if the AI is fighting at all, it must at some point decide that it wants to attack a ship. otherwise it would never happen. likewise, at some point it must recognise / decide that there are no more enemy ships and stop attacking / executing combat moves / scanning.

survey ship: depends on the plan. if the AI wants to attack it, sensor checks are needed. if the AI does not want to attack it (for whatever reason), no sensor checks are needed. there must already be AI code for those decisions. in the last case, there'll be another sensor check at the set interval again, allowing the AI to decide again. if the AI does not want to fight, but wants to keep watching (this one is probably not in the AI code right now, but it's not that important), sensor checks are needed.

an intruding ship that is faster than the defenders: if the defenders want to fight (even though it's unlikely or impossible that they will succeed - but you won't know this unless you calculate the whole combat), sensor checks are needed. if the defenders do not want to engage (for example because they realise it's hopeless), and the intruder does not want to fight either, no sensor checks are needed. if the AI does not want to fight, but keep watching in case the intruder gets closer, sensor checks are needed. it all depends on the requirements of the AIs planned actions. there must already be code for most of this, otherwise the AI wouldn't ever do anything. it's a matter of adding the requirement for sensor checks to those decisions. of course i don't know how easy that is.

as long as there are missiles running, combat is not over. there might be a problem with special missiles (mines, buoys), but those should be solved one way or the other.

it's not about making it perfect, the goal is an improvement, right? if we can't solve all possible solutions, it's better to fix it at least for most of them.

again, i don't know how much time of the current sensor phase is needed for such things anyways. my suggestion makes only sense if most of the sensor phase calculations are needed for other things that would get filtered out.
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 09, 2014, 06:45:31 AM
i don't know how the AI handles such things. but surely, if the AI is fighting at all, it must at some point decide that it wants to attack a ship. otherwise it would never happen. likewise, at some point it must recognise / decide that there are no more enemy ships and stop attacking / executing combat moves / scanning.

The AI makes decisions on whether to close on targets, or avoid them or shadow them or remain at a set range, and continually checks for firing opportunities. It does not make a specific decision to 'attack' at some point in that process apart from the decision to fire (even that is judged by whether the target could evade missiles before they arrive even if in range at that moment).

Quote
survey ship: depends on the plan. if the AI wants to attack it, sensor checks are needed. if the AI does not want to attack it (for whatever reason), no sensor checks are needed. there must already be AI code for those decisions. in the last case, there'll be another sensor check at the set interval again, allowing the AI to decide again. if the AI does not want to fight, but wants to keep watching (this one is probably not in the AI code right now, but it's not that important), sensor checks are needed.

an intruding ship that is faster than the defenders: if the defenders want to fight (even though it's unlikely or impossible that they will succeed - but you won't know this unless you calculate the whole combat), sensor checks are needed. if the defenders do not want to engage (for example because they realise it's hopeless), and the intruder does not want to fight either, no sensor checks are needed. if the AI does not want to fight, but keep watching in case the intruder gets closer, sensor checks are needed. it all depends on the requirements of the AIs planned actions. there must already be code for most of this, otherwise the AI wouldn't ever do anything. it's a matter of adding the requirement for sensor checks to those decisions. of course i don't know how easy that is.

as long as there are missiles running, combat is not over. there might be a problem with special missiles (mines, buoys), but those should be solved one way or the other.

it's not about making it perfect, the goal is an improvement, right? if we can't solve all possible solutions, it's better to fix it at least for most of them.

again, i don't know how much time of the current sensor phase is needed for such things anyways. my suggestion makes only sense if most of the sensor phase calculations are needed for other things that would get filtered out.

I guess what I am driving at is the 'combat' is a lot more than shooting. If you are faced with a faster ship then you might still be able to intercept if its is moving toward you, or you can set a converging course, or if another unit may be closer and could damage it to slow it down or if it decides to hold position for some reason, or it suddenly changes course, or you can attract it with another unit and pull it into range, or some of the other situations you mentioned above, etc. etc.. You would have to do this for any combination of ships in the system.

You would end up with a longer, slower piece of code deciding whether sensors should be used than just running the sensor phase anyway. Aurora is very complex in terms of interactions so what sounds simple, usually isn't.

Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 09, 2014, 07:03:52 AM
I guess what I am driving at is the 'combat' is a lot more than shooting. If you are faced with a faster ship then you might still be able to intercept if its is moving toward you, or you can set a converging course, or if another unit may be closer and could damage it to slow it down or if it decides to hold position for some reason, or it suddenly changes course, or you can attract it with another unit and pull it into range, or some of the other situations you mentioned above, etc. etc.. You would have to do this for any combination of ships in the system.
any of these would require short sensor intervals. but all these examples have one thing in common: they start with a successful detection, which is the core of my suggestion to start using short sensor phase intervals once it matters.[/quote]

You would end up with a longer, slower piece of code deciding whether sensors should be used than just running the sensor phase anyway.
that's possible but only you can judge if it's worth it. setting a bool flag shouldn't take much time though, compared to calculating all sensor ranges to all ships.

besides this, maybe it's possible to optimize the sensor phase in some other way. for example stop it when all ships in the system are detected or similar tricks.

another thought concerning civilian shipping lines: much thought and effort went into these, but from a gameplay point of view they are just a nice little feature, which doesn't justify a CPU usage high enough to significantly slow down the game. i wouldn't mind if they could (optionally) be removed alltogether. maybe as compensation wealth from taxes should be increased accordingly.

i maybe unable to post for a few days, but i'll check back here.
Title: Re: Idea for Speeding Up Aurora
Post by: alex_brunius on September 09, 2014, 07:16:46 AM
I think I understand what letsdance is getting at. Basically splitting up sensor checks into two levels.

High level - Initially detecting hostile ships at all ( only needed perhaps every 24 hours or every 5 days? )
Low level - Tracking of hostile ships and missiles ( at subpulse like presently )

So before any NPR ship is activly tracking other NPR ships ( or after they lost contact ), you can skip subpulse sensor steps. This should mean less combat, but still ensure that they can't both camp the same body or jump point without detecting and engaging.
Title: Re: Idea for Speeding Up Aurora
Post by: Steve Walmsley on September 09, 2014, 07:48:06 AM
I'm going to make this my last reply on this particular line of thinking because we are starting to go in circles a little.

any of these would require short sensor intervals. but all these examples have one thing in common: they start with a successful detection, which is the core of my suggestion to start using short sensor phase intervals once it matters.
that's possible but only you can judge if it's worth it. setting a bool flag shouldn't take much time though, compared to calculating all sensor ranges to all ships.

Obviously setting a bool flag is easy. As I've tried to explain - calculating the logic that allows you to know when to set it is a little trickier. Also, if the intervals are 5 days for example, you run into the major issue raised by objections to the non-detection option. Ships may be very close (in orbit for example) when detected.

Quote
besides this, maybe it's possible to optimize the sensor phase in some other way. for example stop it when all ships in the system are detected or similar tricks.

Ships can be detected using four different methods - you would have to continue until all ships are detected with all methods (at least those available) but part of detection is also knowing when you lose contact so if you stop sensor phases you wouldn't know when that happened.

Quote
another thought concerning civilian shipping lines: much thought and effort went into these, but from a gameplay point of view they are just a nice little feature, which doesn't justify a CPU usage high enough to significantly slow down the game. i wouldn't mind if they could (optionally) be removed alltogether. maybe as compensation wealth from taxes should be increased accordingly.

i maybe unable to post for a few days, but i'll check back here.

Whether civilian shipping lines are fluff or a useful part of the game is really a matter of opinion. For me, they add a lot to the game by simulating the civilian traffic that would be affected when hostilities break out. If you have to start protecting defenceless, scattered merchantmen that could be a key part of your economy, it makes the game much more interesting that defending a few key bases or choke points. They could also be the first things attacked - even starting the war in the first place by being in the wrong place at the wrong time.
Title: Re: Idea for Speeding Up Aurora
Post by: alex_brunius on September 09, 2014, 08:34:59 AM
I also agree that civilian traffic adds alot to the environment, story and combat situation with civilian ships in every corner of your empire that needs to be protected.

Wouldn't want to take them away even if it made the game 4 times faster.  ::)
Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 09, 2014, 08:38:18 AM
steve if you think disussing a point is... pointless, stop it. i'm just throwing out ideas that i would check. since i don't know the code it's hard to tell what would make sense and what doesn't.

but part of detection is also knowing when you lose contact so if you stop sensor phases you wouldn't know when that happened.
why? if i don't detect anything anyways it doesn't matter if i run all detection checks first. the AI doesn't know if it didn't detect anything because there is nothing, or because all detection checks failed.

other ideas would be to auto-detect all ships once at least one ship is detected (in a system), or to detect all ships/missiles of a certain class once one of that class is detected. this would let missiles and small ships keep part of their small size advantage (they usually travel in groups anyways), while probably reducing sensor calculations by alot (since small ships are usually numerous).

since small ships usually come in higher numbers it would probably reduce sensor time if higher resolution sensors searched first. (assuming that already found ships are not checked again for other sensors, which would then be another optimization idea).

civilian shipping lines: seeing the effort you put into them, i assumed that you like them alot. i think they're nice, but if it significantly reduces calculation time, i'd rather not have them. effects of war could be simulated by providing an income per system (slowly building up), and constantly reducing this while hostile ships are in the system.
Title: Re: Idea for Speeding Up Aurora
Post by: tenim on September 10, 2014, 07:38:44 AM
why not use multithreading in aurora? every thread could calculating on star system separately (they would be interact, or?) and if one thread is finished, he takes the next system that isn´t already calculated.  on a 4 core cpu with hyperthreading: 8 simultaneously threads possible -> speedup 8x.  ok, not exact, because of the synchronisation overhead, but the gain speed would be enormous.  on problem could be the database update if this happens during calculation.  in this case the update might me happen at the end of the calculation.

on vb6 multithreading is possbile:

hxxp: www. freevbcode. com/ShowCode. asp?ID=1287

no need to use a new language.  :-)
otherwise,  if a rewrite of aurora is planned, i strongly recommend lazarus (hxxp: www. lazarus. freepascal. org/) -a free delphi clone and sqlite (hxxp: www. sqlite. org/) as database.  i´m using this combination since many years and it works great and fast.  the other advantage is: both is free and runs under windows, linux and macos.

tenim
Title: Re: Idea for Speeding Up Aurora
Post by: Ninetails on September 10, 2014, 01:57:43 PM
I have already been thinking about the problem of detection in aurora (and other systems like it) before I saw this thread.  In general, one first need to realise that detections are a type of colition detection, and can make use of many of the relevant algorithms for dealing with this.
My suggestion is to try to predict when the next change of detection status could occure.  There are many ways to go about this, so I will just present the most simple one, and leave more powerfull and complex versions until another time.  The general idea is to come up with some prediction of when 2 objects (either detector or detectable) can either lose sensor contact or gain sensor contact.  In general this can only happen when 1 of 2 things occure: Either sensors/emission is turned on or off by some event (like activating active sensors, changing speed, jumps between systems, damaging components and such) or an object enters or leaves the detection range of a sensor.  Since the change events can be handled when they happen (and as such does not need to be handled by the sensor phase), we just need to predict when objects enter or leave the detection range.
In general we can group all objects into 3 catagories of movement: stationary objects (such as jump points and stranded ships), orbiting objects, and freely moving objects.  For each set of object types, we simply need to assign some prediction of when they could happen to detect each other.  The most conservative and simple option would simply be to assume that any 2 objects will be heading strait toward each other (ignoring that they are probably following another trajectory) and by using their current speed calculate when they will get within the range for each other.  If we are tracking disengagement, we would instead assume they were moving away frome each other.  For stationary objects, their speed will be 0, while orbitig objects could be described as bands of detection, though this can easily get quite complicated with nested orbits or orbit to orbit detection.  Anyway, since we can always atleast use the "move dirrectly toward or away" model, we can always come up with an estimate which will never be too late, and will work for all types of movement.

When we have such detection times, we can simply put them into an ordered queue (such as a min-heap), and simply only check for detection when we arrive at the time of the event at the top of the queue.  There are 2 speed ups form this.  First off, events could be many many subpulses away from each other, and we would not need to do any sensor checks in those phases.  Second off, it would only really be necessary to do a single sensor check when an event fires off, since it was only really relevant with one pair of sensor/detectable.  Ofcause one could do a full check and reset update the positions in the queue, if there is some reason to do it that way instead.  This could potentially significantly reduce the amount of sensor checks needed to be done, since it will only try it when it is relevant.  Since it is possible to ensure detection at the same accuracy as before with the conservative models metioned above, it would not even have to be limited to NPR vs NPR detection, though one could definitely get away with using some less conservative models, which would generate fewer events but risk being slightly inaccurate.

Concerning the civiliance, I would personally be happy if there was an option to disable them.  I would rather have to do most of the jobs they do myself.  While they do add something to the game, their potential high performance cost makes them a decent target for performances optimization (just as grafic heavy games often have options which makes the game much more ugly but significantly increases performance).

There are a lot of things that could be potentially be done to the NPR movements and civilian trade runs, which could improve performance.  But those would depend on what type of system is already implemented, and they would only really be improvements if they are upgrading from a slower system.
Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 10, 2014, 02:44:40 PM
would it be possible to make detection checks only against one ship per class per task group (same for missle volleys)? and use this result for all ships of this class in this task group /volley. this would reduce the detection checks by alot without changing much else (and contrary to my suggestion above, also works when nothing is detected). active sensors on/off should probably be distinguished, too.
Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 12, 2014, 06:13:47 PM
Probably the most common complaint about Aurora (besides the UI and the install :)) is the speed at which the game runs. This is particularly an issue for larger long-term games. Another element of this is the 5-second slowdown which can occur during NPR combat.
by the way, it's not just the speed in general, it's also that often the game stops earlier than i wanted, without anything new happening for me. a better turn sequence like i suggested in the semi-official suggestion thread http://aurora2.pentarch.org/index.php/topic,5896.msg75601.html#msg75601 (http://aurora2.pentarch.org/index.php/topic,5896.msg75601.html#msg75601) would also be a great improvement - together or independent from a general speedup.
Title: Re: Idea for Speeding Up Aurora
Post by: Barkhorn on September 15, 2014, 06:21:12 PM
I had an idea about civilian shipping lines.  I think they should be abstracted most of the time.  You only really care where each individual ship is when an enemy is detecting them.  You don't need to know precisely where that ship full of merchandise is 24/7, so why bother even tracking it most of the time?  Civilian lines should just generate income and move colonists, goods, and installations on a monthly or 5-day basis, calculated based on the profitability of the routes they run.  When an enemy ship shows up, you could use the density of the civilian lines in that system to decide how many civilian ships might get spotted, and only then do you actually spawn non-abstract ships.
Title: Re: Idea for Speeding Up Aurora
Post by: letsdance on September 19, 2014, 07:37:16 AM
You only really care where each individual ship is when an enemy is detecting them.  (...) so why bother even tracking it most of the time?
because if you don't know where it is, you don't know if an enemy can detect them.
Title: Re: Idea for Speeding Up Aurora
Post by: Barkhorn on September 19, 2014, 10:56:41 AM
But you could know if the route is within alien sensor range.  Then it's just a weighted chance on if they detect one.  My idea is that the route has a ship density calculated from the length of the route and the number of ships that use it; and the higher that density is, the more likely any aliens are to detect a ship on said route.  Then if aliens do roll high enough and detect one, that's when you spawn a ship.
Title: Re: Idea for Speeding Up Aurora
Post by: Shipright on October 07, 2014, 04:35:16 PM
I brought this up in another thread and nobody commented but since this one is dedicated to the topic I'll post it again:

I feel like you could probably make this easier. If the event doesn't trigger a player log entry (other than the interrupt message itself), ie something that affects the player's assets, the game continues to auto cycle through whatever increments the AI needs until you reach the turn increment the player chose. Technically you are still going to run through all the interrupts for AI purposes but at least the player himself doesn't have to sit there hitting next turn.

I don't personally mind long turn processing times. What I do mind is hitting the button, assuming and accepting its going to take a few minutes to process, go get a snack/make dinner/kiss the wife, come back and find out instead of running a turn for ten minutes to get to the increment I selected it got interrupted one minute in and was just sitting there waiting for me to hit "next" for the other nine.

I realize this does not address the delay due to calculations of the various game assets but I would weather those better if I wasn't getting interrupted all the time.
Title: Re: Idea for Speeding Up Aurora
Post by: MarcAFK on October 09, 2014, 10:03:23 AM
Auto Detection when NPRs are in the same system, then normal detection when a player enters would be my vote.

I like exploring and finding wrecks at battle sites :)
This seems pretty reasonable to me.
Title: Re: Idea for Speeding Up Aurora
Post by: ardem on October 26, 2014, 09:51:11 PM
Not understanding the code, I do not know if this is possible. The issue of the really is not so much the detection I think it is the battle that comes afterwards that really slows everything to a grind.

I think leave the detection and make pseudo battles, give ships a rating score based on speed, armour, tech and weapons systems. Then if you want add a random variable in there and blow up the ships on the losing side this is done with one calculation. Now if the speed of the enemy vessel is higher and then decide to run, then give them an instant jump ability to the next system to stop further sensors sweeps.

I do not think players have an issue with one or two 5 sec interrupts the issue is when you see 50-60 of them.

Yes it is not as realistic but from a player point of view it is about as realistic as it gets.


Title: Re: Idea for Speeding Up Aurora
Post by: CharonJr on December 18, 2014, 11:36:33 AM
About the detection flags for systems (which sounds good to me): How about having no detection checks as long as there are no foreign ships in the system and switch to normal once they are. But I have no idea if a check for any occupants whenever a ship changes systems will actually improve performance,  but it seems likely.
Title: Re: Idea for Speeding Up Aurora
Post by: Theodidactus on December 18, 2014, 12:33:04 PM
I brought this up in another thread and nobody commented but since this one is dedicated to the topic I'll post it again:

I feel like you could probably make this easier. If the event doesn't trigger a player log entry (other than the interrupt message itself), ie something that affects the player's assets, the game continues to auto cycle through whatever increments the AI needs until you reach the turn increment the player chose. Technically you are still going to run through all the interrupts for AI purposes but at least the player himself doesn't have to sit there hitting next turn.

I don't personally mind long turn processing times. What I do mind is hitting the button, assuming and accepting its going to take a few minutes to process, go get a snack/make dinner/kiss the wife, come back and find out instead of running a turn for ten minutes to get to the increment I selected it got interrupted one minute in and was just sitting there waiting for me to hit "next" for the other nine.

I realize this does not address the do calculatioelay due tns of the various game assets but I would weather those better if I wasn't getting interrupted all the time.


I think this is true of me too.

I really have no problem with the speed of the game, because usually in aurora you're not "doing" anything so much as winding up and letting things go. In the rare instances where you are making minute-by-minute decisions, the space-time bubble fixes everything. So it doesn't matter that advancing by 30 days takes an hour, or whatever. I can read, cook dinner, grade stuff, watch movies, ect. The inconvenience comes from interrupts and having to constantly check the game to see if its finished a loadcycle.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 18, 2014, 05:52:29 PM
I think this is true of me too.

I really have no problem with the speed of the game, because usually in aurora you're not "doing" anything so much as winding up and letting things go. In the rare instances where you are making minute-by-minute decisions, the space-time bubble fixes everything. So it doesn't matter that advancing by 30 days takes an hour, or whatever. I can read, cook dinner, grade stuff, watch movies, ect. The inconvenience comes from interrupts and having to constantly check the game to see if its finished a loadcycle.
...I am so glad I have not encountered these kind of speeds anymore since my second game.^^ It was always less than a minute for 5 day intervals since then, but after I recently discovered the manual processor assignment trick, I have it down to about 10 seconds for one production turn at year around 2280.
Title: Re: Idea for Speeding Up Aurora
Post by: MarcAFK on December 19, 2014, 12:11:52 AM
I've already forgotten where the time bubble option is :(
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 19, 2014, 01:54:54 AM
You need to be in SM mode, then reload a fresh F3 window, and then you will find it at the bottom of one of the tabs left. I have never used it before though.
Title: Re: Idea for Speeding Up Aurora
Post by: MarcAFK on December 19, 2014, 06:02:52 AM
You need to be in SM mode, then reload a fresh F3 window, and then you will find it at the bottom of one of the tabs left. I have never used it before though.
Oh, I think it's in the galaxy map window, and barely visible on my screen resolution.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 19, 2014, 10:16:18 AM
Possibly it is in the galaxy map simultaneously, but here it is under contacts in the classical system view:
(http://abload.de/img/unbenannttfsb4.jpg)
Title: Re: Idea for Speeding Up Aurora
Post by: 83athom on December 19, 2014, 11:25:22 AM
Yup, for laptop users that check box cannot be seen.
Title: Re: Idea for Speeding Up Aurora
Post by: MarcAFK on December 19, 2014, 11:25:32 AM
Ah, that's right, the screen is cut off just under the tac intelligence button.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 19, 2014, 11:52:06 AM
Yup, for laptop users that check box cannot be seen.
My computer is a laptop, but with 1080p. ;)
Title: Re: Idea for Speeding Up Aurora
Post by: Zenrer on December 27, 2014, 09:05:42 AM
Quote from: Vandermeer link=topic=7477. msg77342#msg77342 date=1418946749
. . . I am so glad I have not encountered these kind of speeds anymore since my second game. ^^ It was always less than a minute for 5 day intervals since then, but after I recently discovered the manual processor assignment trick, I have it down to about 10 seconds for one production turn at year around 2280.

Could you elaborate a little more on this? The eventual slowdown of the game is starting to get to me  :-\
Title: Re: Idea for Speeding Up Aurora
Post by: MarcAFK on December 27, 2014, 05:41:20 PM
Could you elaborate a little more on this? The eventual slowdown of the game is starting to get to me  :-\
you can use task manager to change processor assignment too.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 28, 2014, 10:27:32 AM
test - I wonder how many signs I can put in here before I get an error message. Zenrer, sorry, I cannot reply right now, because the Aurora forums had some problems, and currently it is bugging again. I tried to post stuff without extras - no thick or cursive text, no links, no quotes, just text, but it seems after just a small amount it wouldn't get accepted anymore. I could maybe make like 100 little replies to circumvent, but I'd rather wait.
Can we make it to 500 characters? Yes we can! -500-
And 600? That would be like birthday on christmas, with the easter bunny handing out shots --600---
Now the real achievement: 700. The well is twice as deep as in 300, and more spit on camera -700--

Ok, the other post down proved that 978 is possible, so new goal is 1000. Who bids higher? Do I hear 1001?

Hmm, I am just gonna fill this with nothing: abcdefghijklmnopqrstuvwxyz1234567890ß?*+~.;:-_#'´`\/()&%$§"!²³{[]}F1F2F3F4F5F6F7F8F9F10F11F12jackQueenKingAce god does this not end?!?!?!-999here-1008here-

New theory: This is all completely random, because at around 999 I could at first not post without error. Then I removed brackets, and it worked. Then I could go over 1000 even. I added those brackets again, but they aren't the culprit: ())(()(
So? Gremlins!
Still cannot post my original reply though, no matter how often I try. :(
Title: Re: Idea for Speeding Up Aurora
Post by: JacenHan on December 28, 2014, 10:46:15 AM
Could you elaborate a little more on this? The eventual slowdown of the game is starting to get to me  :-\
If you have multiple cores, you can assign Aurora to use one and let other functions use the rest. I think Vandermeer posted about it somewhere else, but I can't seem to remember where.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 28, 2014, 10:56:35 AM
If you have multiple cores, you can assign Aurora to use one and let other functions use the rest. I think Vandermeer posted about it somewhere else, but I can't seem to remember where.
The post was swallowed with the 1 week rollback of the forum it seems. :(
Well, there was not much more to it. I personally assign some cores further down, like 5 and 6, and then it flies. Aurora always seems to select the first one, even if that one is already busy with whatever runs in your background. I figured that out, because I like to watch movies or something on the side whenever running Aurora, because even though that has been proven to enlarge loading times, I'd rather be entertained through inevitable waiting periods rather than stare at the screen for some minutes.
However, manual assignment takes care of that, and it goes freely as if you were doing nothing else than focusing on Aurora.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 28, 2014, 11:26:33 AM
Hopefully this gets through now. The forum sometimes doesn't accept certain words, but only when they come after other certain words. There is no way to identify the rule behind this, so I will just black out words that the forum does not accept, so you will have to guess sometimes what should be there. See it as a game. :(
Could you elaborate a little more on this? The eventual slowdown of the game is starting to get to me  :-\
Well, aside *black* the only recently discovered processor trick, I've gotten my speeds by making sensible cuts to the game, which probably a lot of people wouldn't go with, since it might take away too much from their play-style. I've tested multiple theories for this to figure out what actually causes slowdown in later game stages, because it could be many things: Count of moving objects (ships, planets etc.), Count of total systems (there might be a calculation routine for every one of the, causing degrading the more you have), Count of officers, number of active sensors in the galaxy (seemed likely to be a calculation maelstrom), amount of accumulated resources (incl. fuel), Count of outpost(/colonized) worlds (again, seems likely to demand an extra calculation each).

By rigorous testing through my long past 2nd and 3rd Aurora game, I could rule out the following as being basically of no influence for production turn speed:
Resource amount, thankfully the count of found star systems, and surprisingly also the count of outposts. -Maybe that gets worse if you really overdo it, but I never experienced any frowning problems up to 30 settlements or what not, including around 8 1billion+ inhabited worlds. The only thing that happens is actually that big, not many colonies slow down the loading when selecting such a colony in the production window. This is however not due to bare size, but because of a surprise guest on the list of possible factors: ground units. Having more ground units on a colony significantly slows down the load-on-selection time. It starts to get annoying around 6 divisions, and with 10 it is becoming a problem.
...Can be avoided by putting them in cryo-modules until they are needed though, and suddenly the biggest colonies run nearly at 100% speed again. This is of course of utmost importance for your main colony to have (the others not too much), because the biggest one sits on top, and gets thus loaded when you summon up either production, research or economics windows, slowing that down by a lot. If you herd your ground forces adequately, you may load almost as on day one though. :)

So what stayed as factors are: officer count, sensor count, count of moving objects.

Officer count you will only notice at two moments: Whenever an assignment cycle is through (happens normally once a year or so, unless you change it), because then elaborate calculations determine where every officer goes on for the next assignment cycle, who is fired, and then some promotions happen too. Of course this becomes more demanding the more officers you have, but it is only once in a what time, and even then not as bad.
What I found is more of a problem is that the officer menu slows down quite a lot once you have around 500 in naval only, and that may become a problem. I've learned through this to not go over 20 naval academies if possible, because on that stage it already needs a few seconds on every window summoning.(this already gives potential for over 1000 naval officers, so unless one sports expansive fighter fleets, it should be enough for a long time)
This isn't really necessary though if one doesn't bother with selection loading time. I may break my own limitation in my most recent game later one for example, if I really go into megaton carriers and need lots of fighters.

Sensor Count I could not fully nail down to determine just how much it does, because you basically need a hostile NPR to test this out. I can confirm general slowdown for sure (does not extra influence production turns however), but if it may become really crippling if you let the NPR around,..I could not sort out individually. This is because I deal with any NPR I encounter quickly, and do not let them leave their system for exploration, because of the last and most important factor:

Moving objects. The one and only absolutely deciding factor in any game's pace are the amount of objects that fly through your game. This does not only include your and any possible NPR or whatever ships, but also most notably the civil fleets, and the orbiting bodies in systems.
- Orbiting bodies can be avoided by turning that option of in the game menu, though I must confess I am not sure on how strong it actually influences, because I never tested it separately (difficult to do), and had it turned of ever since after my 2nd game. Some people might find it breaks some important atmosphere with the realistic body motion turned off, and I can understand that. I just saw the potential that this may slow down the game with pretty much every newly discovered system, even if there is not much living in it, so a game would naturally slow down through exploration alone, and I didn't want to take the risk. I always play usability before presentation.

- Your own ships you can of course easily have under control, but NPR might spawn fast after being discovered. In one older thread here I presented that I played as  the Warhammer 40k human empire, because I noticed that the only sensible philosophy towards aliens is to violently exterminate them before they can spread. Even small NPR start spawning ships fast very soon, and then they start flying through jump points and discover worlds you have not seen yet (including even more NPR possibly about which you can do nothing - exponential slowdown trap). This might then cause the feared 'distant battle' slowdown, where you really cannot do much but watch 5 second intervals for some hours, because your master yoghurt senses a disturbance in the force. This is so threatening, because you never know if the game didn't possibly bug out, and those turns will never end. No joking, I once in my second game had a period of those slowdowns that lasted more than 10 hours. I watched the entire alien trilogy, then reviews on every movie of the trilogy, then parodies, all in that time, and it wasn't even nearly finished. They ended, but I decided I would never deal with this ever again, turned NPR on new games off by default, so that I can only find them, and if I do... Exterminatus! (http://www.greensmilies.com/smile/smiley_emoticons_xmas4_aufsmaul.gif)
Well, I have relativized that decision a bit since, and allow them to live in controlled environment peacefully with me, but I prepare measures to hinder them from leaving their origin, or grow too fast. ("..uhhh, no, this ship was infested by some ancient long extinct empire's AI virus, switched sides, and now goes rampage destroying your fresh jump fleet. Well, we couldn't have known.")
Once they are allies, and one can see all their ships and position, one might let them roam around even more freely, but always beware of letting them discover new systems still, because in the end they are just armed civil fleets with the same problems...

- Keeping you civil fleet under control is an absolute must. Many good things happened to the civil fleet management since 6.3, so it might now be feasible to have one without completely breaking your game on year 100, but you must always be knowing how to treat those to prevent disastrous multiplication. (for example you do not move infrastructure to venus, or else your civils will at first go filthily rich with that, becoming too many easily, and then also only ever to those jobs and not help you out with whatever you really need. All in all it is pretty easy to mess up your game with some AI trap or miscalculation, and it definitely costs you some processor resources in any case for those hundreds of ships to individually figure out where to go on every increment)
The most problematic thing about civils is that they can get out of control still, because there is no way to restrict their growth or even scrap them by empire decisions. In 6.4 they update their design efficiently, and scrap ships after some intended life time by themselves, so that does help out preventing this terrible exponential explosion. There is no guarantee though if you overlook something, and so you still often see threads on the forum where people got their games crippled to backwards time compression because of civil fleets with hundreds of ships in eternal meaningless transit.
A side problem is of course too that these ships might also get intercepted again, so there is potential for a lesser case of the 'distant-battle' slow down. You see them, but it will be a long encounter (a whole freighter trail), about which you can do nothing.

In 6.3 I had concluded civil is impossible to bear, and yes, it turned out to be basically the only most responsible factor of game slowdowns for later years, as when
Title: Re: Idea for Speeding Up Aurora
Post by: Erik L on December 28, 2014, 11:54:51 AM
Could you elaborate a little more on this? The eventual slowdown of the game is starting to get to me  :-\
Well aside from Vandemeer not being able to use "from" in this post.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on December 28, 2014, 12:02:43 PM
Well aside from Vandemeer not being able to use "from" in this post.
Now it is the letter "I". (http://www.greensmilies.com/smile/smiley_emoticons_headbash.gif) But only at that position where my post just ended. (sign number 9187...) (/edit: Correction, I cannot write anything more in the post above. That it ended on not being able to write "I" was strange coincidence. I am not at the sign limit though, so it is still shady, and still goes to the same error page)

I will continue here:

...I started to blow up the first ships of freshly created shipping lines (/preventing them ever making profit, and thus instantly bankrupt them at the root) , I suddenly had a fluent running game with my 3rd one and every other after that. I handled all my freight jobs and fuel mining per hand, which is really not that hard to do once you got the procedures down, and know how to intelligently make use of the stored order templates and order copy/+repeat options.
6.4 again is better with civils, and I will try them out again for the idea that I have for my next game in some future. It is always an argument that civils help you automate the game, so it goes faster, but even with the upgrades nobody can really cut out the performance drop that a couple hundred AI handled ships will inevitably bring with them, so one will always have to accept that trade of immersion with processor work. Automated, yes, faster, no, actually the opposite.

So to sum up the measures I took:
- store (most) ground units in some cryo modules if possible to prevent colony menu loading slowdown
- restrict academy count to either 10-16, or then as low as possible to man all your ships (+some extra) to prevent officer window slowdown
- turn orbiting bodies of to prevent mere exploration from gradually interfering with your game, and have huge star maps later on, yet still normal game progression. (not verified and sufficiently tested)
- do not ever generate NPR on game start, because of horrible slowdowns when they go to battle, which might even become eternal in some bugged sensor related AI trap cases.
- control found NPRs in some way, so that at least you know the position of all their ships all the time and do not risk their multiplication or 'distant-battle' slowdowns.
- Most importantly: Keep your civil fleet under control. Either know exactly what you are doing and live with a small performance impair, or just get rid of all of them and do the stuff manually.(requires designer mode password though)

There you have a recipe for a good and cleanly running game. Whether one decides that he can accept all those points, or maybe drop one here or there for the sake of his own game experience, is of course personal decision.

To prove it, here is a slightly older save from my recent game at year 2221, which still runs smoothly.(I think the ground units are not currently in their cryos {/barque ships} though, so beware :P)
http://www62.zippyshare.com/v/79549364/file.html (http://www62.zippyshare.com/v/79549364/file.html)

I've played other games to well beyond year 300 with the same results, and can safely extrapolate that this is how Aurora stays running and smooth.
Title: Re: Idea for Speeding Up Aurora
Post by: Barkhorn on January 06, 2015, 06:19:50 PM
I've had an idea for awhile now on how to deal with the civilian fleet problem.

Just stop simulating them perfectly 100% of the time.  Switch to a trade route system instead, abstract the whole system 90% of the time.  The routes would have a density, i.e. number-of-ships / length of route.  Trade and installation shipment rate should be computable without calculating exactly where every cargo ship is all the time.

Then, if a hostile ship's sensors are within range of the route, you roll some dice and decide based on the density of the route if one or more civilian ships are there.  If yes, spawn them and then its business as usual.  If no, sorry alien raider, no juicy freighters for you.  Any civilian ships that survive an encounter get despawned and put back in the abstraction once the alien leaves.

This idea would cut the civilian shipping CPU cost down from O(s) per increment where s is the number of ships, to O(r) where r is the number of trade routes.
Title: Re: Idea for Speeding Up Aurora
Post by: rcj33 on January 07, 2015, 05:17:45 PM
Vandermeer, do you mean that the GU should be loaded into cryo drop modules only? Or will troop transports/PDC barracks work as well?

(Still haven't started my 15B pop game due to deleting and recreating 500ish GU being something i only feel like doing incrementally)
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on January 08, 2015, 03:38:28 AM
Normal troop transports would work as well, though I just prefer cryo modules because there the troops don't lose morale while being in it. I don't know if morale loss applies when a normal transporter is in planetary orbit though, so it could be ok.(always skipped the tech right for cryogenic transporters)

I am not sure if PDC barracks would work either, because I never had them bigger than for 1 division so far, so the difference wasn't apparent. I see this as risky, because the troops never really get unassigned from their colony here, as they are found just under a dropdown-menu in the troop tab still. Maybe that doesn't factor into the loadup calculations though, so it could be fine.
There is a 13 division fortress PDC currently in the production line of my recent game, so maybe I can report later if it does help or not. Having fortresses is still useful either way, because you can just station your troops there in case of assault then.
Title: Re: Idea for Speeding Up Aurora
Post by: rcj33 on January 19, 2015, 08:20:30 AM
Afaik, troop transport modules don't cause GU to lose morale, only normal drop modules. I wonder if they prevent morale gain? Anyway, I tested and neither orbiting troop transports nor PDC barracks seem to work for reducing the lag. Seems the troops are still counted as being on planet. Do you think something would change with a multi-division PDC? That could be my problem (two divisions each in nine transports, one each in 19 PDCs).
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on January 19, 2015, 01:37:45 PM
The orbiting ones should work definitely. Do I get that right that you have 18 divisions (not brigades, but full scale divisions)? I was going to bet that you had too few troops to notice the effect, but 18 would be guaranteed to be enough.
Just to confirm this again, I went into my game, where there are currently 7 divisions and 5 brigades stationed on my home planet, and one 5 division transporter to lift most of them away. With ~8 divisions lag is not too bad, but I could stop the clock and found to gain 1-2 seconds easily here.(before 3-5, now 2) To hold against that I went into SM mode and changed all other possible factors gradually, first population to 100b - nothing, then 100mt+ of every mineral + millions of MSP and 100b fuel - no difference again, then thirdly about 20k of every building there is, including naval academies again - a second of drop maybe here, but could be wrong and is nowhere close to the up to 10 seconds you could wait when having well over 10 divisions stationed.

So my best bet is that in fact the troop transports do not work here for some reason. There is a tab under the ground units that lets you display troops from ships in orbit, but that also displays troops in cryo bays... . I think I have two theories for this now.
1. The lesser theory is that the slow-down has to do with morale calculation, because since those stop with cryo bays, that could explain the difference.
2. Much stronger, this all is probably just about the calculation that Aurora needs to make to sort the long list of troops. The way sub-groups like the brigade and divisions get sorted is probably complex, flawed and thus too demanding, meaning the longer the list, the longer (exponentially) the drag. Having troops in PDC or ships can reduce this problem as long as they are partially in there, because that would give you two partial list (and 2x1²<1x2² in matters of loading time). The impact is however gone if you assign all the troops to one of the list, no matter if ground, ships, or maybe even PDC (double unsure here though), because then a single list is just as long again.

Would you mind testing that theory 2 in your game? Easy disprove would be to carry away filled troop transporters a bit from the planet, so they cannot be listed anymore. If that does however work good for the loading times (check the load-up multiple times please, because the first is often always longer), then as second testing stage assign only half the divisions to the transporters, and see if that makes a difference then still.
If it doesn't, then either 2. is wrong or troop bays do indeed not work out the same way, since cryos definitely made a difference here.(even in orbit)
Title: Re: Idea for Speeding Up Aurora
Post by: rcj33 on January 20, 2015, 02:54:34 PM
Yes, the 18 (now 19) divisions were fully manned. I think they are gone now though :(

I decided to make things a bit easier on myself and built something to hold a few more troops than normal:

Code: [Select]
ARMYWIDGET class Troop Transport    793,600 tons     5110 Crew     27089 BP      TCS 15872  TH 32000  EM 0
2016 km/s     Armour 1-763     Shields 0-0     Sensors 1/1/0/0     Damage Control Rating 1     PPV 0
MSP 21    Max Repair 100 MSP
Intended Deployment Time: 3 months    Spare Berths 0    
Troop Capacity: 210 Battalions    Cargo Handling Multiplier 4000    

AARC 500320 VASIMR-E+ (100)    Power 320    Fuel Use 2.53%    Signature 320    Exp 4%
Fuel Capacity 250,000 Litres    Range 2.2 billion km   (12 days at full power)

This design is classed as a Commercial Vessel for maintenance purposes

Here's what I found in terms of average refresh times:

Nessus (Alpha Centauri B-I, a listening post with no garrison yet): <1s; I counted "one-missi" consistently so probably about one-half. I couldn't measure it accurately with the stopwatch.
Luna with all troops on colony: not timed but seemed like a good while
Luna, troops in TT: 2.6s
Luna, troops in TT with move order but insufficient fuel: 2.5s
Luna, troops in TT at random waypoint: <1s; about the same as Nessus.

Then I put two brigades of each division into a PDC and the other two into the TTs.
Luna, troops split among PDCs and orbiting TTs: 3.4s
Luna, troops split among PDCs and TTs in near space: 2.1s
Luna, troops split among PDCs and TTs at random waypoint: 2.2s

Normal troop transport modules do indeed work for reducing troop lag, as long as they aren't at the colony in question. I shaved off about 2s of load time, which is great news. I'm definitely starting my next campaign with a normal sized pop and adding GU later - 3000 or so operations on the troop management screen killed most of my Sunday evening.

However, the second set of tests didn't prove or disprove your other theory conclusively. To be sure whether distributing storage locations within Stevefire matters, I had to get a control for those tests too, namely a trial where the troops in the TTs didn't exist in the database at all; since I didn't want to disband 210 or so units for 10 minutes, I deleted the TG containing them. So I don't really know the answer to this one, and I don't recommend testing it, either, since I wasn't able to restore from backup for some reason.

Edit: OK. the restore did work, I just wasn't in SM mode. Whew.
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on January 20, 2015, 06:49:06 PM
Clean examination there. :) Glad it worked out, but strange that troop transports may not work in orbit like the cryos do. As far as I can see in your data, there is only one test with Luna needed where all the troops you used in the test are neither in PDC or TT, but on the colony itself. If that results in larger loading times than the 3.4s you had for the split, then the problem comes indeed from the list length, and TTs may still help a bit when being in orbit.(as would PDCs)
Title: Re: Idea for Speeding Up Aurora
Post by: Nathan_ on January 20, 2015, 11:27:13 PM
1) No detection
2) Automatic detection

Interested to hear comments and feedback

I'd have to say roll randomly for detection based on distance(via empire tech maybe? and if something is detected have it stay detected) is the simplest thing to do. How does aurora's sensor model work by the way?
Title: Re: Idea for Speeding Up Aurora
Post by: linkxsc on January 21, 2015, 11:33:08 AM
Best advice i can give anyone for keeping your game fast. Gen another empire, give then 1 or 2 destroyers, and stick them in some dead end system that isnt worth much. Then every couple years, have them fly out and start culling your civilian shipping fleet. Personally i do this by genning the empire in sol, having them make a colony on 2009 y sedna. And throw a bit of minerals on there for them. Then periodically go out hunting. Specifically i try to take out all the small and medium ships, the only 1s allowed to live are the large and huges.

Made the terrible mistake in my current game of turning the 7x10^6 duranium that genned on luna into infrastructure/underground infs.

Took a while, but you often have a lot of spare time early on in conventional starts. Then i went and put human colonies on every asteroid, several dwarf planets. Now i have a single shipping line, 39 colony ships, 25 freighters, 19 fuel harvesters, and 2 "luxury liners". And the slowdown is definitely substantial.
Title: Re: Idea for Speeding Up Aurora
Post by: rcj33 on January 22, 2015, 11:03:40 AM
Clean examination there. :) Glad it worked out, but strange that troop transports may not work in orbit like the cryos do. As far as I can see in your data, there is only one test with Luna needed where all the troops you used in the test are neither in PDC or TT, but on the colony itself. If that results in larger loading times than the 3.4s you had for the split, then the problem comes indeed from the list length, and TTs may still help a bit when being in orbit.(as would PDCs)

Yeah, I believe that would've been the last test after unloading all the troops to Luna from the TT (since they're supposed to be there in the actual campaign). I didn't feel much like doing it at the time though ::)
The good news is that the results are in from this morning:

Luna, troops half in PDC and half dead from bombardment elsewhere (much faster than click and wait X 200, and no risk of database corruption - should've done this instead): 1.3s
Luna, 19 divisions present on colony (half in PDC): 3.1s
Luna, 19 divisions present: 3.9s

So it seems like splitting where your troops are to increase the speed at which Aurora can deal with them might work to some extent (inconclusive due to human error of ~ +-.5s), but I can't be sure without using way more divisions. 10x would do it for sure but for obvious reasons I won't be trying that without database access  :P It doesn't seem to make much difference for me, though. The good news is the ~4x faster load time without all those units on the colony, which is even better than I thought, though YMMV depending on your troop count. Feel free to point out any mistakes or make suggestions - I've done what I could but my STEM skills have gotten a bit rusty.

edit:I forgot to mention that I arrived at the error figure by waiting until the stopwatch said 4 seconds had passed and then pressing stop...it averaged to about 4.25s, which I doubled for safety
Title: Re: Idea for Speeding Up Aurora
Post by: Vandermeer on January 23, 2015, 03:10:04 AM
Ok, thank you for the completion. The data with half troops dead and other half in PDC seems off, because that is essentially the same as having half in PDC and half taken away by leaving transports like in one of the tests before, which had different results. But the rest seems to be good indication that the split up does something, so it may be the sorting problem.
I will soon have a great PDC HQ in my current game too, so I may add to that data. Though I currently only have around 10-11 divisions strength, which makes testing inaccurate, but there should be something to see still.
Another thing that I will test is assigning all those troops from their division, and see how that makes a difference or not. If it noticeably does, then this would be proof too.
Title: Re: Idea for Speeding Up Aurora
Post by: rcj33 on January 23, 2015, 10:52:39 AM
I'm a bit hazy on why I did that myself ;D  Let me know if you find anything