Post reply

Warning - while you were reading 8 new replies have been posted. You may wish to review your post.
Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Note: this post will not display until it's been approved by a moderator.

Name:
Email:
Subject:
Message icon:

shortcuts: hit alt+s to submit/post or alt+p to preview

Please read the rules before you post!


Topic Summary

Posted by: wedgebert
« on: August 04, 2018, 10:04:07 PM »

The one place that would be "relatively easy" (at least from a multithreading stand point) is computing the orbits of objects with fixed orbits. As of right now, there's nothing you can do in-game to affect the orbit of Venus aside from disabling orbits altogether. For objects with moons, once the parent position has been computed, the moons can be calculated.

So, while you're designing your next missile, the game could be calculating the next movement phase (or three) of orbits. If your turn ends before the calculations are done, you just have to wait for them to finish their maths because that's how the game would have worked anyways.

I'm not sure how much of a benefit this would provide, and I don't know how the current orbital mechanics work. It's just the most separated game system I can think of. Other systems, like ship movement, AI, etc have too many factors that could affect their logic to easily put in their own threads. But planetary orbits are just "I move this direction, this much".

And even if it's not precalculated, C# has a simple Parallel.ForEach method that can do the threading during the increment processing. You just have to be careful with your data manipulation (calculate the orbital position in the thread, return those values to the main thread, and the main thread is the one that actually sets the position values).


Posted by: SimonS3
« on: August 04, 2018, 08:15:12 PM »

guess we have to invest in a faster processor then ;), at least if we have the option to load the entire database into memory that should significantly cut down on the  read writes.  is there any consideration being made to change the supported database? PostgreSQL  gives Oracle a run for its money performance wise and is happy enough on a windows platform.  Btw Love this game!

cheers
Posted by: Neophyte
« on: August 04, 2018, 11:54:45 AM »

Yeah, regardless of whether or not this changes anything in Aurora this total idiot found it interesting.   Thanks!
Posted by: Jovus
« on: August 04, 2018, 11:00:54 AM »

Apologizes if the stuff above is stuff you already knew about.  Also, I'm not advocating you jumping in and parallelizing, just making a Public Service Announcement that it's available :)  Happy to discuss more if you're interested.

John

I didn't know all this, which is funny because I'm working with FEM approximations of thermodynamic fluids, but I found it both fascinating and helpful! The general gist is something I'd never been told, but more-or-less stumbled into myself in an inexpert and unexamined way. That means I really appreciate having the big picture laid out like this, because now I can consciously consider that in my work.

Thanks a ton!
Posted by: sloanjh
« on: August 04, 2018, 09:25:21 AM »

Currently the program is single-threaded. Due to the sequential nature of events, the scope for multi-threading is limited. Also, multi-threading is complex and can create difficult to pin down bugs, so there needs to be sufficient upside to go down that route.

Having said that, if I find an area that has performance issues and has potential for multi-threading, I will look into it at that point.

One thing that can cut down on the robustness issues is to do "SIMD" (Single-Instruction, Multiple Data) parallelism rather than "MIMD" (Multiple-Instruction, Multiple Data) parallelism when using threads. 

MIMD is what most people think of/use when they're doing multi-threading: every thread is running its own little chunk of program, with different stack pointers and (potentially) different branching in the code.  As you say, this is hard to keep robust, since the individual threads don't have Turing behavior, so you have to be careful about globals causing deadlock etc.

An example of SIMD is GPU programming, where you only do "data parallel" operations.  The parallelism comes from running a small (in lines of code) loop that does the same operation on arrays of objects (rather than individual objects).  For example foreach(ii){c[ii] = a[ii]*b[ii];}.  The idea is that the CPU is running the same instruction on all the data in parallel, so you can't put any branches in the loop that depend on the data.  These terms originally came from supercomputing; Connection Machine 2 was an example of a SIMD machine, while intel clusters can be configured to do MIMD (in a multi-processing context).  The old "vector" machines like Crays also fall into the SIMD category in terms of structuring the program, although under the covers a vector machine is performing the operations in the loop in a slightly different way than a SIMD machine would - the CPU is able to start the next floating point operation in a vector loop before it's finished the one it's working on. ("vector" was synonymous with "array", and a vector operation was an operation performed in parallel on arrays of data).

The thing you have to be careful with in vector/SIMD programming is that the different iterations of the loop are truly independent - the 9th iteration can't depend on the results of the 5th iteration.  For things like sensor detection this shouldn't be a problem; for movement you would need to be careful to separate out TGs that are following other TGs (and have better initiative) and do those in a second, follow-up loop.  Otherwise the thread updating the following TG might execute before the TG it's following has moved and it might aim at the old location.  This is MUCH simpler to conceptually manage, however, than the usual stuff with multi-threading involving global variables etc.

The reason I'm going here is that C# has a lovely little "threadpool" mechanism and "System.Threading.Tasks.Parallel" class that makes it dead simple to code up data parallel loops https://docs.microsoft.com/en-us/dotnet/standard/parallel-programming/data-parallelism-task-parallel-library.  The example they use in the link is overly ambitious in terms of MIMD vs. SIMD (since they're calling functions inside the body of the loop), but use can use the same stuff to code up small loops like I listed above and the use of the threadpool makes the overhead for launching the threads small.  I realized a few years back that vector code (written for e.g. a Cray) could be directly transcribed to multi-threaded code simply by switching all the parallel loops over to using Parallel.  Disclaimer:  I don't think I've ever actually used the Parallel stuff in C#, but I've got colleagues who have.

The downside of the above is that the structure of the code tends to be very different - your core data objects when writing the logic of a function are vectors/arrays rather than elements of the vectors that are used in "scalar" code.  In other words, scalar code tends to have a big outer loop over an index, then have very complicated logic with branching etc. inside that loop that is only working on that one slice of the data corresponding to the loop index.  In vector code this is turned inside out (again think GPU programming) - the complicated logic is on the outside and the individual operations in that logic are performed on entire arrays at a time.  So for example a double temporary variable in scalar code is a single double; in vector code it's an array of doubles.  So this can lead to big rearrangement of code for e.g. scientific programs that need to be data parallel everywhere.

The good news is that Aurora probably has fairly isolated loops that can be rearranged in isolation, so if you find hotspots in e.g. sensors or movement, it should be fairly easy to parallelize them.

Apologizes if the stuff above is stuff you already knew about.  Also, I'm not advocating you jumping in and parallelizing, just making a Public Service Announcement that it's available :)  Happy to discuss more if you're interested.

John
Posted by: Steve Walmsley
« on: August 04, 2018, 04:52:04 AM »

Currently the program is single-threaded. Due to the sequential nature of events, the scope for multi-threading is limited. Also, multi-threading is complex and can create difficult to pin down bugs, so there needs to be sufficient upside to go down that route.

Having said that, if I find an area that has performance issues and has potential for multi-threading, I will look into it at that point.
Posted by: Profugo Barbatus
« on: August 04, 2018, 04:43:07 AM »

Not so sure about that. Multithreading only offers noticable advantages if the processes are capable of being paralleled, and if its programmed to support it. I don't know enough about how Steve's built things out and what order of operations his code relies on for the assumptions it may make, to know if he could have, or did, thread anything.
Posted by: SimonS3
« on: August 03, 2018, 02:11:47 PM »

I think a major benefit of the new C version is that it will be able to support  Multiple processors and take advantage of loading almost the entire universe into memory that alone would give one a significant improvement multi threading on a quad core system with a few GB's of memory. 
Posted by: waresky
« on: June 16, 2018, 11:33:30 AM »

My campaign :

1 Unknow Alien Empire (out there)
3 NPR on Earth (China,East Coalition,Russia Hegemony)
1 NATO player

5-Day turn : 15seconds per turn..

Aurora standard its outdated. unplayable,at last.

Amen.

Am waitn for #C.
Posted by: sloanjh
« on: May 30, 2018, 07:11:56 AM »

The vast majority of lag in VB6 aurora is from vessels moving, either pathfinding calculations or sensor calculations.  It's a function of how many ships you have, not AI.

Wasn't it also because the VB6 version reads and right from the database at every single increments which made it super slow? I remember the C# version will only read from the DB at start and write to DB when saving. So the game will mostly run from ram?

This optimization was already put into VB6 years ago by Steve.  It gave a huge speedup.

John
Posted by: GeaXle
« on: May 30, 2018, 04:20:41 AM »

The vast majority of lag in VB6 aurora is from vessels moving, either pathfinding calculations or sensor calculations.  It's a function of how many ships you have, not AI.

Wasn't it also because the VB6 version reads and right from the database at every single increments which made it super slow? I remember the C# version will only read from the DB at start and write to DB when saving. So the game will mostly run from ram?
Posted by: Steve Walmsley
« on: May 27, 2018, 04:49:04 AM »

The vast majority of lag in VB6 aurora is from vessels moving, either pathfinding calculations or sensor calculations.  It's a function of how many ships you have, not AI.

The game likely won't be much slower when AI is implemented.

Working on AI at the moment, so we should know soon :)
Posted by: Father Tim
« on: May 26, 2018, 11:30:23 PM »

Will C# be more stable and take less time to process increments?

This is the entire reason for the C# rewrite.  Granted, Steve has taken the opportunity to change things that ended up not working the way he wanted them to, or that he now has better ideas for. . .  and we won't know until C# Aurora is actually released. . .  but yes.
Posted by: MarcAFK
« on: May 26, 2018, 10:00:52 PM »

It's not going to be a significant difference untill you have several hundred ships.
Posted by: QuakeIV
« on: May 26, 2018, 09:51:30 PM »

I think task groups pathfind collectively so not quite, but with sensor calculations yes that is always better.