Quasar4x - An Aurora4x VB6 clone q4x

[Kyle (OP)]

Progress Update 2020-01-29:

No new features in this update but I made some important optimizations that I'm excited to share!

In my mission to add missiles to q4x, I had completed the first two phases which were deploy missiles and move missiles.  The next logical step was to implement missile detection since it happens before any impacts occur.  But at about this time, I saw this post in this forum regarding concerns about the apparent slowness of 5-day auto turns.  The animation posted showed a rate of roughly 1 second a turn, maybe a bit longer.  I actually hadn't checked how long turns took on the "days" scale for awhile, having been focused on the "seconds" scale for my recent work.  I was vaguely aware that the things I was adding were going to inevitably lead to slowdowns I'd eventually have to address, but I hadn't actually seen the effects yet.  The missile animation in my previous post does include the slow code but it's hard to tell because each frame of the gif is a single 5 second pulse, not 60 of them.

My response to the post was that I agreed this was way too slow for the amount of work the program is doing, and that it would be fixed in a later optimization phase after more features were done.  But the slowdown nagged at me in the back of my mind, and I decided I wanted to make sure that significant optimization was indeed possible as I had claimed.  This would also be a perfect time to finally test a plan I laid out in the OP of this thread, which was to "implement C++ optimizations where profiling suggests it's needed".  I hadn't actually done this yet, it was still just a theoretical improvement.  So the first thing I did was profile my current test game, running a 5 minute turn consisting of 60 sub-pulses at 5 seconds each.  Here were the results:

Code: [Select]

Total      Avg Name
-------- -------- ----
     597     9.95 detections done for all systems (gdscript)
     170     2.83 handle lost contacts (gdscript, but mostly SQL)
      66     1.10 Combat
      59     0.98 Movement
      23    23.00 other
      23     0.38 detection prep (gdscript, but mostly SQL)
     ...
Total: 1005 ms
The detection phase was by far the clear offender, and not suprising due to the nature of all the factors that go into detection.  There's even a 'Disable Detection in this System' in A4X 7.1 for this reason.   So I rewrote the detection code in C++, and integrated the resulting .dll (and .so for linux) with the Godot side.  That's obviously a lot easier said then done;  It took quite a bit of effort to figure out all the ins and outs of how to do the integration part.  But the good news is that now I know how to do it, so any future C++ integrations will be easy!  I opted to keep the logic itself intact, so I could compare logs and database changes between the C++ and gdscript versions of detection to ensure the results were identical.  And speaking of results, check out the new stats on the same save file after switching detection to C++:

Code: [Select]

Total      Avg Name
-------- -------- ----
      67     1.12 Combat
      64     1.07 Movement
      61     1.02 detections done for all systems (C++)
      25    25.00 other
      23     0.38 detection prep (C++, but mostly SQL)
      18     0.30 handle lost contacts (C++, but mostly SQL)
...
Total: 339 ms
I'm super happy to see such huge differences!  "Detections done for all systems" is where the bulk of the work is done.  Check out the improvement in that section, nearly a 10x speed boost!  I also took a few minutes to adjust an SQL query in the "handle lost contacts" section that I realized was written in a correct but poorly optimized way, resulting in another nice speed boost.  There are other optimizations in the detection logic itself that I know are possible if necessary, and of course plenty of optimizations possible in other areas, but those can be saved for later.

The original post that brought the speed issues to my attention was running 5 day auto turns with interrupts disabled.  Here is an recording of the 5 day turn speed on my own test game before and after the above optimizations.

Before:
.........

After:


In general, it's best to save most optimizations until the section you're working on is feature complete, but this was a super important step to do right now.  I now have a proof of concept that it is indeed possible to transition any problematic parts of the Quasar4x code into C++ and that the performance improvements are very good.  And I now have notes and an example to go on for C++ migrations I make later on.  It's not even that bad that I still have missile detection code to add, because now it can be added to the C++ side and never has to be transcribed later on.  And speaking of missile detection, that's up next!  Onward!

[edit: it's not 2019 any more. doh.]

[joansam]

Wow, nice work! There's something super satisfying about optimizing code. And it's really good to know that those kinds of optimizations are possible and make a real difference. Looking forward to missile detection!

[Kyle (OP)]

Progress update 2020-02-03:

Missile detection is done!

Latest build has been pushed.  The following items were added:

- Active sensors on missiles
- Firing missiles at waypoints, other missiles, populations, ground forces, and shipyards
- Thermal and active sensors detect missiles
- EM sensors detect active sensors on missiles
- EM and Thermal Planetary sensors fully functional
- Active, EM, and thermal sensors on missiles fully functional
- Missile geological surveys
- Missiles are destroyed if they are moving and have no time remaining
- Handling of missiles that lose their targets, including retargeting
- 'Launch Missiles At 'fleet order

And a few more UI elements finished:
- Combat Overview > Missiles in Flight list, Destroy missile, Destroy All missiles
- Fleet Orders > Msl Launch
- Ships window > Msl Launch




The above demonstrates active sensors on missiles, and very briefly you can see the detection of those active sensors in the enemy's POV.

Nice framerate on these animations now, huh?





Animation PAUSED at key points.

The above demonstrates missiles with EM sensors detecting shields, missiles with thermal sensors, and the opponent's active sensors detecting the missiles.  Also note that shields continue to show after the EM missiles disappear because at that point they have charged enough so that the opposing ships can detect them.  And I updated the code after making the animation so that the correct 200 in shields shows at the end.  And as mentioned in an earlier post, no editing necessary for the side-by-side views.  That's done with the in-game 'Second System Map' option.

2 or 3 phases to go.  Impacts, damage to the various types of targets, Secondary stages, and a few other bits are still to-do.  Onward!

[Garfunkel]

Great job Kyle!

[Kyle (OP)]

Progress update 2020-02-06:

Missiles hit ships!  (And miss too)



Here's an excerpt from the event log when a ship is destroyed:

Off-Topic: events • show

2025 February 2, 01:44:21,The Roman Empire,Chance To Hit,Rome,Salvo of 2x Size 4.3 Missile has intercepted its target. Chance To Hit 100% (Target Speed Zero)
2025 February 2, 01:44:21,The Roman Empire,Damage,Rome,Chtrfiak  009 hit by 16 points of damage from Size 4.3 Missile
2025 February 2, 01:44:21,The Orcs,Damage,Ol,Block of Metal 009 hit by 16 points of damage from Size 4.3 Missile
2025 February 2, 01:44:21,The Orcs,Shields Inactive,Ol,The ship has no shields
2025 February 2, 01:44:21,The Orcs,Crew Grade Increase,N/A,As a result of combat, the crew grade bonus of Block of Metal 009 has increased to 0.28%
2025 February 2, 01:44:21,The Orcs,Damage Absorbed,Ol,All damage absorbed by armor.
2025 February 2, 01:44:21,The Roman Empire,Damage,Rome,Chtrfiak  009 hit by 16 points of damage from Size 4.3 Missile
2025 February 2, 01:44:21,The Orcs,Damage,Ol,Block of Metal 009 hit by 16 points of damage from Size 4.3 Missile
2025 February 2, 01:44:21,The Orcs,Shields Inactive,Ol,The ship has no shields
2025 February 2, 01:44:21,The Roman Empire,Intelligence Update,Rome,Chtrfiak  009 is streaming atmosphere
2025 February 2, 01:44:21,The Orcs,Crew Grade Increase,N/A,As a result of combat, the crew grade bonus of Block of Metal 009 has increased to 0.32%
2025 February 2, 01:44:21,The Orcs,Damage Absorbed,Ol,15 damage absorbed by armor.
2025 February 2, 01:44:21,The Orcs,Shock Damage,Ol,1 additional internal damage due to the shock caused by the incoming fire.
2025 February 2, 01:44:21,The Orcs,System Destroyed,Ol,Crew Quarters - Tiny destroyed on Block of Metal 009 after receiving 0 points of damage. 0 Casualties.
2025 February 2, 01:44:21,The Orcs,Ship Destroyed,Ol,Block of Metal 009 suffers complete structural failure due to the amount of damage it has received. The ship is destroyed.
2025 February 2, 01:44:21,The Roman Empire,Enemy Ship Destroyed,Rome,Chtrfiak  009 has been destroyed by FF Shark M 001
2025 February 2, 01:44:21,The Roman Empire,New Wreck,Rome,New Wreck Detected in Rome
2025 February 2, 01:44:21,The Roman Empire,Target Lost,Rome,2x Size 4.3 Missile cannot find its target. It will home on the previous target location and use onboard sensors to search for a new target
2025 February 2, 01:44:21,The Roman Empire,Target Lost,Rome,2x Size 4.3 Missile cannot find its target. It will home on the previous target location and use onboard sensors to search for a new target
2025 February 2, 01:44:21,The Roman Empire,Target Lost,Rome,2x Size 4.3 Missile cannot find its target. It will home on the previous target location and use onboard sensors to search for a new target
2025 February 2, 01:44:21,The Roman Empire,Target Lost,Rome,2x Size 4.3 Missile cannot find its target. It will home on the previous target location and use onboard sensors to search for a new target
2025 February 2, 01:44:21,The Roman Empire,Target Lost,Rome,2x Size 4.3 Missile cannot find its target. It will home on the previous target location and use onboard sensors to search for a new target
2025 February 2, 01:44:21,The Roman Empire,Explosion Detected,Rome,Nuclear Detonation: Strength 16 detected! (x2)
2025 February 2, 01:44:21,The Orcs,Explosion Detected,Ol,Nuclear Detonation: Strength 16 detected! (x2)

The log for the whole fight is at https://pastebin.com/raw/ugUsErZt.

Most of the changes in this update are evident above but additionally:
- I did some testing with a 3rd observer race, and made quite a few fixes to who sees what events in the log so that it's now in line with A4X 7.1.  3rd party observers receive much less technical information in the event log about what goes on in a fight.
- I added a couple new messages I had overlooked in previous combat testing, which are reports to the attacking race of streaming atmosphere and the 'enemy ship destroyed' message.
- In addition to nuclear detonations, engine explosions are also detected. 

Next up is missile-missile collisions!

P.S.: just noticed the speed is missing from the defending contacts. Fixing that first

[Kyle (OP)]

Progress update 2020-02-07

AMM's are done! And missile-based PD!!



Note that I temporarily modified the code to hide the speed, endurance, and salvo ID's to reduce the clutter in the recording. 

The full event log for the above animation is at https://pastebin.com/raw/xScXVqfD.  The fire control setups were:
AMM+Scout TG:
Missile Fire Control FC38-R1 #1 (2v1 PD Mode 300):   11 Launchers
Missile Fire Control FC38-R1 #2 (1v1 PD Mode 300):   3 Launchers
Missile Fire Control FC38-R1 #3 (1v1 PD Mode 300):   1 Launcher
Tank TG and Tank TG2:
Missile Fire Control FC38-R1 #1 (2v1 PD Mode 300):   11 Launchers
Missile Fire Control FC38-R1 #2 (2v1 PD Mode 300):   3 Launchers
Missile Fire Control FC38-R1 #3 (1v1 PD Mode 300):   1 Launcher

Latest build has been pushed.  Items added:
- Missiles can hit other missiles. (Or miss)
- Chance for missile armor to nullify incoming damage
- Warhead strength must be 1 or higher for an explosion to occur
- Overkills are reported
- All missile-based point defense modes

I studied how A4X missile-based PD behaves in general, but rather than trying to match its decisions frame by frame, I rolled my own point defense logic.  It seems to be doing alright

Going to take the weekend off from Quasar, then beam-based point defense is up next!

[Breadabix]

Kyle you definitely deserve a weekend off, made some great progress, and for that I thank you.  Cant wait to play

[Kyle (OP)]

Progress update 2020-02-11

Beam-based PD is done!

Here is Area PD mode:



And the log for the Area PD: https://pastebin.com/raw/N6XZAFQB

Tank TG contains 5 ships with the following config:

PD Fire Control S01.2 96-7812.5 #1 (Area PD Mode 15):   No Target Assignment
PD 25cm C6 Soft X-ray Laser #1 (Ready To Fire)
PD 25cm C6 Soft X-ray Laser #2 (Ready To Fire)

PD Fire Control S01.2 96-7812.5 #2 (Area PD Mode 15):   No Target Assignment
PD 25cm C6 Soft X-ray Laser #3 (Ready To Fire)

PD Fire Control S01.2 96-7812.5 #3 (Area PD Mode 15):   No Target Assignment
PD 25cm C6 Soft X-ray Laser #4 (Ready To Fire)



And here is Final Fire PD mode:



I slowed the animation down to 1 FPS.  This is the event log for the above:  https://pastebin.com/raw/0quzsBRx

All four fleets are providing FFPD and have 2 or 3 ships with the following config:

PD Fire Control S01.2 96-7812.5 #1 (Point Blank PD Mode 10):   No Target Assignment
PD 25cm C6 Soft X-ray Laser #1 (Ready To Fire)
PD 25cm C6 Soft X-ray Laser #2 (Ready To Fire)
PD 25cm C6 Soft X-ray Laser #3 (Ready To Fire)

PD Fire Control S01.2 96-7812.5 #2 (Point Blank PD Mode 10):   No Target Assignment
PD 25cm C6 Soft X-ray Laser #4 (Ready To Fire)



Latest build is pushed, Version 84 according to Itch.  I figure I'll start using Itch's version numbers for now.  Items included with this build are:
- Beams can hit (or miss) missiles
- Area PD mode
- Final Fire PD mode
- Final Fire (Self Only) PD mode

As in Aurora 7.1, final fire PD shots occur in the movement phase where missile collision is resolved and require an Active Sensor contact (otherwise there's no confirmation the signature is in fact a missile!).  Since I use the same sequence of play as Aurora, it means an Active Sensor detection is required in the previous pulse or FFPD will not occur.

As previously reported in this thread, tracking penalties don't seem to always apply when beams shoot missiles in A4X.  I've made sure to include the tracking penalty, and I route every single situation where a beam weapon fires through a single routine so calculations are guaranteed to be identical no matter the scenario. 

As in missile-based PD, the decision making logic is my own and won't necessarily match Aurora's.  Generally, distance takes first priority, then enemy missile size.  In area mode, some attempt is made to use fire controls with the most weapons on enemy salvos with the most missiles.

That's all for now.  Lots more combat to work on! 

[EvadingHostileFleets]

In Progress update 2020-02-06 I see what seems to be missiles changing target mid-course if original one is destroyed.  I may be wrong for I am newbie, but does Aurora behave like that? I observe so far that missiles come to the wreck and only then search for new target, provided that they have sensors.

[Gabethebaldandbold]

In Progress update 2020-02-06 I see what seems to be missiles changing target mid-course if original one is destroyed.  I may be wrong for I am newbie, but does Aurora behave like that? I observe so far that missiles come to the wreck and only then search for new target, provided that they have sensors.

As far as I know that is the case in Aurora

[Sunmannus]

Hi Kyle, is there a technical reason why you can't create a 32-bit version of quasar?
I used to play Aurora on a laptop with a 32-bit OS, so if possible, I would appreciate it if you could create a win32-compatible version (for the final versions I mean, not during development . . . )

Regards.

[obsidian_green]

This is amazing. I love the current version of Aurora 4x, but I was never going to buy a supercomputer to play it, which meant death by slowdown that eventually became even worse than the combination of pathing and exponential dirty clothes that overwhelmed my DF experience. This will be a speedy 7.1 with bug fixes? Terrific!

I'd love to play Aurora again and this looks like it will make it possible. 

[Kyle (OP)]

In Progress update 2020-02-06 I see what seems to be missiles changing target mid-course if original one is destroyed.  I may be wrong for I am newbie, but does Aurora behave like that? I observe so far that missiles come to the wreck and only then search for new target, provided that they have sensors.

As far as I know that is the case in Aurora

You're right, but it was no extra work on my part to just allow retargeting before they reach the wreck, and I feel it makes more sense this way.  And in some situations, less tedious to manage. 

Hi Kyle, is there a technical reason why you can't create a 32-bit version of quasar?
I used to play Aurora on a laptop with a 32-bit OS, so if possible, I would appreciate it if you could create a win32-compatible version (for the final versions I mean, not during development . . . )

Regards.

Nope, no reason.  I can certainly do that later on, for now I'm just saving time by only building for a couple of platforms.

[Sunmannus]

Hi Kyle, is there a technical reason why you can't create a 32-bit version of quasar?
I used to play Aurora on a laptop with a 32-bit OS, so if possible, I would appreciate it if you could create a win32-compatible version (for the final versions I mean, not during development .  .  .  )

Regards. 

Nope, no reason.   I can certainly do that later on, for now I'm just saving time by only building for a couple of platforms.

Good to know, thanks.

[joansam]

Given that most missiles today can retarget in flight, Aurora’s limitations never made much sense to me. Good tweak, from both a gameplay and realism perspective.