Disappearing Ground Units

[skoormit]

I've never seen a save problem in my own games without an error message.

I have started to change some save functions to have a second try/catch for each record rather than the whole insert, so maybe I need to complete that for every save function.

I think that's a good idea, but maybe not needed to fix this particular bug. If that would be time consuming, maybe just force the savegroundunits function to throw an exception and see if you actually get the expected error message.

[Erik L]

How about updating the code to...

        public void SaveSystems()
        {
            try
            {
                // establish a database connection
                using (SQLiteConnection AuroraDB = new SQLiteConnection(GlobalValues.AuroraConnectionString))
                {
                    AuroraDB.Open();

                    // delete the existing records
                    SQLiteCommand del = new SQLiteCommand("DELETE FROM FCT_System WHERE GameID = " + GameID, AuroraDB);
                    del.ExecuteNonQuery();

                    using (var cmd = new SQLiteCommand(AuroraDB))
                    {
                        using (SQLiteTransaction transaction = AuroraDB.BeginTransaction())
                        {
                            // insert new records
                            foreach (StarSystem obj in SystemList.Values)
                            {
                                try
                                {
                                    cmd.CommandText =
                                        @"INSERT INTO FCT_System (SystemID, SystemNumber, Age, AbundanceModifier, Stars, GameID, JumpPointSurveyPoints, SystemTypeID, DustDensity, SolSystem, NoSensorChecks )
                                VALUES ( @SystemID, @SystemNumber, @Age, @AbundanceModifier, @Stars, @GameID, @JumpPointSurveyPoints, @SystemTypeID, @DustDensity, @SolSystem, @NoSensorChecks)";

                                    cmd.Parameters.AddWithValue("@SystemID", obj.SystemID);
                                    cmd.Parameters.AddWithValue("@SystemNumber", obj.SystemNumber);
                                    cmd.Parameters.AddWithValue("@Age", obj.Age);
                                    cmd.Parameters.AddWithValue("@AbundanceModifier", obj.AbundanceModifier);
                                    cmd.Parameters.AddWithValue("@Stars", obj.Stars);
                                    cmd.Parameters.AddWithValue("@GameID", GameID);
                                    cmd.Parameters.AddWithValue("@JumpPointSurveyPoints", obj.JumpPointSurveyPoints);
                                    cmd.Parameters.AddWithValue("@SystemTypeID", obj.SystemTypeID);
                                    cmd.Parameters.AddWithValue("@DustDensity", obj.DustDensity);
                                    cmd.Parameters.AddWithValue("@SolSystem", obj.SolSystem);
                                    cmd.Parameters.AddWithValue("@NoSensorChecks", obj.NoSensorChecks);
                                    cmd.ExecuteNonQuery();
                                }
                                catch (Exception error) { GlobalValues.ErrorHandler(error, 3249); return; }
                            }

                            transaction.Commit();
                        }
                    }

                    AuroraDB.Close();
                }
            }
            catch (Exception error) { GlobalValues.ErrorHandler(error, 1430); return; }
        }

I'd move the AuroraDB.Close(); to the finally section of the try/catch

[skoormit]

How about updating the code
...

That looks exactly right.

[skoormit]

I'd move the AuroraDB.Close(); to the finally section of the try/catch

That's good form in general, but the end of the using statement will force a Dispose(), which will also close the connection.

[Steve Walmsley (OP)]

How about updating the code
...

That looks exactly right.

Thanks - some code grinding ahead

I'll start with the ones I know had issues (ships, ground units, etc.) and add the rest when the enthusiasm reservoir is high

[SpikeTheHobbitMage]

One thing I'd like to point out (and I'm sorry if I'm getting annoying) is that the delete operation is outside of the transaction.  This means that if the insert is interrupted and gets rolled back, the delete still gets committed.

I'd move the AuroraDB.Close(); to the finally section of the try/catch

Shouldn't the 'using' clause take care of that?  Edit: ninjas everywhere.

[Erik L]

One thing I'd like to point out (and I'm sorry if I'm getting annoying) is that the delete operation is outside of the transaction.  This means that if the insert is interrupted and gets rolled back, the delete still gets committed.

I'd move the AuroraDB.Close(); to the finally section of the try/catch

Shouldn't the 'using' clause take care of that?  Edit: ninjas everywhere.

Maybe, but I don't use using in C#

[Steve Walmsley (OP)]

One thing I'd like to point out (and I'm sorry if I'm getting annoying) is that the delete operation is outside of the transaction.  This means that if the insert is interrupted and gets rolled back, the delete still gets committed.

I'd move the AuroraDB.Close(); to the finally section of the try/catch

Shouldn't the 'using' clause take care of that?

Good advice is never annoying

Now I am torn. If I prevent the delete then I could end up with a table that is out of sync with all the others, which might cause harder to find bugs. With the try/catch on the record level, would I not be better deleting and then only losing any individual records that fail?

[Bughunter]

I'd move the AuroraDB.Close(); to the finally section of the try/catch

Yes agree with making sure it closes. Just be aware if it failed to open earlier it may throw an error trying to close it there. I don't know that happens with sqlite but have seen with other db:s. And not that it matters in this example since you would have an earlier error anyway.

[db48x]

if you took out the return from the inner catch block, then it would have the advantage that if it fails to insert any single record, then only that one record is lost. If you logged the information about the failing record(s), you could track down the bug more easily.

[Earthrise]

Steve, I've just had this issue happen to me, in v8. 0 .  The two police battalions I built more recently have disappeared, but the first two I built are still there (though their ORBAT has gone, which is also odd).    I am only about 15 years into this game.    My PC is set up with a British decimal separator.   

Please find a dropbox link with the DBs from just before it happened and just after it happened.   

As other people have reported, I too was AFK for a couple of hours inbetween the two saves, though the game was not advancing time.   

Hope this helps.   
https://www. dropbox. com/sh/de44pyqzo3hprni/AAAB0pL-1fvDFHMjKJ1xZzita?dl=0

(hyperlink is not working, sorry)

[skoormit]

If I prevent the delete then I could end up with a table that is out of sync with all the others, which might cause harder to find bugs. With the try/catch on the record level, would I not be better deleting and then only losing any individual records that fail?

Which way is better depends entirely on your design decisions about error resilience.
The stance that "errors will happen and we should minimize their impact on a particular game instance" is reasonable, but the effort (to localize errors and minimize impacts) requires dev time.

[skoormit]

...
As other people have reported, I too was AFK for a couple of hours inbetween the two saves, though the game was not advancing time.   
...

I'm curious: did any game-time pass between the two saves?

[Second Foundationer]

Maybe you've already narrowed it down by now, I haven't read all the recent posts.   But my unqualified shot in the dark: Is it absolutely certain that it is the saving rather than the loading process that causes the error?

I began a new game with 4 distant starting NPRs briefly after 1.  8 release and kept manual db copies periodically.   A slow setup, two encounters with Trappist aliens, and, I don't remember exactly, about 35, in any case less than 40 (now very quick, thanks to C# Aurora) game years later, I had an unrelated program in a buggy script debug mode crash while saving aurora, and I thought that might have caused a save corruption: Next time I started Aurora, all ground units were gone (in total maybe around 40, most of them on Earth, a handful on fledgling colonies, and possibly 1 or 2 in transit on commercial transports, plus several garrisons on CMC rocks).   As for saving, Aurora had been running idle in the background for longer periods at times, so the save intervals were irregular.   And it is possible that I saved twice without advancing time. 

I have no experience with SQLite or C#, but I took a brief look into the db with an SQLite DB Browser, and it seemed that most if not all of the units were still basically listed in the formation table with their names, template numbers, etc.   Yet Aurora couldn't load the ground units, everything else seemed to be fine.   But as I had somewhat botched my desired setup (a gloomy pacifist Antarctic Union of WWIII survivors that came out way too militarized and also too populous), didn't suspect yet that this might be a general bug and feared there might be more serious, less obvious save corruption elsewhere, I decided to start over when the next version is out .  .  .   and wiped the manual saves, so I cannot pass on the db in question. 

But if it is just possible from other bug reports that the cause might be in the input rather than the output, it might be worth to glance over the startup/load/initialization function/s or whatever it is that gets the data into the running program.   In case it's relevant: I had deleted the "Federated Nations" (? or something like that) example/default game after my own Antarctica setup. 

Although I've been drawn into a descending orbit around Aurora for years and browse the forums periodically, I just registered now.   So, with my first post: Many thanks for sharing a marvelous game with us and your continued efforts to make us spend many more hours with it.   And to the generally helpful community here for answering all of my Aurora 4x questions so far, before I had ever asked them. 

[Earthrise]

. . .
As other people have reported, I too was AFK for a couple of hours inbetween the two saves, though the game was not advancing time.   
. . .

I'm curious: did any game-time pass between the two saves?

Yes some time had passed inbetween the two saves - I don't remember exactly how long.  Feel free to download my saves for further details