Aurora 4x
C# Aurora => Development Discussions => Topic started by: Steve Walmsley on April 03, 2020, 12:52:44 PM
-
I've encountered a very weird issue. In debug mode, Aurora takes about 10 seconds before the tactical map comes up and it runs. I've built an install program and installed Aurora on a laptop. However, it now takes about five minutes before the tactical map appears. Once running, it is fine. However, I have no idea why the change from debug to release mode changes that startup from 10 seconds to five minutes.
On the other hand, I'm new to creating C# install programs :)
Anyone seen anything similar before?
-
Do you see this when you run from your IDE in release mode as well? Or is it still around 10 seconds there?
-
At the risk of stating the obvious, does it behave the same way on a different computer?
-
Do you see this when you run from your IDE in release mode as well? Or is it still around 10 seconds there?
Running from IDE in Release mode takes about 10 seconds as well.
-
At the risk of stating the obvious, does it behave the same way on a different computer?
Yes, tried my PC and a laptop.
-
Five minutes is a really long time. It's long enough that my immediate suspicion isn't that any actual work is happening, but instead some sort of timeout is occurring. Given that you don't have any web code I think the most likely culprit would be some sort of database connection happening (or attempting to happen) on startup that it's waiting for.
I don't know what you ended up using for your DB handling, is it possible that Entity Framework or whatever else is initialized to have different behavior in debug or release mode? I'd look through your startup.cs or program.cs or whatever initialization with a really fine eye for any configuration differences based on the Environment.
I don't know what sort of logging you've implemented, unfortunately several standard solutions for that aren't fully initialized during early program initialization, but in cases like this you can hack in a temporary file writer to write out trace statements before the rest of your logging comes on line. It's not ideal but for this sort of thing I've done the same.
-
I've started adding popup messages during startup. The problem is happening once the DB starts to load.
Anyway, time for food and wine - will come back to it later :)
-
Ok, once you've pinned it down that far it's probably best to switch to SQL Server Profiling.
Given that at this point you shouldn't have a lot of activity if there is a request that hangs for a long time you'll hopefully be able to spot it without too much trouble. Hopefully.
-
Dumb question, but it's always best to be bone-headed with this kind of thing: I take it this is happening before you've attempted obfuscation? So that's not likely to be part of the problem?
-
Dumb question, but it's always best to be bone-headed with this kind of thing: I take it this is happening before you've attempted obfuscation? So that's not likely to be part of the problem?
Yes, decided to take a break from testing Diplomacy and work on the install. I'll tackle obfuscation once I can successfully install and run, so I can identify any obfuscation-specific issues.
-
Any sort of anti-virus, real-time protection etc kicking in?
When running the game from your IDE you probably have the exe, db etc in your project folder under your user directory and maybe the process is also running under your IDE process.
After installing it you have it somewhere under C:Program.. and maybe it is also writing to there. That could possibly trigger some kind of lookup trying to compare your newly commissioned exe against various malware signatures from the cloud or whatever antiviruses do these days.
-
Oh, if you have forgotten to mark any resources to "publish when built" or whatever that setting is, then they could also not have come along to the new directory, which could cause a timeout when trying to locate them.
-
I've started adding popup messages during startup. The problem is happening once the DB starts to load.
Anyway, time for food and wine - will come back to it later :)
Really random and "Out there" thought, but if it's potentially a DB related issue after the installation process, given debug and IDE release profile attempts are fine, what happens when you simply take the DB from a environment that works fine and cut it out into a release build that has gone through the installation process?
On the off chance that the installation process is smegging up the DB somehow in some bizarre obscure way, does the DB need to be bundled in to the process or can it be provided individually without having gone through the process?
-
The problems seem to be DB-related. The program is simply progressing very slowly through all the queries that load tables into object structures. I need to figure out why that is taking so much longer in release than debug. Its an SQLite database.
Anyway, it will probably have to wait until tomorrow now. Programming ability decreases at an exponential rate vs linear alcohol consumption :)
-
I've started adding popup messages during startup. The problem is happening once the DB starts to load.
Anyway, time for food and wine - will come back to it later :)
Really random and "Out there" thought, but if it's potentially a DB related issue after the installation process, given debug and IDE release profile attempts are fine, what happens when you simply take the DB from a environment that works fine and cut it out into a release build that has gone through the installation process?
On the off chance that the installation process is smegging up the DB somehow in some bizarre obscure way, does the DB need to be bundled in to the process or can it be provided individually without having gone through the process?
I'll try that tomorrow. As far as I can tell though, the install program is just placing the DB file in the installation directory.
-
The problem doesn't seem to be debug vs release but running inside the IDE vs runing outside the IDE, given that running in release mode inside the IDE does not show the problem.
-
Simple fix: just package your IDE with the executable.
What? Why isn't that a good idea? :P
-
There's no way to know if the difference is truly because of the IDE. So I recommend that an image of Steve's OS installation should come with it instead.
-
This sounds like it could be an anti-virus or Windows Defender issue. I would try excluding the folder from Windows Defender to see if it makes a difference:
- Windows Defender -> Virus & threat protection settings -> Manage Settings -> Exclusions -> Add or remove exclusions
-
Assuming a file from a known good environment works, does the installer understand the sqlite file is binary, not ascii.
-
There's no way to know if the difference is truly because of the IDE. So I recommend that an image of Steve's OS installation should come with it instead.
More weirdness.
If I close the IDE and run the exe that sits within the release folder of the build, it runs fine.
If, rather than installing, I simply move those same files from the release folder into a new location on the C drive, it is back to 5 minutes.
-
This sounds like it could be an anti-virus or Windows Defender issue. I would try excluding the folder from Windows Defender to see if it makes a difference:
- Windows Defender -> Virus & threat protection settings -> Manage Settings -> Exclusions -> Add or remove exclusions
I tried disconnecting from internet and then turning off the anti-virus. No difference.
-
5 minutes first run in new folder, or any next run ?
Maybe some database files created outside of "release" or "debug" folder. After you copy folder content to another location, SQLite must create new database (Or maybe your database useabsolute patch for files)
1. Try run executable and inspect his activity (CPU, Memory, Disk usage) by "Resources Inspector" on Windows - for "bottleneck" detecting.
2. Make Virtual Mashine with clean Windows, and test your installer on VM.
-
It is folder related. If I move the install files from C:/Program Files/Aurora to C:/Aurora it works fine. That is true for both my main PC and my laptop.
No idea what is going on there.
Anyway, that means the installation process is working, but it can't be installed in program files.
-
Windows doesn't like things in C:/Program Files being edited, so if the database is in the same folder as the rest of the program then it's likely that protection is what is causing the long waits. Making sure it doesn't get installed in program files would fix it, as would having the database be somewhere else. %appdata% is commonly used for this sort of thing.
-
Do you have x64 Windows ? And you compile x64 Aurora ?
If Windows x64 and Aurora compiled by 32-bit (x86) compilator, he must be install to C:/Program Files (x86) folder
-
Windows doesn't like things in C:/Program Files being edited, so if the database is in the same folder as the rest of the program then it's likely that protection is what is causing the long waits. Making sure it doesn't get installed in program files would fix it, as would having the database be somewhere else. %appdata% is commonly used for this sort of thing.
Yes, got this as my prime suspect too.
-
It is folder related. If I move the install files from C:/Program Files/Aurora to C:/Aurora it works fine.
Please, check %APPDATA%\Local\VirtualStore folder.
-
Do you have x64 Windows ? And you compile x64 Aurora ?
If Windows x64 and Aurora compiled by 32-bit (x86) compilator, he must be install to C:/Program Files (x86) folder
I'm going to create both 32 and 64 bit versions. In testing, the 32 bit is actually faster but I suspect that will change for larger games.
For this example, I compiled 32 bit and installed to C:/Program Files (x86) folder.
-
Great to hear you found a cause.
Personally I can't imagine many people still put things in program files unless it's simply the default path directory and they can't be bothered to change it. Heck I'd be surprised if many people even install Aurora on their system drive outside of those where that's their only option.
When I first started playing VB Aurora I had it installed on my system drive as I wasn't sure on the amount of read/writing the game would involve due to how it's structured and figured the SSD on my system drive may help in certain cases, but after a while realised it really makes no difference with Aurora, I can only imagine C# with the removal of the constant writing state to disk will make it matter even less, so it's definitely going on my secondary HDD rather than my SSD so I can keep it stored alongside my usual game library collection.
-
Windows doesn't like things in C:/Program Files being edited, so if the database is in the same folder as the rest of the program then it's likely that protection is what is causing the long waits. Making sure it doesn't get installed in program files would fix it, as would having the database be somewhere else. %appdata% is commonly used for this sort of thing.
This is most likely the correct reason. Windows really doesn't like tinkering with the c:/Program Files folder.
In a certain sense it's a good thing (it can protect users, especially the less computer-savvy ones), but then you have these problems cropping up.
Probably do write it somewhere in the installer, or propose a different folder by default. And I would surely use 64bits, for systems that support it.
-
It is folder related. If I move the install files from C:/Program Files/Aurora to C:/Aurora it works fine. That is true for both my main PC and my laptop.
No idea what is going on there.
The "program files" directories in modern (post-xp) windows versions is meant for immutable program data. It has different file permissions and backup behavior than the rest of the filesystem, and consequently is very slow to write.
Assuming that you have a base database file(s) that are meant to be changed during use, the correct way to use them in modern windows systems is to have a clean copy of them in the install folder, but never open them from this folder, instead on startup you check whether %appdata%/aurora4x(example_name)/ exists, and if not, create it, and then copy over all the files that are meant to be changed during use, and open them in %appdata%.
-
It is folder related. If I move the install files from C:/Program Files/Aurora to C:/Aurora it works fine. That is true for both my main PC and my laptop.
No idea what is going on there.
The "program files" directories in modern (post-xp) windows versions is meant for immutable program data. It has different file permissions and backup behavior than the rest of the filesystem, and consequently is very slow to write.
Assuming that you have a base database file(s) that are meant to be changed during use, the correct way to use them in modern windows systems is to have a clean copy of them in the game folder in in the install folder, but never open them from this folder, instead on startup you check whether %appdata%/aurora4x(example_name)/ exists, and if not, create it, and then copy over all the files that are meant to be changed during use, and open them in %appdata%.
Things like this bug me immensely when trying to keep my System drive dedicated to the OS and only specific software that benefits from using a SSD.
If I have something installed on my secondary HDD, I expect it to keep as much of its dependant files stored relative to the drive I've installed it on, not to dump additional files onto another drive.... I get why developers do it, but so much of it seems to stem from outdated "Good practise" that really isn't necessary 9 times out of 10.
Just like all those games out there that despite being installed on my secondary HDD, still insist that settings, mod data, saves and such all be tossed onto my system SSD in a directory within the users Documents hierarchy, which ends up meaning my system drive has gigs of misc data sprawled around on it for applications installed on my secondary drive..... Yes Paradox.... you suck big time for doing it for every game.
-
Putting stuff in program files… does the . exe have a digital signature?
-
I think it will be easiest just to have the database in the installation directory and warn players not to install in program files. That reduces the potential for bugs outside my control.
-
If I have something installed on my secondary HDD, I expect it to keep as much of its dependant files stored relative to the drive I've installed it on, not to dump additional files onto another drive.... I get why developers do it, but so much of it seems to stem from outdated "Good practise" that really isn't necessary 9 times out of 10.
This isn't some outdated "good practise", it's following the proper guidelines on how to store data on Windows set by Microsoft. Putting all the things relevant to one program into one folder is the traditional way of doing things in windows, separating the program and static data into %programfiles% and putting everything that might get modified into %appdata% is the new, proper way of doing things.
Whether this is the right approach is another discussion, but on this Paradox (and basically all other modern games) are just doing what Microsoft orders them to do on their platform.
-
Is there a need to have an install program vs. just providing a zip file of the folder structure that can be placed where we want it to live in our machine?
-
If you don't have the required runtimes to run the game, you will need the installer, yes. (Source (https://gamedev.stackexchange.com/questions/35461/what-exactly-does-an-installer-do-and-why-might-i-need-one))
-
If you don't have the required runtimes to run the game, you will need the installer, yes. (Source (https://gamedev.stackexchange.com/questions/35461/what-exactly-does-an-installer-do-and-why-might-i-need-one))
Yep, I just figured that is a paragraph on the download post with a link to the. Net 4 runtime installer for anyone with a version of Windows to old to have it. I believe anyone with Windows 7 and up who has updates enabled should have the. Net 4 runtime installed.