Last weekend I was reluctant to release 512.1449 right away because of the sweeping changes under the hood, so it was going through some private testing. That turned out to be a good thing because a few items cropped up right away that I was able to fix before release.
However as I think anyone could have predicted, there were still a few issues to deal with. Most of the problems that testing caught were server-side, and it turned out that the client and Linux compiler had a couple of problems that jumped out in release mode. This led to a few more releases, culminating in the Linux-only 512.1451 build (Windows didn't need to update past 1450) which is where we're at now. I did a little more work in this refactoring area for the next release, updating the server-side color matrices like I mentioned and making some changes to color matrix handling in the MapIcon struct.
With things holding steady for the moment, I decided to start in on the compiler profiling I'd been talking about for a while. I promised this to the SS13 devs some time ago and hadn't gotten around to it yet because 1) there were too many other things on my plate at the time, and 2) profiling sucks. Profiling tends to produce inconsistent reports based on the instrumentation and how the system happens to be feeling at any given time, and in newer compilers it also produces massive files that take up a lot of space and take forever to analyze--which among other things, meant I couldn't simply profile an entire compilation process from start to finish, but had to start and stop the profiling to catch specific time slices of interest.
Well, it turned out that profiling did highlight an optimization target, and it was a big one. It uncovered that a lot of time was being spent in an O(N) loop in a function used to convert info about a var (its name, some definition info, etc.) into a variable ID. In other words, this was a lookup, and it was trying to find out if the info already had an ID or needed a new one by going through an entire list one at a time. In small projects these O(N) loops are no big deal, but in something the size of SS13 it gets bad. Remember, this loop is O(N) but as N rises it means you're dealing with O(N) references to vars in the code; which means the compiler is effectively seeing O(N2) performance. My solution to that loop was to replace it with the CBag template class that we use for appearances and many other structures, which allows a binary search on a self-balancing tree.
After dealing with that I tried some more tests and broke into compilation with the debugger here and there. This highlighted two more O(N) loops: one for adding ID arrays which are a very big deal, and one for unique var paths (an internal compiler concept). These, and the var IDs above, each numbered in the tens of thousands, and I believe the number of ID arrays ended up somewhere around 50K. So I replaced these two loops with CBag handlers as well.
The results I got for those efforts were little short of miraculous. I didn't get a baseline time for compiling SS13 before, but it took something like 3 or 4 minutes at least on my machine. The new compilation (in release mode) takes 33 seconds; sometimes 32. I was saying initially that I got about a 300%+ improvement just by eyeballing it, but based on my recollection of how long compilation used to take, it's probably more accurate to say I got about 500% or more. Obviously the improvement will be smaller for smaller projects, but it's still a huge deal.
Also I added some additional output to the compiler: total elapsed time in seconds, and also notification of when it includes files. I tried to add something fancy to estimate % of completion, but... nope, that did not work. Anyway the good stuff will be coming in 512.1452, but ain't no way am I taking chances with the Friday curse.
If you haven't gotten to work yet on your killer Halloween game, what are you waiting for? This is pumpkin season! With the filters in 512 you can have some fun with ghostly effects, like maybe a nice eerie mist or a blurry figure in the corner.
Thanks as always to our Members, donors, and Patrons for their support!
Oct 5 2018, 9:09 am