I appreciate the kind words, but as far as typical game performance is concerned I do expect it to be a much less dramatic result. I saw about about a 75-80% reduction in running time on your stress test on my end, but that's keeping in mind that it demonstrates worst-case behavior. What the impact will be on actual games, I can't say. My gut says the obj/mob improvement will matter only to games that churn them very frequently, whereas strings--which churn more frequently in more games--may have a slightly bigger edge.

Interestingly strings were part of your stress test's running time because of the \ref macro you used. Strings however had the advantage of being released immediately, so there probably wasn't a huge issue there. Where this change will impact strings most will be in cases where the string is allocated and held onto for a while, so that lower-numbered IDs get reused first.

Writing A New One wrote:

We're testing the new version out in Eternia at the moment. Given the changes, I'm assuming that there's a good chance the consistent rise in CPU over time will no longer be a problem?

Difficult to say for sure, as there could be any number of factors in play for that sort of thing. Certainly the O(N) loops to get fresh IDs for many types are no longer going to be an issue.

I do think the string tree would benefit from periodic rebalancing, which was an idea I had tried in the past that got scrapped. It only rebalances now in extreme cases.

I'd say try it again with threading disabled, cpu readings aren't the same in threaded mode vs. non-threaded mode.

Yep, this change is definitely a dramatic improvement on performance in bigger games like Eternia and SS13, I can say that with certainty now. Eternia is running better than it ever has, even with things running that normally bog it down and cause performance loss.

Raids with lots of monster creation and deletion used to spike the CPU above 100% no matter what, even if barely anyone was involved. We just had a raid run through start to finish with a somewhat decent amount of players involved without ever going above 60% and with none of the normal lag spiking as monsters were created and deleted. After said raids we'd usually be hovering around 60%-90%, 30% of that or so being CPU we never got back, but now, after a few minutes we're back to normal CPU values.

I'm not seeing any of the gradual increase in CPU (and decrease in performance) I was seeing before, and the CPU values we are getting are about as low as they are when we first start the server up. It's been about six hours since we started up and I'll post another update after it's been running a while.

My thanks go out to Ter13 for bringing this issue to light, it's been plaguing Eternia for quite some time and it would have been a mess to try and fix manually -- and of course to Lummox and Tom for prompting tracking this down and nailing it. You may not see the significance with this change but it is indeed significant and allows large-scale games to operate far more effectively without bogging themselves down.

Still runs like crap in threaded mode though, sorry =\

Since deletion is still absymally slow, custom garbage collectors are still necessary

I think most of the improvement comes from that strings in the past were hilariously slow, which I assume this sort of fixes

Not necessarily stupidity

They run multiple passes to try to clean up references in some form of intelligent manner other than a dumb sweep, along with that something, i.e a power cell, can be inside of basically anything.

Custom garbage collectors also clean up any visibility of the object within the game (in theory) and, if it cant be refcounted away, schedules it for deletion with precise timing to avoid causing a CPU spike.

Making BYOND games perform well is hard.

That and not to mention there is absolutely no other way to catch GC misses right now other than overriding del(), which is awful

I don't think Tobba's wrong about strings being a big factor, though Nadrew's description of mass obj creation also would be involved quite a lot. I'm thrilled to hear that non-threaded mode, at least, has been improved significantly by these changes.

Honestly I've been looking at those O(N) loops with a skeptical eye for a while now, but I felt I would be chasing only the tiniest of gains.

On this subject, I actually discovered another O(N) loop that has not yet been removed, which is that internal ID arrays such as those used for overlay/underlay/verb lists are always checked for uniqueness. While getting a new ID for the array is done in a nicer fashion now, and the main memory struct (though not the internal list of IDs) has been recycling for longer than that, optimizing this check might possibly show some gains in games that modify overlays and verbs very frequently. If I came up with some kind of very fast hash, I could implement a hash table approach to deal with that. There's also the matter of frequent malloc/free on the internal list of IDs, which I'd love to improve on but it's not such low hanging fruit.

I assumed you were already using a hashing function to associate appearances and overlay lists... The more you know.

Still, I think the next biggest gain to performance besides eliminating that particular setup would be in getting rid of the per-client, per-frame, per-atom(culled) appearance checking loop in favor of a more event-oriented approach. After that, it really is up to the developer to determine their own CPU usage, as that's the last big resource hog that's out of our control I can think of at the moment.

At least on one end, when the HTML5 client is out, we should be able to pretty effectively program our own client-side stuff a little more deeply into the client than before.

Nadrew wrote:

Raids with lots of monster creation and deletion used to spike the CPU above 100% no matter what, even if barely anyone was involved. We just had a raid run through start to finish with a somewhat decent amount of players involved without ever going above 60% and with none of the normal lag spiking as monsters were created and deleted. After said raids we'd usually be hovering around 60%-90%, 30% of that or so being CPU we never got back, but now, after a few minutes we're back to normal CPU values.

The CPU is back to its usual levels now, after a good 12 hours.

It's high again (over 100% constantly) with nothing in the profiler showing any major resource usage, and performance has dropped pretty heavily. So the issue with over-time performance loss hasn't handled yet but the performance increase before this was dramatic and I'm still noticing even now that the CPU values are lower than usual (usually 150-200) but still acting like the game is struggling to keep up, movement is sluggish, command response is slow.

It's still a difference than previously, just not a total resolution.