ID:2214654
 
Resolved
Garbage collection didn't trigger often enough in some cases, and appearance IDs being reused before client-side deletion caused a memory leak.
BYOND Version:510
Operating System:Windows 7 Pro
Web Browser:Firefox 51.0
Applies to:Dream Seeker
Status: Resolved (511.1375)

This issue has been resolved.
Descriptive Problem Summary:

Dream Seeker's memory usage continually increases as the colors (most likely appearances in general) of visible objects change. Reconnecting, even after rebooting Dream Daemon, only frees some of the memory in use.

Sometimes, after a reconnect, a "Dream Daemon killing connection due to network read error." message would show on the client, causing a disconnection. This may be related to the number of objects (over 12k) I had visible and changing for some tests.

Once Dream Seeker's memory usage reaches about 1.8GB, the map control stops updating, and Dream Seeker may crash.

I should note here that, in addition to 510, I tested with 511.1374, which fixed the turf problem found in my previous post. This problem seems to be unrelated.

Code Snippet (if applicable) to Reproduce Problem:
The objects_per_turf var below can be increased to make Dream Seeker consume memory faster; 100 makes the increasing memory usage quite obvious, but even 1 (121 objects) will cause a slow, constant increase.
This code snippet does not use icons, but whether icons were set or not did not seem to make a difference.
/world
maxx = 11
maxy = 11

var/list/objects = list()
/turf/New()
var/static/objects_per_turf = 1 // More objects consume memory faster
for(var/i in 1 to objects_per_turf)
objects += new /obj(src)

/mob/New() {x = 6; y = 6}

/mob/verb/stress_objects()
spawn()
usr << "Stressing [objects.len] objects..."
while(usr && usr.client)
for(var/obj/O in objects)
O.color = rgb(rand(255),rand(255),rand(255))
sleep(1)


Expected Results:
After using the verb, Dream Seeker would use only as much memory as needed for the current or recent appearance(s) of each object.
Actual Results:
Memory usage continually rose over time, eventually reaching the limit and crashing Dream Seeker.

Does the problem occur:
Every time? Or how often?
Every time.

When does the problem NOT occur?
When not changing the appearances of visible objects.

Did the problem NOT occur in any earlier versions? If so, what was the last version that worked?
I tested 507, and it exhibited the problem. Testing older versions did not seem worthwhile.
I also tested 511.1374, which also exhibited the problem.
Did you test this with cases where the objs weren't visible, like in the 2-z-layer situation we discussed previously? This snippet as-is brings up the exact situation I described where the appearances are held onto because a client has seen them and hasn't discarded them yet.
In response to Lummox JR
I didn't, because this is a Dream Seeker memory leak, so it's only affected by what the client can see. Dream Daemon's memory usage barely changes over time, suggesting that the client is telling the server that it's done with the appearances, but never de-allocates the memory for them (or something related to them).
Wow, this was a very good find. It took some work for me to track down, after initially trying to figure out if the garbage collection was doing its job properly--which it is, as it turns out. Mostly.

The leak I did find was a problem with appearances on the client being recycled--that is, the appearances were marked as garbage, the server reused the ID, and before the client ever deleted the appearance it got a message telling it to reuse that ID. Rather than delete the appearance and create a new one, which was wasteful, DS was simply clearing its data--but it cleared that data incorrectly, and therein lies the bug! Strings and a few other items were not being freed.

With that bug fixed, I still saw memory creeping up, apparently because the garbage collection was only triggering after so many new appearances and it wasn't counting the recycled ones as new. Having fixed that, the memory stabilized.
Lummox JR resolved issue with message:
Garbage collection didn't trigger often enough in some cases, and appearance IDs being reused before client-side deletion caused a memory leak.
Bruh thank God for ss13 devs

And lummox