Improve memory reporting

ID:2401776
 
Sep 27 2018, 12:05 pm
Resolved
Memory usage reporting has been improved for greater accuracy:
  • The associative portions of list memory were not counted.
  • The main array of values for various types (objs, mobs, etc.), although not a large contributor to memory usage, was not previously counted.
  • The memory used for image lists attached to objects, mob.group, vis_contents and vis_locs, and animations is now counted.
  • Appearance memory was counted inaccurately, resulting in a slightly lower value than it should. This has been fixed.(Some minor data like color matrix usage still isn't counted.)
  • Filter memory is now included.
Applies to:Dream Daemon
Status: Resolved (512.1449)
This issue has been resolved.
Memory reporting doesn't currently include some things like associative lists.
Sep 27 2018, 12:14 pm
Lummox JR resolved issue with message:
Memory usage reporting has been improved for greater accuracy:
  • The associative portions of list memory were not counted.
  • The main array of values for various types (objs, mobs, etc.), although not a large contributor to memory usage, was not previously counted.
  • The memory used for image lists attached to objects, mob.group, vis_contents and vis_locs, and animations is now counted.
  • Appearance memory was counted inaccurately, resulting in a slightly lower value than it should. This has been fixed.(Some minor data like color matrix usage still isn't counted.)
  • Filter memory is now included.
Oct 1 2018, 4:05 pm
 
Server mem usage:
prototypes:
        obj: 2.72 MB (17,647)
        mob: 6.52 KB (417)
        proc: 15.7 MB (37,566)
        str: 9.17 MB (202,520)
        appearance: 1.03 MB (33,388)
        filter: 16.3 KB (1)
        id array: 42.7 MB (92,972)
        map: 104 MB (255,255,12)
objects:
        mobs: 329 KB (317)
        objs: 49.1 MB (181,503)
        datums: 66.1 MB (573,198)
        images: 7.1 MB (36,362)
        lists: 190 MB (1,647,670)


This adds up to roughly 490 MB, yet the OS reports 873 MB of memory usage. So there's still 400 MB hidden somewhere.
Oct 1 2018, 11:38 pm
Yeah, absolutely accurate memory reporting is never going to be possible, plus fragmentation is a thing. This will however give you a gist and it's better for some types than it was before.

Adding clients is a future target.
Oct 2 2018, 11:04 am
how are you calculating packed types? Would Value count as 8 bytes or 5?


edit
(Some minor data like color matrix usage still isn't counted.)

also a potential source. I know that lighting uses a fair amount of color matrices
Oct 2 2018, 12:45 pm
Value counts as 8, because nothing is packed. I'm using the sizeof() operator on structures.

The reason I didn't add counting for color matrices yet is that currently server-side appearances are using a very weak shared-reference system for them, so the same matrix might be counted more than once. I want to replace that with what the client is now using, in fact the exact same hash, so that all color matrices are canonical. That will vastly similify counting matrix memory since I can just query the hashtable (although I do need to add a routine to get the total hashtable memory).