ID:1484647
 
Resolved
Certain games had timer issues that could cause CPU racing.
BYOND Version:504.1229
Operating System:Linux
Web Browser:N/A
Applies to:Dream Daemon
Status: Resolved (504.1230)

This issue has been resolved.
Descriptive Problem Summary:
The CPU seems to be taking considerably longer to process things. Before I switched to the beta, it worked file. But since I switched to 1229, CPU spikes from 5 to 70+, and my Master Controller (Counts howlong things takes) goes from it's standard 2-2.5 to 3-8

Numbered Steps to Reproduce Problem:
Launch SS13 server, watch CPU and Master Controller in Status panel.

Expected Results:
Faster game, since things are now done on separate threads.


Actual Results:
Laggy game, much laggier than working off of a single core.

Does the problem occur:
Every time? Or how often?
The entire time. (See below for tests performed)

When does the problem NOT occur?
On the stable builds. (See below for tests performed)

Tests done:
Threading turned off in daemon.txt:
Situation returned to normal, MasterController sticks to 1.9-2, CPU doesn't go over 30 in that case.
Map-threading turned off in daemon.txt:
CPU continues to spike between 30 and 80, even hit 150 a bit ago. Master Controller goes from 2 to 4+

My guess here is that it's an issue with the interface threading.
Operating system is Ubuntu 12.04 64-bit
Is Master Controller another program on the system, or just something you've built into the game?

The measurement from world.cpu is a little misleading, because it's actually more accurate in multithreaded mode as it can measure more of what's going on. However, if you're seeing actual lag then that's a bad sign.

When threads are off, is performance identical to the stable builds or do you see a difference one way or another?
It's something built into the game to process objects, and mobs at once. Basically one big loop. I merely used it as another example on how performance was affected.

Any way, threads off, no problems, same performance as regular, stable builds. Threads on/threads on with map-threads off gets quite laggy. Both visible and invisible (Invisible being the CPU variable), map movement stutters, movement is slow, etc.
I'll see what I can find to explain this. About how many players did this happen with, and did you see any difference when there were no players or just one?

Your big loop itself is going to remain on a single server thread all the time of course, so threaded or not it should remain about the same. world.cpu will appear higher in threaded mode even when it's not actually doing more, because it's counting user commands, whereas unthreaded mode has no proper way to handle that; in essence the unthreaded mode is far less accurate, though perhaps I should change threaded mode to have a similar inaccuracy.
This happened with less than 5 players. It was early. I'm not sure if I could test this with more than that, since it's still a playerbase I wish to keep.

Although it was meant to be a reference point. No threads kept it on it's standard 2 seconds, some times 1.9, or 2.2. But with threading it got close to 3-6, which is quite a lot.
Htop showed both Dream Daemon processes to be on 100 to 101% at these times.

Let me know if you need any more information/tests. I'd be happy to perform them.
If I were to start up an SS13 test using your same codebase, is there something I could start up on an automated basis that would help me see this kind of thing in action? If I could run tests on my own it would help a great deal. (I'd need at least just the hosting files, preferably setup so I had admin access and could see the same kinds of things you would, and since I'm not an SS13 player I'd need some instructions on what to do at the beginning.) If this is happening with 5 players then I assume the same issue would happen just as much with only myself, and being able to run that myself I could add the appropriate debugging to find out where it's slowing down.

Source code would probably help as well, if only so that I could tell which procs were running more often and what kinds of things they were doing, but it's not necessarily as important as being able to host it on my own and have it in this heavy usage state.
I am hoping the same thing occurs on every build, and it wasn't just my own. Which shouldn't really be possible. Any way.

Simple as it is, source code: https://github.com/Baystation12/Baystation12/archive/ master.zip

Move config's "examples" folder contents into the config folder, change admins.txt to hold username - Host. And that's all, I believe. From there it's just a matter of starting the round. And checking out the rest.
Thanks. I'll get that compiled and run some tests on it. Is there anything I need to know about how to start the round or character choices or whatnot?
IIRC, starting mode is extended. So all you really need to do is just hit "Declare ready" and type in 'now' to force it to start early.
If it's any help, I'm familiar with BS12 (I was a coder on it), and I might be able to assist if needed - I'm available all day tomorrow if it's convenient.
With two keys logged in and observing, I'm seeing a steady 1.9 for the master controller. But I'm not doing anything else. Is there something I need to trigger that would cause more to happen?
Depends. If people were manning engineering or walking around, they'd probably be causing the atmospherics simulation to kick in a fair amount. The station's power core probably also uses up some amount of CPU, especially if they'd accidentally let it loose (assuming TGStation uses the singularity).
I think this is going to be a problem, then. I need to at least know how to get those things to kick in and cause problems, preferably in an easily repeatable way where I can compare results. Observing obviously doesn't cut it, but I don't know the first thing about the game.
We installed BYOND 504.1229 on our vps with linux which has an 8 core i7 cpu and tried to run our game Dragon Universe on it, and it wouldn't even start up. No idea why. So we reverted back to the stable release.

But Dragon Universe wasn't compiled for 504.1229 so maybe that was why. So does the game need to be compiled for that version to be hosted using it?
In response to Tens of DU
Tens of DU wrote:
But Dragon Universe wasn't compiled for 504.1229 so maybe that was why. So does the game need to be compiled for that version to be hosted using it?

This, is this the case? Because if so, it may be my source of lag.

Recompiled using 1229, uploaded. Decided to host it on both my local computer and on my server, my computer's CPU range (BYOND's var) seems to be between 0 to 20. Looking at the server.. It's hopping between 30 to 90.

Local PC is windows, Server is Linux.
Both have multi-threading on fully. But my local PC just seems to be smoother.


EDIT: Running some more tests, on my local PC, but still.
I'm looking into something I discovered with Topkasa last night in testing, where I found a problem in our hub connection that led to a crash. This may or may not be the issue. One thing of note is that with that code, I was never able to see the MasterController go beyond about 3.5, and that was only when Topkasa was moving around the station a lot; otherwise it was a steady 1.9-2. So I'll need more information on how to really push that number higher and see stress conditions I think, if this hub thing doesn't yield any answers.

The compiled version does not matter at all.

I would however be very interested in obtaining Dragon Universe source code for testing--and it must be the source in this case, not just the host files, because I can't trust that game at all and would want to verify the source was clean. Failure to startup is something SilkWizard is seeing in NEStalgia on just one Windows machine, and that's something I'm trying to diagnose. It's quite possible DU's problem could be related.
Lummox JR resolved issue with message:
Certain games had timer issues that could cause CPU racing.
I didn't feel the need to make a new thread but this is still very much a problem in 1230.