Why does my server lag so much? in Developer Help

ID:517267
 
Mar 14 2012, 6:38 pm
(See the best response by JunK Games.)
My game is still in the alpha stages so it doesn't have much code. It's got all the basic functions of an RPG but most of those functions aren't even implimented yet.

So, when I log in and walk around, it doesn't go one whole minute before I get a 5 to 15 second lag spike. About every 40 seconds I'll get another lag spike. Not only does this effect the game, it's also effecting the Dream Daemon itself. When I go to the players tab or any other tab, or when I minimize it, sometimes it lags a lot. It lasts about as long as it does in-game.

My computer is more than sufficiant to host even the most demanding Byond game, so specs aren't the issue. I'm thinking maybe I set something up wrong or maybe there's an internet setting I should change. I tried disabling my anti-virus so it's not that.

Any help would be much appreciated. My game is an ORPG so the server is always online if you'd like to see what I'm talking about.
#1 Mar 14 2012, 6:53 pm
I'm convinced that it's either:
A. Poor programming habits or an obscenely large/populated map.
B. Your system isn't a strong as you think.

I'm partial towards A, though. No offense!
#2 Mar 14 2012, 6:56 pm
If it does it when running the game locally, try profiling your game to see if anything is taking a ridiculous amount of CPU time.
#3 Mar 14 2012, 7:05 pm
It runs great when running locally. When running the server, it never uses more than 2% of my CPU. The code is very clean. There's only one map and every area of the game is on that one map. That doesn't say much though, since there's hardly anything to the game at this point.

I've been playing a lot of Crysis 2 lately (great game btw) and it runs at 60 FPS (synced, could go higher) on max settings so I very highly doubt the lag is due to a lack of sufficient processing power.
#4 Mar 14 2012, 7:08 pm
Do you get any run time errors that might cause it lag so much?
#5 Mar 14 2012, 7:10 pm
So when Dream Daemon "freezes", the CPU levels are normal? Did you try profiling just to see if anything stood out?

If you're not maxing out your CPU (and keep in mind that BYOND can only use one of your processor's cores), then that probably means it's a network issue. Are you hosting the server on the same machine, on the same local network, or is it remote? If you have it host a separate, very basic project, does it exhibit the same symptoms?
#6 Mar 14 2012, 7:45 pm
Thanks for the replies guys.

I'm not getting any errors. Below is the profile after running the server for about 10 minutes and refreshing the profile approximately 10 times.

I'm hosting on the same machine that I'm playing on, so network lag shouldn't be an issue unless there's a problem with my router or something (with oddly only effects Byond).

                                        Profile results (total time)
Proc Name                                                    Self CPU    Total CPU    Real Time        Calls
--------------------------------------------------------    ---------    ---------    ---------    ---------
/mob/Move                                                       0.309        6.532      399.292        11603
/mob/monster/UpdateTurnController                               6.040        6.041        6.047        11646
/mob/proc/Step                                                  0.003        0.242        5.511          132
/mob/pc/Move                                                    0.001        0.236        5.504           97
/mob/monster/proc/AIMove                                        0.228        4.898        4.898         8731
/client/North                                                   0.002        0.102        2.527           59
/client/South                                                   0.002        0.092        2.260           55
/client/West                                                    0.000        0.028        0.366            9
/client/East                                                    0.000        0.024        0.362            9
/mob/proc/UpdateTurnController                                  0.147        0.222        0.222           98
/obj/LoadChar/Click                                             0.002        0.055        0.055            1
/mob/pc/proc/LoadPlayer                                         0.005        0.052        0.052            1
/mob/pc/Read                                                    0.011        0.047        0.047            1
/TurnController/proc/Action                                     0.000        0.033        0.033           95
/obj/object/New                                                 0.000        0.033        0.033            2
/icon/proc/RscFile                                              0.033        0.033        0.033            2
/mob/proc/UpdateLevelController                                 0.026        0.026        0.026        11605
/LevelController/proc/ManageLevel                               0.000        0.018        0.018           58
/mob/pc/proc/DoPlayerAttack                                     0.006        0.012        0.012           12
/mob/pc/Bump                                                    0.000        0.012        0.012           14
/TurnController/MonsterController/proc/AddMonsterToLevel        0.006        0.009        0.009           60
/TurnController/MonsterController/proc/SpawnMonsters            0.000        0.009        0.009           14
/mob/pc/proc/KillMonster                                        0.006        0.006        0.006            2
/mob/pc/proc/DoPlayerHit                                        0.000        0.006        0.006            6
/mob/pc/proc/FDeadDoDamageToMonster                             0.000        0.006        0.006            6
/atom/movable/proc/Teleport                                     0.000        0.003        0.003            1
/TurnController/proc/CheckAction                                0.003        0.003        0.003           97
/TurnController/MonsterController/proc/CreateMonster            0.000        0.003        0.003           60
/mob/monster/New                                                0.001        0.002        0.002           60
/mob/monster/New                                                0.000        0.001        0.001           60
/client/Topic                                                   0.001        0.001        0.001            1
/mob/monster/New                                                0.000        0.001        0.001           60
                                                                0.001        0.001        0.001           60
/mob/pc/proc/PlayerIsSaved                                      0.001        0.001        0.001            1
/mob/New                                                        0.001        0.001        0.001           61
/mob/monster/other/Spider/New                                   0.000        0.001        0.001            1
/mob/monster/Dragon/ADGreenDragon/New                           0.000        0.001        0.001            2
/mob/monster/Reptile/Viper/New                                  0.000        0.000        0.000            2
/mob/monster/Dragon/YORedDragon/New                             0.000        0.000        0.000            2
/mob/monster/Dragon/ADRedDragon/New                             0.000        0.000        0.000            1
/mob/monster/Dragon/YABlueDragon/New                            0.000        0.000        0.000            1
/mob/monster/Dragon/YAWhiteDragon/New                           0.000        0.000        0.000            2
/mob/monster/Dragon/ADWhiteDragon/New                           0.000        0.000        0.000            2
/mob/monster/Dragon/YOGreenDragon/New                           0.000        0.000        0.000            3
/mob/monster/other/Bear/New                                     0.000        0.000        0.000            5
/mob/monster/Zombie/New                                         0.000        0.000        0.000            2
/mob/monster/other/Ghost/New                                    0.000        0.000        0.000            2
/mob/monster/other/BarrowWight/New                              0.000        0.000        0.000            1
/mob/monster/other/PaleWraith/New                               0.000        0.000        0.000            1
/mob/monster/other/DarkWraith/New                               0.000        0.000        0.000            2
/mob/monster/other/Shadow/New                                   0.000        0.000        0.000            3
/mob/monster/other/CarrionCrawler/New                           0.000        0.000        0.000            2
/mob/monster/other/Wolf/New                                     0.000        0.000        0.000            1
/mob/monster/other/WhiteWolf/New                                0.000        0.000        0.000            1
/mob/monster/Reptile/Snake/New                                  0.000        0.000        0.000            1
/mob/monster/other/CaveBear/New                                 0.000        0.000        0.000            1
/mob/monster/other/Ant/New                                      0.000        0.000        0.000            1
/mob/monster/proc/ACFromMonster                                 0.000        0.000        0.000           12
/mob/monster/other/Scorpion/New                                 0.000        0.000        0.000            3
/mob/monster/other/IceElemental/New                             0.000        0.000        0.000            1
/mob/monster/other/DustElemental/New                            0.000        0.000        0.000            1
/mob/monster/other/WoodStatue/New                               0.000        0.000        0.000            2
/mob/pc/proc/FPlayerImmune                                      0.000        0.000        0.000           10
/mob/monster/Human/Warrior/New                                  0.000        0.000        0.000            1
/mob/pc/proc/FPlayerHit                                         0.000        0.000        0.000           12
/mob/monster/Human/Bandit/New                                   0.000        0.000        0.000            3
/mob/monster/Human/Thief/New                                    0.000        0.000        0.000            2
/mob/monster/Bump                                               0.000        0.000        0.000          103
/mob/pc/proc/GainExperience                                     0.000        0.000        0.000            2
/mob/pc/proc/ExpForNextLevel                                    0.000        0.000        0.000            3
/mob/pc/proc/GainLevel                                          0.000        0.000        0.000            1
/mob/monster/NewTurnController                                  0.000        0.000        0.000           62
/Page/proc/DisplayPage                                          0.000        0.000        0.000            1
/mob/pc/Bump                                                    0.000        0.000        0.000           14
/mob/pc/Login                                                   0.000        0.000        0.000            1
/mob/monster/proc/Alert                                         0.000        0.000        0.000           60
/mob/monster/proc/FMonsterResists                               0.000        0.000        0.000            6
/mob/pc/Stat                                                    0.000        0.000        0.000          405
/mob/monster/proc/FMonsterImmune                                0.000        0.000        0.000            6
/mob/monster/proc/ExpFromMonster                                0.000        0.000        0.000            2
/mob/monster/Humanoid/BullMan/New                               0.000        0.000        0.000            1
/mob/proc/NewTurnController                                     0.000        0.000        0.000            2
/mob/monster/proc/FDeadDoDamage                                 0.000        0.000        0.000           10
/mob/monster/Humanoid/RatMan/New                                0.000        0.000        0.000            2
/mob/Stat                                                       0.000        0.000        0.000          405
/mob/monster/proc/DoMonsterDamage                               0.000        0.000        0.000           10
/ATTACK/New                                                     0.000        0.000        0.000          121
/mob/monster/proc/FMonsterHit                                   0.000        0.000        0.000           10
/TurnController/MonsterController/AddMob                        0.000        0.000        0.000          122
/TurnController/MonsterController/RemoveMob                     0.000        0.000        0.000           66
/TurnController/MonsterController/proc/MonsterDeath             0.000        0.000        0.000            2
/TurnController/MonsterController/proc/MonsterBirth             0.000        0.000        0.000           60
/mob/monster/proc/DoMonsterAttack                               0.000        0.000        0.000           10
/proc/MonLevelFromLevel                                         0.000        0.000        0.000           60
/proc/FindPage                                                  0.000        0.000        0.000            1
/TurnController/New                                             0.000        0.000        0.000            2
/TurnController/proc/AddMob                                     0.000        0.000        0.000            6
/TurnController/proc/RemoveMob                                  0.000        0.000        0.000            3
/proc/StartsWith                                                0.000        0.000        0.000            5
/proc/FNormalDamage                                             0.000        0.000        0.000           20
/turf/GetDenseObject                                            0.000        0.000        0.000           60
/turf/outside/houses/church/a2/Entered                          0.000        0.000        0.000            1
/turf/outside/floor/dungeonexit/Entered                         0.000        0.000        0.000            1
/turf/outside/floor/fence/fencecent/Entered                     0.000        0.000        0.000           25
/Page/AdminPages/CreateMonster/GetPage                          0.000        0.000        0.000            1
/Page/AdminPages/GetPage                                        0.000        0.000        0.000            1
/Page/AdminPages/CreateTurf/GetPage                             0.000        0.000        0.000            1
/Page/LoginForm/GetPage                                         0.000        0.000        0.000            1
/Page/About/GetPage                                             0.000        0.000        0.000            1
/Page/TempleOfOdin/GetPage                                      0.000        0.000        0.000            1
/Page/TempleOfOdin/ProcessPage                                  0.000        0.000        0.000            1
/Page/proc/GetPage                                              0.000        0.000        0.000            1
/mob/monster/Humanoid/GoblinFighter/New                         0.000        0.000        0.000            2
/Page/proc/UnDisplayPage                                        0.000        0.000        0.000            1
/client/proc/base_Initialize                                    0.000        0.000        0.000            1
/icon/New                                                       0.000        0.000        0.000            2
/mob/monster/Human/Berserker/New                                0.000        0.000        0.000            3
/icon/proc/Blend                                                0.000        0.000        0.000            4
                                                                0.000        0.000        0.000            1
                                                                0.000        0.000        0.000            2
                                                                0.000        0.000        0.000           61
/client/New                                                     0.000        0.000        0.000            1
                                                                0.000        0.000        0.000            1
#7 Mar 14 2012, 7:49 pm
Here's the top few lines after a few more minutes.

                                        Profile results (total time)
Proc Name                                                    Self CPU    Total CPU    Real Time        Calls
--------------------------------------------------------    ---------    ---------    ---------    ---------
/mob/Move                                                       0.705       14.957     1166.634        26787
/mob/monster/UpdateTurnController                              14.038       14.039       14.048        26830
/mob/monster/proc/AIMove                                        0.580       11.965       11.969        21083
/mob/proc/Step                                                  0.003        0.242        5.511          132
/mob/pc/Move                                                    0.001        0.236        5.504           97
/client/North                                                   0.002        0.102        2.527           59
/client/South                                                   0.002        0.092        2.260           55
/client/West                                                    0.000        0.028        0.366            9
/client/East                                                    0.000        0.024        0.362            9
/mob/proc/UpdateTurnController                                  0.147        0.222        0.222           98
/mob/pc/Logout                                                  0.004        0.065        0.065            1
#8 Mar 14 2012, 8:03 pm
and a few more minutes.

                                        Profile results (total time)
Proc Name                                                    Self CPU    Total CPU    Real Time        Calls
--------------------------------------------------------    ---------    ---------    ---------    ---------
/mob/Move                                                       1.989       40.921     4645.064        77348
/mob/monster/UpdateTurnController                              38.623       38.627       38.644        77391
/mob/monster/proc/AIMove                                        1.608       35.159       35.179        64472
/mob/proc/Step                                                  0.003        0.242        5.511          132
/mob/pc/Move                                                    0.001        0.236        5.504           97
/client/North                                                   0.002        0.102        2.527           59
/client/South                                                   0.002        0.092        2.260           55
/client/West                                                    0.000        0.028        0.366            9
/client/East                                                    0.000        0.024        0.362            9
/mob/proc/UpdateTurnController                                  0.147        0.222        0.222           98
/mob/proc/UpdateLevelController                                 0.144        0.148        0.167        77350
/LevelController/proc/ManageLevel                               0.001        0.078        0.078          264
/mob/pc/Logout                                                  0.004        0.065        0.065            1
#9 Mar 14 2012, 8:57 pm
Best response
I'm no expert on these things, but I do know it takes well over 10 minutes for any proc in my game to reach 1.000 Total or self CPU. And I really didn't think my game was all that CPU friendly so far.

My Real Time for procs takes forever to go above 1 as well
#10 Mar 14 2012, 10:24 pm
Ah, thanks. I was told by one of the previous devs that the AI system would need to be replaced and I suppose he wasn't kidding. I didn't know what to think of the high numbers cause I didn't even know what was high or not.

Anywho, case closed. It's the AI system. Thanks again guys.
#11 Mar 14 2012, 10:33 pm
I assume these two are part of the AI system?

/mob/Move                                                       1.989       40.921     4645.064        77348
/mob/monster/UpdateTurnController                              38.623       38.627       38.644        77391


Any NPC should lay dormant until they sense a player nearby.

We can't have 1000 NPC's moving constantly now can we? :)
#12 Mar 14 2012, 10:42 pm
Yeah those two are part of the AI system. They control turn-based mob movement, unique to this game. It's weird that they use so much CPU yet they actually use hardly any. I mean, opening up task manager to check it uses more than the server does.

Buuut now that I know what the problem is I can fix it. Or replace the mob spawn system. Idk, either way it'll work eventually.
#13 Mar 15 2012, 12:06 am (Edited on Mar 15 2012, 12:13 am)
Sorting the profiler by Self CPU is usually most helpful. Also, use world.cpu to check for meaningful usage. The BYOND engine maxes out far sooner than the physical system gets anywhere near being stressed.

Also, as ATHK pointed out, UpdateTurnController is using a ridiculous amount of CPU compared to anything else, and the real time of /Move would make me think that there was some type of infinite loop there, or long sleep(s), perhaps.

A.T.H.K wrote:
We can't have 1000 NPC's moving constantly now can we? :)
Sure we can, we just can't have them doing complex things like searching for targets or path finding =P
The issue isn't really that you can't do that, but that doing so is usually a waste of power. No point in having 1,000 NPCs wandering around if they're 10 miles away from any player and have no real purpose.
#14 Mar 15 2012, 1:02 am
In response to Falacy (#13)
Falacy wrote:
Sure we can, we just can't have them doing complex things like searching for targets or path finding =P
The issue isn't really that you can't do that, but that doing so is usually a waste of power. No point in having 1,000 NPCs wandering around if they're 10 miles away from any player and have no real purpose.

:) That was my point I just didn't explain it so well!

#15 Mar 15 2012, 2:09 am (Edited on Mar 15 2012, 2:22 am)
I added one single, simple line of code to make the mobs spawn only when a player is near and it made the server run 1000x better! lol

But I still need to change the AI completely. It uses way too much CPU.