Linux Based Server Crashing

ID:757779
 
May 21 2012, 10:28 am (Edited on May 21 2012, 10:34 am)
BYOND Version:494
Operating System:Linux
Web Browser:Chrome 19.0.1084.46
Applies to:Dream Daemon
Status: Unverified
Thus far we've been unable to verify or reproduce this bug. Or, it has been observed but it cannot be triggered with a reliable test case. You can help us out by editing your report or adding a comment with more information.
-I am not good on reporting but i will try my best-
Descriptive Problem Summary: Everytime server reaches above 5 players and uptime gets over 10 mins server just crashes

Numbered Steps to Reproduce Problem:Almost every times

Code Snippet (if applicable) to Reproduce Problem:











Expected Results:



Actual Results:Welcome BYOND! (4.0 Public Version 494.1131)

BUG: Out of memory.

The BYOND hub reports that port 7777 is not reachable.

BUG: Crashing due to an illegal operation!

Mon May 21 18:18:23 2012

World opened on network port 3948.

Welcome BYOND! (4.0 Public Version 494.1134)

BUG: Out of memory.



Does the problem occur:

Every time

Tested with older version of source as well



When does the problem NOT occur?



Did the problem NOT occur in any earlier versions? If so, what was the last version that worked? (Visit http://www.byond.com/download/build to download old versions for testing.)



Workarounds:





    


      





	



	

	












































  

  


































	
	
#1 May 21 2012, 11:10 am












  

  

  





Thanks for the report, Hassanjalil. We may need a little more information to investigate this fully.



1) Can you confirm whether this happens in newer versions?



2) Do you have the stack trace that is printed out by the crash? In Linux you should have a printout that shows the problem. I don't expect this to tell us much under the circumstances but it can't hurt.



3) What game were you trying to run? (Please provide a link to the hub entry if possible.)



Because your issue is that the server is running out of memory, I'm not positive this is something we can deal with, unless something in a recent version has caused memory to balloon out of control under special circumstances. Most likely the issue is simply that you're running a very large world on a machine that has far too little memory, in which case the best option is to get more memory.



If this is a game of your own design, I'd also look at anything that might possibly be taking up too much memory. There are cases where game design can have an impact on this, although generally speaking it's hard for a server to let memory get out of control by accident. BYOND is usually very good at cleaning up after itself, but there are some kinds of data structures you could use that would evade garbage collection if not handled properly. The simplest thing you can do is periodically run a routine to print out the number of objs, mobs, datums, and possibly lists in use; you should be able to see if any of those are growing out of control.





  





	




	












































  

  


































	
	
#2 May 22 2012, 3:12 am (Edited on May 22 2012, 3:49 am)












  

  

  





1) Can you confirm whether this happens in newer versions?



1A)It happened in both v493 and 494 just the difference was that 494 was on for 30 minutes before it decided to crash.



2) Do you have the stack trace that is printed out by the crash? In Linux you should have a printout that shows the problem. I don't expect this to tell us much under the circumstances but it can't hurt.



2A)I have no idea because i am not much of linux user



3) What game were you trying to run? (Please provide a link to the hub entry if possible.)



3A) the game is based on DBZ(which is banned but still continued) it is also a mod version of Dragon Ball Z:Heroes United which was released by Falacy



If this is a game of your own design, I'd also look at anything that might possibly be taking up too much memory.



I tried running my game aswell and samething happened, so it cant be both game doing the same thing.





  





	




	












































  

  


































	
	
#3 May 22 2012, 5:53 am (Edited on May 22 2012, 6:26 am)












  

  

  





I doubt it runs out of memory because my machine has 32GB RAM.



edit I might not be a BYOND developer but this must not be good. DreamDaemon: symbol lookup error: DreamDaemon: undefined symbol: _ZN10DungServer20PrintProcDiagnosticsEv





  





	




	












































  

  


































	
	
#4 May 22 2012, 6:55 am












  

  

  





Here is another bug









Sun May 20 02:11:58 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 493.1116)
The BYOND hub reports that port 3948 is not reachable.
The BYOND hub reports that port 3948 is reachable.

******************************************
Caught SIGTERM, attempting clean shutdown.

Sun May 20 02:56:54 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 493.1116)
The BYOND hub reports that port 3948 is reachable.
Sun May 20 03:47:36 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 493.1116)
The BYOND hub reports that port 3948 is reachable.
Sun May 20 05:09:06 2012
Dream Daemon FAILED to open port 3948!
Sun May 20 05:40:41 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 493.1116)
The BYOND hub reports that port 3948 is reachable.
Sun May 20 07:13:15 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 493.1116)
BYOND Error: corrupt map data in world file
Sun May 20 07:16:04 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 493.1116)
The BYOND hub reports that port 3948 is reachable.
BUG: Unexpected hub certificate (65535)
BUG: Unexpected hub certificate (65535)
BUG: Removing corrupt rsc entry 'Dialga.dmi'
BUG: Removing corrupt rsc entry 'Deoxys.dmi'
BUG: Finished erasure with refcount=1 (ref=5:3) DM (Verbs.dm:143)
BUG: Finished erasure with refcount=1 (ref=5:0) DM (:0)
BUG: Crashing due to an illegal operation!
DreamDaemon: symbol lookup error: DreamDaemon: undefined symbol: _ZN10DungServer20PrintProcDiagnosticsEv
Mon May 21 22:55:24 2012
World opened on network port 3948.
Welcome BYOND! (4.0 Public Version 492.1115)
The BYOND hub reports that port 3948 is reachable.
BUG: Finished erasure with refcount=1 (ref=5:3) DM (:0)












  





	




	












































  

  


































	
	
#5 May 22 2012, 8:19 am












  

  

  





I recommend keeping an eye on memory usage as it goes along--the out of memory bug means it's having trouble allocating memory. Since this is a 32-bit application, it's never going to see all 32 GB of memory to use; it can only use up to 4 GB (and I wouldn't rule out the possibility it'd have issues using more than 2). That's still way more than it should ever need though, so if the memory is jumping up something has to be wrong.



The errors that stand out the most to me here are "corrupt map data in world file", suggesting the .dmb itself is damaged and needs to be recompiled (this could be a major source of problems), and the corrupt .rsc entries. Neither of these should really be happening, which suggests your files are messed up.





  





	




	












































  

  


































	
	
#6 May 22 2012, 8:43 am












  

  

  





The errors that stand out the most to me here are "corrupt map data in world file", suggesting the .dmb itself is damaged and needs to be recompiled (this could be a major source of problems), and the corrupt .rsc entries. Neither of these should really be happening, which suggests your files are messed up.

I know that i hosted the game at wrong time(when it was updating)







I recommend keeping an eye on memory usage as it goes along--the out of memory bug means it's having trouble allocating memory. Since this is a 32-bit application, it's never going to see all 32 GB of memory to use; it can only use up to 4 GB (and I wouldn't rule out the possibility it'd have issues using more than 2). That's still way more than it should ever need though, so if the memory is jumping up something has to be wrong.



2 game using was hosted on same vps but with different type of gaming but it was still crashing even after turning off every possible loop at the start up





  





	




	












































  

  


































	
	
#7 May 22 2012, 8:52 am




	
In response to Hassanjalil (#6)










  

  

  





You know, now that I think about it the "undefined symbol" error makes me think there's something wrong with the installation. That's where you'd normally get a stack trace, but obviously that's not happening. However this also happened right after your issue with corrupt .rsc entries, so if that was update-related then I wouldn't give it much credence. If you're always seeing that error in place of a stack trace though, then I'd try updating the BYOND installation.



On the subject of memory, 494 did make some improvements in that area and might be worth trying. If you use pixel movement, 494 fixed a couple of leaks related to that.





  





	




	












































  

  


































	
	
#8 May 22 2012, 1:07 pm












  

  

  





I installed version 492 to test whether it crashes and the response was same game kept crashing so the





  





	




	












































  

  


































	
	
#9 May 22 2012, 2:03 pm




	
In response to Hassanjalil (#8)










  

  

  





494 is the one I asked about, because it has a couple of memory leak fixes not present in 493. I suspect the problem will still exist anyway if this project does not use pixel movement though.





  





	




	












































  

  


































	
	
#10 May 23 2012, 3:14 am












  

  

  





Project was made before v490 so there is no pixel movement being used as it would take longer to implement it.

Edit: Flashed new operating system on vps and installed byond again so i will be testing if it bugs





  





	




	












































  

  


































	
	
#11 May 23 2012, 7:58 am












  

  

  





Okay. To be clear, I'm not ruling out a BYOND bug, which would likely be in the form of a memory leak. But I need a lot more data to investigate. I'll need to get a look at how your server's memory usage is progressing over time up until the crash, and a stack trace may be helpful.



The trouble is, "out of memory" errors are just what it says on the tin, and they can be as much over-ambitious design or system constraint as anything else. If BYOND's memory allocations are growing, I'd like to know why: such as if more and more objects are being allocated but not deleted. Having an ongoing count of mobs, objs, datums, images, etc. would probably be a good idea. You can get a list of some of that by sending the SIGUSR2 signal to your server, which will spit out diagnostics. If you do that several times leading up to the crash you may be able to get a better picture of what resources the server is using. I expect more likely than not that this will point out a programming error in the project for you to address, rather than a BYOND bug, but again I can't rule that out.





  





	




	












































  

  


































	
	
#12 May 24 2012, 7:15 am












  

  

  





ok lummox i understand that problem will be more likey with my programming btw what does this indicates?









BUG: Finished erasure with refcount=1 (ref=5:6) DM (:0)
BUG: Finished erasure with refcount=2 (ref=5:4) DM (:0)
BUG: Finished erasure with refcount=1 (ref=5:6) DM (:0)












  





	




	












































  

  


































	
	
#13 May 29 2012, 2:24 pm












  

  

  





I've seen those come up before; I believe those warnings are related to some of the skin functions like winget(), but I've never been able to determine that for certain. They are unlikely to be related to the crash.





  





	




	












































  

  


































	
	
#14 May 29 2012, 2:24 pm












  

  

  





Lummox JR changed status to 'Unverified'