In response to Falacy (#19)
As far as the networking code itself goes, I'd say it's pretty well optimized for being a generic system (where most of the processing needs to be client-side). The server will only send data that has been changed, and compression is done as-needed.

Since neither your bandwidth nor cpu are being maxed out, I have to believe you are getting lag spikes (where your cpu is instantaneously maxed) rather than a continuous lag. This may occur when some world event forces updates on all users simultaneously or something to that effect. Does this seem likely, given what you've seen of the game, or are things basically sluggish all the time?

To properly diagnose this, I think we'll have to host a busy server from a debug version so we can see where the bottlenecks are. But any further specifics you can provide would help. Like I said, we can run some simple tests to see how we do (vs, say, FTP) in mostly idle-states and work from there.
In response to Tom (#20)
Tom wrote:
Since neither your bandwidth nor cpu are being maxed out, I have to believe you are getting lag spikes (where your cpu is instantaneously maxed) rather than a continuous lag. This may occur when some world event forces updates on all users simultaneously or something to that effect. Does this seem likely, given what you've seen of the game, or are things basically sluggish all the time?

The games have occasional lag spikes, but I don't worry much about them. Every online game I've ever played has its lag spikes every now any again, I don't consider it out of the ordinary. I'm more worried about the continual lag buildup as player count increases. 30 70 and 90 seem to be main lag increase points. Before 30 players the game will play as lag free as it will get when connected to a BYOND server. At 70 the lag will start to get noticeable. At 90+ players in HU I'd say the game is nearly half as responsive as it should be.

To properly diagnose this, I think we'll have to host a busy server from a debug version so we can see where the bottlenecks are. But any further specifics you can provide would help. Like I said, we can run some simple tests to see how we do (vs, say, FTP) in mostly idle-states and work from there.

I run my games in debug mode if that's what you're referring to, any steps I should take to get you additional information?
In response to Falacy (#21)
Falacy wrote:
The games have occasional lag spikes, but I don't worry much about them. Every online game I've ever played has its lag spikes every now any again, I don't consider it out of the ordinary. I'm more worried about the continual lag buildup as player count increases. 30 70 and 90 seem to be main lag increase points. Before 30 players the game will play as lag free as it will get when connected to a BYOND server. At 70 the lag will start to get noticeable. At 90+ players in HU I'd say the game is nearly half as responsive as it should be.

What I'm wondering is if the game is continuously laggy or just having more frequent lag spikes as the player count increases. I don't know if this is something you can easily determine from the cpu monitor. It seems odd that you'd have lag if cpu and bandwidth are continuously available. My best theory is that frequent world events (such as the sending of the map or possibly an internal event to your code) are, at some number of players, taking more than a tick to complete, and this is resulting in lag. But then I'd think that'd show up on the monitor.

Have you tried increasing your tick lag to see if that has any effect at all? Even if it's not a direct improvement, if the behavior is different it might give us some insight into the problem. For instance, it might worsen the behavior for less players but improve it for more.

I run my games in debug mode if that's what you're referring to, any steps I should take to get you additional information?

I mean we'd have to run the server in our debugger, so we could properly profile it. This would require running your .dmb and having a large number of players connect, or finding a comparable test case. But we can try this (with your approval obviously) as a last resort if we can't figure it out with smaller test cases.
In response to Tom (#22)
Tom wrote:
Have you tried increasing your tick lag to see if that has any effect at all? Even if it's not a direct improvement, if the behavior is different it might give us some insight into the problem. For instance, it might worsen the behavior for less players but improve it for more.

HU has a system that modifies tick lag dynamically based on world.cpu. If world.cpu reaches a certain % (set by the host) the tick lag will increase to limit player's actions until the server can catch up. From what I've seen the gameplay only slows as the tick_lag increases. Though it rarely kicks in on the new server since its minimum setting is 50%

Also, for a good, simple example of the lag I'm talking about when connecting to a server compared to playing locally:
Create an icon with 2 icon states that have the same name, mark one for movement. ie: A flying state, one hovering for standing still, one marked as movement where the arms are out in front flying like superman.
When you fly around when playing locally the game will correctly display the superman position as long as you fly. But, when connected to a host, while flying around you'll constantly swap between the superman position and the hover state.
In response to Tom (#18)
Tom wrote:
From the developer's perspective, the best thing to do to accomodate large volumes of players is, as others have mentioned, using link() and multiple servers. In recent updates we should have removed any of the old prompts that bugged players when they were moving between servers, so the experience is seamless. Correct me if I'm wrong about that and we will fix it.

This works fairly well, FYI. The delay between being linked seems to be approximately 3-5 seconds, which isn't bad.

One thing that could improve is the fact that the interface reverts to default state when you switch to a new world. It also moves the physical position of the window to some static location, every time you link(). The map also goes black. Also, you get two connection messages in the default output window which are impossible to suppress and quite annoying. Ideally, the games windows and their contents stay exactly the same while being linked. The map should basically be a snapshot of how it looked when you got link()ed. At least in the case of being link()'ed to the same DMB. At the moment, I do a few things to try and make this as seamless as possible:

All interface elements that require modification at runtime to look how they shouild before being shown, are never visible by default - Otherwise they'd show in their default state when being linked.

The map is not visible by default. At the same position and size as the map is a label with the text 'Loading', which is visible by default. This has the effect of basically showing 'Loading' there in place of the map.

The default input is resized to 1x1 pixel and hidden in a corner - This window will 'absorb' the default messages that aren't nice.

Of the above, I can see some of the issues here might not be possible to solve (Or be difficult). But things are not quite seamless yet, even though what I have now looks pretty close - The resizing/repositioning is very bad; and if your game operates with multiple windows, some of which are not always open, that will look really bad as well as they will poof. I suppose in that case you'd do what I did with the map for those windows, using a 'Loading' label as visible across the entire window.

On networking: I'd like to see isolated testcases, as opposed to referring to GOA and the like - As Stephen said, and I don't mean this as criticism either, but optimizing games for many players is a pretty difficult thing that not many people in BYOND grasp.
We investigated this somewhat this afternoon.

Our preliminary tests are showing that BYOND is performing reasonably as far as packet transfer goes, so we need to figure out why you are getting such lag with apparently available cpu. Since you run a multi-processor system it's possible that at peak times one or more of your processors is simply maxed out (single-threaded DD only able to use one processor at a time). On an 8-processor system the conventional cpu monitor would just show each such process as using 12.5% cpu, even though it is maxed out. So we need to get some better diagnostics by using a tool that can analyze multiple cores. Here's one possibility:

http://www.softpedia.com/get/Windows-Widgets/ System-Utilities/Multiple-Processor-Monitor.shtml

Try that or an alternative and let us know what it reports with a DD that appears to be overloaded. If each core still shows consistently available cpu, then we have a different issue (possibly related to bandwidth-- we'll have to check on that separately). Also, I'm interested to see if world.cpu aligns with this-- from your reports, it too is reporting available cycles but if the core is truly maxed out, that is wrong.

We'd appreciate similar feedback from anyone else experiencing similar issues with a large volume of users.
In response to Tom (#25)
Tom wrote:
http://www.softpedia.com/get/Windows-Widgets/ System-Utilities/Multiple-Processor-Monitor.shtml

What am I supposed to be doing with that thing? It seems to just do exactly what the task manager does...
http://www.angelfire.com/hero/straygames/ByondBugs/ YoureAWidget.png

EDIT: That's with 5 servers of HU and a server of PC being hosted from DDs.
Also, I don't have a multi-processor system, I have a single-processor multi-core.
In response to Falacy (#26)
Falacy wrote:
EDIT: That's with 5 servers of HU and a server of PC being hosted from DDs.
Also, I don't have a multi-processor system, I have a single-processor multi-core.

OK, that's interesting data. Are these servers showing lag yet? If they are, do any of the cores (effectively same thing as a processor) show spikes where they are using 100%?
Page: 1 2