Image crash still in 512.1411

ID:2348506
 
Feb 24 2018, 12:50 pm (Edited on Feb 24 2018, 2:13 pm)
BYOND Version:512.1411
Operating System:Windows 10 Pro 64-bit
Web Browser:Firefox 58.0
Applies to:Dream Seeker
Status: Open
Issue hasn't been assigned a status value.
Descriptive Problem Summary:
Can still cause client to crash when some images come onto the screen.
(4f08.47c4): Access violation - code c0000005 (first/second chance not available)
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for byondcore.dll -
eax=6e6f6c5f ebx=345bf008 ecx=345bf008 edx=0db55e08 esi=345bf008 edi=00000000
eip=59568425 esp=0085c128 ebp=0085c244 iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00210202
byondcore!DMIcon::clear+0x15:
59568425 8b1cb8          mov     ebx,dword ptr [eax+edi*4] ds:002b:6e6f6c5f=????????
0:000> k
 # ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
00 0085c244 5955e024 byondcore!DMIcon::clear+0x15
01 0085c26c 5968f49d byondcore!DungClient::BeginDelay_CIO+0x1594
02 0085c2a8 5968f49d byondcore!ClassyCallback::Callback+0xd
03 0085c34c 5954a07e byondcore!ClassyCallback::Callback+0xd
04 0085c364 5953bc28 byondcore!FilterChain::Interpolate+0x22e
05 0085c7cc 5953bb68 byondcore!SharedFilter::operator!=+0x6b98
06 0085c89c 595235cb byondcore!SharedFilter::operator!=+0x6ad8
07 0085d140 59522ff4 byondcore!DMTextPrinter::BodyTag+0xc7b
08 0085d154 596cfc5a byondcore!DMTextPrinter::BodyTag+0x6a4
09 0085d16c 596cdf79 byondcore!ByondHttpServerLink::WriteBuffer+0x2a0a
0a 0085d1b0 59524108 byondcore!ByondHttpServerLink::WriteBuffer+0xd29
0b 0085d1c4 596cf868 byondcore!DungServer::GetServerPort+0x5b8
0c 0085d1ec 596ce2d5 byondcore!ByondHttpServerLink::WriteBuffer+0x2618
0d 0085d1fc 596e0351 byondcore!ByondHttpServerLink::WriteBuffer+0x1085
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for dreamseeker.exe -
0e 0085d224 0097f812 byondcore!SocketLib::Event_io+0x1f1
0f 0085d254 592f540a dreamseeker+0x4f812
10 0085d320 592f50ca mfc120!CWnd::OnWndMsg+0x31d [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 2272]
11 0085d340 592f36ad mfc120!CWnd::WindowProc+0x22 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 2094]
12 0085d3b0 592f38cf mfc120!AfxCallWndProc+0x99 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 285]
13 0085d3d0 591f3a36 mfc120!AfxWndProc+0x34 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 434]
14 0085d40c 74dce0bb mfc120!AfxWndProcBase+0x34 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\afxstate.cpp @ 299]
15 0085d438 74dd8849 user32!_InternalCallWinProc+0x2b
16 0085d45c 74ddb145 user32!InternalCallWinProc+0x20
17 0085d52c 74dc90dc user32!UserCallWinProcCheckWow+0x1be
18 0085d598 74dc38c0 user32!DispatchMessageWorker+0x4ac
19 0085d5a0 592e2d8c user32!DispatchMessageA+0x10
1a 0085d5b0 592f7f80 mfc120!AfxInternalPumpMessage+0x3e [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\thrdcore.cpp @ 183]
1b 0085d5d4 5929a745 mfc120!CWnd::RunModalLoop+0xc6 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\wincore.cpp @ 4644]
1c 0085d5ec 5929a9c8 mfc120!CWnd::CreateRunDlgIndirect+0x3e [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\dlgcore.cpp @ 474]
1d 0085d640 0096202e mfc120!CDialog::DoModal+0x109 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\dlgcore.cpp @ 633]
1e 0085fee0 59306300 dreamseeker+0x3202e
1f 0085fef4 00991d3e mfc120!AfxWinMain+0x47 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\winmain.cpp @ 37]
20 0085ff40 76c98654 dreamseeker+0x61d3e
21 0085ff54 77b24a77 kernel32!BaseThreadInitThunk+0x24
22 0085ff9c 77b24a47 ntdll!__RtlUserThreadStart+0x2f
23 0085ffac 00000000 ntdll!_RtlUserThreadStart+0x1b


I have a largeish (11mb) client memory dump if you'd like it at the time of the exception. Appears to be access violation exception, so maybe the return of that 'reading off the end of the master appearance list' bug again? I'd link it here but I assume client memory contains sensitive session information or something. Is there somewhere else I can send it, if you want the full one?

Numbered Steps to Reproduce Problem:
On our SS13 server, just move around until the client crashes.

Code Snippet (if applicable) to Reproduce Problem:
Not sure of the cause.

Expected Results:
Less crashing.

Actual Results:
Crashing.

Does the problem occur:
Every time? Or how often? I can reproduce it within 20 seconds of joining the server.
In other games? Not sure, don't play other games.
In other user accounts? Yes, many.
On other computers? Yes, many.

When does the problem NOT occur?
While remaining completely still and nothing nearby moving.

Did the problem NOT occur in any earlier versions? If so, what was the last version that worked? (Visit http://www.byond.com/download/build to download old versions for testing.)

Workarounds:
None that I'm aware of.
Feb 24 2018, 9:09 pm
Well a crash is not a hang, so at least that's an improvement. I'll do a trace on this to see what I can find, but a test project would help even more. I might need one anyway in the long run.
Feb 24 2018, 9:11 pm
It does crash, unfortunately. Hangs for about 2 seconds, then throws up the normal Windows exception handler. I don't have the slightest idea what causes this, though.

I can give you the larger memory dump if you think it would help. It's about 11mb, so if you give me somewhere to send it, I can.
Feb 24 2018, 10:08 pm
Just looking briefly at that output, the very close proximity to FilterChain::Interpolate makes me wonder if in fact the problem is filter-related. However the calls to ClassyCallback::Callback and DMIcon::clear are more suggestive that this could be an issue related to icons, or to some heap corruption that's touching the icons.

I'm very likely to need a test case for this. Since you said this happens "when some images come onto the screen", is there a specific reliable point where this always happens? That would be a huge help.
Feb 24 2018, 10:13 pm
I'll admit, this occured on a server running AFTER http://www.byond.com/forum/?post=2348509 had occured on it. Is it possible that there's a server bug that causes corruption but doesn't quite kill the server, and a client bug that is incapable of handling the results?
Feb 24 2018, 11:13 pm
I don't see it as likely that the client would choke on something that went wrong on the server end. Not to crash, anyway.

Have you gotten this issue to happen in any consistent way, though?
Feb 24 2018, 11:15 pm
I was able to get it to crash at-will for that server run, but not really after as we updated the server to avoid touching anything that seemed related. I can give you the client full crash dump if you want. Maybe it contains more info, I dunno.
Feb 25 2018, 7:58 pm
I now have a 500mb dump at exception-time with stack, of the client and all associated memory, if you'd like it. I'm still not able to reproduce it in any sort of test case, but our live server randomly has the issue and I can reproduce it there.

EDIT: I see that this time, on server start, there was a:
BUG: Bad ref (2:286118) in DecRefCount(DM living.dm:1135)


living.dm:1135 is:
animate(client, color = null, time = 10)
Feb 26 2018, 9:19 am
If you have the stack traces from each crash, I'd like to compare them all to see if I can find any common threads. If the errors are happening in DMIcon::clear(), though, then I know for sure I can't fix this without a reliable test case--not without finding it accidentally or solving it as a result of dealing with a separate issue.

Server-based errors definitely won't have an impact unless the game is being hosted in DS and the client that crashes is also the server.
Feb 26 2018, 9:37 am
I sent links to you in a forum message, since I'm not sure I want to post that 500mb dump in public.
Feb 26 2018, 9:46 am
The 500 MB dump isn't anything I have a use for; it won't tell me anything helpful. The stack traces might from them might be helpful, and will be easier for you to grab.

The trace you shared here tells me that there is a crash in DMIcon::clear(), and it's happening in the process of reading an icon from a cache file, or individual resource, sent by the server. But that's really all I can tell. I expect your other traces will show the same or similar, although it's possible they won't.

Ultimately what I really need is a way to catch this reliably on my end.
Feb 26 2018, 9:50 am
I'm never really able to recreate it locally, it only shows up on our prod server, which makes me wonder why that is. I'll poke at it s'more if it shows up again.
Feb 26 2018, 9:03 pm (Edited on Feb 26 2018, 10:33 pm)
I'm actually experiencing this again. It only happens when you are playing the game for some time. Occurred for me three times in about 30 minutes.

I see no blatant cause of effect, except for maybe one clue: each time my client froze, it was at the time of (or immediately close to) the triggering of some objects in view having their loc nulled and subsequently being deleted (doesn't matter whether garbage collection does it, or if the delete is forced).
Feb 26 2018, 11:59 pm
In response to FKI
I'm not certain this specific report relates to images.

If you're having image trouble again though, as usual a test case would help a ton. I'll look into the image code relating to objects being deleted and see if there's any pattern.
Feb 28 2018, 8:37 pm (Edited on Mar 1 2018, 3:07 pm)
Perhaps. Regardless, I don't think I'm any help at this point. The issue only occurs when playing with others.

Edit: Just as a mention, the problems don't occur in the stable build (511.1385).