ID:1821482
 
BYOND Version:507
Operating System:Windows 8 64-bit
Web Browser:Chrome 41.0.2272.101
Applies to:Dream Maker
Status: Deferred

This issue may be low priority or very difficult to fix, and has been put on the back burner for the time being.
Descriptive Problem Summary:
When an icon's file size is too big, it has a chance of crashing Dream Maker during the saving process and will always crash when you compile before saving. The icon in question that I used to discover this bug was a 480x270 icon with three icon states containing 300 frames each for an animation.
The icon doesn't crash when saving at two icon states, but still takes a long time and crashes if I don't save before compoiling. The overall icon size is 64.7mb with two icon states of 300 frames each.

Numbered Steps to Reproduce Problem:
1. Open an icon in Dream Maker
2. Give it enough frames of animation at a large enough icon size until the icon is above 70 megabytes.
3. Try saving the icon.
4. Crash succeeded!

Code Snippet (if applicable) to Reproduce Problem: No coding involved.

Expected Results:
Saves the icon properly.

Actual Results:
Crashes Dream Maker. It doesn't just say "Not Responding" it outright says "Dream Maker has stopped working" every time.

Does the problem occur:
Every time? Or how often? Every time.
In other games? Any time you save with a large icon file size.
In other user accounts? Yup.
On other computers? I only have one computer to test it on. This computer has an Intel i7-4500CPU Dual Core with 1.8/2.4 Ghz and 8GB of RAM. It's also a regular Windows 8.1 64-bit.

When does the problem NOT occur?
When the icon size is small enough.

Did the problem NOT occur in any earlier versions? If so, what was the last version that worked?
Not sure about ancient BYOND builds, but all of the BYOND builds that support opening multiple files on tabs in Dream Maker crashes with this bug.

Workarounds:
Split up the icon into multiple icons and save them separately.

EDIT: Apparently this workaround actually has a chance of still crashing. I have 2 files with 2 icon states each and 300 frames per icon playing at 0.3 speed. When saving the first icon state to the third icon, it saved properly. Then upon compiling it seemed to crash again.
I can see how that would be a problem; even one of these states would be ginormous. You're talking about the equivalent of 7200x5400 per icon state. Each one would take up almost 150 MB uncompressed. At a minimum I would recommend using only one icon state per file at that size, just for sanity's sake.

But I have to ask if what you're doing can be done a simpler way that isn't so taxing on memory. DS is going to have trouble displaying those icons. Most GPUs will have trouble loading that many textures at that size, especially as their inclination is usually to round texture sizes up to power-of-2 increments, making each one actually 512x512. At 4 bytes per pixel that works out to 300 MB exactly in texture memory.
In response to Lummox JR
Lummox JR wrote:
I can see how that would be a problem; even one of these states would be ginormous. You're talking about the equivalent of 7200x5400 per icon state. Each one would take up almost 150 MB uncompressed. At a minimum I would recommend using only one icon state per file at that size, just for sanity's sake.

But I have to ask if what you're doing can be done a simpler way that isn't so taxing on memory. DS is going to have trouble displaying those icons. Most GPUs will have trouble loading that many textures at that size, especially as their inclination is usually to round texture sizes up to power-of-2 increments, making each one actually 512x512.

It's actually only around 60-70MB per icon. Also I am using browse_rsc() to load them in first. Then I load them through HUD images having a one-frame one-state icon at the same exact size as the animations on a higher layer, and filled black so they don't see it when the files load since the animations are supposed to overlay the screen and the background is black anyways.

It actually doesn't have much trouble on my end rendering the icons because the game forces it to be added to the cache and load the data before-hand.

EDIT2: I figured out the problem with what I did wrong in terms of it showing the wrong icon and fixed it accordingly. I still get the crashing error sometimes even at 300 frames on one icon, one state, and the same size.
In response to GameAnalysis
GameAnalysis wrote:
It's actually only around 60-70MB per icon.

But as I said, that's compressed. Uncompressed, a 450x270 icon with 300 frames is just over 139MB. When Dream Seeker or Dream Maker load the icon, it ends up in an uncompressed format.

Also I am using browse_rsc() to load them in first. Then I load them through HUD images having a one-frame one-state icon at the same exact size as the animations on a higher layer, and filled black so they don't see it when the files load since the animations are supposed to overlay the screen and the background is black anyways.

browse_rsc() is meaningless when it comes to displaying icons on the map. It has relevance only to displaying files in the browser.

I don't want to imply that the crash doesn't matter or this isn't a bug, because it is, but when you're running up against size numbers like this it's important to consider economic factors in your design. Memory is a limited resource, whether it's in Dream Maker or Dream Seeker, or even on your video card which ultimately Dream Seeker will try to enlist via DirectX. A few images of that size aren't a big deal, but 300 of them and you're dealing with something mondo ginormous. 300 frames is a lot for any animation; 450x270 is well beyond a normal sprite size. How much of that is truly necessary, and is there another approach that would use far less memory?
In response to Lummox JR
Lummox JR wrote:
GameAnalysis wrote:
It's actually only around 60-70MB per icon.

But as I said, that's compressed. Uncompressed, a 450x270 icon with 300 frames is just over 139MB. When Dream Seeker or Dream Maker load the icon, it ends up in an uncompressed format.
How is that a compressed file size? It isn't inside of an archive and the file sizes are being reported by Windows Explorer file properties. Keep in mind that the file size reported by Windows Explorer considers all data inside of the file, does it not?
How is that a compressed file size? It isn't inside of an archive and the file sizes are being reported by Windows Explorer file properties. Keep in mind that the file size reported by Windows Explorer considers all data inside of the file, does it not?

The DMI format natively uses the PNG file format. PNGs inherently are compressed by a number of methods to reduce redundant data inside of the file. PNGs don't need to have 4 bytes per pixel in the image.

GPUs work a little differently. They expect that an image should have 3-4 bytes per pixel. When an image is loaded off of the hard drive, the PNG compression is undone, and each pixel is given 4 bytes of memory to be contained within. 3 bytes are the color channels, and 1 byte is the alpha channel.

GPUS absolutely must use uncompressed textures, further, some GPUs require a power of 2 texture size. This means that if your image is 129x129 pixels, the texture will be loaded as a 256x256 texture and extra space will be buffered onto the edges. This extra space still takes 4 bytes per pixel in memory.

If you have a 450x270 icon file with 300 frames, you've already done something seriously wrong. Most 2D game art really shouldn't have animations that proceed over more than a handful of frames. It sounds like you are trying to make some kind of a video. This is really something that would be better done by actually embedding a video into a browser element rather than using a DMI.

If someone approached me asking me to make a 300 frame animation, as a pixel artist that freelances from time to time, I'd turn the job down, simply because I would know that whoever was paying me to do it wouldn't appreciate the amount of time it would take (and thus would be unwilling to pay the wage I would request). If you are generating that art using some kind of a tool, then it's entirely probable that you can perform the effect in a better way entirely.
In response to Ter13
Ter13 wrote:
How is that a compressed file size? It isn't inside of an archive and the file sizes are being reported by Windows Explorer file properties. Keep in mind that the file size reported by Windows Explorer considers all data inside of the file, does it not?

The DMI format natively uses the PNG file format. PNGs inherently are compressed by a number of methods to reduce redundant data inside of the file. PNGs don't need to have 4 bytes per pixel in the image.

Oh okay, I didn't know it used PNG as it's format. I thought it was just simply uncompressed all the time since it's used mainly for sprite work.

If you have a 450x270 icon file with 300 frames, you've already done something seriously wrong. Most 2D game art really shouldn't have animations that proceed over more than a handful of frames. It sounds like you are trying to make some kind of a video. This is really something that would be better done by actually embedding a video into a browser element rather than using a DMI.

I actually tried to do that at one point, but there were a lot of troubles doing it through the browser. It just so happened that every embed code I tried never worked properly in BYOND's browser: Some wouldn't hide video controls, some wouldn't autoplay but hid video controls, some showed both controls and didn't autoplay, and some just outright didn't function properly.
If BYOND supported HTML5 in it's browser or natively supported video files it would make these kinds of things a lot easier.
http://www.byond.com/forum/?post=1820325#comment14547769

You can force the browser into IE9/10/11 using a doctype depending on what version of IE the client has available and gain access to HTML5.

http://caniuse.com/#feat=video

If you gate people who have IE<9 installed out of your game, you can use the HTML5 VIDEO tag:

http://www.w3schools.com/tags/att_video_autoplay.asp

Then there's simply the autoplay tag.

Also, to hide the controls:

http://stackoverflow.com/questions/14161516/ html5-video-completely-hide-controls
In response to Ter13
Ter13 wrote:
If you gate people who have IE<9 installed out of your game, you can use the HTML5 VIDEO tag:
Why should I assume that everyone who is downloading my game has IE9+ when most people I know actually remove IE from their computer to begin with or have outdated machines?
Or would there be a way to bundle the browser into the files in case they don't have IE9+?
Why should I assume that everyone who is downloading my game has IE9+ when most people I know actually remove IE from their computer to begin with or have outdated machines?
Or would there be a way to bundle the browser into the files in case they don't have IE9+?

You could just not use DreamSeeker and rely on the webclient. Even then, guess what? It's HTML5, so IE9+ or another browser entirely.

I'd prefer it if it were possible to use a CEF browser in DS, but it's just not going to happen. You don't really have many other options.

I suppose you could distribute your game with an MSI installer that actually checked the version of IE, but that wouldn't help you much if you wanted to rely on the BYOND service itself to start your game.

Honestly, there's not a really good solution other than telling people running pre-2011 hardware/software to get updating.

Just for reference, Windows 7 was released in 2009. Windows Vista was released in 2007. XP was released in 2001. There's no point in supporting XP anymore. It's almost a decade and a half old.
In response to Ter13
Ter13 wrote:
Why should I assume that everyone who is downloading my game has IE9+ when most people I know actually remove IE from their computer to begin with or have outdated machines?
Or would there be a way to bundle the browser into the files in case they don't have IE9+?

You could just not use DreamSeeker and rely on the webclient. Even then, guess what? It's HTML5, so IE9+ or another browser entirely.

So I've relied on DreamSeeker so much in the past (I actually started BYOND when I was around 12) and I am a complete noob to the webclient. Would you mind sending over a chat message on how to use that since this bug report is getting way off topic?
If you've already received a reply then you might not need to pay attention to this, but i figured it was worth leaving this here:

You mainly just need to get yourself learning how html/css and javascript work, to be comfortable with the working with the webclient (php and jquery stuff[just specialised javascript, iirc] might also be worthwhile considering in general but hey!). You could go to sites like w3schools or just do a google search for some preferable resource to get you started.

Of course, there is more to the process though, particularly in the specifics of byond and it's webclient. For that i'd suggest some handy links as a good place to start on covering certain necessary areas i.e. what you can do with the webclient, how to do what you've done with ds so far with it -and, perhaps, likely even better- how to manage programming something that works for both ds and the webclient, etc. etc.
Beyond that though, utilising the webclient -and other related tags- within the forum search should, i believe, get you fairly far in terms of what the byond -> webclient transition entails for a developer (apart from learning how to use/becoming comfortable with the relevant languages :D).
In response to Turboskill
Turboskill wrote:
If you've already received a reply then you might not need to pay attention to this, but i figured it was worth leaving this here:
Yeah, I figured out that webclient wasn't the way to go since this game is a single-player game.

Thanks for the references though! I could use some Javascript when learning BYOND!
Ah, alright then. No problem.

Out of curiosity by the way, would you be able to upload a gif or video recording -just wondering about the size- of your attempt at a manual constructed 'video' for in game use? I'm just interested in seeing the results of what you attempted to do. This is if it isn't too much hassle (i'm assuming the icon editor runs the animations fine and allows you to view them, and you only get crashes on compile/game load?)
In response to Turboskill
Turboskill wrote:
Out of curiosity by the way, would you be able to upload a gif or video recording -just wondering about the size- of your attempt at a manual constructed 'video' for in game use? I'm just interested in seeing the results of what you attempted to do. This is if it isn't too much hassle (i'm assuming the icon editor runs the animations fine and allows you to view them, and you only get crashes on compile/game load?)

Sorry for the wait time. I was uploading the files but MEGA kept crashing.
I just used this clip, isolated when the bgm track started through video editing, exported as a png sequence, opened as a sequence in an image/GIF editing software as 30FPS, resized to 480x270, and created a GIF for every every 10 seconds (300 frames) on the video.
To get the crash, I just pasted the frames of three GIFs into one icon (three separate icon_states). Then try saving or compiling.

BYOND isn't designed to handle large icon sizes, so it can cause issues doing this anyways. When talking with Ter13, he said it was best to use HTML5 through Javascript to make a video for the game. The only problem with me doing that is it's too advanced for my level at the moment.
Lummox JR changed status to 'Deferred'
Ah ok so you reverse engineered from what was already a video in he first place, gotcha. I was more interested to see if you'd gone through the trouble of/managed to complete manual rendition of a video using pixel as is from scratch lol, kind of like how flip books(?) work. Would have just liked to see how that turned out, but indeed doing as you did -when you thought it your only option- does seem much more feasible. lol.
In response to Turboskill
Turboskill wrote:
Ah ok so you reverse engineered from what was already a video in he first place, gotcha. I was more interested to see if you'd gone through the trouble of/managed to complete manual rendition of a video using pixel as is from scratch lol, kind of like how flip books(?) work. Would have just liked to see how that turned out, but indeed doing as you did -when you thought it your only option- does seem much more feasible. lol.

Yeah, I was reading through Ter's Snippet Sundays and finally learned how to utilize HTML5 for video. Still doesn't erase the bug, just a workaround to it.

Also I'm kinda having troubles using some Javascript to call verbs for some reason:
http://www.byond.com/forum/?post=1823146