Allow browse_rsc to export an entire DMI file at once

ID:2365093
 
Apr 27 2018, 9:04 pm
Applies to:Dream Daemon
Status: Open
Issue hasn't been assigned a status value.
Currently, calling browse_rsc('icons.dmi', "icons.dmi") will export not the entire file icons.dmi to the cache as with (I believe) every other resource type, but only the first frame of the default icon state. The DM reference does not mention this detail and I could not find anything else on the forums mentioning it.

It would be extremely useful to either change this behavior or add a flag or something else to allow exporting the .dmi file wholesale.

The specific usecase is using HTML+CSS to crop a region from the spritesheet rather than icon operations, which should reduce the total number of icon operations, fcopy_rsc calls, and browse_rsc calls and therefore reduce both essential and accidental inefficiency in the process of showing a lot of icons in browser windows.
Apr 28 2018, 12:13 am
on that note, since byond already uses a http server on the client, a way to access a certain icon state, direction, and frame/moving frame via some url method so we don't have to do css and sprite sheets (like <img src="file.dmi?icon_state=blah....") if no params are sent in the request serve the dmi file up wholesale
Apr 28 2018, 12:29 am
Actually that would be even better to not have to browse_rsc at all. Even without icon_state support, serving up the .dmi file wholesale would be AMAZING.
Apr 28 2018, 12:30 am
if you don't have browse_rsc then somebody could use such a system to extract resources out of the client's rsc cache.

While ss13 doesn't have to care about that, it is a feature byond still has to cater to because not every game uses cc-by-sa assets.

The idea here is a way for the server to mark a dmi as readable by the client's webserver for browser asset reasons. Browse_rsc'ing it to the client is the best way to do that
Apr 28 2018, 12:36 am
The attacker in this scenario would still have to know the asset names beforehand. It'd be very unfortunate if obscurity measures were really in the way of performant asset loading, but I understand it's a part of keeping asset-rip riff-raff out.

Maybe a flag to enable everything to be accessed? More performance for less security, with default-secure? But I don't want to impose too much of an implementation burden. Waiting for a browse_rsc or multiple calls to go through can still sometimes be a bottleneck.
Apr 28 2018, 12:44 am
honestly, the biggest thing asset loading needs is to just have byond pipeline/slipstream groups of assets in the browse queue.

Most the performance issue is the fact that byond does a bit of a back and forth with respect to sending assets, and it waits for one of these RTTs to finish before doing the next. This is done to preserve load order, but the fact is only the browse() calls matter, it doesn't matter what order browse_rsc() calls get to the client, as long as they happen before the next browse() call.

If byond could just pipeline the asset sends or even group up the initial message to the client asking if they need the file so that it sent multiple requests at once and got a list back of files to send it would be a massive improvement to the delay sending assets add to the following browse() call. There is no harm in doing this if byond just adds a flag to the browse queue for if a file is a ##action=browse/browse_text/browse_file call vs a ##action=browse_rsc call.
Apr 27 2020, 8:22 pm
Bumping this. It would be pretty nice to have access to DMI files, or the whole RSC file directly via the browser, because the amount of hacks we employ around this limitation is immeasurable.
May 1 2020, 10:21 am
Are you sure browse_rsc() doesn't export the whole DMI? I ask because I believe I've used it in the past to do just that, but I have never bothered to include the naming argument.

Just browse_rsc('icon.dmi')

May 1 2020, 6:03 pm
In response to Major Falcon
Major Falcon wrote:
Are you sure browse_rsc() doesn't export the whole DMI? I ask because I believe I've used it in the past to do just that, but I have never bothered to include the naming argument.

Just browse_rsc('icon.dmi')

Yes, I just checked. It exports only the first frame.
May 2 2020, 9:11 am
You're right!
Must be a security feature. I am sure I pulled icons right out of the byond.rsc file using browse_rsc() and filenames I grabbed from text strings inside. I don't think it could have been another proc.

I heard encryption was added at some point. That would explain the security measures you folks were talking about. BYOND begged for this back in the day because RSC extractors were being passed around.

MrStonedOne is onto something nice but in the meantime couldn't you make copies as PNGs and browse_rsc() those?
May 2 2020, 9:34 am
In response to Major Falcon
Major Falcon wrote:
couldn't you make copies as PNGs and browse_rsc() those?

It works, but it's expensive enough that its use has to be limited. The server has to actively strip the .dmi metadata then re-transmit the stripped PNGs to the client (even though it already has the non-stripped versions).
Dec 18 2020, 9:38 pm
bumping this for 514, since this proc is already being touched
Mar 27 2022, 2:45 pm (Edited on Mar 27 2022, 6:25 pm)
Bump

So currently you can upload an entire icon_file with all of its icon_states into the browser via 
var list/imgCACHE = new

proc/CacheIcon(icon_file)
    for(var/state in icon_states(icon_file))
        if(!(icon_file in imgCACHE))
            imgCACHE[icon_file] = list()
        imgCACHE[icon_file][state] = icon(icon_file,state)

mob/proc/PreloadBrowserCache()
    var i,j
    for(i in imgCACHE)
        for(j in imgCACHE[i]){src << browse_rsc(imgCACHE[i][j],"[j].png")}

world/New()
    CacheIcon('items.dmi')

mob/Login()
    PreloadBrowserCache()


The issue now is that the amount of icon_states inside of the icon_file. Via the above I have a icon_file with 571 icon_states within it, it takes around 8 seconds with a frozen screen to load.(The size is 16x16)

If this could be improved upon it would be welcomed, as of now the window being frozen is unavoidable I believe. Haven't tested it in a multiplayer setting so idk if the entire server freezes when someone connects butt.