ID:2853190
 
Resolved
json_decode() was allocating way too much space for strings, which was more apparent with large files.
BYOND Version:515.1599
Operating System:Windows 11 Pro 64-Bit
Web Browser:Chrome 109.0.0.0
Applies to:Dream Daemon
Status: Resolved (515.1600)

This issue has been resolved.
Descriptive Problem Summary:

So I was testing the new CLI DD and noticed we now have an issue with JSON decoding, specifically it can no longer work on large datasets (I havnt tested small but it definitely breaks on 2MB JSON files)

Occurs in CLI DD: https://i.imgur.com/ryQmxJD.png
Occurs in windowed DD: https://i.imgur.com/R92GgaI.png

Numbered Steps to Reproduce Problem:

1. Decode massive JSON
2. Observe

Code Snippet (if applicable) to Reproduce Problem:
    var/json_data = file2text("./large_json.json") // Profiler data from paradise. Its a big json.
var/json_object = json_decode(json_data)


The JSON in question can be found at https://mocha.affectedarc07.co.uk/large_json.json

Expected Results:

It to decode

Actual Results:

It did not decode

Does the problem occur:
Every time? Or how often? Every time
In other games? No idea, probably
In other user accounts? No idea, probably
On other computers? No idea, probably

When does the problem NOT occur?

Any version prior to this

Did the problem NOT occur in any earlier versions? If so, what was the last version that worked? (Visit http://www.byond.com/download/build to download old versions for testing.)

Worked fine in any version prior to this

Workarounds:

Dont try decode 2 megabytes of JSON
You're saying this only happens in 1599, not in 1598?

I'm super confused, because no changes were made to the JSON decoding code at all in this release.

[edit]
Okay, I discovered this is running into an out-of-memory condition. I find that extremely strange since I can't see any reason even this large file would result a situation like that. I'm researching to find out why the memory is going way up. If you can narrow down for sure that this only happens in 1599 or if it's happening in earlier versions too, that'd be helpful.

[edit 2]
I see the issue and this absolutely can't be new to 1599. I know the code responsible was touched in the 515 period but it definitely wasn't touched in the last few weeks.
Lummox JR resolved issue with message:
json_decode() was allocating way too much space for strings, which was more apparent with large files.