ID:2215619
 
Resolved
json_encode() produced code that was valid for JavaScript but not for JSON in the case of certain special characters.
BYOND Version:511.1374
Operating System:Windows 7 Home Premium 64-bit
Web Browser:Firefox 51.0
Applies to:Dream Daemon
Status: Resolved (511.1375)

This issue has been resolved.
Descriptive Problem Summary:
Text containing unicode characters are encoded as \x(hex) with json_encode(). This is invalid JSON and won't be parsed by JS.

Code Snippet (if applicable) to Reproduce Problem:
/client/proc/sendUni()
var/list/test = list("foo" = "Toupé")
src << output(json_encode(test), "someWindow:someFunction")


And in the browser...
function someFunction(test) {
try {
test = JSON.parse(test);
} catch (e) {
alert(e);
}
}


Expected Results:
Success parsing the JSON, clearly

Actual Results:
JS throws a JSON parse exception of "Error: JSON parse error for: (json)".
The JSON received by the JS function in the example above is: "{"foo": "Toup\xe9"}".

Workarounds:
Sanitize the JSON before parsing:
    test = test.replace(/\\x[\dA-F]{2}/ig, function(match) {
return String.fromCharCode(parseInt(match.replace(/\\x/g, ''), 16));
});


Hrm. I wasn't aware that \x would be invalid for JSON, considering it's valid in JavaScript. I can make a change for that.
Both Chromium (56.0.2924.87, ie the same version listed in your report) and NodeJS (6.9.2) parse the following JSON object as expected:
{"foo": "Toup\xe9"}


Neither, however, parse the following as expected, though in a different way (both give the string as "Toupà ©" minus the space; space added because the forums tries to be smart and make it an é):
{"foo": "Toup\xc3\xa9"}


I suspect this is a character encoding problem, not a JSON format problem; "\xe9" is invalid UTF-8. Perhaps your system is encoding as Windows-1252/ISO 8859-1 then decoding as UTF-8 (and throwing errors since "\xe9" is invalid UTF-8), and mine is encoding as UTF-8 and decoding as 1252/8859-1?
Are you aware that byond uses IE. Like, at all. I guess I should have removed the automatic browser version from my post but dang dude.
So it's not parsed by Internet Explorer's JS implementation.

That's not the way the issue read to me; yes, BYOND uses IE, but that's not the only place you can send json_encode()'s results.

Does it parse the following as expected?
{"foo": "Toup\u00e9"}
Presumably yes, because that's valid according to the spec. That certainly isn't what json_encode spits out though.

A note: when in doubt, shove your crap in http://jsonlint.com/
Just gonna leave this here:



Notably, no \x
In response to GinjaNinja32
It "smartly converts it to an é" because the forums use utf8 encoding.
It works in browsers(and ditto to v8) because its a compliant javascript string, not a compliant JSON string.
Lummox JR resolved issue with message:
json_encode() produced code that was valid for JavaScript but not for JSON in the case of certain special characters.