Repository navigation
dump(): Non-conforming with JSON-spec escape of strings? #4088
Description
Description
This library seems to not be in compliance/non-conformant with the JSON spec, though I'm not 100% on that. None the less, it shows serious differences from almost any other library or "first-class citizen" of languages with built-in support of JSON (as is the case in Javascript, or Python for instance).
Reproduction steps
json obj;
obj["foo"] = 10;
obj["str"] = "\nhello world!\n";
obj["baz"] = 30;
const auto dump = obj.dump();
In javascript, if we create the following object and serialize it using JSON.stringify(obj) we get the following:
let obj = { a: 10, foo: 20, str: `
hello world!
`, baz: 30};
JSON.stringify(obj);
// we can also do
let obj = { a: 10, foo: 20, str: "\nhello world!\n", baz: 30};Python example:
obj = {}
obj["a"] = 10
obj["str"] = "\nhello world!\n"
import json
json.dumps(obj)Expected vs. actual results
The outputs from the library is
// output
// "{ "foo": 10, "str": "\nhello world!\n", "baz": 30 }"Where as Javascript produces
// output
// '{"a":10,"foo":20,"str":"\\nhello world!\\n","baz":30}'and Python produces
'{"a": 10, "str": "\\nhello world!\\n"}'Notice the correct escaping of the string. This library seems to only actually do the actual escape and not the "doubly" escape, which as far as I am aware, is not JSON-compliant?
I came across this particular bug (if it is a bug - it sort of feels like a bug at least?), because my application communicates with javascript over the "wire" (whether that is a socket, pipe, stdio, doesn't matter) - and fails, because nlohmann doesn't produce proper JSON string output that Javascript's JSON-functionality can parse.
I've used many languages before, using JSON as a communication protocol, and they all exhibit the same behavior, so I was under the assumption that this library would also be able to function in that manner. Perhaps there is some other feature in this library that produces the behavior seen above? (Python's json.dumps() for instance ,works in this way).
JSON spec says something like:
A string is a sequence of Unicode code points wrapped with quotation marks (U+0022). All characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark (U+0022), reverse solidus (U+005C), and the control characters U+0000 to U+001F. There are two-character escape sequence representations of some characters.
Here's a stack overflow post discussing this issue, as well:
How do I handle newlines in JSON?
Minimal code example
No response
Error messages
No response
Compiler and operating system
clang++-15
Library version
v3.11.2
Validation
- The bug also occurs if the latest version from the
developbranch is used. - I can successfully compile and run the unit tests.
Activity
[-]Non-conforming escape of strings?[/-][+]Non-conforming with JSON-spec escape of strings?[/+][-]Non-conforming with JSON-spec escape of strings?[/-][+]dump(): Non-conforming with JSON-spec escape of strings?[/+]nlohmann commentedon Jul 28, 2023
I am confident the escaping of this library is correct. There are thousands of tests and compliance checks, and also many projects where the library is used together with other libraries. Please check again if there is some error in your setup.
theIDinside commentedon Jul 28, 2023
I'll check again. Could you please inform me where the tests are (if there are any) where
.dumpis verified to produce escaped escape-sequences (as per JSON spec and as Javascript & Python examples above), perhaps that will guide me in what I'm doing wrong.Edit:
It could potentially be that another library is simply eating the
\\when doing it's formatting (thus producing the incorrect\instead. So if that test exists, it would be greatly helpful as this would conclude this issue entirely).theIDinside commentedon Jul 28, 2023
Yeah, I'm closing this. Another library was indeed eating that extra escape character.