Skip to content

Add dump() and comparisons to json_view - #5624

Open
nlohmann wants to merge 22 commits into
json-view/11-view-accessfrom
json-view/13-view-dump
Open

nlohmann wants to merge 22 commits into
json-view/11-view-accessfrom
json-view/13-view-dump

Conversation

@nlohmann

@nlohmann nlohmann commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Adds dump(), operator<<, operator== and operator!= to basic_json_view, and reads floats from the parser's digit layout; part of the zero-copy view stack for #5295, on top of #5622.

Summary

  • basic_json_view::dump(indent, indent_char, ensure_ascii, number_format) writes a value the way ordered_json::parse(text).dump() writes it, for the same arguments:
    • members in document order; all of them, should a key occur more than once;
    • strings escaped by the same rules, using the library's scanning kernels;
    • floats written with the library's conversion (detail::to_chars), so the output equals basic_json's byte for byte;
    • integers copied from the source, where they are already canonical, except -0, which parse() reads as 0.
    • There is no error_handler argument: the view only holds valid UTF-8.
    • With number_format::source, numbers are copied exactly as they appear in the source, e.g. 1.50, 1E2, -0, or all digits of a long integer. basic_json cannot provide this.
    • operator<< takes the indentation from the stream width, as for basic_json.
    • The writer (detail/view/serializer.hpp) writes through a raw pointer into a string. The walk is iterative, so the nesting depth is limited by memory only.
    • The output buffer is sized from the extent of the value itself (not from the rest of the document), and it is not shrunk afterwards.
  • operator== and operator!= compare two views, or a view and a basic_json value in either order. They answer whether the values parse() would produce are equal by basic_json's operator==:
    • numbers compare by value across their types (1 == 1.0);
    • objects compare by their members, with duplicate keys resolved as parse() resolves them: the last value, at the position of the first key. (Lookups such as operator[] still return the first member; this is documented in both places.)
    • member order matters where the object type keeps one (ordered_json) and not otherwise, exactly as basic_json compares;
    • discarded views compare as discarded basic_json values do, following JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
    • Nothing is materialized except single scalars, and the walk is iterative. There is no operator<.
  • While parsing, the view records where the integer digits, the fraction digits and the exponent of a float token are. For float and double tokens with at most 19 digits, the value is now read from that layout instead of rescanning the token:
    • the digits are read eight at a time;
    • the result is rounded by the library's conversion core, decimal_to_float() from Speed up the lexer: own float parser, string scan, and \u table #5738: Clinger's fast path where both operands are exact, and Eisel-Lemire otherwise, which needs no fallback for up to 19 digits.
    • Both round correctly, so the values are those of parse(). Longer tokens, and other float types, keep the library's conversion of the whole token.
    • get<double>(), materialize(), dump() and the comparisons all use it.
  • Docs: new pages for dump, number_format, operator==, operator!= and operator<<; the view article has a "Writing a view back" section; the pages of basic_json::dump and basic_json::operator== link to the view's counterparts. The ViewDump benchmark joins tests/benchmarks.

Performance

Measured on Apple M1 Max (Apple clang -O2) and an x86-64 KVM VPS (GCC 13 / Clang 18 -O2, pinned to one core, about ±5-10% noise), µs, best of 3 interleaved rounds of 15 runs; files from nativejson-benchmark.

Traversing canada (every number converted), previous PR → this PR: Apple M1 2074 → 1501 (-28%), x86-64 GCC 4296 → 3505 (-18%), Clang 4331 → 3348 (-23%).

dump() at this PR (a faster writer comes later in the stack):

Apple M1 x86-64 GCC x86-64 Clang
canada 5319 8180 8063
citm_catalog 379 835 623
twitter 238 454 403

The core library (json::parse / dump) is unaffected: the code is view-internal (detail/view/serializer.hpp, detail/view/compare.hpp, additions to detail/view/number.hpp, small changes to materialize.hpp, value.hpp and lookup.hpp, and new members of basic_json_view).

Tests

The tests of dump() and the comparisons are in a new file, tests/src/unit-json_view_dump.cpp (with helpers shared through tests/src/json_view_test_helpers.hpp), so that no test object exceeds the 32,767 sections that the MinGW linker of the Windows clang job accepts.

  • dump(): 2,000 generated documents compared with ordered_json::dump() for indentations -1/0/2, space and tab, and ensure_ascii; strings with every kind of escape; numbers (5,000 random doubles, and float as number_float_t); duplicate keys, 100,000 levels of nesting, streams and discarded views; the buffer of a small value stays small, and output that outgrows the estimate is handled.
  • Comparisons: pairs of generated documents, also written differently (sorted keys, canonical numbers), give the same result as basic_json, for json and for ordered_json; numbers, duplicate keys, member order, discarded values and 100,000 levels of nesting.
  • Float layout: tokens around the limits (19 and 20 digits, 2^53, 10^22, underflow, and those of float) join the bit-for-bit comparison with parse().

Public API

No breaking changes. The view API is new in this stack, so nothing here changes a released interface. New members of basic_json_view: dump, operator<<, operator==, operator!= and number_format. Reading floats from the digit layout changes no observable behavior, only how the value is computed.

User-visible decisions in this PR:

  • dump() writes members in document order and writes every occurrence of a repeated key, so for a json_view it can differ from materialize().dump().
  • operator== resolves duplicate keys as parse() does (last value), while lookups find the first member.
  • There is no operator< and no error_handler argument for dump().

Written by Claude Code.

🤖 Generated with Claude Code

@nlohmann
nlohmann added this pull request to stack #5636 September 29, 2026 14:20
@nlohmann nlohmann changed the title json view/13 view dump Add dump() to json_view Sep 29, 2026
@nlohmann

Copy link
Copy Markdown
Owner Author

Besides merging the lower CI fixes (see #5617 to #5623), commit 5484f2e reformats docs/mkdocs/docs/examples/basic_json_view__dump.cpp with the pinned astyle 3.4.13. The "check" job runs astyle over the examples once it gets past the amalgamation step (see #5621).

— posted by Claude Code on behalf of @nlohmann

@nlohmann
nlohmann removed this pull request from stack #5636 September 30, 2026 13:19
@nlohmann
nlohmann force-pushed the json-view/13-view-dump branch from 3e2b810 to 7c179b3 Compare September 30, 2026 13:20
@nlohmann
nlohmann added this pull request to stack #5739 September 30, 2026 13:21
@nlohmann
nlohmann force-pushed the json-view/13-view-dump branch from 7c179b3 to bd22b6e Compare September 30, 2026 15:34
@nlohmann
nlohmann force-pushed the json-view/13-view-dump branch 2 times, most recently from d1e2563 to 48307dc Compare September 30, 2026 18:06
@nlohmann nlohmann added the review needed It would be great if someone could review the proposed changes. label Sep 30, 2026
@nlohmann
nlohmann marked this pull request as ready for review September 30, 2026 18:17
@nlohmann
nlohmann force-pushed the json-view/13-view-dump branch from 48307dc to bc56ac6 Compare September 30, 2026 19:15
Comment thread include/nlohmann/detail/view/serializer.hpp Dismissed
Comment thread single_include/nlohmann/json_view.hpp Dismissed
@nlohmann
nlohmann removed this pull request from stack #5739 October 6, 2026 09:26
@nlohmann
nlohmann changed the base branch from json-view/12-view-values to json-view/11-view-access October 6, 2026 09:26
@nlohmann
nlohmann force-pushed the json-view/13-view-dump branch from dfa17bd to d26cbef Compare October 6, 2026 09:28
@nlohmann nlohmann changed the title Add dump() to json_view Add dump() and comparisons to json_view Oct 6, 2026
@nlohmann
nlohmann added this pull request to stack #5768 October 6, 2026 09:29
Add basic_json_view::dump() and the comparison operators, and read
floats from the parser's digit layout instead of rescanning the
token.

dump(indent, indent_char, ensure_ascii, number_format) writes a
value the way ordered_json::parse(text).dump() writes it for the
same arguments: members in document order, all of them should a
key occur more than once; strings escaped by the same rules, using
the library's scanning kernels; floats written with the library's
to_chars conversion, so the output equals basic_json's byte for
byte; integers copied from the source, where they are already
canonical, except -0, which parse() reads as 0. There is no
error_handler argument, because the view only holds valid UTF-8.
number_format::source copies numbers exactly as they appear in the
source (e.g. "1.50", "1E2", "-0"), which basic_json cannot provide.
operator<< takes the indentation from the stream width, as for
basic_json. The writer walks iteratively, so nesting depth is
limited by memory only.

operator== and operator!= compare two views, or a view and a
basic_json value in either order, by the rules basic_json's
operator== uses: numbers compare by value across their types,
objects compare by their members with duplicate keys resolved as
parse() resolves them, member order matters only where the object
type keeps one, and discarded views compare as discarded basic_json
values do, including under JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
Nothing is materialized except single scalars.

While parsing, the view now records where the integer digits, the
fraction digits, and the exponent of a float token are, so floats
and doubles with at most 19 digits are read from that layout with
the library's decimal_to_float() instead of rescanning the token.
Both round correctly, so the values are those of parse(). get<double>(),
materialize(), dump(), and the comparisons all use it.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@nlohmann
nlohmann force-pushed the json-view/13-view-dump branch from d26cbef to da1ca7f Compare October 7, 2026 14:44
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated and/or formatted correctly, or BUILD.bazel is out of date.

📎 A ready-to-apply patch is attached to the failed workflow run as the amalgamation-patch artifact. Download it, then apply it locally from the repository root with:

git apply amalgamation.patch

This does not require installing astyle yourself.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The source extent of a value was read from the next node, falling back to the rest of the document when that node held a decoded string. dump() of a small value could thus allocate a buffer as large as the document. Skip a few such nodes, cap the fallback estimate, and shrink a buffer that is much larger than its output.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
root() of a temporary document is deleted: its views would dangle.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
finish() copied the whole output when the buffer was more than twice as
large as the result (citm dump +11%). The tighter source_extent()
estimate already keeps the buffer of a small value small.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
dump() writes every member and == resolves duplicate keys as parse()
does, whereas lookups find the first member of a duplicate key.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
{
case 0:
std::snprintf(buf.data(), buf.size(), "%.17g", d); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>(std::snprintf(buf.data(), buf.size(), "%.17g", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
break;
case 1:
std::snprintf(buf.data(), buf.size(), "%.15g", d); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>(std::snprintf(buf.data(), buf.size(), "%.15g", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
break;
case 2:
std::snprintf(buf.data(), buf.size(), "%.3e", d); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>(std::snprintf(buf.data(), buf.size(), "%.3e", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
break;
case 3:
std::snprintf(buf.data(), buf.size(), "%.25g", d); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>(std::snprintf(buf.data(), buf.size(), "%.25g", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
break;
default:
std::snprintf(buf.data(), buf.size(), "%.0f", d); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
static_cast<void>(std::snprintf(buf.data(), buf.size(), "%.0f", d)); // NOLINT(cppcoreguidelines-pro-type-vararg,hicpp-vararg)
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view.cpp reaches that limit as the stack
grows, so its "json_view dump" and "json_view comparison" test cases move
into unit-json_view_dump.cpp. The test generator and has_duplicate_keys()
that both files use move into json_view_test_helpers.hpp.

The new file mentions JSON_HAS_CPP_17, so it is built for C++17 like the file
it was split from.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation L review needed It would be great if someone could review the proposed changes. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants