Fix stack overflow and exponential runtime when comparing nested values by nlohmann · Pull Request #5390 · nlohmann/json

nlohmann · 2026-08-20T19:30:24Z

What & why

Comparing two values compares their containers, which compare their elements, which brings the comparison back once per nesting level. Two values nested deeply enough exhaust the call stack and terminate the process — the same bug as #5387, in the last operation of basic_json that still had it. (The copy constructor is #5389, dump() is #5285, the destructor was #1436.)

While fixing that, a second and worse problem turned up.

An ordered comparison took exponentially long before C++20

std::vector::operator< is a lexicographical comparison: for each element it asks whether the element is less than its counterpart, and then whether the counterpart is less than it. Both questions recurse into everything below that element, so every level doubles the work.

Measured on develop, comparing two equal values nested n levels deep with <:

depth	`develop`
20	5.7 ms
26	237 ms
30	3778 ms
40	≈ 1 hour (extrapolated)

Nothing about such a value is pathological — 30 levels of nesting is ordinary, and reaching this needs no crafted input, just a < b on two parsed documents. C++20 is unaffected: std::lexicographical_compare_three_way asks once, and operator< is derived from <=> there.

The change

A value nested too deeply to descend into is compared on an explicit stack instead, in a single pass that yields less, equal, greater or unordered at once.

Equality and the three-way comparison descend as they always did for the first 128 levels — nothing measurable changes for them — and finish iteratively below that.
An ordered comparison never descends, which is what takes the exponent out of it.

Equality needs no ordering, so it no longer asks for any: the iterative path is templated on whether the values are being ordered, so a key or string type that can only be compared for equality still compiles.

Reproducing the results exactly

Two subtleties had to be reproduced, both found by differential testing rather than by reading:

std::lexicographical_compare steps over a pair it cannot order — a NaN, say — and carries on with the next element, where std::lexicographical_compare_three_way stops at it. The iterative pass does whichever the caller needs.
An object compares its keys with < where its entries are ordered, but with == where they are only checked for equality — not with the object's own comparator, which for nlohmann::ordered_map is std::equal_to and would report every equal key as "less".

Measured

Medians of 7 interleaved runs, clang -O3, C++11:

workload
two equal values nested 30 deep, `<`	3778 ms → 0.002 ms
ordering flat objects	−33.6%
equality, every shape	unchanged
ordering flat arrays of numbers	+27.3%

The last row is the one shape that pays for the single pass, and it buys the row above it.

Public API impact

No breaking changes. No public signature, type, or exception changes; the comparison helpers are private members. Results are unchanged for every pair of values — see below. Nothing is rejected that was accepted before.

Verification

Differential vs. develop: 68,121 comparisons — every pair drawn from a corpus covering NaN, discarded values, mixed number types, binary values, empty containers, both object types and every operator (==, !=, <, <=, >, >=) — identical in C++11, C++17 and C++20, with and without thread_local storage, and with JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON.
unit-comparison.cpp (4097 assertions) green in all three standards and in legacy mode; full suite green in every configuration; ASan + UBSan clean; warning-clean under the CI clang flag set.
New tests compare values nested 100,000 deep, for arrays and objects, with every operator.

JSON_NO_THREAD_LOCAL earned its keep here: it forces the iterative path for every value, and it is what caught the ordered_map key-comparator bug above — a divergence that would otherwise only appear below depth 128 and be practically untestable.

Note for reviewers

This is stacked on #5389, which introduces the depth counter this reuses. Retarget to develop once that lands.

binary_writer (to_cbor/to_msgpack/to_ubjson/to_bson) is not in this PR because the four formats need four separate iterative writers, and BSON additionally computes each document's length up front through a second recursive walk. Worth its own change.

What still recurses after this PR

what	where	status
copy constructor (and everything built on it)	`json.hpp`	fixed in #5389
`dump()`	`serializer.hpp`	fixed in #5285
`operator==`, `operator<`, `operator<=>`	`json.hpp:3669`	this PR
`to_cbor` / `to_msgpack` / `to_ubjson` / `to_bson`	`binary_writer.hpp`	open
`basic_json::diff`	`json.hpp:5089`	open
`basic_json::merge_patch`	`json.hpp:5231`	open
`json_pointer::flatten`	`json_pointer.hpp:861`	open

diff, merge_patch and flatten were not previously called out anywhere; they recurse once per nesting level on user-controlled data exactly as copying and comparison did. #5387 should not be closed as fully fixed until they are dealt with too.

The changes are described in detail, both the what and why.
If applicable, an existing issue is referenced. (Stack overflow in copy constructor and dump() on deeply nested json (destructor was fixed in #1436) #5387)
The Code coverage remained at 100%. A test case for every new line of code.
If applicable, the documentation is updated. (the JSON_NO_THREAD_LOCAL page and the macro overview now cover comparison as well as copying, including what the macro costs a comparison)
The source code is amalgamated by running make amalgamate.

Written by Claude Code.

gregmarr · 2026-08-21T04:49:33Z

~~Haven't looked at all of this in detail yet.~~
Done.

nesting_depth_limit() and nesting_depth() stay behind #ifndef JSON_NO_THREAD_LOCAL, since a descent cannot be bounded without a per-thread count. But the guard itself now always exists, becoming a no-op that is never okay() under that macro - the same way the bound is already reached on every call without one. copy_structured() no longer needs to know which case it is in. This is what lets #5390 reuse the guard for comparison, which cannot test JSON_NO_THREAD_LOCAL where the macro-based operators use it: the guard now carries that distinction itself instead of requiring every caller to. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Comparing two values now bounds its descent with a thread_local counter just as copying does, so the JSON_NO_THREAD_LOCAL page, the macro overview and the ci_test_no_thread_local target cover both rather than copying alone. Also record what switching the macro on costs a comparison: on the benchmark documents, comparing two equal values takes 10% to 90% longer. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

MSVC reports the test of a constant as C4127 ("conditional expression is constant"), which the Windows builds treat as an error: may_descend is false for operator<, so the operand short-circuits the whole condition. Passing it to compare_descent_exhausted() puts the test where the value is an ordinary parameter, and leaves the call sites with no condition of their own. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

The macro page describes what the library defines JSON_NO_THREAD_LOCAL for by itself in terms of copying alone; comparing falls back the same way. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

clang-tidy reports the mixed * and + as readability-math-missing- parentheses, as it does for the identical line in the copy test. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Comparing kept a thread_local count, a limit and a guard of its own beside the ones copying already had, all three the same thing under a different name. They are gone; the shared count, limit and guard do the work. The guard grows a second constructor here, because the comparison operators are written as a macro and a macro cannot use the preprocessor: it cannot look the count up behind an #ifdef the way copy_structured does, so the guard looks it up for it. nesting_depth_exhausted() arrives for the same reason - whether an operator descends at all is a constant at every call site, and testing it there is what MSVC reports as C4127. Also say in compare_leaves what happens to a pair that is an array on one side and an object on the other, since the answer is not obvious from the code: an operator only descends into two values of the same type, so such a pair is told apart by its types alone - unequal, and ordered the way the types are - exactly as it is above the bound. And record what the explicit stack costs: the comparison operators are noexcept and the container comparison this replaces allocated nothing, so running out of memory here ends the process instead of throwing. It takes a value nested past the bound and an exhausted heap to reach, and the same comparison used to exhaust the call stack, but it is a new way to fail. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

@gregmarr

* Bound the descent of the copy constructor basic_json's copy constructor copied objects and arrays by handing the container to its own copy constructor, which copy-constructs every element and so reaches this constructor again, once per nesting level. A value nested deeply enough exhausted the call stack and terminated the process with a segmentation fault - no exception, nothing the caller could catch. Parsing such a value works, as the parser is iterative, and so does destroying one, as #1436 made destruction iterative. Bound how far the copy descends rather than take the call stack away from it. The first levels are copied exactly as they were - the containers copy their own elements, which is by far the fastest way to fill them - and only once the copy has descended 128 levels is the value below it finished without the call stack, through an explicit worklist. Copying can therefore no longer exhaust the stack, however deeply a value is nested, while a value nested less deeply than the bound - all but a vanishing minority - is copied by the very same code as before and pays only for one counter. That counter lives in thread_local storage, as one shared between threads would be raced. JSON_NO_THREAD_LOCAL switches it off for toolchains without thread_local; copying then goes through the worklist right away, which yields the same values but is measurably slower. The deferred values are completed before the copy they belong to returns, so a value copied while another copy is going on - by a custom base class, say - is unaffected by the copy it is nested in. operator= takes its argument by value, so copy assignment is fixed as well. Copying is as fast as it was, within measurement noise (medians of 9 interleaved runs, clang -O3): -1.3% for an array of strings, +0.0% for a flat object, +0.1% for a flat array of numbers, +0.3% for nested arrays, +0.6% for nested objects and +1.2% for a twitter-like document. Copying a three-key object costs about ten nanoseconds more, the counter. Deferring every level instead, rather than only those below the bound, measured between 3% and 9% slower depending on the shape of the value. This fixes #5387 for the copy constructor. dump() is still recursive. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Test the copy constructor's iterative path in CI The copy constructor descends into 128 levels before it finishes a value without the call stack, so the iterative path is otherwise only reached by the few tests that nest deeper than that. JSON_NO_THREAD_LOCAL switches the descent off, which sends every value down that path. Running the whole test suite that way covers it with every object type, string type, allocator, and base class the suite already exercises. The new ci_test_no_thread_local target does that; the macro had no build coverage at all before. Copying a nested value also has to carry over what the element-wise copy constructor would have copied: the parents that JSON_DIAGNOSTICS relies on, and the positions that JSON_DIAGNOSTIC_POSITIONS reports. Both are now checked on either side of the descent bound, for objects and arrays. Neither was tested before, and dropping either one makes the new tests fail. Also quantify what JSON_NO_THREAD_LOCAL costs a copy instead of calling it "measurably slower". Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the regression tests so that they keep linking Linking test-regression2 fails with "relocation truncated to fit: IMAGE_REL_AMD64_REL32 against `.rdata'" once its object grows past what the MinGW linker copes with, and the copy constructor's helpers push it over: the object grows by 6.3%, from 4,654,128 to 4,944,920 bytes at -O0, and develop links at the smaller of the two. Building the tests optimized shrinks the object enough to link, but the binaries clang 11.0.1 and clang 18.1.8 then produce crash before doctest prints its first line - 39 of 102 tests on clang 18 - so the objects have to become smaller rather than denser. Moving the test cases that follow "regression tests 2" into a file of their own brings that object to 4,687,888 bytes, which is 0.7% above the size that links today rather than 6.3%. Both files still build for C++11, C++17 and C++20, and run the same 9 test cases and 135 assertions as before, now spread over two binaries. New regression tests belong in unit-regression3.cpp from here on, which is what CONTRIBUTING.md now says. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Do not use thread_local storage with Clang targeting MinGW Every test that copies a value segfaults there - 42 of 105 on clang 11.0.1, 39 of 102 on clang 18.1.8 - while the same tests pass with GCC targeting MinGW, with Clang targeting MSVC, and with every other toolchain the library is tested on. The counter that bounds the copy constructor's descent is the library's first use of thread_local, so that job had never exercised it before. JSON_NO_THREAD_LOCAL already covers toolchains without thread_local storage, and copying yields the same values with it, only more slowly. Define it for this one automatically. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Balance the warning suppression the split separated unit-regression2.cpp opens a DOCTEST_CLANG_SUPPRESS_WARNING_PUSH block at the top and closed it at the very bottom, which the split moved into unit-regression3.cpp: one file was left with a push and no pop, the other with a pop and no push, which clang reports as an error. Give each file the pair it needs. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Check both shapes without a C-style array clang-tidy rejects the array the two shapes were iterated over (cppcoreguidelines-avoid-c-arrays). The array only existed because astyle reformats a range-for over a braced initializer list into something unreadable; naming the two cases avoids both. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Split the regression tests far enough to leave room The first split left unit-regression2.cpp 0.7% below the size develop links at, which the comparison change in the follow-up immediately used up: the MinGW linker fails on test-regression2_cpp20 again, naming copy_shallow and to_partial_ordering among the relocations it cannot fit. Move the sections from "issue #2067" on, and the helper types they use, so that the file stops being the one that decides whether the tests can be linked at all. At -O0 and C++20, unit-regression2.cpp is now 2,964,944 bytes against develop's 4,708,248, and 3,070,568 bytes with the follow-up applied - roughly a third smaller either way, rather than a fraction of a percent larger. The 135 assertions are the same ones as before, now spread over three test cases in two files. Also silence the clang-tidy findings the deep-nesting tests draw: the copies they make are what is being tested, and the reserve() computation gets its parentheses. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Move the #4804 alias to the file that uses it The split left the json_4804 alias behind in unit-regression2.cpp while the test case that uses it went to unit-regression3.cpp, which does not build for C++17 and C++20 as a result. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Include <span> where the split moved its only use The #2546 test case guards itself with __has_include(<span>), but the include itself sat in unit-regression2.cpp's preamble and stayed behind, so the section compiled without a declaration wherever the guard passed - which nvhpc reported and libc++ builds do not, as they skip the section altogether. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the descent bookkeeping in one place Copying carried a depth count, a depth limit and a guard of its own, and the comparison in the follow-up added a second set beside them. Neither operation needs its own: they are never nested inside one another by the library - copying a value does not compare one, and comparing two values does not copy them - and where user code nests them anyway, sharing the count only ends a descent sooner than it had to. So there is now one nesting_depth(), one nesting_depth_limit() and one nesting_depth_guard, which the follow-up uses instead of adding its own. Inverting the test in copy_structured leaves the too-deep case and the no-thread-local case as the same code. The guard takes the count rather than looking it up, because the caller has looked it up already to test it against the limit, and reaching thread-local storage twice on the path that is taken almost every time is worth avoiding. The switch that copies the value of anything that is not an object or an array was written twice - once in the copy constructor, once in copy_shallow - so that adding a value_t meant editing both, and missing one would have been silent. It is copy_leaf_value now, and inlined: both callers have already sorted the containers out, and folding that test into the switch is what keeps a value made mostly of numbers copying as fast as it did. Copying canada.json, citm_catalog.json and twitter.json is within 0.6% of what it was before, measured as a paired ratio over 18 interleaved rounds against a run-to-run spread of 0.3%. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Check that an abandoned copy can still be destroyed Copying a value without the call stack builds the copy from the top down, and every value whose own copy has not been made yet stays a null value until it is. That is what lets a copy be abandoned half-built: the destructor finds nothing but complete values and null ones. Nothing tested it. Failing an allocation part-way through a copy of a deeply nested value does, with the allocator the file already has for exactly this kind of test. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Name the test's locals so Flawfinder stops matching them The code scanning job reports CWE-362 - "check when opening files" - for a test that opens no files: Flawfinder matched a local variable called open. Rename it and its partner. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep the descent guard's bookkeeping self-contained nesting_depth_limit() and nesting_depth_guard were only used inside the JSON_NO_THREAD_LOCAL-guarded branch of copy_structured(), but were defined unconditionally. Move them inside the #ifndef, and have the guard look up the depth and test it against the limit itself (via okay()) instead of making the caller do it - the caller no longer needs to touch nesting_depth() at all. Also shrink the thread-local counter to std::uint8_t, matching what its own doc comment already argued. Addresses gregmarr's review comments on #5389. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make nesting_depth_guard usable regardless of JSON_NO_THREAD_LOCAL nesting_depth_limit() and nesting_depth() stay behind #ifndef JSON_NO_THREAD_LOCAL, since a descent cannot be bounded without a per-thread count. But the guard itself now always exists, becoming a no-op that is never okay() under that macro - the same way the bound is already reached on every call without one. copy_structured() no longer needs to know which case it is in. This is what lets #5390 reuse the guard for comparison, which cannot test JSON_NO_THREAD_LOCAL where the macro-based operators use it: the guard now carries that distinction itself instead of requiring every caller to. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Silence VS2015's C4503 for the custom-base-class test The deep-copy support added for #5387 lengthened the mangled name of std::allocator_traits<...>::construct for the test's map type past VS2015's limit, which /WX turns into a build failure even though the name is only used for (now-truncated) debug info. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove dead unused-parameter casts from copy_metadata() @gregmarr asked whether the static_cast<void> pair in the JSON_DIAGNOSTIC_POSITIONS-off branch was needed for an empty json_base_class_t. It isn't: src and dst are already referenced unconditionally by the base-class copy above, so no -Wunused-parameter warning fires either way (checked with -Wall -Wextra -Wunused-parameter, JSON_DIAGNOSTIC_POSITIONS 0 and 1). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix CI: build custom array types without a fill constructor, re-amalgamate copy_array_level() built the destination array with the fill constructor array_t(count, value), which is not part of the array container interface the library otherwise assumes (e.g. custom ArrayTypes that only provide a default and an iterator-pair constructor, as covered by unit-custom-array-type.cpp). Default- construct the array and resize() it instead, matching how the rest of the codebase already grows array_t. Also re-run the amalgamation, which had fallen out of sync with include/nlohmann/json.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

diff() descended into both values once per nesting level, and compared them with operator== on every level on the way, which recurses as well. Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted the call stack and terminated the process, although parse() accepts them without complaint. On such a chain the per-level comparisons and path strings also made diff() quadratic in time and memory. Both the recursion and operator== only descend as far as the source is nested. So diff() first checks, recursing at most diff_depth_limit() (128) levels, whether the source is nested more deeply than that. If not - all but a vanishing minority of values - the recursive algorithm diffs it exactly as before, now as diff_recursively(). Otherwise diff_iteratively() walks the two values on an explicit stack, emitting the same operations in the same order. It does not compare arrays and objects with operator== up front (equal ones yield no operations anyway), keeps the path in one buffer instead of a new string per level, and hands every subtree that is not nested too deeply back to diff_recursively(), so equal parts are still skipped quickly. The check costs one pass over the source. On a 3,000-object document that is about 30% of diffing two equal values (which is just an operator== call), about 10% of diffing values that differ in a few places, and noise when arrays change length. Once operator== no longer recurses (#5390), the check can go. Tests check that the patch reproduces the target at every depth up to 300, for json and ordered_json, including reordered members. They also check the exact operation for a difference deep inside, and diff values nested 100,000 levels deep. Fixes #5393 for diff(). Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Resolve the conflict in json.hpp by keeping both the comparison helpers and set_parents_after_object_erase() from #5552, and regenerate the amalgamation. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

github-actions · 2026-09-25T12:51:22Z

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated and/or formatted correctly, or BUILD.bazel is out of date.

📎 A ready-to-apply patch is attached to the failed workflow run as the amalgamation-patch artifact. Download it, then apply it locally from the repository root with:

git apply amalgamation.patch

This does not require installing astyle yourself.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

…endency policies (#5580) Answer the OpenSSF Best Practices criteria that asked for policies the project follows but had not written down: - SECURITY.md: a first response within 14 days, publishing an advisory with credit once a fix is released, and that only the latest release receives security fixes. - Governance: who has access to the project's resources, how write or admin access is granted, and how CI secrets are stored and rotated. - Quality assurance: how dependencies of the build, test, and documentation tooling are pinned, scanned, and kept free of known vulnerabilities. Also update the assurance case, since comparison no longer recurses per nesting level (#5390), and point the best practices badge and links to bestpractices.dev under the program's current name. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Now that operator== no longer recurses (#5390), diff() can keep its per-level equality shortcut all the way down. It diffs recursively for the first detail::recursion_depth_limit() levels, as merge_patch() does, and hands anything deeper to diff_iteratively(). The nesting_exceeds() scan, which cost about 30% on equal documents, is gone, and diff() is on par with develop again. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

@gregmarr

The regression came from #5390, which is not in any release. Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

* Diff deeply nested values without recursing per nesting level diff() descended into both values once per nesting level, and compared them with operator== on every level on the way, which recurses as well. Values nested deeply enough - 25,000 levels on an 8 MiB stack - exhausted the call stack and terminated the process, although parse() accepts them without complaint. On such a chain the per-level comparisons and path strings also made diff() quadratic in time and memory. Both the recursion and operator== only descend as far as the source is nested. So diff() first checks, recursing at most diff_depth_limit() (128) levels, whether the source is nested more deeply than that. If not - all but a vanishing minority of values - the recursive algorithm diffs it exactly as before, now as diff_recursively(). Otherwise diff_iteratively() walks the two values on an explicit stack, emitting the same operations in the same order. It does not compare arrays and objects with operator== up front (equal ones yield no operations anyway), keeps the path in one buffer instead of a new string per level, and hands every subtree that is not nested too deeply back to diff_recursively(), so equal parts are still skipped quickly. The check costs one pass over the source. On a 3,000-object document that is about 30% of diffing two equal values (which is just an operator== call), about 10% of diffing values that differ in a few places, and noise when arrays change length. Once operator== no longer recurses (#5390), the check can go. Tests check that the patch reproduces the target at every depth up to 300, for json and ordered_json, including reordered members. They also check the exact operation for a difference deep inside, and diff values nested 100,000 levels deep. Fixes #5393 for diff(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Make diff_frame a member struct that declares its special members GCC's -Weffc++ (an error in CI) asks a class with pointer members, a user constructor and a non-trivial destructor to declare its copy constructor and copy assignment; diff_frame's vector and basic_json members make its destructor non-trivial. Declare all five as defaulted, which also satisfies clang-tidy's special-member-functions check. Leave their exception specifications implicit: GCC 4.8 rejects an explicit one that differs from the implicit one, as it does for flatten_task in #5517. The converting constructor cannot throw, and is now declared noexcept for GCC's -Wnoexcept, which flags the emplace_back() under C++26 otherwise. The struct also moves from diff_iteratively() into the class, like dump_frame in the serializer. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Use the shared recursion limit in diff() diff_depth_limit() is gone in favor of detail::recursion_depth_limit(). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Diff fewer nesting depths so the test does not time out under Valgrind Checking every depth up to 300 made test-json_patch exceed the 1500 s ctest timeout in ci_test_valgrind. Check the depths up to 16, those around the recursion limit of 128, and 300 instead. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Mark the diff frame's value-initialized members for clang-tidy The braces are kept for GCC's -Weffc++, as in json_sax.hpp. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Bound diff()'s descent with a depth count instead of scanning the source Now that operator== no longer recurses (#5390), diff() can keep its per-level equality shortcut all the way down. It diffs recursively for the first detail::recursion_depth_limit() levels, as merge_patch() does, and hands anything deeper to diff_iteratively(). The nesting_exceeds() scan, which cost about 30% on equal documents, is gone, and diff() is on par with develop again. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Note that the diff frame reference is invalidated by pop_back() too Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Keep diff()'s recursive levels small and its result elided diff_recursively built every patch operation in place from initializer lists. Unoptimized builds give each of those temporaries its own stack slot, so every level of the bounded descent cost kilobytes of stack (about 6 KB with clang -O0), and the 128 recursive levels overflowed the 1 MB stack of MSVC Debug in the "deeply nested values" test. The operations and the key comparison of two objects are now built by separate functions, which diff_iteratively shares, and both diff functions append to one result instead of returning a patch per level that the caller copies. With clang -O0, diffing values nested 300 levels deep now peaks at about 190 KB of stack instead of 880 KB. Since diff() now owns the only returned value, clang's -Wnrvo no longer reports the returns of diff_recursively, which alternated between the local patch and diff_iteratively's result. Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Copy the diff frame's members instead of holding a reference to it The loop in diff_iteratively held a reference to the top frame, which enter() invalidates when it pushes and the end of the loop invalidates when it pops. Nothing used it afterwards, but a later change could. As in the other iterative walks, the members the loop reads are now copied out as constants and the ones it advances are changed through stack.back(). The frame as a whole is not copied: it holds the common keys and the "add" operations of an object. Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>

github-actions Bot added L tests labels Aug 20, 2026

nlohmann marked this pull request as draft August 20, 2026 19:46

nlohmann force-pushed the claude/issue-5387-comparison-binary branch from e8e9f5f to b1bb90b Compare August 20, 2026 19:55

nlohmann mentioned this pull request Aug 20, 2026

Fix stack overflow when copying a deeply nested value (#5387) #5389

Merged

5 tasks done

nlohmann force-pushed the claude/issue-5387-comparison-binary branch from b1bb90b to 2797ae1 Compare August 20, 2026 20:58

github-actions Bot added documentation CMake labels Aug 20, 2026

nlohmann force-pushed the claude/issue-5387-comparison-binary branch 7 times, most recently from ae3aeb5 to 4c590dd Compare August 21, 2026 01:20

gregmarr reviewed Aug 21, 2026

View reviewed changes

Comment thread include/nlohmann/json.hpp Outdated

Comment thread include/nlohmann/json.hpp Outdated

Comment thread include/nlohmann/json.hpp Outdated

Comment thread include/nlohmann/json.hpp

nlohmann force-pushed the claude/issue-5387-comparison-binary branch from 0d4138c to 835a5e6 Compare August 21, 2026 06:56

This was referenced Aug 21, 2026

Stack overflow in binary format writers (CBOR/MessagePack/UBJSON/BSON) due to unbounded recursion #5392

Open

Stack overflow in diff(), merge_patch(), and flatten() due to unbounded recursion #5393

Closed

nlohmann force-pushed the claude/issue-5387-comparison-binary branch from 835a5e6 to 596e33c Compare August 21, 2026 10:31

gregmarr reviewed Aug 21, 2026

View reviewed changes

Comment thread include/nlohmann/json.hpp

gregmarr reviewed Aug 21, 2026

View reviewed changes

Comment thread include/nlohmann/json.hpp

nlohmann force-pushed the claude/issue-5387-comparison-binary branch from 596e33c to 1527b0f Compare September 2, 2026 16:58

nlohmann added 5 commits September 24, 2026 17:01

Note the comparison fallback in the no-thread-local documentation

ead6a66

The macro page describes what the library defines JSON_NO_THREAD_LOCAL for by itself in terms of copying alone; comparing falls back the same way. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Parenthesise the reserve() computation in the comparison test

169edf2

clang-tidy reports the mixed * and + as readability-math-missing- parentheses, as it does for the identical line in the copy test. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

nlohmann force-pushed the claude/issue-5387-comparison-binary branch from ffca65f to c5e380c Compare September 24, 2026 15:01

gregmarr approved these changes Sep 24, 2026

View reviewed changes

nlohmann added 🚀 ready to merge Ready to merge - just waiting for CI to complete. and removed review needed It would be great if someone could review the proposed changes. labels Sep 24, 2026

nlohmann added this to the Release 3.13.0 milestone Sep 24, 2026

Merge branch 'develop' into claude/issue-5387-comparison-binary

6659373

Resolve the conflict in json.hpp by keeping both the comparison helpers and set_parents_after_object_erase() from #5552, and regenerate the amalgamation. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Amalgamate

310da0b

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

nlohmann merged commit 02dd3e6 into develop Sep 25, 2026
3 of 159 checks passed

nlohmann deleted the claude/issue-5387-comparison-binary branch September 25, 2026 15:59

nlohmann mentioned this pull request Sep 25, 2026

Document response times, supported versions, access, secrets, and dependency policies #5580

Merged

This was referenced Sep 25, 2026

Add tests for uncovered code paths #5581

Merged

Compare unordered objects by key below the nesting bound #5582

Merged

nlohmann added a commit that referenced this pull request Sep 30, 2026

Drop the version history note for a bug that was never released

8ab6355

The regression came from #5390, which is not in any release. Addresses review comment by @gregmarr. Signed-off-by: Niels Lohmann <mail@nlohmann.me>

Navigation Menu

Uh oh!

Fix stack overflow and exponential runtime when comparing nested values - #5390

Fix stack overflow and exponential runtime when comparing nested values#5390
nlohmann merged 8 commits into
developfrom
claude/issue-5387-comparison-binary

nlohmann commented Aug 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gregmarr commented Aug 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

nlohmann commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What & why

An ordered comparison took exponentially long before C++20

The change

Reproducing the results exactly

Measured

Public API impact

Verification

Note for reviewers

What still recurses after this PR

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gregmarr commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Sep 25, 2026

🔴 Amalgamation check failed! 🔴

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nlohmann commented Aug 20, 2026 •

edited

Loading

gregmarr commented Aug 21, 2026 •

edited

Loading