Repository navigation
Restore a duplicate key's prior value when the callback rejects its new value - #5466
Merged
nlohmann merged 2 commits intoSep 22, 2026
Merged
Conversation
nlohmann
changed the base branch from
develop
to
fix/ordered-json-diff-order
September 5, 2026 14:53
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
2 times, most recently
from
September 5, 2026 15:06
77390de to
0ec4bdb
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 5, 2026 16:43
9ca179e to
740c847
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 5, 2026 20:01
740c847 to
658f9a2
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 9, 2026 11:17
658f9a2 to
3d12ee7
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 11, 2026 08:30
3d12ee7 to
c03f8df
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 11, 2026 15:56
c03f8df to
d4fd004
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 14, 2026 07:17
d4fd004 to
a9f487f
Compare
4 tasks done
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 14, 2026 19:21
a9f487f to
a0f2dfc
Compare
gregmarr
approved these changes
Sep 15, 2026
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 15, 2026 18:46
a0f2dfc to
1ef3677
Compare
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 16, 2026 06:43
1ef3677 to
65a3f96
Compare
…ew value json_sax_dom_callback_parser::key() unconditionally overwrote the object slot for a key with a `discarded` placeholder as soon as the key was accepted by the parser callback. For a duplicate key (legal JSON), this destroyed the pre-existing value from an earlier occurrence of the same key before the new value was even parsed. If the new value was then rejected by the callback, remove_discarded_value() erased the member entirely instead of leaving the original value in place, contradicting the documented behavior that a discarded value behaves as if it was never read. Add a small stash of (slot pointer, previous value) pairs so that when key() overwrites an existing member with the discarded placeholder, the previous value can be restored later if the corresponding value (scalar, object, or array) is rejected, instead of being erased. The stash entry is dropped without restoring once the new value is definitively accepted (in handle_value() for scalars, end_object()/end_array() for containers), so a duplicate key whose new value is accepted still keeps the last value as before. Non-duplicate keys are unaffected: rejecting their value still removes the member entirely, since there is nothing to restore. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
GCC's libstdc++ std::function move assignment evaluates a noexcept
check that invokes a wrapped callable in an unevaluated context; a
non-noexcept parser_callback_t lambda then trips -Wnoexcept ("noexcept-
expression evaluates to 'false'"), which CI's ci_test_gcc job builds
with -Werror. The pre-existing parser_callback_t test lambdas in this
file already work around this by declaring themselves noexcept; apply
the same fix to the three added lambdas that didn't.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
nlohmann
force-pushed
the
fix/parser-callback-duplicate-key
branch
from
September 16, 2026 18:12
65a3f96 to
31b281f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In
json_sax_dom_callback_parser::key(), when the callback accepts a key, the code does:object_element = &(ref_stack.back()->m_data.m_value.object->operator[](val) = discarded);For a duplicate key (legal JSON), this immediately overwrites the already-stored member with a
discardedplaceholder before the new value is even parsed. If the callback then rejects the new value,remove_discarded_value()finds the (still)discardedslot and erases the member entirely — losing the original value, not just skipping the new one.parser_callbacks.mddocuments that "the parser will behave as if the discarded value was never read", which for a duplicate key should mean the first value stays in place:The same problem occurs when the rejected duplicate value is a container, rejected at
object_end/array_end.Fix
Add a small stash (
duplicate_key_stash, a vector of slot-pointer/previous-value pairs) thatkey()populates when it detects the key already exists. The stash entry is dropped (superseded, no restore) once the new value is definitively accepted — for a scalar, inhandle_value(); for a container, inend_object()/end_array()'s "kept" branch. It's consumed with a restore instead of an erase wherever a rejected value's placeholder would otherwise be erased —remove_discarded_value()(scalar rejection) and the discard branches ofend_object()/end_array()(container rejection). Brand-new (non-duplicate) keys are completely unaffected, since no stash entry ever exists for them — that path is byte-for-byte unchanged.Note: an separate, currently open PR (#5457) also touches
remove_discarded_value()in this same file (an unrelated performance optimization, converting a linear scan to an erase-by-key lookup). Whichever of the two lands second will likely need a small rebase against the other.Test plan
tests/src/unit-regression2.cppcoverage for: both reproducers above (scalar and object), an array variant, accepted-duplicate regression guards (last value still wins, scalar and container), a brand-new-key-rejected regression guard, a two-level-nested duplicate key, and a three-occurrence key case.tests/src/unit-class_parser.cppsuite offline (10299 assertions, unaffected).parser_callbacks.mddoc examples (reject_duplicate_keys.cpp,parse__string__parser_callback_t.cpp) against the patched header — output byte-identical to the committed expected output.single_include/nlohmann/json.hppviamake amalgamate.Breaking change?
No breaking changes to the public API. This only changes behavior for the specific, previously-buggy case of a duplicate object key whose second occurrence's value is rejected by a parser callback — the member now correctly retains its first value instead of being erased, matching the documented "as if never read" contract. All other parsing/callback behavior (including the well-known "duplicate key keeps last value" rule when the callback accepts the value) is unchanged.
🤖 Generated with Claude Code