Skip to content

Add editable json_documents: set, push_back, insert, and erase - #5630

Open
nlohmann wants to merge 5 commits into
json-view/16-view-simdfrom
json-view/19-edit-set
Open

nlohmann wants to merge 5 commits into
json-view/16-view-simdfrom
json-view/19-edit-set

Conversation

@nlohmann

@nlohmann nlohmann commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Part of the stack for the zero-copy view (#5295). This PR combines #5630 and #5631 (without the bench_edit.cpp benchmark, which moves to the comparison-harness PR at the top of the stack).

Summary

basic_json_document gets a second template parameter, Editable (false by default). The new aliases json_editable_document, json_editable_view, ordered_json_editable_document and ordered_json_editable_view can change the document they hold:

  • set(view, value) replaces a value;
  • set(object, key, value) assigns a member or adds it, and a null becomes an object. With duplicate keys, the first member is assigned and the others are removed;
  • set(array, index, value) assigns an element;
  • set(json_pointer, value) sets the member or element that a pointer names; "-" or the size of the array appends;
  • push_back(array, value) appends, and a null becomes an array;
  • insert(array, index, value) inserts before an element (index <= size()) and returns a view of the new element;
  • erase(object, key) removes all members with the key and returns their number;
  • erase(array, index) removes an element;
  • erase(json_pointer) removes the member or element a pointer names and returns the number of removed values.

A value can be a view (of any document; it is copied), a BasicJsonType value, or anything BasicJsonType can be constructed from. A view of an erased value keeps its last value; views of other values keep referring to them when elements move.

How edits are stored

  • The source text is never written.
  • The parsed index never moves. New values and element sequences go to storage that the document owns and that never moves (detail/view/edit_storage.hpp). So views stay valid, and a view keeps referring to its value: after an assignment, it sees the new value.
  • An array or object whose elements change gets its elements in a separate sequence. Entries of that sequence link to the values, which stay where they are.
  • Views of read-only documents walk the plain node array through detail::view::navigation<false> and compile without any of this. That policy does not change behavior.

Errors

  • They are those of basic_json where the operation corresponds: type_error.305/307/308/309, out_of_range.401/403/405, parse_error.106/109.
  • A view of another document is invalid_iterator.202.
  • Strings are checked for UTF-8 when they enter the document, with the same type_error.316 message as basic_json::dump() for the same string. So an editable document only ever holds valid UTF-8, and dump() never throws.
  • Binary values cannot be stored: this is the new type_error.319.
  • Edits of 4 GiB or more end with out_of_range.416, as documents of that size do.

Views of editable and read-only documents compare with each other.

Performance

json_view

Measured on the regrouped stack

Not re-measured separately: the read-only paths are unchanged by this PR (see the numbers of the next PRs).

Earlier measurements (on the old stack)

Read-only documents, before vs. after this change are within about 1% for parse, traverse and dump().

Parse + edit + serialize (bench_edit, measured in the harness PR once it is rebuilt on this change): 1.1–5.9x faster than yyjson mutable documents, RapidJSON and nlohmann::json, on twitter and citm_catalog.

Core library (json::parse / dump)

Not affected: this PR only adds new code to the view.

Tests

unit-json_view_edit.cpp holds a seeded differential test that applies random assignments, member and element changes, inserts, erases (directly and through JSON pointers), copies within and between documents, and pushes, both to an ordered_json_editable_document and to the equivalent ordered_json value, and compares serialized, materialized, compared and read-back results after every edit.

Further tests cover the errors; strings that stay valid while the edit arena grows; numbers (NaN, infinities, extremes, number_format::source); nulls that become containers, and a replaced root; duplicate keys, values of other documents, and large objects (the index is not used for moved objects); views across inserts and erasures; and documents reused with read().

Public API

No breaking change:

  • basic_json_document and basic_json_view get a defaulted template parameter. Existing code that names them with one argument, or uses the existing aliases, is unaffected.
  • New aliases json_editable_document, json_editable_view, ordered_json_editable_document, ordered_json_editable_view.
  • New exception ids type_error.319 (binary values cannot be stored) and the already-existing basic_json error ids now also thrown by the view's set/push_back/insert/erase.

Written by Claude Code.

🤖 Generated with Claude Code

@nlohmann
nlohmann added this pull request to stack #5636 September 29, 2026 14:20
@nlohmann nlohmann changed the title json view/19 edit set Add editable documents to json_view Sep 29, 2026
@nlohmann

Copy link
Copy Markdown
Owner Author

CI fixes for this PR (commit f72fe8d, plus the lower fixes merged in). CI had not run yet; these were found locally with gcc:16 (linux/amd64) and the CI flags:

  • ci_test_gcc: in unit-json_view_edit.cpp, std::mt19937::result_type is std::uint_fast32_t (unsigned long on Linux), so returning it as std::uint32_t failed -Werror=conversion. It is now converted explicitly through a named variable. static_cast<std::uint64_t>(18446744073709551615u) and static_cast<std::int64_t>(-9223372036854775807 - 1) were useless casts there and are now std::numeric_limits<...>::max()/min().
  • ci_test_noexceptions: exception_of_call() catches outside a CHECK_THROWS, so the invalid-UTF-8 checks aborted with JSON_NOEXCEPTION. They are now compiled only with exceptions.

Verified: unit-json_view_edit builds and passes (C++11/C++20 with the CI flags, and a JSON_NOEXCEPTION build run as --no-throw). #5631 got only a propagation merge.

— posted by Claude Code on behalf of @nlohmann

@nlohmann
nlohmann removed this pull request from stack #5636 September 30, 2026 13:19
@nlohmann
nlohmann force-pushed the json-view/19-edit-set branch from 711ddc4 to c675bcd Compare September 30, 2026 13:20
@nlohmann
nlohmann added this pull request to stack #5739 September 30, 2026 13:21
@nlohmann
nlohmann force-pushed the json-view/19-edit-set branch 3 times, most recently from 05b6ce8 to d52db33 Compare September 30, 2026 18:06
@nlohmann nlohmann added the review needed It would be great if someone could review the proposed changes. label Sep 30, 2026
@nlohmann
nlohmann marked this pull request as ready for review September 30, 2026 18:16
@github-actions

Copy link
Copy Markdown

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated and/or formatted correctly, or BUILD.bazel is out of date.

@nlohmann
nlohmann force-pushed the json-view/19-edit-set branch from d52db33 to 467ccfd Compare September 30, 2026 19:19
Comment thread include/nlohmann/detail/view/edit_storage.hpp Dismissed
Comment thread include/nlohmann/detail/view/edit_storage.hpp Dismissed
Comment thread include/nlohmann/detail/view/edit_storage.hpp Dismissed
Comment thread include/nlohmann/detail/view/materialize.hpp Fixed
Comment thread include/nlohmann/detail/view/materialize.hpp Dismissed
Comment thread include/nlohmann/detail/view/materialize.hpp Dismissed
Comment thread include/nlohmann/detail/view/materialize.hpp Dismissed
Comment thread include/nlohmann/detail/view/node.hpp Dismissed
Comment thread include/nlohmann/detail/view/node.hpp Dismissed
const string_t no_token{};
// the ends of the open containers, and whether they are objects
std::vector<std::pair<const node*, bool>> open;
std::vector<frame> open;
@nlohmann
nlohmann removed this pull request from stack #5739 October 6, 2026 09:26
@nlohmann
nlohmann changed the base branch from json-view/18-view-object-index to json-view/16-view-simd October 6, 2026 09:26
@nlohmann
nlohmann force-pushed the json-view/19-edit-set branch from e0c29a1 to a0b9832 Compare October 6, 2026 09:28
@nlohmann nlohmann changed the title Add editable documents to json_view Add editable json_documents: set, push_back, insert, and erase Oct 6, 2026
@nlohmann
nlohmann added this pull request to stack #5768 October 6, 2026 09:29
basic_json_document gets a second template parameter, Editable
(false by default), plus the aliases json_editable_document,
json_editable_view, ordered_json_editable_document and
ordered_json_editable_view.

Editable documents can change values and structure without
rewriting the source text: set()/push_back() on values, keys,
array indices and JSON pointers; insert() before an array
element; erase() of an object key, array index or JSON pointer.

New values and element sequences go into edit storage that the
document owns and never moves, so views keep referring to their
value across edits and a parsed node never moves. Read-only
documents walk the plain node array and are unaffected.

Strings are checked for UTF-8 on entry, so dump() of an editable
document never throws type_error.316. Binary values cannot be
stored (type_error.319).

A seeded differential test applies random edits to an editable
document and to the equivalent ordered_json and compares both
after every step.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@nlohmann
nlohmann force-pushed the json-view/19-edit-set branch from a0b9832 to c6ee5a6 Compare October 7, 2026 14:44
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated and/or formatted correctly, or BUILD.bazel is out of date.

📎 A ready-to-apply patch is attached to the failed workflow run as the amalgamation-patch artifact. Download it, then apply it locally from the repository root with:

git apply amalgamation.patch

This does not require installing astyle yourself.

Value-initialize the const std::less in find_parent (clang 3.4/3.6 do not
implement DR 253), and test the Editable template argument through a
function to avoid MSVC C4127.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

# Conflicts:
#	include/nlohmann/detail/view/document_data.hpp
#	single_include/nlohmann/json_view.hpp
Signed-off-by: Niels Lohmann <mail@nlohmann.me>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI documentation L review needed It would be great if someone could review the proposed changes. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants