Repository navigation
Keep external headers as #include when amalgamating - #5615
Merged
Merged
Conversation
A header that builds on json.hpp (such as the planned json_view.hpp) must not inline json.hpp: its single-header version would contain a second copy of the library, and that copy would change with every library change. The optional config key "external" lists include paths that are kept as #include directives. Only the first directive per path is kept; repeated ones are commented out, as the tool already does for inlined headers. config_json_view.json uses it for json_view.hpp; the existing configs do not set it, and json.hpp and json_fwd.hpp regenerate byte-identically. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
nlohmann
added this pull request to stack #5636
September 29, 2026 14:20
github-actions
Bot
added
documentation
M
python
gregmarr
approved these changes
Sep 29, 2026
nlohmann
added
the
🚀 ready to merge
nlohmann
marked this pull request as ready for review
September 29, 2026 14:36
This was referenced Sep 29, 2026
nlohmann
removed this pull request from stack #5636
September 30, 2026 13:19
nlohmann
added this pull request to stack #5739
September 30, 2026 13:21
nlohmann
added a commit
that referenced
this pull request
Sep 30, 2026
tools/amalgamate/README.md is the unmodified upstream text and no longer matches how the tool is used here: - It named a Bitbucket origin that no longer exists; CHANGES.md already tracks the GitHub mirror commit this copy is based on. - It asked for Python 2.7, but CI and the Makefile run the script with python3. - It told readers to run ./test.sh (not vendored) and install to /usr/local/bin; in this repository the tool runs through `make amalgamate`. - Its usage synopsis showed `-v` taking no argument, but the script's own argparser requires `choices=["yes", "no"]`, so that form fails with "argument -v/--verbose: expected one argument". The Makefile calls it as `--verbose=yes`. - It pointed at test/source.c.json and test/include.h.json, which are not vendored; the configs actually used are config_json.json and config_json_fwd.json. Rewrote only the Installing and Using sections to match; left the "Here be dragons" caveats and the rest of the vendored code untouched to avoid diverging further from upstream. Overlaps #5615, which edits amalgamate.py, this README and CHANGES.md. #5717 item 6 Signed-off-by: Niels Lohmann <mail@nlohmann.me>
nlohmann
added a commit
that referenced
this pull request
Sep 30, 2026
…er (#5735) * Remove dead doctest help entry and pretty_format target from Makefile The top-level Makefile still carried three leftovers: - The help text listed a "doctest" target that was removed in #4560, so "make doctest" fails with "No rule to make target". The example check now runs as "make check_output -C docs". - "pretty_format" ran clang-format on all sources, but .clang-format was deleted in #4573, so the target reformatted everything in the default LLVM style, against the Artistic Style formatting that "make pretty" applies and CI enforces. - "clean" removed benchmarks/files/numbers/*.json, a directory that no longer exists since the benchmarks moved to tests/benchmarks (#3462). Only maintainer tooling changes; the library is not affected. Part of #5717 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Document tools/macro_builder and tidy up serve_header.py tools/macro_builder generates the NLOHMANN_JSON_EXPAND, NLOHMANN_JSON_GET_MACRO and NLOHMANN_JSON_PASTE* macros in macro_scope.hpp, but nothing referred to it. Add a README that explains what it generates, how to run it and where the output goes, and which dependent tables (NLOHMANN_JSON_DOUBLE_PASTE, NLOHMANN_JSON_TYPE_BODY) are maintained by hand. Point to it from a comment above NLOHMANN_JSON_EXPAND. The generator itself is unchanged; following the README reproduces the header byte for byte. In serve_header.py, drop the LGTM suppression (LGTM.com shut down in 2022), replace the """.""" placeholder docstrings with real ones, and import socket and ssl at module level. DualStackServer.server_bind uses socket, which was only imported under __main__; when the module was imported instead, the NameError was swallowed and IPV6_V6ONLY was not cleared. The header change is a comment only; behavior, API and ABI are unchanged. Part of #5717 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Hash every release_files artifact, not a hardcoded subset The `release` target signed and copied json_fwd.hpp into release_files alongside json.hpp, but the shasum line that writes hashes.txt only listed json.hpp, include.zip and json.tar.xz. Users could not verify the published json_fwd.hpp against hashes.txt. Hash every file in release_files except the .asc signatures instead of naming files by hand, so a newly shipped header (such as the json_literals.hpp that #5610 adds to this target) cannot be missed again. Only affects the generated hashes.txt release artifact; the library itself is unaffected. Overlaps #5610, which touches the same lines to add json_literals.hpp to the release target. #5717 item 1 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove the broken fuzz_testing* Makefile targets fuzz_testing and fuzz_testing_{bon8,bson,cbor,msgpack,ubjson} seeded fuzz-testing/testcases from tests/data, which was removed in dbf1a1f (2020) when the test data moved to the external json_test_data repo. The find command found nothing, but the pipeline's exit status was that of xargs, so the recipe still reported success with an empty corpus, and the printed afl-fuzz command would refuse to start. The recipes were also six near-identical copies with unquoted -name patterns, used the legacy CXX=afl-clang++, and fuzzing-start/stop were missing from both the help output and .PHONY. tests/fuzzing.md already documents the working flow (download json_test_data, then `make -C tests fuzzers`), so replace the six broken targets and their help lines with a single pointer to that document instead of trying to keep six copies of a fragile shell pipeline in sync. This does not affect OSS-Fuzz, which builds through tests/Makefile. Overlaps #5621, which adds a seventh copy of the same broken line for fuzz_testing_json_view. #5717 item 2 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Remove stale Travis comment above check-amalgamation check-amalgamation carried "Note: this target is called by Travis", left over from before the project switched off Travis CI. The prior Makefile cleanup commit removed the other stale Travis-era leftovers (the doctest help entry, pretty_format, and the benchmarks/ path in clean) but missed this comment. #5717 item 4 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix stale install/usage instructions in the vendored amalgamate README tools/amalgamate/README.md is the unmodified upstream text and no longer matches how the tool is used here: - It named a Bitbucket origin that no longer exists; CHANGES.md already tracks the GitHub mirror commit this copy is based on. - It asked for Python 2.7, but CI and the Makefile run the script with python3. - It told readers to run ./test.sh (not vendored) and install to /usr/local/bin; in this repository the tool runs through `make amalgamate`. - Its usage synopsis showed `-v` taking no argument, but the script's own argparser requires `choices=["yes", "no"]`, so that form fails with "argument -v/--verbose: expected one argument". The Makefile calls it as `--verbose=yes`. - It pointed at test/source.c.json and test/include.h.json, which are not vendored; the configs actually used are config_json.json and config_json_fwd.json. Rewrote only the Installing and Using sections to match; left the "Here be dragons" caveats and the rest of the vendored code untouched to avoid diverging further from upstream. Overlaps #5615, which edits amalgamate.py, this README and CHANGES.md. #5717 item 6 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Derive generate_natvis.py's ABI tag list and version from abi_macros.hpp generate_natvis.py hard-coded abi_tags = ['_diag', '_ldvcmp', '_dp', '_bics', '_psp', '_snul'] and required --version on the command line. The source of truth is include/nlohmann/detail/abi_macros.hpp: the NLOHMANN_JSON_ABI_TAG_* defines, the argument order of NLOHMANN_JSON_ABI_TAGS_CONCAT, and NLOHMANN_JSON_VERSION_MAJOR/MINOR/ PATCH. Nothing checked that the copies stayed in sync, and they have drifted apart before: _dp was added in #4517 but missed here until #5544, and 3.11.3 shipped json_abi_v3_11_2 namespaces (#4340). Parse the tag list (in NLOHMANN_JSON_ABI_TAGS_CONCAT order) and the version from abi_macros.hpp instead of hard-coding them. Make --version optional (falling back to the parsed version) and default the output directory to the repository root the script lives in. Add a "natvis" Makefile target that runs the script, and extend check-amalgamation to regenerate nlohmann_json.natvis and fail on a diff, the same way it already does for the amalgamated headers and BUILD.bazel. Wire the same regeneration into check_amalgamation.yml, using the tool copy checked out from develop (as the workflow already does for amalgamate.py) and installing jinja2 from tools/generate_natvis/requirements.txt. Update the tool's README to say it must be re-run after adding an ABI tag or bumping the version. Verified: a run against develop produces no diff (with either the default or an explicit --version 3.12.0); adding a dummy NLOHMANN_JSON_ABI_TAG_* without a matching #define makes the script fail loudly instead of silently omitting the tag; xmllint --noout passes on the regenerated file; and running the script from a directory other than the one being checked (simulating the workflow's separate tool checkout) against this repository root also produces no diff. Overlaps #5600, which added _ekmo to the same hand-written abi_tags line and regenerated the file. #5717 item 3 Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Strip the leading "./" find(1) prefix from release hashes.txt entries bc6e7db (#5717 item 1) switched the release target's shasum line from naming files by hand to $$(find . -type f -not -name '*.asc' | sort), so a newly shipped header is hashed automatically. Run from inside release_files, that find prints paths as "./json.hpp" instead of "json.hpp", so hashes.txt lists "./json.hpp" etc. instead of the plain filenames it always used. shasum -c still verifies "./json.hpp" fine, but it is a needless cosmetic regression for anyone reading the file or matching it against release notes. Strip the "./" prefix with sed before sorting, keeping the filenames exactly as before while still hashing every artifact automatically. Review fix for #5717 item 1 (PR #5735). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Fix check_amalgamation.yml: pass --version to generate_natvis.py bff45f1 (#5717 item 3) made --version optional in tools/generate_natvis/generate_natvis.py and wired the workflow's new "Regenerate nlohmann_json.natvis" step to call it without --version, relying on the script deriving the version from abi_macros.hpp itself. But NATVIS_TOOL_DIR is checked out from develop, the same way TOOL_DIR already is for amalgamate.py, precisely so an in-flight PR's tooling changes cannot mark themselves clean. Until this PR (or an equivalent) merges to develop, that checkout is the old generate_natvis.py, whose --version argument is still required=True. The new step's invocation of "generate_natvis.py $MAIN_DIR" (no --version) then fails argparse on this PR's own CI run with "the following arguments are required: --version", before the check ever gets to compare output. Extract the version from $MAIN_DIR's own abi_macros.hpp in the workflow and always pass it as --version. That satisfies the old script's required argument and is accepted as an explicit override by the new one, so the step behaves the same whether NATVIS_TOOL_DIR holds the pre- or post-merge tool, and stays correct for later PRs that bump the version. Verified by running the workflow step's shell logic locally against both the pre-#5717 generate_natvis.py (checked out at 633de8e) and the new one: both produce the identical nlohmann_json.natvis as the committed file. Review fix for #5717 item 3 (PR #5735). Signed-off-by: Niels Lohmann <mail@nlohmann.me> * Extend tools/macro_builder to also generate DOUBLE_PASTE and TYPE_BODY tools/macro_builder only emitted NLOHMANN_JSON_EXPAND..PASTE64, so two other tables that scale with the same max_args stayed hand-maintained with nothing checking them: NLOHMANN_JSON_DOUBLE_PASTE (added by hand in #4563 for the *_WITH_NAMES macros) and the 64-slot dispatch table of NLOHMANN_JSON_TYPE_BODY (the #4041 zero-member/one-or-more-member switch). Both tables pass one macro name per slot to the same NLOHMANN_JSON_GET_MACRO dispatch as PASTE, so they can drift out of sync with max_args exactly the way _dp did in the ABI tag list fixed by #5544. Extend main.cpp with build_double_paste_code() (same recursive-doubling shape as build_paste_code(), but DOUBLE_PASTE consumes two arguments per member, so an even slot index falls back to the next lower odd DOUBLE_PASTE<N>) and build_type_body_table() (max_args - 1 MEMBERS slots and one trailing EMPTY slot, 8 per line, matching how it is written by hand today). Add a "type_body" argument that selects the TYPE_BODY block, since it lives at a separate location in macro_scope.hpp from the EXPAND..DOUBLE_PASTE63 block; plain invocation is unchanged apart from covering the extended range. No longer emit the tool's old trailing blank line, so its output is directly diffable without post-processing. Verified with c++ -std=c++11: running the tool (with and without "type_body") and piping the raw output through the pinned astyle reproduces both blocks of the current macro_scope.hpp byte for byte. tests/src/unit-udt_macro.cpp (all NLOHMANN_DEFINE_TYPE_*/_WITH_NAMES/ zero-member variants) passes unchanged under -std=c++11 and -std=c++17 with -fsanitize=address,undefined. Add a "macro_builder_check" Makefile target that builds main.cpp, regenerates both blocks into a scratch directory inside the repository (astyle's --project lookup needs the target files under the same tree as .astylerc, unlike an external /tmp directory), and diffs them against the corresponding ranges of macro_scope.hpp; wire it into check-amalgamation next to the natvis check. Wire the same regeneration into check_amalgamation.yml, splicing the (still unindented) generated blocks back into the PR's own macro_scope.hpp before the existing astyle/amalgamation step runs, so that step's own tree-wide astyle pass both indents them and folds any drift into the amalgamation patch/diff the workflow already produces. Unlike amalgamate.py and generate_natvis.py, this step builds tools/macro_builder/main.cpp from the pull request's own checkout ($MAIN_DIR) rather than a separate checkout of tools/ at develop: this tool has no independent source of truth to regenerate against (its README documents that it must reproduce macro_scope.hpp byte for byte), so a develop-pinned copy would only reproduce the generate_natvis.py trap fixed in a previous commit on this branch, where a PR that teaches the tool to cover more of the file fails its own CI until that PR merges and updates the develop copy. Add tools/macro_builder/README.md documentation for both new tables and the two-invocation usage, and a short pointer comment above NLOHMANN_JSON_TYPE_BODY (the EXPAND pointer already covered the first block; extended its wording to include DOUBLE_PASTE63). Closes #5717 item 5 in full, completing what the documentation-only "Document tools/macro_builder..." commit already on this branch left open (that commit's README/pointer-comment half stands; it also covers item 7). Signed-off-by: Niels Lohmann <mail@nlohmann.me> --------- Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the first PR of the stack for the zero-copy view (#5295).
Summary
The PRs further up add
include/nlohmann/json_view.hpp, a header that includes<nlohmann/json.hpp>. Amalgamated with the current tool,single_include/nlohmann/json_view.hppwould inline all of json.hpp: a second copy of the library (1.2 MB) that would change with every library change.This PR adds an optional
externalkey to the amalgamation config. Include directives of the listed paths are kept as they are, and only the first one per path is kept; repeats are commented out, as the tool already does for inlined headers. The newtools/amalgamate/config_json_view.jsonuses it.This PR sits at the bottom of the stack because
check_amalgamation.ymlruns develop'stools/on pull requests. The PR that addssingle_include/nlohmann/json_view.hppcan only pass that check once this config is on develop.Changes
tools/amalgamate/amalgamate.py: theexternalkeytools/amalgamate/config_json_view.json: the config for the new header (not used yet)tools/amalgamate/README.md,tools/amalgamate/CHANGES.mdTests
make amalgamateregeneratesjson.hppandjson_fwd.hppbyte-identically; neither config setsexternal.Public API
No change: only the amalgamation tool is affected.
Written by Claude Code.
🤖 Generated with Claude Code