CI: jobs that check less than they claim (dead Cirrus arm64 job, libc++ without warning flags, ci_cmake_flags wiring, static analysis that cannot fail, duplicated amalgamation checks, dead targets)

[nlohmann]

These items came out of a technical-debt review of develop at 633de8e. They share one theme: CI jobs and helper targets that pass while checking less than their names, the README, or the QA page suggest. Each item is independent and can be fixed in its own PR that references this issue. None of them changes library behavior, the public API, or the ABI. The exceptions to "CI config only" are item 1, which also edits the README badge and the QA page; item 4c, which may adjust #includes in public headers; and item 5, which changes which files make pretty / make amalgamate format. Items 2, 3, and 4 make currently silent jobs stricter, so they may surface existing warnings or findings that have to be fixed in the same PR.

Checklist

• 1. Cirrus CI has stopped reporting, so Linux arm64 is untested: move the job to ubuntu-24.04-arm (user-visible)
• 2. The six ci_test_clang_libcxx_cxx* jobs build without -Werror -Weverything
• 3. ci_cmake_flags checks only JSON_BuildTests on CMake 3.31.6, never runs its current-CMake targets, and skips JSON_StrictNulHandling
• 4. Static-analysis jobs (scan-build, Infer, IWYU) cannot fail on findings
• 5. The amalgamation/format check exists three times with different file sets — overlaps Move the user-defined string literals to <nlohmann/json_literals.hpp> and add JSON_NO_AUTOMATIC_UDLS #5610, Add json_document and json_view: parse, accept, types, materialize #5621
• 6. ci_cmake_flags builds three CMake versions from source instead of downloading prebuilt binaries — overlaps Add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION to fix std::tuple conversions #5598 (step 3 only)
• 7. Remove the unused ci_oclint and ci_pvs_studio targets and the always-built single_all executable
• 8. Remove the no-op AppVeyor with_win_header job
• 9. Stop Dependabot from proposing astyle bumps that break the deliberate 3.4.13 pin

Items

1. Cirrus CI has stopped reporting, so Linux arm64 is untested

.cirrus.yml is the repository's only Linux/aarch64 job (gcc:latest on an arm_container, ctest with -DJSON_FastTests=ON). The cirrus-ci app reported a check run on every develop commit up to d10879b (2026-05-26). From e4bdf1b (2026-06-07) through 633de8e, develop commits only have github-actions check runs. #5218 (merged 2026-06-30) bumped CMake in the file without anyone noticing that the job no longer reported. A missing check fails silently, while these places still claim the coverage exists:

• README.md L10: Cirrus CI badge (added in 3935, removed lgtm badge and added Cirrus CI badge #3937)
• FILES.md L36-L45: documents the pipeline
• quality_assurance.md L93: GNU 16.1.0 | arm64 | Linux 6.1.100 | Cirrus CI

The repository already uses a native GitHub ARM runner (msvc-arm64 on windows-11-arm).

Proposed change:

1. Append a ci_test_arm64 job at the end of ubuntu.yml. It runs on runs-on: ubuntu-24.04-arm with container: gcc:latest (multi-arch), uses the same pinned actions/checkout (with persist-credentials: false) and lukka/get-cmake as the other jobs, runs a uname -m step, then cmake -S . -B build -DJSON_CI=On and cmake --build build --target ci_test_compiler_default (ci.cmake L727).
2. Delete .cirrus.yml. Remove its section from FILES.md and the badge from README.md.
3. Change the arm64 GNU row in quality_assurance.md to GitHub, using the OS that the new job's log reports.
4. Optional: disconnect the Cirrus CI GitHub app.

Verification: The new job passes and uname -m prints aarch64, which shows that it runs natively. Afterwards, git grep -i cirrus matches only ChangeLog.md. As with windows-11-arm, the runner label works only for public repositories.

2. The six ci_test_clang_libcxx_cxx* jobs build without -Werror -Weverything

cmake/ci.cmake L131-L142 passes the warning flags through the CXXFLAGS environment variable and also passes -DCMAKE_CXX_FLAGS="-stdlib=libc++". CMake uses CXXFLAGS only to initialize CMAKE_CXX_FLAGS when that cache entry is unset, so the explicit -D wins. A minimal project on CMake 4.3.4 confirms this: CXXFLAGS="-Weverything -Werror" cmake ... -DCMAKE_CXX_FLAGS=-stdlib=libc++ produces FLAGS=[-stdlib=libc++]. As a result, the six ci_test_standards_clang (…, libcxx) jobs (ubuntu.yml L248-L270) never use the CLANG_CXXFLAGS set. Their libstdc++ siblings (L120-L129) do use it, so libc++-only warnings go unnoticed.

A smaller problem: both variants share the build directory build_clang_cxx${CXX_STANDARD}. In CI each target runs in its own job, so this is harmless there. Locally, running both targets one after the other reuses the first target's CMake cache.

Proposed change:

-    COMMAND CXX=${CLANG_TOOL} CXXFLAGS="${CLANG_CXXFLAGS}" ${CMAKE_COMMAND}
+    COMMAND CXX=${CLANG_TOOL} CXXFLAGS="${CLANG_CXXFLAGS};-stdlib=libc++" ${CMAKE_COMMAND}
 ...
-        -DCMAKE_CXX_FLAGS="-stdlib=libc++"

Also rename the libc++ build directory to build_clang_libcxx_cxx${CXX_STANDARD} in all three places.

Verification: The inner build's CMakeCache.txt contains both -Weverything and -stdlib=libc++ in CMAKE_CXX_FLAGS. Then run the six libc++ jobs. These jobs have never been built with -Werror, so expect libc++-only warnings in the tests that need fixes or targeted suppressions in the same PR.

3. ci_cmake_flags checks only JSON_BuildTests on CMake 3.31.6, never runs its current-CMake targets, and skips JSON_StrictNulHandling

The ci_cmake_options (ci_cmake_flags) job is meant to configure the project with every JSON_* option on CMake 3.5.0, 3.31.6, 4.0.0, and the CMake the job installs. The wiring in ci.cmake L628-L701 comes from #4709 and has these problems:

• 3.31.6 list refers to itself (L660): set(JSON_CMAKE_FLAGS_3_31_6 JSON_BuildTests ${JSON_CMAKE_FLAGS_3_31_6}). The variable is still undefined at that point, so CMake 3.31.6 checks one option instead of eleven. The 4.0.0 line correctly uses ${JSON_CMAKE_FLAGS_3_5_0}.
• Current-CMake targets are never run. ci_add_cmake_flags_targets() creates ${flag_target}_${min_version}_2 targets with ${CMAKE_COMMAND}, but L681 appends the never-set ${JSON_CMAKE_FLAG_TARGET}. None of the _2 targets becomes a dependency of ci_cmake_flags, and all three versions would share the build directory build_${flag_target}.
• JSON_StrictNulHandling (CMakeLists.txt L63) is missing from the list at L658-L659.
• Leftovers: the COMMENT at L675 uses the caller's ${JSON_CMAKE_FLAG} (dynamic scoping) instead of ${flag}. The comment at L632-L635 still calls 4.0.0 "the latest release". ci_clean (L857) removes cmake-3.5.0-Darwin64, which nothing creates, and never removes the cmake-<version> directories that ci_get_cmake() creates.

Proposed change: Add JSON_StrictNulHandling to JSON_CMAKE_FLAGS_3_5_0, and base the 3.31.6 list on ${JSON_CMAKE_FLAGS_3_5_0}. Either delete the _2 targets, or create one ci_cmake_flag_<flag> target per flag outside the per-version function, with its own build directory, and append it to JSON_CMAKE_FLAG_TARGETS. Use ${flag} in the COMMENT, update the stale comment, and change ci_clean to remove cmake-3.5.0 cmake-3.31.6 cmake-4.0.0.

Verification: ninja -t query ci_cmake_flags lists every option for each version, and the job log shows one "Check CMake flag ..." line per option and version. Options that were never checked on CMake 3.31.6 or current CMake may surface real -Werror=dev warnings. Item 6 also edits ci_clean L857, so whichever of the two lands second has to rebase.

4. Static-analysis jobs (scan-build, Infer, IWYU) cannot fail on findings

These three targets report findings but pass whenever the build succeeds. The QA page L133 still lists IWYU as a check that the code passes. The logs below are from green jobs of run 35829411620 (commit 1054b20).

• scan-build (ci_clang_analyze) (L437) runs without --status-bugs. The job runs in silkeh/clang:dev, but ubuntu.yml L86-L87 installs Debian's clang-tools, and the find_program list at L72 falls through to plain scan-build. The log shows "Using '/usr/lib/llvm-14/bin/clang' for static analysis". It currently reports "No bugs found".
• Infer (ci_infer) (L535) runs without --fail-on-issue and logs "Found 174 issues". Of those, 129 are PULSE_UNNECESSARY_COPY_ASSIGNMENT, 22 PULSE_UNNECESSARY_COPY, 15 PULSE_UNNECESSARY_COPY_INTERMEDIATE, 5 PULSE_RESOURCE_LEAK, 2 PULSE_CONST_REFABLE, and 1 PULSE_UNNECESSARY_COPY_OPTIONAL. infer-out/report.txt is not uploaded, so the full list cannot be seen.
• IWYU (ci_single_binaries) (L591) runs without -Xiwyu --error, so CMake ignores IWYU's exit status. The job prints add/remove reports for json.hpp, json_fwd.hpp, and ordered_map.hpp. For example, ordered_map.hpp should add <new> and nlohmann/detail/abi_macros.hpp and should drop <memory>.

Proposed change (independent steps):

• a. Add --status-bugs --use-analyzer=${CLANG_TOOL} to the scan-build command. Optionally, prune the irrelevant osx.*, webkit.*, fuchsia.*, and optin.mpi.* checkers from L430.
• b. Upload build/build_infer/infer-out/report.txt as an artifact and triage the findings. Then add --fail-on-issue, with the triaged issue types disabled or a baseline in place.
• c. Fix the IWYU reports or silence them with pragmas. Then use -Xiwyu --error -Xiwyu --max_line_length=300.

Verification: On a scratch branch, inject a finding and confirm that each job fails. For (a), the log must name the dev clang instead of llvm-14, and a newer analyzer may report findings that need triage first. For (c), changed includes in public headers must pass the full compiler matrix.

5. The amalgamation/format check exists three times with different file sets

1. Makefile pretty / amalgamate / check-amalgamation (L156-L188) is what CONTRIBUTING.md tells contributors to run. pretty formats $(SRCS) $(TESTS_SRCS), the single headers, and docs/mkdocs/docs/examples/*.cpp. The comment at L176 says check-amalgamation "is called by Travis", but Travis is gone and no job calls this target.
2. Workflow check_amalgamation.yml L59-L88 runs only on: pull_request. It regenerates BUILD.bazel and runs astyle on find docs/mkdocs/docs/examples include tests (*.hpp, *.cpp, *.cu), excluding tests/thirdparty/* and tests/abi/include/nlohmann/*.
3. CMake ci_test_amalgamation (ci.cmake L365-L396) runs via ubuntu.yml L60 on pushes as well. For direct pushes to develop, master, and release/*, it is the only amalgamation check. Its glob covers only include/nlohmann/*.hpp, tests/src/*.{cpp,hpp}, tests/benchmarks/src/benchmarks.cpp, and docs/mkdocs/docs/examples/*.cpp, and it never checks BUILD.bazel.

On develop, 28 files are checked by the workflow but not by the CMake target, whose file set is a strict subset of the workflow's. The 28 files are the four examples/custom_*_type.hpp headers, 13 files under tests/abi/, 10 under tests/cmake_*/project/, tests/cuda_example/json_cuda.cu, tests/fmt_formatter/project/main.cpp, and tests/module_cpp20/main.cpp. As a result, make amalgamate leaves the custom_*_type.hpp headers unformatted, although the workflow rejects them. A misformatted file or a stale BUILD.bazel can also reach develop through a direct push. Every new header has to be added in three places (#5610 and #5621 each edit all three).

Proposed change:

• Keep the workflow running amalgamate.py from the develop checkout (L52-L57). This is deliberate, so that a PR cannot change the tool that checks it.
• Make ci_test_amalgamation match the workflow: glob the same roots and extensions, list(FILTER ... EXCLUDE REGEX "/tests/thirdparty/|/tests/abi/include/nlohmann/"), and regenerate and diff BUILD.bazel the same way the target already handles json.hpp. Alternatively, remove the target, although the workflow does not cover push events.
• In the Makefile, add docs/mkdocs/docs/examples/*.hpp to pretty (and pretty_format) and drop the Travis comment.

Verification: Run the target on develop with the pinned astyle 3.4.13 and fix any existing findings in the same PR. It must fail on a misformatted tests/abi/ file and on a new include/ header without a regenerated BUILD.bazel. make amalgamate must leave a clean tree.

Sequencing: #5610 and #5621 change the same hunks (pretty, INDENT_FILES / ci_test_amalgamation, and check_amalgamation.yml). Land this change after them or rebase it onto them. If the target is removed, update any required-check list that names it.

6. ci_cmake_flags builds three CMake versions from source

ci_get_cmake() (ci.cmake L637-L651) downloads the source tarballs of CMake 3.5.0, 3.31.6, and 4.0.0 and compiles each of them completely, including CMake's own test helpers, with -DCMAKE_POLICY_VERSION_MINIMUM=3.5 as a workaround. In run 35829411620 the ci_cmake_options (ci_cmake_flags) job took about 11 minutes, most of it spent building CMake. The source build is also why libssl-dev is on the apt line (ubuntu.yml L106). The whole 12-target matrix still runs in container: ubuntu:focal (L98-L115), and Ubuntu 20.04 left standard support in May 2025. Prebuilt archives were used until b6dcf3e (2022-12, "fix Ubuntu build"). The commit gives no reason for the switch, and the stale cmake-3.5.0-Darwin64 entry in ci_clean is left over from that time. Kitware publishes cmake-3.5.0-Linux-x86_64.tar.gz, cmake-3.31.6-linux-x86_64.tar.gz, and cmake-4.0.0-linux-x86_64.tar.gz. The platform name is lowercase from 3.20 on.

Proposed change:

1. On Linux x86_64, download and unpack the prebuilt archive and point ${var} at its bin/cmake. Remove the configure/build steps and CMAKE_POLICY_VERSION_MINIMUM. Keep the source build as a fallback for macOS and aarch64.
2. Make ci_clean remove the new directory names.
3. As a separate commit, drop libssl-dev and move the job off ubuntu:focal (for example to ubuntu:24.04). This changes the GCC used by the other 11 option targets.

Verification: All ci_cmake_options entries pass, and ci_cmake_flags drops to a minute or two. The 2016 3.5.0 binary must run in the chosen container.

Sequencing: Step 3 conflicts with #5598, which edits the ci_cmake_options matrix line (ubuntu.yml @@ -100,7), so wait until #5598 is merged. Steps 1 and 2 touch only cmake/ci.cmake. The ci_clean line is shared with item 3.

7. Remove the unused ci_oclint and ci_pvs_studio targets and the always-built single_all executable

No workflow calls ci_oclint or ci_pvs_studio (grepping .github/ for either name finds nothing). The ci_oclint setup also costs time on every -DJSON_CI=ON build. L475-L480 copies the single header to src_single/all.cpp, appends int main() {}, and adds single_all without EXCLUDE_FROM_ALL:

add_executable(single_all ${PROJECT_BINARY_DIR}/src_single/all.cpp)

As a result, a plain cmake --build of a JSON_CI tree compiles the whole ~1.2 MB single header as its own translation unit, only to give ci_oclint (L482-L497) a compile_commands.json entry. The OCLint discovery (L58-L62) also runs on every configure. ci_pvs_studio (L513-L526, discovery at L70-L71) duplicates the Makefile's pvs_studio target (L134-L141), which is the one used for local runs. Separately, ci_cppcheck passes --check-level=exhaustive twice (L450 and L453).

Proposed change: Delete the OCLint discovery and the "Check code with OCLint" section (L471-L497), including single_all. src_single/ is still created by the file(WRITE ...) in the ci_single_binaries loop (L599). Delete the ci_pvs_studio section (L512-L526), and remove the PLOG_CONVERTER_TOOL and PVS_STUDIO_ANALYZER_TOOL lookups if nothing else uses them. Remove the duplicate --check-level=exhaustive at L453. Keep the Makefile pvs_studio target.

Verification: Diffing the ci_* target list before and after the change should show only ci_oclint and ci_pvs_studio removed. ninja -t query all should no longer list single_all. The ci_cppcheck and ci_single_binaries jobs should still pass.

8. Remove the no-op AppVeyor with_win_header job

The with_win_header entry (appveyor.yml L33-L39), added in d28b4b9 (2019), prepends #include <Windows.h> to single_include/nlohmann/json.hpp in before_build (L77-L79). Since #3532 (2022), JSON_MultipleHeaders defaults to ON (CMakeLists.txt L60, L88-L93). The entry passes CMAKE_OPTIONS: "", so the patched header is never compiled. The job is an exact duplicate of the VS2015 x86 Release entry at L26-L31 and costs one slow AppVeyor slot per build. The intended coverage already comes from tests/src/unit-windows_h.cpp (#3631), which every MSVC job builds. That test file is why the claim at quality_assurance.md L135 still holds.

Proposed change: Delete the matrix entry (L33-L39) and the comment plus the two ps: lines (L77-L79). Optionally, point quality_assurance.md L135 at unit-windows_h.cpp. Do not switch to /FIWindows.h or -DJSON_MultipleHeaders=OFF: that would require test-only edits (for example, parenthesizing min()/max() in unit-32bit.cpp and unit-bjdata.cpp) and would test nothing new.

Verification: The remaining AppVeyor jobs are green, and test-windows_h still runs in the AppVeyor and windows.yml MSVC jobs. #5605 also edits appveyor.yml, but only test_script (L85-L91).

9. Stop Dependabot from proposing astyle bumps that break the deliberate 3.4.13 pin

The /tools/astyle pip entry in .github/dependabot.yml L21-L26 runs daily and has no ignore rule. tools/astyle/requirements.txt pins astyle==3.4.13 on purpose, because that version defines the formatting that make pretty produces and that check_amalgamation.yml and ci_test_amalgamation enforce. Each astyle release therefore opens a PR that is closed unmerged: #4580 (3.6.6), #4942 (3.6.9), #5445 (3.6.10), and #5448 (3.6.18, which failed check and ci_test_amalgamation). Each one costs a full CI run. Closing such a PR suppresses only that exact version.

Proposed change:

     cooldown:
       default-days: 7
+    # astyle is deliberately pinned at 3.4.13: newer versions reformat the code,
+    # and the pinned version defines the formatting enforced by check_amalgamation.yml.
+    ignore:
+      - dependency-name: astyle

Add a matching comment to requirements.txt.

Verification: GitHub validates dependabot.yml on push. After the next astyle release, no Dependabot PR for /tools/astyle should appear. #5638 adds an npm block just before this entry in the same file, but the hunks do not overlap.

Related

• Reduce test suite runtime and run the Unicode tests everywhere #5605, Add JSON_DISABLE_TUPLE_REFERENCE_CONVERSION to fix std::tuple conversions #5598, and Review and extend the documentation, and check it in CI #5638 edit other parts of cmake/ci.cmake, ubuntu.yml, appveyor.yml, and dependabot.yml. Only the overlaps listed under the items above touch the same hunks. For the rest, expect at most trivial rebase conflicts.
• Move the user-defined string literals to <nlohmann/json_literals.hpp> and add JSON_NO_AUTOMATIC_UDLS #5610 and Add json_document and json_view: parse, accept, types, materialize #5621 each add a new amalgamated header and have to update all three amalgamation check variants (item 5).

---

This issue was written by Claude Code on behalf of @nlohmann.