Convert floats with Eisel-Lemire when std::from_chars is unavailable - #5617
Merged
Merged
Conversation
Owner
Author
|
CI fixes for this PR (commit a4556e8):
Verified with Homebrew GCC 16 using the CI's The fix was merged upward through the whole stack (merge commits only: #5618 to #5635). — posted by Claude Code on behalf of @nlohmann |
This was referenced Sep 29, 2026
gregmarr
approved these changes
Sep 30, 2026
nlohmann
marked this pull request as ready for review
September 30, 2026 05:22
nlohmann
removed this pull request from stack #5636
September 30, 2026 13:19
nlohmann
added this pull request to stack #5739
September 30, 2026 13:21
Float tokens that Clinger's fast path cannot convert (e.g. the 17-digit coordinates of canada.json) went to strtod unless std::from_chars was available. It is not used in C++11/14, and not with libc++, which does not define __cpp_lib_to_chars. The Eisel-Lemire algorithm (after fast_float's compute_float) now converts them with integer arithmetic, correctly rounded for any token with at most 19 significant digits. Longer tokens are truncated; the result is used if w and w + 1 round alike, else strtod decides as before. Overflow still yields infinity (out_of_range.406). The table of powers of five (fast_float's) lives in pow5_table.hpp; a unit test recomputes every entry with big-integer arithmetic. Further tests: known values generated with Python (whose float() is correctly rounded), 200,000 round trips through to_chars, and the 128-bit multiplication and leading-zero count against big-integer references (both with and without a 128-bit type). Checked against strtod on 6.5 million tokens, among them 60,000 exact halfway cases: no difference. json::parse on canada.json: -8.6% (C++11), -7.6% (C++17, Apple clang); other files unchanged. Compile time of a TU including json.hpp: +0.7%. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
GCC (-Werror=unused-result) rejected CHECK_THROWS_WITH_AS(json::parse(...))
because parse() is [[nodiscard]]; assign the result to a dummy json as the
other tests do. clang-tidy flagged longer.find('.') == npos with
abseil-string-find-str-contains; store the position in a variable first.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
nlohmann
force-pushed
the
json-view/02-eisel-lemire
branch
from
September 30, 2026 18:06
a4556e8 to
3ad6a88
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the stack for the zero-copy view (#5295). This change speeds up
json::parseon its own; the view uses it later.Summary
Some float tokens are too long for Clinger's fast path, e.g. the 17-digit coordinates of
canada.json. Such tokens go tostrtodunlessstd::from_charsis available, and it is not:__cpp_lib_to_chars, so macOS builds never used it.These tokens are now converted with the Eisel-Lemire algorithm, adapted from fast_float's
compute_float. It uses integer arithmetic only.wandw + 1round to the same double; otherwisestrtoddecides, as before.out_of_range.406.Changes
detail/input/number_parse.hpp:eisel_lemire()andparse_float_eisel_lemire(), fordouble;floatandlong doubletake the existing path.detail/input/pow5_table.hpp: the 128-bit powers of five from fast_float, in a function-local static.detail/bit_ops.hpp:count_leading_zeros(), a 64×64→128-bit multiplication, andread_eight_bytes(). Each has a portable fallback where there is no builtin or__int128, and none depends on the byte order.license.md: credit for fast_float (MIT, Apache 2.0, or BSL; used under MIT).Tests
New test case in
unit-class_lexer.cpp(622,082 assertions):float()is correctly rounded;to_chars;__int128.Also checked outside the test suite: 6.5 million tokens compared with
strtod, among them 60,000 exact halfway cases. There was no difference.Benchmarks
tests/benchmarks: Apple M1 Max, Apple clang 21,-O3 -flto, C++11, median of 10 repetitions, before → after:json::parseofcanada.jsonin a separate process, best of 5: −8.6% (C++11), −7.6% (C++17).Compile time (clang
-O2): +0.7% for a translation unit that only includes json.hpp, +0.9% for one that also instantiatesparse.x86-64: not measured yet; the numbers will follow in a comment.
Public API
No change. Parsed values are the same, since both paths round correctly.
Written by Claude Code.
🤖 Generated with Claude Code