from_bon8(ptr, len) and from_bjdata(ptr, len) compile without a warning and read ptr as a NUL-terminated string #5648

Description

json::from_bon8(ptr, len) and json::from_bjdata(ptr, len) compile without a warning, but they do not read len bytes. They read ptr as a NUL-terminated string (strlen) and pass len on as the strict flag. Data that contains a 0x00 byte is cut off at that byte. Data that contains no 0x00 byte is read past the end of the buffer.

Root cause. from_cbor, from_msgpack, from_ubjson and from_bson each have a deprecated overload (const T* ptr, std::size_t len, ...) that forwards to (ptr, ptr + len, ...), for example from_cbor. from_bjdata (added in v3.11.0, #3336) and from_bon8 (added on develop, #2998) have no such overload, only
from_bon8(InputType&& i, const bool strict = true, const bool allow_exceptions = true) and the iterator/sentinel overload (from_bjdata is the same). For from_bon8(ptr, len):

• The iterator/sentinel overload drops out, because a pointer and a std::size_t cannot be compared with !=.
• The call binds to from_bon8(InputType&&, bool strict) with InputType = const std::uint8_t*, converting len to bool.
• input_adapter(CharT b) treats a pointer to a byte type as a C string and determines its length with std::strlen.

The documentation does not list a (ptr, len) overload for either function. Its input list for from_bon8 does include "a pointer to a null-terminated string of single byte characters", which describes what happens. However, the other four binary readers accept the same call with a deprecation warning (from_cbor.md). For these two functions nothing warns that the call means something else. BON8 data contains 0x00 bytes whenever it contains an integer from 40 to 67637031 (its second byte is 0x00..0x7F), and BJData data contains them for small integers and lengths.

Since when. from_bjdata(ptr, len) reproduces on v3.12.0 (the program below without the from_bon8 lines, against v3.12.0's single header). from_bjdata has had no (ptr, len) overload since it was added in v3.11.0 (#3336). from_bon8 is new on develop (#2998, 1e101ec).

Related. #2426 discussed the deprecation of from_*(ptr, len) in favour of from_*(ptr, ptr + len). According to from_cbor.md, the deprecated overloads "will be removed in version 4.0.0". Once they are removed, from_cbor(ptr, len) and the others would bind the same way as described here. Separately, json::accept(ptr, len) also compiles and passes len as ignore_comments; that is not covered by this issue.

Possible fix. Three options:

• Add a deleted overload template<typename T> static basic_json from_bon8(const T* ptr, std::size_t len, bool strict = true, bool allow_exceptions = true) = delete; (the same for from_bjdata), so that the call does not compile.
• Add the same overload as a deprecated one that forwards to from_bon8(ptr, ptr + len, strict, allow_exceptions), like from_cbor.
• Add it as a regular, non-deprecated overload.

I tried the first two for from_bon8 in a patched copy of include/ with this program only; I did not run the unit tests. With the deleted overload, from_bon8(v.data(), v.size()) fails with "call to deleted function". With the deprecated one, it returns the value and warns with -Wdeprecated-declarations. In both cases, from_bon8(ptr, true), from_bon8(ptr, ptr + n, false) and from_bon8(vec, false, true) still compile and behave as before (C++11, C++17, C++20). With either overload, a length of type int makes the call ambiguous, which is also the case for from_cbor(ptr, int_len) today.

Reproduction steps

1. Save the program below as b1_from_bon8_ptr_len.cpp.
2. From the repository root, compile it with clang++ -std=c++11 -Wall -Wextra -g -fsanitize=address,undefined -I include b1_from_bon8_ptr_len.cpp -o b1. There are no warnings.
3. Run ./b1.

Expected vs. actual results

• Expected: Either from_bon8(ptr, len) and from_bjdata(ptr, len) read len bytes like from_cbor(ptr, len) does, returning {"a":40,"b":"x"}, {"a":0} and "abc", or the calls do not compile.
• Actual: The calls compile without a warning, and len is used as strict.
  1. Data with a 0x00 byte is cut off there, so parse_error.110 is thrown for valid input.
  2. Data without a 0x00 byte is read past the end of the buffer by strlen (heap-buffer-overflow with ASan).

Minimal code example

#include <nlohmann/json.hpp>
#include <iostream>

using json = nlohmann::json;

int main()
{
    // case 1: data that contains a 0x00 byte (the BON8 integer 40 is C2 00)
    const std::vector<std::uint8_t> v = json::to_bon8({{"a", 40}, {"b", "x"}}); // 88 61 C2 00 62 78 FF

    // expected (like from_cbor(ptr, len)): reads v.size() bytes and returns {"a":40,"b":"x"},
    //           or the call does not compile
    // actual:   compiles without a warning as from_bon8(InputType&&, bool strict), reads
    //           v.data() only up to the first 0x00 byte, and throws parse_error.110
    try
    {
        std::cout << json::from_bon8(v.data(), v.size()) << std::endl;
    }
    catch (const json::parse_error& e)
    {
        std::cout << e.what() << std::endl;
    }

    // the same happens with from_bjdata
    const std::vector<std::uint8_t> d = json::to_bjdata({{"a", 0}}); // 7B 69 01 61 69 00 7D
    try
    {
        std::cout << json::from_bjdata(d.data(), d.size()) << std::endl;
    }
    catch (const json::parse_error& e)
    {
        std::cout << e.what() << std::endl;
    }

    // case 2: data without a 0x00 byte (json::to_bon8("abc")):
    // actual: strlen() reads past the end of the buffer (heap-buffer-overflow with ASan)
    const std::vector<std::uint8_t> w = {0x61, 0x62, 0x63, 0xFF};
    std::cout << json::from_bon8(w.data(), w.size()) << std::endl;
}

Error messages

[json.exception.parse_error.110] parse error at byte 4: syntax error while parsing BON8 number: unexpected end of input
[json.exception.parse_error.110] parse error at byte 6: syntax error while parsing BJData number: unexpected end of input
=================================================================
==26369==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000ab4 at pc 0x000102a8e804 bp 0x00016dd456d0 sp 0x00016dd44ea0
READ of size 5 at 0x602000000ab4 thread T0
    #0 0x000102a8e800 in strlen+0x22c (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x16800)
    #1 0x000102125a38 in nlohmann::json_abi_v3_12_0::detail::iterator_input_adapter<char const*, char const*> nlohmann::json_abi_v3_12_0::detail::input_adapter<unsigned char const*, 0>(unsigned char const*) input_adapters.hpp:839
    #2 0x0001020bb25c in nlohmann::json_abi_v3_12_0::basic_json<...>::from_bon8<unsigned char const*>(unsigned char const*&&, bool, bool) json.hpp:5559
    #3 0x0001020b9dc4 in main b1_from_bon8_ptr_len.cpp:38

0x602000000ab4 is located 0 bytes after 4-byte region [0x602000000ab0,0x602000000ab4)
SUMMARY: AddressSanitizer: heap-buffer-overflow input_adapters.hpp:839 in nlohmann::json_abi_v3_12_0::detail::input_adapter<unsigned char const*, 0>(unsigned char const*)

Compiler and operating system

Apple clang 21.0.0 (clang-2100.3.34.2), macOS 27.0 (arm64)

Library version

develop @ 633de8e (the from_bjdata part also on 3.12.0)

Validation

• The bug also occurs if the latest version from the develop branch is used.
• I can successfully compile and run the unit tests.

This issue was written by Claude Code on behalf of @nlohmann.