Skip to content

Fix to_msgpack() reading the inactive number union member #5694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nlohmann merged 2 commits into from
Sep 30, 2026
Merged

Fix to_msgpack() reading the inactive number union member #5694

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4b027c9
Fix to_msgpack() reading the inactive number union member
nlohmann Sep 29, 2026
9d415f0
Cast number_integer to number_unsigned_t only once in to_msgpack()
nlohmann Sep 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/mkdocs/docs/api/basic_json/to_msgpack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,6 @@ Linear in the size of the JSON value `j`.

- Added in version 2.0.9.
- Throws `out_of_range.412` and `out_of_range.415` since version 3.13.0.
- Fixed in version 3.13.0 to serialize `number_integer_t`/`number_unsigned_t` pairs of different width correctly;
before, integers could be serialized with the wrong value if `number_integer_t` was narrower than
`number_unsigned_t`.
19 changes: 10 additions & 9 deletions include/nlohmann/detail/output/binary_writer.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,24 +337,25 @@ class binary_writer
// MessagePack does not differentiate between positive
// signed integers and unsigned integers. Therefore, we used
// the code from the value_t::number_unsigned case here.
if (j.m_data.m_value.number_unsigned < 128)
const auto value_as_unsigned = static_cast<typename BasicJsonType::number_unsigned_t>(j.m_data.m_value.number_integer);

@gregmarr gregmarr Sep 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be static_cast<std::make_unsigned<typename BasicJsonType::number_integer_t>> to prevent truncation if number_unsigned_t is smaller than number_integer_t?

@nlohmann nlohmann Sep 30, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thought, but that configuration can no longer be instantiated: since #5443, basic_json has static_assert(sizeof(NumberUnsignedType) >= sizeof(NumberIntegerType), ...), so every non-negative number_integer_t value fits in number_unsigned_t and the cast can't truncate. std::make_unsigned<number_integer_t> would be the same width or narrower, so it wouldn't add anything here. I verified this by trying to add a test with number_integer_t = std::int64_t, number_unsigned_t = std::uint32_t — it fails to compile on that assertion.

(Reply written by Claude Code on behalf of @nlohmann.)

@gregmarr gregmarr Sep 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thought, but that configuration can no longer be instantiated:

Perfect.

if (value_as_unsigned < 128)
{
// positive fixnum
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
else if (value_as_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
{
// uint 8
oa.write_character(to_char_type(0xCC));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
else if (value_as_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
{
// uint 16
oa.write_character(to_char_type(0xCD));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
else if (value_as_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
{
// uint 32
oa.write_character(to_char_type(0xCE));
Expand Down Expand Up @@ -410,31 +411,31 @@ class binary_writer
if (j.m_data.m_value.number_unsigned < 128)
{
// positive fixnum
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
{
// uint 8
oa.write_character(to_char_type(0xCC));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
{
// uint 16
oa.write_character(to_char_type(0xCD));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
{
// uint 32
oa.write_character(to_char_type(0xCE));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_unsigned));
}
else
{
// uint 64
oa.write_character(to_char_type(0xCF));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
}
break;
}
Expand Down
19 changes: 10 additions & 9 deletions single_include/nlohmann/json.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20667,24 +20667,25 @@ class binary_writer
// MessagePack does not differentiate between positive
// signed integers and unsigned integers. Therefore, we used
// the code from the value_t::number_unsigned case here.
if (j.m_data.m_value.number_unsigned < 128)
const auto value_as_unsigned = static_cast<typename BasicJsonType::number_unsigned_t>(j.m_data.m_value.number_integer);
if (value_as_unsigned < 128)
{
// positive fixnum
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
else if (value_as_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
{
// uint 8
oa.write_character(to_char_type(0xCC));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
else if (value_as_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
{
// uint 16
oa.write_character(to_char_type(0xCD));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_integer));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
else if (value_as_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
{
// uint 32
oa.write_character(to_char_type(0xCE));
Expand Down Expand Up @@ -20740,31 +20741,31 @@ class binary_writer
if (j.m_data.m_value.number_unsigned < 128)
{
// positive fixnum
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint8_t>::max)())
{
// uint 8
oa.write_character(to_char_type(0xCC));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint8_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint16_t>::max)())
{
// uint 16
oa.write_character(to_char_type(0xCD));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint16_t>(j.m_data.m_value.number_unsigned));
}
else if (j.m_data.m_value.number_unsigned <= (std::numeric_limits<std::uint32_t>::max)())
{
// uint 32
oa.write_character(to_char_type(0xCE));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint32_t>(j.m_data.m_value.number_unsigned));
}
else
{
// uint 64
oa.write_character(to_char_type(0xCF));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.number_integer));
write_number(static_cast<std::uint64_t>(j.m_data.m_value.number_unsigned));
}
break;
}
Expand Down
57 changes: 57 additions & 0 deletions tests/src/unit-msgpack.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2475,3 +2475,60 @@ TEST_CASE("MessagePack lengths beyond UINT32_MAX cannot be serialized")
}
#endif
}

TEST_CASE("MessagePack numbers use the active union member (see #5644)")
{
// when number_integer_t is narrower than number_unsigned_t, to_msgpack()
// used to read the union member that was not the active one, writing
// wrong bytes for some values; std::int64_t/std::uint64_t (the default
// types, where both members have the same width) were not affected
using int32_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int32_t, std::uint64_t, double>;
using int16_json = nlohmann::basic_json<std::map, std::vector, std::string, bool, std::int16_t, std::uint64_t, double>;

SECTION("number_integer_t = std::int32_t")
{
SECTION("6442450944 (uint 64; the low 32 bits used to be sign-extended)")
{
const int32_json j = 6442450944ULL;
CHECK(j.is_number_unsigned());

std::vector<uint8_t> const expected{0xcf, 0x00, 0x00, 0x00, 0x01, 0x80, 0x00, 0x00, 0x00};
const auto result = int32_json::to_msgpack(j);
CHECK(result == expected);
CHECK(int32_json::from_msgpack(result) == j);
}

SECTION("4294967496 (uint 64; the low 32 bits used to be the whole value)")
{
const int32_json j = 4294967496ULL;
CHECK(j.is_number_unsigned());

std::vector<uint8_t> const expected{0xcf, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0xc8};
const auto result = int32_json::to_msgpack(j);
CHECK(result == expected);
CHECK(int32_json::from_msgpack(result) == j);
}
}

SECTION("number_integer_t = std::int16_t, 98304 (uint 32)")
{
const int16_json j = 98304ULL;
CHECK(j.is_number_unsigned());

std::vector<uint8_t> const expected{0xce, 0x00, 0x01, 0x80, 0x00};
const auto result = int16_json::to_msgpack(j);
CHECK(result == expected);
CHECK(int16_json::from_msgpack(result) == j);
}

SECTION("default types (std::int64_t/std::uint64_t) are unaffected")
{
const json j = 4294967496ULL;
CHECK(j.is_number_unsigned());

std::vector<uint8_t> const expected{0xcf, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0xc8};
const auto result = json::to_msgpack(j);
CHECK(result == expected);
CHECK(json::from_msgpack(result) == j);
}
}
Loading