Skip to content

fix(logger): keep string messages that are valid JSON scalars as strings - #8838

Open
saquibjawedbit wants to merge 1 commit into
aws-powertools:developfrom
saquibjawedbit:fix/logger-json-scalar-messages
Open

saquibjawedbit wants to merge 1 commit into
aws-powertools:developfrom
saquibjawedbit:fix/logger-json-scalar-messages

Conversation

@saquibjawedbit

Copy link
Copy Markdown

Issue number:
closes #8828

Summary

Changes

LambdaPowertoolsFormatter._extract_log_message (aws_lambda_powertools/logging/formatter.py) tried json.loads on every string message and kept whatever came back. It now keeps the decoded value only when it is a dict or a list; for any other result (null, bool, number, JSON string) it keeps the original string. Decoding JSON objects and arrays into structured output, the documented feature, is unchanged.

Tests added in tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py, through the public Logger API only:

  • test_with_json_scalar_string_message_stays_string, parametrized with "null", "true", "123", "1e3", '"quoted"', "NaN"
  • test_with_json_object_or_array_string_message_is_decoded, a JSON object string and a top-level JSON array string are still decoded

No existing tests were changed.

User experience

Before:

logger.info("null")  # no "message" key in the output
logger.info("true")  # "message": true
logger.info("123")   # "message": 123
logger.info("1e3")   # "message": 1000.0

After, all four log "message": "<the original string>". logger.info('{"a": 1}') and logger.info('[1, 2]') still log structured {"a": 1} / [1, 2].

This is the bug fix as triaged in the issue. Other user-visible effects of the same rule, all covered by tests or checked by hand:

  • logger.info('"quoted"') now keeps the quotes instead of logging quoted.
  • logger.info("NaN") now logs "message": "NaN" instead of a bare NaN, which made the whole line invalid JSON.
  • A custom json_deserializer that returns something other than a dict/list (e.g. a tuple) now leaves the message as the original string.
  • Whitespace-padded objects (' {"a": 1} ') are still decoded; bytes messages are untouched. DatadogLogFormatter inherits the method, so it gets the same fix.

Verification

AI-assisted; I ran the tests below myself and pasted the output.

Local run on macOS, Python 3.13.15, environment from make dev.

New tests on develop, before the fix (uv run --locked pytest tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py -k "json_scalar or json_object_or_array"):

FAILED tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[null]
FAILED tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[true]
FAILED tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[123]
FAILED tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[1e3]
FAILED tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string["quoted"]
FAILED tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[NaN]
============ 6 failed, 2 passed, 39 deselected, 1 warning in 0.06s =============

Same tests after the fix (same command with -v):

tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[null] PASSED [ 12%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[true] PASSED [ 25%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[123] PASSED [ 37%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[1e3] PASSED [ 50%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string["quoted"] PASSED [ 62%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_scalar_string_message_stays_string[NaN] PASSED [ 75%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_object_or_array_string_message_is_decoded[{"order_id": 123, "items": ["a", "b"]}-expected0] PASSED [ 87%]
tests/functional/logger/required_dependencies/test_logger_powertools_formatter.py::test_with_json_object_or_array_string_message_is_decoded[["a", "b"]-expected1] PASSED [100%]
================= 8 passed, 39 deselected, 1 warning in 0.02s ==================

Logger unit + functional suites (uv run --locked pytest tests/unit/logger tests/functional/logger):

======================= 212 passed, 7 warnings in 0.38s ========================

Lint, format, types (make format-check lint-check, make mypy):

1417 files already formatted
uv run --locked ruff check aws_lambda_powertools tests examples
All checks passed!
uv run --locked mypy --pretty aws_lambda_powertools examples
Success: no issues found in 963 source files

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@saquibjawedbit
saquibjawedbit requested a review from a team as a code owner November 15, 2026 17:14
@boring-cyborg

boring-cyborg Bot commented Nov 15, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 15, 2026
@sonarqubecloud

Copy link
Copy Markdown

@leandrodamascena leandrodamascena added this to the 3.42.0 milestone Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logger size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: string messages that are valid JSON scalars are coerced or dropped

2 participants