Skip to content

fix(logging): preserve JSON logs on serialization errors - #8173

Merged
Vidit-Ostwal merged 2 commits into
crewAIInc:mainfrom
Winoren:fix/preserve-json-logs-on-serialization-error
Oct 23, 2026
Merged

Vidit-Ostwal merged 2 commits into
crewAIInc:mainfrom
Winoren:fix/preserve-json-logs-on-serialization-error

Conversation

@Winoren

@Winoren Winoren commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Related issue

Fixes #8172.

Summary

A failed JSON log attempt can destroy earlier log records: FileHandler opens the destination with w before json.dump discovers unsupported or circular metadata. The partially written file is then treated as malformed by the next call, which replaces the history.

Serialize the complete payload before opening the destination. Serialization errors still surface through the existing ValueError wrapper, with the previous file untouched and no partial file created for a failed first record. The file lock, JSON indentation and trailing newline remain in place. This change addresses serialization failures, without introducing crash-atomic writes for filesystem failures.

Four regression cases use real temporary files and cover unsupported/circular metadata with and without existing history, then verify a later valid log retains earlier records. A normal JSON metadata control and existing pickle tests also pass.

Verification

  • Tests added for the changed behavior.
  • Relevant tests and source checks pass locally.

Python 3.13.12 / macOS arm64, repository-lockfile dependencies:

# New tests + unchanged production: 4 failed, 5 passed.
# Fixed implementation: 9 passed.
uv run --frozen --package crewai pytest lib/crewai/tests/utilities/test_file_handler.py -q

uv run --frozen --package crewai ruff check lib/crewai/src/crewai/utilities/file_handler.py
uv run --frozen --package crewai ruff format --check lib/crewai/src/crewai/utilities/file_handler.py
uv run --frozen --package crewai mypy lib/crewai/src/crewai/utilities/file_handler.py
git diff --check

Tests ran with CREWAI_TELEMETRY_DISABLED=true and OTEL_SDK_DISABLED=true. Full monorepo tests were not run. No provider/model calls. Test-file formatting follows its existing style; repository Ruff configuration excludes tests.

Additional context

AI assistance: Codex investigated the problem, prepared the fix and tests, and ran the checks above.


Note

Low Risk
Narrow change to JSON file logging in FileHandler; successful writes stay the same format, with lower risk of accidental log data loss on bad metadata.

Overview
Fixes JSON log history loss when a log entry cannot be serialized. FileHandler now runs json.dumps on the full append payload before opening the log file for write, so unsupported or circular metadata still raises the existing ValueError without truncating prior records or leaving a partial new file.

Adds regression tests for serialization failures with and without existing history, plus a happy-path check that serializable metadata appends correctly and the trailing newline is preserved.

Reviewed by Cursor Bugbot for commit 8aa19d4. Bugbot is set up for automated code reviews on this repo. Configure here.

@strix-security

strix-security Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 57 commits after the last Strix review (6faba6b). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

No security issues found.

Review summary

Reviewed the two changed files: lib/crewai/src/crewai/utilities/file_handler.py and its test file. The production change is a minimal, well-scoped bug fix that serializes the complete JSON payload (json.dumps) before opening the destination file for writing, so a serialization failure (unsupported or circular metadata) no longer truncates or destroys previously written log records. The change introduces no new input sources, sinks, or data flows: JSON serialization is inherently injection-safe, the existing store_lock file lock is unchanged and still wraps the entire read-modify-write operation, and no authentication, authorization, cryptographic, path-handling, or deserialization logic is affected. The test file only adds regression coverage for the fixed behavior. No security vulnerabilities were identified in the changed code.

Updated for 6faba6b.


Reviewed by Strix
Re-run review · Configure security review settings

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8347085f-63e0-4ddf-b386-d6fab2aeef50
📥 Commits

Reviewing files that changed from the base of the PR and between 6faba6b and 16d4670.

📒 Files selected for processing: 2
  • lib/crewai/src/crewai/utilities/file_handler.py
  • lib/crewai/tests/utilities/test_file_handler.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📝 Walkthrough

FileHandler.log now serializes JSON log entries before writing them to the destination. Tests cover unsupported and circular metadata, preservation of existing files after serialization failures, subsequent valid logging, and trailing newlines.

Changes: JSON logging

Layer / File(s) Summary
Serialize log entries and verify file behavior
lib/crewai/src/crewai/utilities/file_handler.py, lib/crewai/tests/utilities/test_file_handler.py
FileHandler.log serializes the JSON payload before writing it. Tests verify that serialization failures leave existing files unchanged or new files absent, and that valid entries are logged afterward. They also check parsed records and the final newline.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 16d46

Serialization failures occur before the log file is opened, and the added tests cover file preservation and subsequent logging. No actionable merge risk is established.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: preserving JSON logs when serialization fails.
Description check Passed The description includes the related issue, explains the cause and solution, documents verification commands and results, states scope limitations, and provides additional context.
Linked Issues check Passed Issue #8172 requires failed JSON serialization to preserve existing bytes, leave no file after a failed first record, retain history after a later valid record, and preserve the existing wrapper, lock…
Out of Scope Changes check Passed The production change only changes JSON serialization before the write. The test changes directly cover issue #8172 and preserve existing pickle coverage. No unrelated production behavior or crash-ato…
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Oct 15, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@Vidit-Ostwal Vidit-Ostwal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Vidit-Ostwal
Vidit-Ostwal force-pushed the fix/preserve-json-logs-on-serialization-error branch from 16d4670 to f37eebc Compare October 23, 2026 14:09
@Vidit-Ostwal
Vidit-Ostwal merged commit 5e9be90 into crewAIInc:main Oct 23, 2026
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Failed JSON log serialization corrupts earlier log records

2 participants