Skip to content

fix(chatcmpl): clear pending thinking blocks when flushing an assistant message - #4089

Merged
seratch merged 1 commit into
openai:mainfrom
Kaif10:fix/thinking-blocks-cross-turn-leak
Aug 1, 2026
Merged

seratch merged 1 commit into
openai:mainfrom
Kaif10:fix/thinking-blocks-cross-turn-leak

Conversation

@Kaif10

@Kaif10 Kaif10 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Converter.items_to_messages keeps two pieces of cross-item reasoning state. pending_reasoning_content is cleared on flush, with a comment explaining why (prevents stale reasoning_content from contaminating later turns). Its sibling pending_thinking_blocks is not — it is only cleared where it is consumed, at the assistant-message and function-call branches.

So when a reasoning item is not directly followed by its own assistant message, its signed thinking blocks stay pending across an intervening user turn and are prepended to a later assistant message. Anthropic validates signature against the turn the block belongs to, so the next request either fails validation or replays a previous turn's private reasoning back to the model.

This is reachable without anything contrived: message_to_output_items drops the message item when the content is empty, so a Claude turn with extended thinking but no visible text produces only a reasoning item — the next history entry is the user's next message. It affects the paths that set preserve_thinking_blocks=True for Claude (litellm_model.py, any_llm_model.py).

Round-tripping a two-turn history through message_to_output_items → items_to_messages on main:

turn 1 output item types: ['reasoning']

{"role": "user", "content": "u1"},
{"role": "user", "content": "u2"},
{"role": "assistant", "content": [
   {"type": "thinking", "thinking": "TURN-1 PRIVATE THINKING", "signature": "SIG-TURN-1"},
   {"text": "a2", "type": "text"}]},
{"role": "user", "content": "u3"}

Turn 1's thinking block is attached to turn 2's assistant message. With this change the same history produces {"role": "assistant", "content": "a2"}.

Fix

flush_assistant_message now clears pending_thinking_blocks alongside pending_reasoning_content. The single call site that deliberately preserves reasoning state across a flush — the maybe_response_output_message branch, where a reasoning item legitimately belongs to the assistant message that follows it — still passes the opt-out, so that case is unchanged. The keyword is renamed from clear_pending_reasoning_content to clear_pending_reasoning since it now covers both pieces of state; it is a parameter of a local closure, not public API.

Test plan

Added test_thinking_blocks_do_not_leak_across_an_intervening_user_turn to the existing tests/models/test_anthropic_thinking_blocks.py, asserting the later assistant message carries no thinking block from the earlier turn. It fails on main (the assistant content is the thinking block plus the text instead of the text) and passes with this change.

  • tests/models/test_anthropic_thinking_blocks.py: 6 passed with the fix; 1 failed / 5 passed without it.
  • tests/models: 605 passed. Converter/litellm/reasoning-related selection: 341 passed.
  • ruff check and ruff format --check clean on both files; mypy reports no errors in either changed file.

Verification-script note: .agents/skills/code-change-verification/scripts/run.sh could not run here because make is not installed on this machine, so I ran the underlying commands directly. The full parallel suite shows 38 pre-existing failures with the change and 39 on a stashed baseline — all xdist/tracing flakes in testing_processor.py, unrelated to this diff.

Issue number

None — self-reported, found while reading the converter.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh — could not: make unavailable locally (see Test plan); ran the equivalent commands directly
  • I've confirmed all verification steps pass (format, lint, typecheck, affected tests; remaining failures verified pre-existing)
  • If using Codex, I've run /review before submitting this PR — n/a

AI assistance was used in preparing this change; I reviewed it, reproduced the leak and verified the fix locally.

…nt message

Signed thinking blocks belong to the assistant turn that produced them.
A reasoning item that is not directly followed by its assistant message
kept them pending across an intervening user turn, so they were prepended
to a later assistant message.
@seratch seratch added this to the 0.19.x milestone Aug 1, 2026
@seratch
seratch merged commit 29e99da into openai:main Aug 1, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants