Skip to content

fix(core): replay OpenAI reasoning statelessly#34013

Open
kitlangton wants to merge 1 commit into
devfrom
fix/openai-stateless-reasoning-continuation
Open

fix(core): replay OpenAI reasoning statelessly#34013
kitlangton wants to merge 1 commit into
devfrom
fix/openai-stateless-reasoning-continuation

Conversation

@kitlangton

Copy link
Copy Markdown
Contributor

Summary

  • resolve Core catalog @ai-sdk/openai models through the LLM OpenAI provider facade so they inherit its stateless Responses defaults
  • preserve an explicit catalog store override as a semantic provider option
  • cover a deterministic encrypted-reasoning tool continuation and its explicit stateful counterpart

Fixes #33999.

Production root cause

Core resolved catalog models using @ai-sdk/openai directly from the low-level OpenAIResponses.route. That bypassed the OpenAI provider facade, which owns the default store: false policy.

Frank's custom model requested reasoning.encrypted_content, and the first Responses request successfully produced reasoning plus a tool call. The same-turn continuation then reconstructed that reasoning with store still undefined. The Responses lowerer treats every value other than explicit false as stateful, so it emitted only:

{ "type": "item_reference", "id": "rs_..." }

OpenAI rejected that continuation because items are not persisted when store: false.

Why encrypted replay is required

A tool call does not end the user turn. After the local tool result is available, the next Responses request must carry the reasoning state that led to that call.

Under store: false, the provider cannot resolve an rs_* handle from server-side storage. The continuation must therefore replay the complete locally retained item:

{
  "type": "reasoning",
  "id": "rs_...",
  "summary": [],
  "encrypted_content": "..."
}

Routing catalog OpenAI models through the existing provider facade supplies semantic store: false, so the protocol selects that encrypted replay path. Explicit store: true remains supported and continues to emit an item reference.

Tests

Passed:

  • bun run test test/session-runner-model.test.ts test/session-runner-message.test.ts test/session-runner.test.ts in packages/core: 99 passed
  • bun run test in packages/llm: 281 passed, 30 skipped
  • bun run typecheck in packages/core
  • bun run typecheck in packages/llm
  • repository-wide bun turbo typecheck push hook: 29 packages passed
  • bunx prettier --check packages/core/src/session/runner/model.ts packages/core/test/session-runner-model.test.ts
  • bunx oxlint packages/core/src/session/runner/model.ts packages/core/test/session-runner-model.test.ts: zero errors; one pre-existing consistent-return warning in unchanged apiKey code
  • git diff --check

The new regression asserts that a custom catalog model with encrypted reasoning and no explicit store:

  • resolves with store: false
  • replays reasoning.encrypted_content
  • carries the function-call output
  • emits no bare item_reference

It also asserts that explicit store: true still emits the reasoning item reference.

The full Core package run completed 1,023 of 1,024 tests. The unrelated filesystem watcher test failed because macOS reported a rewrite of .git/HEAD as add instead of the expected change; the same failure reproduced when that test file was run alone.

Provider failure preservation

This is complementary to the separate provider-failure preservation work from f8e5b12f41 and its evolved failed-turn metadata handling already on dev. Failed turns continue to discard provider-native continuation metadata; this PR fixes successful same-turn tool continuation by choosing stateless encrypted replay at model resolution.

This PR does not include the transient stream retry work or any files from #34010.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Core catalog OpenAI models skip stateless Responses defaults

1 participant