fix(core): replay OpenAI reasoning statelessly#34013
Open
kitlangton wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@ai-sdk/openaimodels through the LLM OpenAI provider facade so they inherit its stateless Responses defaultsstoreoverride as a semantic provider optionFixes #33999.
Production root cause
Core resolved catalog models using
@ai-sdk/openaidirectly from the low-levelOpenAIResponses.route. That bypassed the OpenAI provider facade, which owns the defaultstore: falsepolicy.Frank's custom model requested
reasoning.encrypted_content, and the first Responses request successfully produced reasoning plus a tool call. The same-turn continuation then reconstructed that reasoning withstorestill undefined. The Responses lowerer treats every value other than explicitfalseas stateful, so it emitted only:{ "type": "item_reference", "id": "rs_..." }OpenAI rejected that continuation because items are not persisted when
store: false.Why encrypted replay is required
A tool call does not end the user turn. After the local tool result is available, the next Responses request must carry the reasoning state that led to that call.
Under
store: false, the provider cannot resolve anrs_*handle from server-side storage. The continuation must therefore replay the complete locally retained item:{ "type": "reasoning", "id": "rs_...", "summary": [], "encrypted_content": "..." }Routing catalog OpenAI models through the existing provider facade supplies semantic
store: false, so the protocol selects that encrypted replay path. Explicitstore: trueremains supported and continues to emit an item reference.Tests
Passed:
bun run test test/session-runner-model.test.ts test/session-runner-message.test.ts test/session-runner.test.tsinpackages/core: 99 passedbun run testinpackages/llm: 281 passed, 30 skippedbun run typecheckinpackages/corebun run typecheckinpackages/llmbun turbo typecheckpush hook: 29 packages passedbunx prettier --check packages/core/src/session/runner/model.ts packages/core/test/session-runner-model.test.tsbunx oxlint packages/core/src/session/runner/model.ts packages/core/test/session-runner-model.test.ts: zero errors; one pre-existingconsistent-returnwarning in unchangedapiKeycodegit diff --checkThe new regression asserts that a custom catalog model with encrypted reasoning and no explicit
store:store: falsereasoning.encrypted_contentitem_referenceIt also asserts that explicit
store: truestill emits the reasoning item reference.The full Core package run completed 1,023 of 1,024 tests. The unrelated filesystem watcher test failed because macOS reported a rewrite of
.git/HEADasaddinstead of the expectedchange; the same failure reproduced when that test file was run alone.Provider failure preservation
This is complementary to the separate provider-failure preservation work from
f8e5b12f41and its evolved failed-turn metadata handling already ondev. Failed turns continue to discard provider-native continuation metadata; this PR fixes successful same-turn tool continuation by choosing stateless encrypted replay at model resolution.This PR does not include the transient stream retry work or any files from #34010.