Repository navigation
[Feature]: Keep the Anthropic prompt cache across mid-conversation reasoning effort changes (per-message effort) #6895
Description
Area
Provider adapters
What are you trying to accomplish?
A Codex conversation routed to an adaptive-thinking Claude model (Haiku 5.5, Sonnet 5.5, Opus 5.5, Fable 5.1) through the Anthropic adapter changes its reasoning effort between turns, for example to raise effort after a failed attempt. The goal is to change effort mid-conversation without paying a cache write for the whole conversation history again.
What prevents this today?
On OpenCodex 2.81.0 (/v1/responses → Anthropic Messages), an effort change only changes the top-level output_config.effort. Thinking stays {type: adaptive, display: summarized}, and tools, system, message order and cache_control placement do not depend on effort. So OpenCodex does not change the prefix itself. The miss comes from the upstream rule that a change of the top-level effort invalidates the messages cache.
Measured through the proxy (direct /v1/responses calls, fixed instructions and prompt_cache_key, same account, single send each):
| Case | Result |
|---|---|
| Single turn, about 12k-token prefix, effort high → xhigh → max → high → medium → low (Haiku 5.5 and Sonnet 5.5) | Every call after the first reads the whole prefix from cache; cache creation 0 |
| Multi-turn, replayed reasoning items, effort constant | Cache grows normally turn by turn |
| Multi-turn, replayed reasoning items, effort changes between turns | Cache reads stop at the first user message (about 11.6k tokens); everything after it is written again on each effort change |
| Multi-turn, reasoning items stripped from the replay, effort changes | Cache keeps growing (not a fix: dropping thinking costs quality) |
A second, smaller effect: with multi-agent guidance enabled and the v1 spawn_agent tool surface, reaching effort max/ultra injects a <multi_agent_mode> developer message into the message prefix (src/server/responses/collaboration.ts, the effort !== 'max' && effort !== 'ultra' early return). Crossing into or out of max therefore changes the prefix as well.
What should OpenCodex do?
Keep the cached prefix across mid-conversation effort changes on models that support Anthropic's per-message effort (beta header mid-conversation-output-config-2026-07-01, adaptive thinking required):
- Per conversation key (conversation id or
prompt_cache_key), remember the effort of the first request and keep the top-leveloutput_config.effortat that value for the rest of the conversation. - When a request's effort differs, insert
{role: "system", content: [], output_config: {effort}}intomessagesright before that turn's user message. Re-insert every earlier change point at the same position on later requests so the prefix stays byte-identical (Codex history carries no per-turn effort, so OpenCodex needs a per-conversation list of change points). - Send the beta header only for
api.anthropic.com, supported models and adaptive thinking (a request with thinking disabled is refused). Fall back to today's behaviour when the account or route does not accept the beta. - Make the v1 multi-agent guidance independent of the per-request effort, or inject it once per conversation, so a max turn does not change the prefix.
Example usage or interface
No new user-facing option is required. An opt-out config flag (for example anthropic.perMessageEffort: false) would help while the beta is new.
Alternatives or workarounds
- Keep effort fixed for the whole conversation (what we do for now).
- Start a new thread on escalation (loses context and writes the whole prompt again).
- Strip replayed reasoning (keeps the cache but drops thinking continuity).
Additional context
Not yet measured: when the cache grows again after one effort change with constant effort afterwards, tool-use loops, and whether the tools/system blocks stay cached on the effort-changing turn (the experiment had no tools). The wire bodies were not retained, so the exact upstream cause was not proven by byte comparison. Whether OAuth accounts accept the beta header also needs a check.
Checks
- I searched existing issues and documentation.
- This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
- I removed secrets and personal data.
Activity
Ingwannu commentedon Oct 10, 2026
The API capability is real, but this should remain a design item rather than a default-on adapter tweak. Anthropic's current per-message effort contract documents the dated beta value, model/platform eligibility, and empty-content system messages. A change takes effect for the applicable user turn with new input; tool-result-only turns do not start a new effort boundary. The mid-conversation message contract also distinguishes effort-only messages from content-bearing system messages and their placement rules. These docs do not establish acceptance by the OAuth subscription route or an arbitrary Anthropic-compatible gateway.
Two pieces need separate owner decisions:
prompt_cache_keyalone is not a sufficient history identity: retries, forks, concurrent requests, compaction, model/provider/account changes and truncated/replaced histories can share a key but require different change points. Specify request ownership, prefix matching, commit/rollback timing, TTL/byte/entry bounds, and behavior when continuity cannot be established. Earlier accepted change points must replay identically; never guess a missing historical effort from the latest global setting.For a focused proposal, reuse the current adapter/continuation owners and add synthetic wire-prefix tests for effort changes, tool loops, retries, concurrent branches, compaction, route/account replacement and unsupported beta responses. Define fallback before sending; a beta rejection is not blanket permission to replay a possibly processed request. Default-on rollout and live cache/quota acceptance are not approved here. Your observations are useful, but without retained comparable wire prefixes they do not isolate every cache miss. I have not made live calls, copied credentials, changed settings, or run a scan.
rkb32 commentedon Oct 11, 2026
I'd like to take this on. I'll post a design proposal here before writing any code. It will cover:
Ingwannu commentedon Oct 12, 2026
Thanks — please post that design proposal before implementing. The proposed scope matches the earlier review: bounded immutable effort history, explicit ownership/rollback/fallback, guidance policy kept separate, synthetic branch/retry/compaction coverage, and default-off opt-in.
Please base the proposal on the current
devand identify the existing adapter/continuation owners it reuses. Include a failure table for missing or replaced history, concurrent forks, account/provider changes and unsupported upstream capability. This is agreement to review a proposal, not approval of a new history store, default rollout, live subscription calls or changing the V1 delegation policy. No need to post private prompts, credentials or full request captures.rkb32 commentedon Oct 13, 2026
Here's the plan, based on the current
devbranch (1edd55a). Opt-in, off by default. @IngwannuFiles I'd change:
src/adapters/anthropic.ts,src/adapters/anthropic/beta-allowlist.ts,src/claude/inbound.tsThe idea: when a chat changes effort partway through, add a small note at that point instead of changing the setting for the whole chat. The earlier part of the chat stays the same, so the cache still works.
Limits: only for Anthropic's direct API and the models that support this. It waits for #7042 to be fixed first.
Measurement: log cache read and creation token counts per request, with and without an effort change, before turning it on by default.
Tests: synthetic only, no live calls.
Open question: should saved history live in memory with an expiry, or in the existing storage? What expiry and size limits do you want?
rkb32 commentedon Oct 13, 2026
Correction to my last comment: #7042 is fixed (#7067), so it’s no longer a blocker. @Ingwannu @lidge-jun
The change I still need to make is adding the per-message effort beta (mid-conversation-output-config-2026-07-01) to the allowlist in src/adapters/anthropic/beta-allowlist.ts. It isn’t in the list on dev yet. I’d add it as part of this work, limited to the models that support it.
rkb32 commentedon Oct 19, 2026
@Ingwannu @lidge-jun Update on #6895.
#7042 is fixed (#7067), so it no longer blocks this. #7341 (merged Oct 19) changes
src/adapters/anthropic.ts,src/adapters/anthropic/passthrough.ts, and addssrc/adapters/anthropic/cache-retention.ts. I'll rebase on it before any change.Proposed next step, once the design is confirmed: add
mid-conversation-output-config-2026-07-01tosrc/adapters/anthropic/beta-allowlist.ts, sent only for Fable 5.1, Mythos 5.1, Opus 5.5, Opus 5, Sonnet 5.5, and Haiku 5.5 (not Fable 5, per Anthropic's effort docs), with a test.The history design from my Oct 13 comment is still the open part. Question: can you confirm that design, including the failure table, before I write the allowlist change?
github-actions commentedon Oct 19, 2026
Automated translation bookkeeping — detected language: English.
Ingwannu commentedon Oct 19, 2026
Rechecked against dev d11f067 and the current official effort contract on 2026-10-19. Your six-model API eligibility list and Fable 5 exclusion match that contract; tool-result-only user turns still must not create a new-input effort boundary. The system-message contract also distinguishes empty effort-only entries from content-bearing system messages.
The Oct 13 failure table is directionally useful but is not yet sufficient to approve the history design. Please make its actions precise before implementation:
prompt_cache_keyalone cannot own this state. Specify concurrent update fencing and commit/rollback timing.Rebase over #7341, keep guidance/Max delegation unchanged, and retain opt-in/default-off behavior. A narrowly tested allowlist change can be proposed separately, but it does not approve a history store, OAuth/gateway acceptance or live quota experiments. The overall design remains needs-design pending those details and owner disposition.
rkb32 commentedon Oct 19, 2026
@Ingwannu Proposal for #6895, following your latest points.
src/adapters/anthropic.ts.src/responses/reasoning-replay-cache.ts, L125–142): provider name, destination identity, adapter name, model ID, and process-local credential identity. Plus the cohort fromsrc/codex/lineage.ts. The credential identity is a hash of the key or account slot, never the raw key or token. It resets on restart, which matches a memory-only store.message_stopor error. Requests arriving while it's set skip the optimization.message_stop. A stream without it is a failure.api.anthropic.com, for non-OAuth accounts.message_stop: error returned, no automatic resend, optimization off for that cohort until the next clean turn.message_stopreceived: change point recorded.claudeCode.perMessageEffort, defaultfalse.The allowlist change would be a separate proposal. This doesn't ask for approval of a history store, OAuth or gateway acceptance, or live quota tests.
Ingwannu commentedon Oct 19, 2026
The new proposal makes ownership/bounds and commit timing more concrete. I do not accept treating an arbitrary missing record as a fresh history: restart, TTL/LRU eviction, truncation/compaction and a genuinely new conversation can all have no record, and the current adapter cannot reconstruct earlier effort breakpoints from that absence. An in-flight marker avoids overlap but does not establish branch/attempt identity or successful rollback after an ambiguous partial turn.
For a reviewable design, distinguish a provably new transcript from missing/evicted prior state using captured request lineage plus an immutable matched-prefix receipt; disable the optimization when that evidence is absent or a change point cannot be placed exactly. Keep a sent-without-message_stop outcome disabled until a subsequent baseline round trip has re-established a complete matched prefix, rather than silently starting a new optimized history. Specify how concurrent/replayed branches and credential-generation changes fence that receipt. The proposed 64-entry/512 KB/1-hour limits are useful bounds, not reconstruction evidence. Please keep the shared routing/credential identity owner outside a monolithic adapter implementation and submit this as the design contract first. This comment does not approve a new history store, model allowlist expansion, OAuth/gateway acceptance or live quota tests.
rkb32 commentedon Oct 19, 2026
@Ingwannu Design contract for review: https://gist.github.com/rkb32/69ee9faf767de6aae0369244aaa0d654 — It answers your Oct 19 points: a named owner and identity, fixed transcript and frame hashes, fencing for concurrent and replayed requests, commit after
message_stop, no restart after an uncertain send, a pre-send check for the beta, and a failure table. Open items are in section 13. For this design, the proxy sets the beta throughmergeAnthropicBetaHeader(defined insrc/providers/anthropic-fast.tsL68, called atsrc/adapters/anthropic.tsL1229), so it needs no change to the allowed-beta list. An OAuth token refresh marks the history unconfirmed, so the next request is off until a baseline round trip. One decision for you: we keep the OAuth generation out of the scope key (per #1926). If you want it in, every refresh turns the history off.Ingwannu commentedon Oct 20, 2026
I read the linked design contract against current dev
12025981e5a304cdb6b1b54f525ba8554f623977. Separating the owner and proxy-set beta path is useful; this does not need a caller passthrough allowlist change merely to add a proxy-owned header. The design is not yet approved:Keep the durable credential scope key aligned with the existing shared identity owner; do not add rotating OAuth tokens to that global key. Capture generation on each physical attempt and fence the local receipt/commit through the existing owner instead. Identifying that concrete propagation seam (your open item 7) is required. Bounds and cache-read counters do not establish transcript equality. Please revise the contract and synthetic failure table first; this is not approval of a new store, OAuth capability, billing assumption or live probe.
rkb32 commentedon Oct 21, 2026
@Ingwannu Revision 5 of the design contract is at https://gist.github.com/rkb32/69ee9faf767de6aae0369244aaa0d654. It covers your Oct 20 comment and PR #7473 (merged 2026-10-21), which adds
configuration_updateitems on OpenAI rows only.Three decisions are needed from you:
Ingwannu commentedon Oct 22, 2026
I read revision 5 and checked the cited identity/parser boundaries against current dev
bf4eb1209672. Keeping the feature completely off without a positive new-transcript receipt fixes the earlier missing-record inference, but the design is not approved for activation.For the requested choices: prefer a positively owned client/lifecycle receipt if one can actually be propagated; do not create a persisted thread-start store merely to infer one from the first request observed. Such a store still cannot prove that an existing/restarted/compacted client transcript is new. Until that seam is demonstrated, an offline contract/test slice that leaves every production request unchanged is the appropriate next step.
A credential-generation mismatch should invalidate the local optimization receipt. Do not weaken the durable shared identity or silently re-anchor the same chat after refresh; a new anchor needs the same positive lineage evidence. This is an acceptable fail-closed constraint for a prototype, not acceptance of permanently degraded production behavior.
Two contract gaps also remain: (1) “pending entries are never evicted” must still obey the 64-entry / 512-KiB hard bounds, including pending transcripts and queued commit data; specify admission refusal and bounded cleanup. (2) section 9's unoptimized-send rule must invalidate any matching old parent receipt at the physical-send boundary, even when that send fails or never yields an echoable next prefix. Not recording the reply alone is not that invalidation. Include a same-parent retry after an ambiguous off send.
Do not silently pin a later caller's smaller explicit output cap upward to an old anchor merely for caching. The cap/thinking contract needs an explicit compatibility decision or optimization-off path. Beta/cache behavior, exact client item echo and developer-role handling remain unverified; no live test, new store, model allowlist or OAuth/gateway capability is authorized by this review.
rkb32 commentedon Oct 22, 2026
@Ingwannu Point-by-point on your Oct 22 comment. Contract revision 8: https://gist.github.com/rkb32/69ee9faf767de6aae0369244aaa0d654
history_initializationvalue (New, ColdResume, and others) in its turn metadata, and sends it in thex-codex-turn-metadataheader (openai/codex main40db507e). The opencodex proxy does not read it yet. Whether real requests carry it is not confirmed, and that needs a live test. Until then the feature stays off. No store is added.Not approving any live probe, new store, OAuth capability, or model allowlist change. Could you confirm which open items (1, 5, 6) you want resolved before the activation review?
Ingwannu commentedon Oct 23, 2026
I read revision 8 of the contract and checked current proxy dev
26415a90e4. All three open items (1, 5 and 6 in your comment) remain prerequisites to activation review. The proxy already carries/parses turn metadata for other purposes, including WebSocket frame headers, but that is not a positivehistory_initialization/compaction receipt for this feature; the current source does not consumehistory_initialization.Two remaining contract inconsistencies should be resolved in an offline test slice first. Rule 4 exempts a physically sent, unoptimized replay from invalidation; sections 9/11 explicitly preserve that exception. An input hash matching an earlier turn does not prove that this new physical attempt cannot diverge or that a later client continuation belongs to the still-stored optimized branch. Either invalidate the applicable receipt at that send boundary too, or provide a concrete branch/attempt proof and regression for the exception. Also section 14's changed-thinking retry expectation conflicts with section 7's first-decision-wins order: rule 3b precedes rule 4, without the otherwise-a-match qualification asserted by the test prose. Define one ordered algorithm and derive the table/tests from it.
Generation fences and accounting pending/streaming data inside the hard limits are useful revisions, but are design text, not executed evidence. Keep production unchanged; an offline state-machine/transport-fixture slice can establish the failure table without a new live store, capability/allowlist change or provider call. No activation approval is given here.
rkb32 commentedon Oct 23, 2026
@Ingwannu Point by point on your Oct 23 comment. Contract and offline slice: https://gist.github.com/rkb32/69ee9faf767de6aae0369244aaa0d654 (the slice has 68 tests passing locally; it is not wired into opencodex and makes no provider calls).
history_initialization. The slice tests exact-match "New" and treats ColdResume, WarmFork, ColdFork, SuppliedHistory, Cleared, absent, malformed, and restart as off. Not shown: what real requests carry. That needs a live capture, which we are not requesting.Decisions for you: the output-limit comparison, committed-output canonicalisation, the compaction marker, the clear-400 criteria, chat lineage in the replay key, and the 128 KB buffer value.
Not requested: live probes, a new store, OAuth capability, or an allowlist change.
Ingwannu commentedon Oct 25, 2026
I read the updated contract and your point-by-point reply. Rule 3b now explicitly precedes replay, rule 4 deletes the matched entry and parent before an unoptimized replay, and section 9 deletes the parent on every physically sent off turn. That resolves the two ordering/receipt-retention objections in my Oct 23 comment at the design-text level. I have not independently run the offline slice or validated its production wiring.
For the next offline revision:
These are offline review constraints. They do not approve activating the feature, live traffic capture, a history store integration, or an OAuth/provider capability claim. Both lifecycle/compaction activation conditions and concrete output canonicalization remain unresolved.
rkb32 commentedon Oct 26, 2026
@Ingwannu Follow-up on your Oct 25 comment.
Changed since last time:
previous_response_idare off.Checked against the sources (verified):
{role: "system", content: [], output_config: {effort}}with betamid-conversation-output-config-2026-07-01. Takes effect from the next user turn with new input (Anthropic effort docs).claude-fable-5-1. Anthropic's pages say Mythos 5.1 is the same model. Fable is off in the slice until you confirm that.id,status, and output text. Byte-identical echo is not verified.Still tests only: the positive control compares the same objects after a JSON round trip. It is a shape check, not a real client echo.
Decisions for you:
history_initializationreaches the proxy on real ordinary and WebSocket turns, and (b) that the client echo is byte-identical. Proposed scope: one test account, three short turns, no credentials in logs. We won't run it without your yes.Not requested: a new store, OAuth capability, or allowlist change.
I'd like to get this ready to build, so could you list the remaining changes you expect, in priority order? That way we work on what matters most first.
lidge-jun commentedon Oct 28, 2026
Maintainer triage design proposal. This is a proposal for discussion, not approval. I fetched
origin/devand reviewedfdc6dd82d930295880dcf6d10bbb9f2c61994574.Current state. Adaptive requests still serialize the current effort at top level and derive an omitted output cap from it, while preserving an explicit caller cap (src/adapters/anthropic.ts:1128–1149). Positional
configuration_updateitems are deliberately dropped on translated routes, with #6895 named as a separate decision (src/responses/parser.ts:332–336). #7341's first-party one-hour cache default is present (src/adapters/anthropic/cache-retention.ts:16–25); it does not implement effort history. @rkb32's latest contract and offline slice and @Ingwannu's reviews already establish the design constraints. In particular, the ordering and unoptimized-send invalidation objections were resolved at the design-text level. They should not be reopened as missing work. The old caller-beta-allowlist proposal is also superseded by the proxy-owned header approach.Options and recommendation. We can keep current stateless behavior; finish the isolated state-machine and real-boundary fixture comparison with activation disabled; or integrate default-off, bounded process-local history after the unresolved ownership contract is accepted. I recommend the second option now. If runtime work later proceeds, a first-party API-key-only slice with unchanged effective output limits would reduce the initial capability surface. Neither OAuth nor compatible gateways inherit acceptance from the public API documentation.
Remaining decisions and risks. Priority one is a positively owned chat/new-transcript and compaction receipt plus one projection binding input to the actual assistant/tool output the client can echo. Raw metadata transport is not that receipt. Priority two is adapter-derived eligibility: the adaptive branch's internal budget calculation is not an emitted
thinking.budget_tokens; distinguish it from the fixed-budget branch before deciding whether a thinking change is compatible. Keep the already stated rule that differing effective output limits disable optimization; never raise a smaller cap for cache reuse. Then settle pending-slot expiry and buffer bounds. Fork contamination, stale credential generations, retained transcript bytes and extra physical sends are the main correctness, privacy and cost risks. The existing replay owner separates durable account identity from rotating generation (src/server/responses/core-replay.ts:269–281).Smallest next slice. Adapt the existing offline artifact to current Bun fixtures and show a real serializer-to-next-request positive control, ordinary/WebSocket metadata negatives, compaction, concurrent forks and ambiguous-send deletion. Reuse
structure/transports/anthropic-thinking-recovery.mdownership; do not repurpose its memo as effort history. Production integration must wait for owner acceptance of receipt propagation, canonicalization and eligibility. The latest live-capture request remains undecided; this analysis authorizes none and claims no cache savings.rkb32 commentedon Oct 28, 2026
@Ingwannu @lidge-jun Thanks for the triage. Agreed, we'll finish the isolated state machine and fixture work first, with activation off. Integration waits until the ownership contract is accepted.
Here's what we're going with, unless you object:
thinking.budget_tokenscounts as a thinking change. The adaptive branch's internal budget doesn't, so that branch stays eligible.Next up is an offline fixture on current Bun, with a real serializer-to-next-request check. I'll cover compaction, concurrent forks, ambiguous sends, stale credential generations, and retained transcript bytes.
Could you assign this issue to me? I'll take the isolated work from here.
No live capture until someone approves it.
Ingwannu commentedon Oct 29, 2026
The isolated/off-by-default fixture work can stay with you, within the owner-review boundary already recorded above. This is ownership of the offline investigation, not approval of the activation/integration contract or a live account capture. Please preserve the emitted
thinking.budget_tokensdistinction and exercise the receipt across real serializer-to-next-request fixtures, compaction, forks, ambiguous sends and stale credential generations. Keep the 10-minute / 128-KB limits explicit as proposed fixture policy until the owner accepts them.I checked GitHub assignment eligibility for
rkb32; the repository assignee endpoint currently returns 404, so I cannot record you as an actual GitHub assignee with the current repository eligibility. I have not sent an invitation or changed access. This comment records your volunteered ownership; formal assignment remains pending repository eligibility. No live capture was authorized or executed.rkb32 commentedon Oct 29, 2026
@Ingwannu Update on the offline work. Activation is still off, and there's no live capture.
I ported the slice to Bun fixtures on
devatfdc6dd82. Every body and output item comes from the real Anthropic adapter, Responses parser, and bridge. That's 21 fixtures plus 95 slice tests, covering the positive control, the emittedbudget_tokensrule, compaction, forks, ambiguous sends, stale credential generations, and retained bytes. Files are in the gist: https://gist.github.com/rkb32/69ee9faf767de6aae0369244aaa0d654Two fixes came out of it. A pending slot now keeps only hashes, never transcript text. Effort
noneis now off: on Opus 5.5 it keeps the samethinkingfield, so the thinking rule alone missed it.One question. Codex never sends
max_output_tokens, so the adapter derivesmax_tokensfrom effort (Opus 5.5: low 12288 up to max 40192). Under the output-limit rule, every effort change is off, so the feature never fires for default Codex users. A fixture shows an alternative: send each turn'smax_tokensexactly as today and compare only the caller's cap. No cap is raised or lowered, andmax_tokensisn't in Anthropic's cache-invalidation list. Does the rule cover only caller or configured caps, or this derived default too?Ingwannu commentedon Oct 29, 2026
The current rule includes the derived effective
body.max_tokens, not only an explicit caller/configured cap. That was the meaning of my Oct 25 comparison against the current request's adapter output, and @lidge-jun retained differing-effective-limit => optimization-off in the Oct 28 proposal.I checked the current adaptive branch in
src/adapters/anthropic.tsat dev9b78a16f4fe0c300527e6b183052407269bb941d: it preserves an explicit cap and otherwise derivesmax_tokensfrom the current effort and configured default. Your conclusion about the default omitted-cap sequence is therefore a real eligibility limitation, not a reason to substitute a caller-only comparison silently. Comparing only the caller cap while retaining each turn's ordinary emitted cap is a distinct compatibility proposal; leaving wire limits unchanged is useful evidence, but absence from a cache-invalidation list does not settle the accepted history/eligibility contract. That relaxation needs @lidge-jun's explicit decision.For the offline slice, retain a negative control for omitted-cap effort changes whose derived effective caps differ, plus a serializer-based positive control where the same explicit cap naturally produces equal effective limits. Do not change defaults or recommend increasing a cap merely to obtain eligibility. Compare emitted
thinking.budget_tokensseparately from an adaptive branch's internal calculation; effortnonemust stay off even when its emitted thinking object happens to match.I read the updated contract's cap and bounds sections and the corresponding adapter path; I have not independently executed the reported 21 fixtures/95 slice tests or validated real client echo. Pending-slot hashes and proposed TTL/buffer numbers remain offline evidence/proposals. Activation, integration and live capture remain unapproved; no need to reopen the previously resolved ordering/invalidation objections.
rkb32 commentedon Oct 31, 2026
@Ingwannu @lidge-jun thanks for the review. Some replies, a couple of corrections, and a test run.
Derived max_tokens: yep, you're right, the derived limit counts. The slice checks the limit the adapter actually sends, derived one included.
Corrections:
Test run today: 29/29 fixtures pass (Bun 1.4.4, on Linux) and 111/111 slice tests pass. The fixtures ran on Linux because the Windows test lock refuses on my machine. Contract and tests are in the gist: https://gist.github.com/rkb32/69ee9faf767de6aae0369244aaa0d654
Not done yet: real client wire behavior, the positive receipt, and real-serializer resume/restart cases. Activation, integration, and live capture are still off.
@lidge-jun quick question: can we accept 10-minute / 128 KB as fixture-only policy, with activation still off? I'd say yes, since it keeps everything offline and easy to undo.
rkb32 commentedon Nov 4, 2026
@lidge-jun @Ingwannu bump on this, plus an update. I'm doing this part alone right now, so I need one of you to pick up the owner-side decisions.
Done since Oct 31:
Not done yet:
Open question for lidge-jun: can I keep 10-minute / 128 KB as fixture-only values for now, with activation still off? If you want different numbers, tell me which and I'll change the fixtures.