Skip to content

fix(provider): keep the Anthropic cache breakpoint off ephemeral turns - #1008

Merged
TheGreatAxios merged 2 commits into
mainfrom
cl-6913-anthropic-conversation-cache-breakpoint-lands-on-ephemeral
Sep 14, 2026
Merged

TheGreatAxios merged 2 commits into
mainfrom
cl-6913-anthropic-conversation-cache-breakpoint-lands-on-ephemeral

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Summary

  • The Anthropic Messages cache breakpoint lands on the last persisted non-assistant turn instead of the ephemeral tail, so director nudges no longer bust the cached prefix.
  • Covers the stock anthropic provider and both first-party Messages adapters (zen-messages, opencode-go-messages); system and tools breakpoints stay untouched.

Verification

  • bun run check passes: lint, typecheck, build, and guard with 7309 tests and 0 failures.
  • New src/provider/anthropic-cache-breakpoint.test.ts pins the breakpoint placement for all three providers, system/tools preservation, byte-identical passthrough without ephemeral turns, and non-Anthropic passthrough.

Fixes CL-6913

Ephemeral turns are appended to the prompt but never persisted, so a breakpoint on the tail re-bills the full prefix every call. Move the breakpoint to the last persisted non-assistant turn instead.
@linear-code

linear-code Bot commented Sep 13, 2026

Copy link
Copy Markdown

CL-6913

@TheGreatAxios

Copy link
Copy Markdown
Collaborator Author

Critic review · no blocking findings

Moves the Anthropic messages cache breakpoint off the ephemeral nudge tail onto the last persisted non-assistant message, via a registry decorator wrapping the three Anthropic-messages providers.

Verified with evidence

  • src/provider/anthropic-cache-breakpoint.ts:23-27 — suffix counts non-system ephemeral turns only, matching the vendored adapter (vendor/intx-inference/src/providers/anthropic.ts:69-70), which extracts system turns out of messages into body.system. System-role ephemeral can never skew the split.
  • src/provider/anthropic-cache-breakpoint.ts:42-55 — backward scan skips assistant, mirroring the vendor's isLastUser rule (anthropic.ts:90-93). Confirmed parity by test: persisted [user, assistant] + nudge puts the breakpoint on index 0, identical to the vendor building persisted-alone.
  • src/provider/anthropic-cache-breakpoint.ts:33-40,71-74 — strip covers the full suffix, placement targets pre-split only; system/tools wire keys are never touched (only body.messages is parsed/mutated). PR test pins both breakpoints surviving.
  • src/provider/anthropic-cache-breakpoint.ts:62,72,74 — fail-open is safe and leak-free: suffix === 0 returns built untouched (byte-identical passthrough, pinned by test); split <= 0 and placement-failure return the original built, discarding the parsed mutation — no half-stripped body ever ships.
  • src/provider/inference-dependencies.ts:99 — decorator applies outside withReplaySanitizer, so the split is computed from post-sanitizer wire length against the ephemeral count. Persisted-side drops by the sanitizer cannot desync it; only the ephemeral tail must survive 1:1, and it does (producers emit user turns only; sanitizer drops assistant turns only).
  • src/provider/anthropic-cache-breakpoint.ts:89-98 — contract is sync AdapterRegistry -> AdapterRegistry, non-Anthropic providers return the inner adapter by reference. No async drift, no signature drift.
  • Position-vs-marks: the decorator trusts count-from-tail rather than verifying turn identity. Sound here — vendor/intx-inference/src/reactor.ts:730-733 always appends ephemeralTurns at the prompt tail, and every producer (src/agent/director.ts, src/subagent/nudge-director.ts) emits user-role nudges. No caller passes ephemeral mid-list.
  • Temp hypothesis tests (since removed): two stacked nudges, assistant-ended persisted tail, all-ephemeral prompt, system-role ephemeral — all 4 pass; PR's own 6 tests pass.

Notes

  • Keeper tests worth adding (routed, not implemented here): stacked multi-nudge breakpoint placement, and the all-ephemeral fail-open byte-identity case. The fail-open branches (split <= 0, unparseable body, no eligible persisted message) are currently unpinned.
  • File-for-later, not this diff: a direct buildRequest caller passing ephemeral turns anywhere but the tail would mis-split. No such caller exists today; if one ever does, the decorator should verify tail identity instead of counting.

No blocking or should-fix findings. Hygiene is clean: two focused files, no dead code, no untouched-file churn, red-then-green commit shape intact.

@TheGreatAxios
TheGreatAxios merged commit d3d8465 into main Sep 14, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant