Skip to content

fix(cache): route opaque call items through compression + strip chaining fields - #75

Merged
ranxianglei merged 2 commits into
masterfrom
fix/cache-opaque-items-and-chaining
Aug 10, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
fix/cache-opaque-items-and-chaining

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

背景

Issue dog/billion-context-pi#15 用户反馈「缓存命不中」问题持续存在(即便 reasoning 修复 PR #71 已合并)。全面审计后排除了 ACP tag 漂移(tag 是确定性函数,字节稳定),找到 2 个真正持续破坏 prompt-cache 前缀的根因。

根因

Fix 1: 7 种 call/action item 被错误钉在 preamble(与 reasoning 同类 bug)

OPAQUE_ITEM_TYPES(src/responses.ts:90)之前把 computer_call、computer_call_output、file_search_call、web_search_call、image_generation_call、code_interpreter_call、mcp_call 当作「宿主指令」原样钉在 input[0] 永不压缩。但它们其实是上一轮 response 的输出,属于对话历史 → 每轮原样累积 → prompt-cache 前缀无限增长 → 缓存永远 miss。

这与 PR #71 修的 reasoning 是同一类 bug,只是 item 类型不同。

影响面:

  • 纯 code_mode Codex → 用 custom_tool_call(本来就分类正确)→ 潜伏,不踩
  • Codex + MCP 工具 → 发 mcp_call → 会踩,累积爆炸
  • Codex + computer-use → 发 computer_call → 会踩

不是 Codex 的锅:Codex 重发这些 item 是 Responses API 正常行为,是中转站分错了类。

Fix 2: previous_response_id + instructions 没剥(server.ts:691)

adapter 是无状态的(把完整对话 replay 进 input),但转发 body 时仍保留 previous_response_id → 上游既解析服务端存储的状态、又拿到完整 rebuilt input → 内容重复 + 前缀每轮漂移 → 缓存破坏。instructions 已被 responsesToCore 提升进 input[1] developer message(server.ts:660-661),再次转发 = 重复发送 + 违反 responses_lite 契约(顶层 instructions 对 code_mode 必须留空以暴露工具)。

改动

src/responses.ts:

  • OPAQUE_ITEM_TYPES 只留 additional_tools + mcp_list_tools(真·定义),移除 7 种 call 类型。
  • 新增 helper pushVerbatimItem(msgs, clusters, it, kind):从 it.id 或 hashId(JSON.stringify(it)) 派生 id,push {role:\"assistant\", contentType:\"reasoning\", text:rawId, rawResponsesItem:it}。kind 命名空间("reasoning" vs "responses-call")保证 id 永不冲突。
  • reasoning case 重构为调用 pushVerbatimItem(..., \"reasoning\");新增 7 个 call 类型的显式 switch 分支 → pushVerbatimItem(..., \"responses-call\")。

为什么复用 contentType "reasoning" 是唯一安全桶(核验 node_modules/acp-kernel/dist/index.js):kernel 对 tool-call/tool-result 按 toolCallId 做 filter/pair(call item 会被错误丢弃/配对);reasoning item 在 text-only 策略下 renderMessage 原样返回不注入 tag,且被压缩块覆盖(coveredMessageIds)。MessageContentType union(types.d.ts:2)是固定的,新增值需要 as-any(被 AGENTS.md 禁止)。

src/server.ts:691: delete rebuilt.previous_response_id; delete rebuilt.instructions;

tests/bili-message-roundtrip.test.ts: +3 测试(//14 call items 进入 msgs[] 不进 preamble + 原样 round-trip;//15 压缩 prune 后 call items 被隐藏;//16 call item 与 reasoning id 命名空间隔离,同 raw id 不冲突)。

验证

  • 186 tests pass / 0 fail
  • typecheck + build clean
  • default case(未知 item 类型)保守保留 push 到 preamble(前向兼容安全),oracle Finding 3 暂缓

需要线上核验的点

  1. Codex 是否真的发 previous_response_id?(防御性删除零副作用)
  2. additional_tools(input[0])字节是否稳定?(代理侧盲区,依赖客户端稳定)
  3. reasoning 的 encrypted_content 是否每 response 轮转?(影响 reasoning 自身字节稳定性)

…ing fields

Two remaining prompt-cache killers found via full audit (ACP tag drift
was cleared — tags are deterministic):

1. OPAQUE_ITEM_TYPES still pinned computer_call / computer_call_output /
   file_search_call / web_search_call / image_generation_call /
   code_interpreter_call / mcp_call in the preamble at input[0..] every
   turn — same unbounded-accumulation bug pattern as reasoning. They are
   prior-response output items (conversation history), not host
   directives. Now routed as tracked BiliMessages via pushVerbatimItem
   (contentType 'reasoning' + rawResponsesItem), so the kernel hides them
   once their turn is summarized. Only additional_tools + mcp_list_tools
   (definitions) stay opaque. Stock Codex code_mode was latent (uses
   custom_tool_call), but MCP/computer-use clients hit this directly.

2. server.ts rebuilt body forwarded previous_response_id + instructions
   verbatim. This adapter replays the FULL conversation in input
   (stateless), so previous_response_id made the upstream resolve stored
   server-side state ON TOP of the input we sent (duplication + a prefix
   that shifts every turn as the id advances) — breaking prompt-cache.
   instructions was already lifted into the developer message at input[1],
   so forwarding it again double-sent it and violated the responses_lite
   contract. Both are now stripped from the rebuilt body.

contentType 'reasoning' is the only safe bucket for the call items: the
kernel filters/pairs tool-call & tool-result by toolCallId and injects
ACP tags into text, while reasoning items pass through untouched (no tag,
no filtering) and are still covered by compression blocks.
@ranxianglei
ranxianglei merged commit 74b2ef0 into master Aug 10, 2026
3 checks passed
@ranxianglei ranxianglei mentioned this pull request Aug 10, 2026
@ranxianglei
ranxianglei deleted the fix/cache-opaque-items-and-chaining branch October 15, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant