ACP dynamic nudges prevent OpenAI Responses prefix cache from growing #5

Activity

1. ranxianglei

   closed this as completedin #6on May 30, 2026
2. ranxianglei commented on May 30, 2026

   

   ranxianglei

   on May 30, 2026

   Owner

   More actions

   @Chikage0o0

   ---

   Prefix Cache Invalidation — Root Cause & Fix

   Background

   The OpenAI Responses API supports prefix caching — if consecutive requests share identical leading content, the API reuses cached token computations and reports cache hits via the cache_read metric.

   Bug Manifestation

   With ACP (and the original DCP), cache_read would stop growing after the first few turns:

   Turn 1:  cache_read = 17,728
   Turn 5:  cache_read = 55,424
   Turn 10: cache_read = 55,424  ← stuck, no further growth
   

   Every subsequent request was wasting the opportunity to cache ~55K tokens of repeated computation.

   Root Cause: A DCP Design Flaw (Not Introduced by ACP)

   DCP (and ACP after forking) injects dynamic per-turn content into historical user messages during the chat.messages.transform hook:

   Injection	Description	Changes each turn?
   applyAnchoredNudges	Compression prompt nudges (context limit, turn, iteration)	✅ Yes
   injectContextUsage	Current token usage info	✅ Yes
   injectVisibleIdRange	Visible message ID range	✅ Yes
   injectMessageIds	Per-message mNNNNN reference tags	❌ No (stable)

   The first three functions generate different text each turn and inject it into historical user messages in the middle of the conversation. Since prefix caching requires the leading content to remain byte-identical across requests, modifying any message invalidates the cache for all content after that message.

   This can be confirmed in the DCP source code — applyAnchoredNudges directly calls injectAnchoredNudge(message, nudgeText) to mutate anchor message content. This pattern existed in DCP from the start.

   ACP added injectContextUsage and injectVisibleIdRange (targeting the last user message) after forking, which exacerbated the issue. But the root cause traces back to DCP's design.

   Fix: Suffix Message Pattern

   All dynamic injections are now consolidated into a single synthetic suffix message.

   Key changes:

   1. New createSuffixMessage() — Creates a synthetic user message appended to the end of the message array. Uses a fixed seed (acp-dynamic-guidance) to generate a deterministic ID (msg_dcp_summary_<sha256-hash>), ensuring it is skipped by assignMessageRefs and invisible to the compress tool.

   2. Dynamic content targets suffix only — applyAnchoredNudges, injectContextUsage, and injectVisibleIdRange now write to the suffix message, leaving all historical messages untouched.

   3. Historical messages stay byte-identical — All real messages before the suffix remain stable across turns, allowing prefix cache to cover up to the last real message.

   Measured results:

   Before fix: cache_read stuck at 55,424 — never grows
   After fix:  cache_read grows 17K → 26K → 35K → 53K, continuously increasing
   

   How to Upgrade to 1.1.1

   Note: The auto-update in previous versions had a bug — update.ts hardcoded the old package name @tarquinen/opencode-dcp, preventing auto-update detection. This is also fixed in 1.1.1.

   Manual upgrade (recommended):

   # Option 1: via opencode plugin management
   opencode plugin install opencode-acp
   
   # Option 2: direct npm update
   npm update opencode-acp --prefix ~/.cache/opencode
   
   # Option 3: install latest
   npm install --save opencode-acp@latest --prefix ~/.cache/opencode

   Restart OpenCode after upgrading. Auto-update will work correctly from 1.1.1 onward.

   ---

   前缀缓存失效问题说明

   背景

   OpenAI Responses API 支持前缀缓存（Prefix Cache）——如果连续请求的前面部分内容相同，API 会复用已缓存的 token 计算结果，跳过重复计算，返回 cache_read 指标反映命中量。

   Bug 表现

   在 ACP（以及原版 DCP）中，cache_read 在会话初始几轮增长后完全停滞。例如：

   第1轮: cache_read = 17,728
   第5轮: cache_read = 55,424
   第10轮: cache_read = 55,424  ← 停滞，不再增长
   

   每次请求都在"浪费"前 55K token 的重复计算。

   为什么会有这个 Bug

   这是 DCP 的固有设计缺陷，不是 ACP 引入的。

   DCP（以及 fork 后的 ACP）在 chat.messages.transform 钩子中，每轮都会向历史用户消息注入动态内容：

   注入内容	说明	每轮变化？
   applyAnchoredNudges	压缩提示 nudge（上下文限制、轮次、迭代）	✅ 是
   injectContextUsage	当前 token 用量信息	✅ 是
   injectVisibleIdRange	可见消息 ID 范围	✅ 是
   injectMessageIds	每条消息的 mNNNNN 引用标签	❌ 否（固定）

   前三个函数每轮生成不同的文本，注入到会话中间或末尾的历史用户消息中。由于前缀缓存要求请求开头的内容逐字节不变，修改中间某条用户消息会使该消息之后所有内容的缓存失效。

   在 DCP 源码中可以确认这一点——applyAnchoredNudges 直接调用 injectAnchoredNudge(message, nudgeText) 修改锚点消息内容，这个模式在 DCP 就已经存在了。

   ACP 在 fork 后新增了 injectContextUsage 和 injectVisibleIdRange（注入到最后一条用户消息），进一步加剧了这个问题，但根本原因来自 DCP。

   修复方案

   将所有动态注入内容集中到一个新建的 suffix 消息中。

   核心改动：

   1. 新建 createSuffixMessage()：创建一条合成用户消息，追加到消息数组末尾。使用固定 seed 生成确定性 ID（msg_dcp_summary_<sha256("acp-dynamic-guidance")>），确保不会被 assignMessageRefs 分配 mNNNNN 引用，对 compress 工具完全透明。

   2. 动态内容全部写入 suffix：applyAnchoredNudges、injectContextUsage、injectVisibleIdRange 改为写入 suffix 消息，不再修改任何历史消息。

   3. 历史消息保持不变：前面所有真实消息的内容逐字节稳定，前缀缓存可以覆盖到最后一条真实消息。

   修复效果（本地实测数据）：

   修复前: cache_read 停滞在 55,424，不再增长
   修复后: cache_read 从 17K → 26K → 35K → 53K，持续增长
   

   如何升级到 1.1.1

   注意：之前的自动更新有 bug——update.ts 中硬编码了旧包名 @tarquinen/opencode-dcp，导致自动更新无法正确检测新版本。1.1.1 同时修复了这个问题。

   手动升级（推荐）：

   # 方法1：通过 opencode 插件管理
   opencode plugin install opencode-acp
   
   # 方法2：直接 npm 更新
   npm update opencode-acp --prefix ~/.cache/opencode
   
   # 方法3：本地安装
   npm install --save opencode-acp@latest --prefix ~/.cache/opencode

   升级后重启 OpenCode 即可生效。1.1.1 之后的版本自动更新将正常工作。
3. ranxianglei commented on Jun 27, 2026

   

   ranxianglei

   on Jun 27, 2026

   Owner

   More actions

   Second part of the prefix-cache fix: pruneToolOutputs (PR #26)

   This issue fixed the nudge-injection half of the "in-place message mutation breaks prefix cache" bug class. PR #26 (merged, published as v1.4.2) fixes the other half — pruneToolOutputs/pruneToolInputs/pruneToolErrors in prune.ts were doing the same in-place mutation on part.state.output, causing periodic cache invalidation every time tool-output pruning fired.

   Evidence: Session with 828 messages had 176 cache misses over 2 days — 89% of all fresh input tokens (50.8M) wasted on re-sending context that should have been cached.

   Fix: Disabled the three in-place mutating calls in prune(). Compression still works via filterCompressedRanges + model-initiated compress tool. The functions are kept as dead code for a future suffix-message approach (same pattern used for nudges in this issue's fix).

   Upgrade: opencode-acp@1.4.2 is published on npm.
4. ranxianglei

   added 2 commits that reference this issue on Sep 11, 2026

   perf: avoid transform work that scales with compression history (#384)

   ...

   68b32dc

   perf: avoid transform work that scales with compression history (#384)

   ...

   0a06c2d