Skip to content

[PR] kernel: fold aged compress receipts (#4348) - #4354

Merged
ranxianglei merged 4 commits into
masterfrom
2026-10-23_receipt-lace-recovery
Oct 23, 2026
Merged

ranxianglei merged 4 commits into
masterfrom
2026-10-23_receipt-lace-recovery

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

ALWAYS_PROTECTED_TOOLS = ["compress", "acp_rule"] gave every compress exchange a permanent, age-independent folding exemption. Successful compression receipts therefore accumulated on the wire forever — on the incident session, 148 receipts (~170K chars) rode every request with no tool able to reclaim them, and any range over them died with Range would split N turn(s) once the receipts interleaved with re-anchored coverage (the #4348 lace).

This lifts the exemption for aged exchanges only:

  • New COMPRESS_EXCHANGE_KEEP_RECENT = 3 (kernel/src/protected.ts): the newest 3 compress exchanges keep the hard-coded exemption — working memory, the model still sees what it just folded.
  • collectAgedCompressCallIds(messages, config) — host dialect recognition (config.isCompressTool, [Bug] MCP compress 历史调用未被清理,长会话累积大量旧摘要与回执 #2970 MCP names) wins over the bare-name check.
  • filterProtectedToolMessages (kernel/src/unit-projection.ts) skips the blanket exemption for aged exchanges; both halves fold (call + receipt), with reasoning runs kept whole by the existing turn-integrity gate — no split, no stranded pairs.
  • Summaries the model wrote live on verbatim in the rendered block anchors, so decompress/search lose nothing by folding the exchange.
  • Explicit host protectedTools / isToolProtected patterns still win (conservative — same philosophy as [PR] kernel: lift the permanent exemption for FAILED compress exchanges (#4289) #4293). acp_rule keeps its blanket exemption.

Root-cause correction

The issue's original framing (re-anchor misalignment) and my first framing (turn-split deadlock) were both wrong about the cause: the turn-split fatal is the symptom. The cause is that successful compress exchanges were never foldable by design — the lace is simply what that design accumulates on a long session. #4291/#4293/#4310 walk the same doctrine back for failed/refused exchanges; this completes it for aged successful ones.

kernel-change: protected.ts aged-compress exemption lift + unit-projection fold filter, 2 kernel tests

Tests

  • kernel/tests/issue4348-receipt-lace.test.ts (new): production-shaped fixture (reasoning + unref'd tool-call + ref'd receipt turns behind an island of coverage) — the lace range now folds with receipts AND their unref'd calls and reasoning all covered, zero withdrawals; control test pins that the newest 3 exchanges stay protected and a fold over them is refused.
  • Full kernel suite: 1430/1430; root tsc clean; straddle/prune/recommend/absorb families green.

Fixes #4348

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重结论(先答)

查重: 不重复。 与 #4291/#4293/#4310(failed/refused 交换的豁免回退)同属「compress 交换豁免教义回退」家族,但本 PR 补的是 aged-successful 这一半——对象不同(成功回执 vs 失败/被拒交换),不重复。与我在 #4348 下落的 host 侧止血 PR #4353(重锚配对对齐)机制也不同:那条修「重锚覆盖岛落 turn 中段」的记账缺口,本条修「成功回执从设计上永不可折」;同事故族、不同失败域,两者可并存。

标签

标签:bug · scope/kernel · p1(缺陷类;领域 = 压缩引擎折叠判定(kernel);严重度按 p1 缓存泄露类口径——成功回执只增不减、每轮原样重发,实测 148 条/~170K 字符。kernel 级跨车道,不打客户端标签。)

受理

开始独立审核(diff 全窗通读 + 既有行为回归清点 + 定向测试)。先报两个已发现的硬情况:

  1. PR 分支落后 master 且冲突:冲突文件 = kernel/package.json、kernel/package-lock.json、kernel/src/protected.ts、kernel/src/unit-projection.ts(master 上 [PR] Refused compress exchanges become foldable like failed ones, no longer permanently stuck on the wire | 被拒绝的压缩调用像失败的一样可折叠,不再永久占着上下文 #4310 refused-compress-unprotect 与 There are two copies of the summary sent to the model: the block's acp_summary rendering and the compress call parameter | 发给模型的摘要有两份:块的 acp_summary 渲染 + compress 调用参数各存一份 — 组合快照时,调用在场就不要再渲染 #4311 kernel bump 0.0.156→0.0.157 已动过同一批文件;PR 父提交 9e2d217 早于这些合并)。GitHub 现报 mergeable_state: dirty。
  2. CI 尚未出结果:head a4a47be8b47d 目前 0 个 check-run。

我会在审核同时把 rebase 到当前 master 做掉(语义并集,不是机械取边——#4310 与本 PR 改的是同一函数的相邻语义,rebase 后以新头为准给八问结论)。

@ranxianglei ranxianglei added bug Something isn't working p1 scope/kernel labels Oct 23, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-23_receipt-lace-recovery branch from a4a47be to a710d36 Compare October 23, 2026 02:21
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

上游冲突已解决,rebase 完成并推送(a4a47be → a710d36)。八问:

  1. 是什么问题:本 PR 分支与 master 冲突(merge state dirty),无法合并。
  2. 什么原因导致的:PR 分叉后,Failed compression invocations are never folded back for reclamation, and the context keeps ballooning under the weight | 失败的压缩调用永远不会被折叠回收,上下文被压缩残渣越撑越大 #4289/Compression calls with lost ACKs are permanently stuck in context: client-side message loss / model-generation switch re | 回执丢失的压缩调用永久滞留上下文:客户端丢消息/切模型世代重置产生孤儿,且无任何回收路径 #4307(失败的 compress 交换可折叠)先合入 master,改动了同一区域——kernel/src/protected.ts(新增 FAILED_COMPRESS_RESULT_PREFIXES / isCompressToolName / collectFailedCompressCallIds)和 unit-projection.ts(isMessageProtected 加第三参 failed 集合);另外 kernel 版本号撞车——PR 基于旧基线从 0.0.153 提到 0.0.155,而 master 已到 0.0.157。
  3. 影响范围与严重程度:只阻塞本 PR 自身合入,无用户面影响;不解决则 Accident re-anchor debris is unrecoverable: the overlay island lands mid-turn, and the turn-split guard means the orphan | 事故重锚碎屑无法回收:覆盖岛落在 turn 中段,turn-split 守卫让孤另一半永远折不了 #4348 的修复持续搁置。
  4. 怎么修复的:rebase 到当前 master,按「两个机制并存」做并集解决——Failed compression invocations are never folded back for reclamation, and the context keeps ballooning under the weight | 失败的压缩调用永远不会被折叠回收,上下文被压缩残渣越撑越大 #4289 的失败交换折叠保留原样,Accident re-anchor debris is unrecoverable: the overlay island lands mid-turn, and the turn-split guard means the orphan | 事故重锚碎屑无法回收:覆盖岛落在 turn 中段,turn-split 守卫让孤另一半永远折不了 #4348 的过期成功交换豁免叠加其上;collectAgedCompressCallIds 改为复用 Failed compression invocations are never folded back for reclamation, and the context keeps ballooning under the weight | 失败的压缩调用永远不会被折叠回收,上下文被压缩残渣越撑越大 #4289 的 isCompressToolName(bare "compress" ∪ 宿主 isCompressTool hook 的并集语义),删掉 PR 里重复的本地闭包;filterProtectedToolMessages 两处判定同时受 failed 集合与 aged 豁免约束。kernel 版本 0.0.157 → 0.0.158(kernel 边界规则:改 kernel/src/** 必须 bump)。
  5. 是否有回归问题:逐项过过既有行为——失败/拒绝 compress 交换的折叠(Failed compression invocations are never folded back for reclamation, and the context keeps ballooning under the weight | 失败的压缩调用永远不会被折叠回收,上下文被压缩残渣越撑越大 #4289/Compression calls with lost ACKs are permanently stuck in context: client-side message loss / model-generation switch re | 回执丢失的压缩调用永久滞留上下文:客户端丢消息/切模型世代重置产生孤儿,且无任何回收路径 #4307)不受影响;acp_rule 保护不变;宿主显式 protectedTools / isToolProtected 仍优先于内置豁免(与原 PR 设计一致);唯一行为变化就是 PR 本意(超过最新 3 条的成功 compress 交换变为可折叠,call+receipt 成对随 turn-integrity 门一起走)。本地验证:kernel typecheck 干净、根 typecheck 干净、定向测试 20/20(issue4348 / issue4289 / issue3236)+ 166/166(保护与折叠相关套件);全量回归交 CI 在推送头上判定。
  6. 是否破坏缓存:不涉及——折叠资格判定不改任何缓存键、重试或增量游标逻辑,前缀缓存行为沿用 PR 原设计。
  7. 是否改了设计方向:没有——仍严格限定在折叠路径(projectUnit / filterProtectedToolMessages),刻意未把 aged 豁免扩散到 absorb/ccr/compress/recommend 等其他判定站点,与 PR 作者的范围意图一致。
  8. 是否引入了新的配置:没有——COMPRESS_EXCHANGE_KEEP_RECENT = 3 是内核内部常量,不进配置面。

收工判定:冲突已清除,分支干净可合,等 CI 全绿后由人合并。

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_receipt-lace-recovery (3959b10)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5174-pr.4354.52f7202

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4354.tgz
npm install -g package

Install instructions are refreshed on each push.

ranxianglei pushed a commit that referenced this pull request Oct 23, 2026
The rebase bumped kernel/package.json 0.0.157 -> 0.0.158 without
regenerating the root lockfile; lock-sync (#2704 drift guard) failed on
the stale version field. Regenerated with the same command CI runs
(npm install --package-lock-only); kernel/package-lock.json was already
in step (idempotency verified).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重结论(先答)

查重: 不重复。 这是本 PR 分支自身的锁文件漂移(rebase 时 bump 了 kernel/package.json 但没重生成根 package-lock.json),不是 master 红风暴、无既有 issue/修复 PR 对应;按几行级小修纪律直接在当前 PR 分支修掉(独立标注 commit 22cfc0a9b),不另开 issue/PR。

lock-sync 修复八问

  1. 是什么问题:CI lock-sync job 红——根 package-lock.json 里 acp-kernel 还记着 0.0.157,而本 PR 的 kernel/package.json 已是 0.0.158(rebase 冲突解决时按 kernel 边界规则 bump 的)。
  2. 什么原因导致的:bump package.json 版本号不会自动重生成 lockfile,只有 npm install 会(package-lock.json version metadata lags package.json (bili 0.1.190 vs 0.1.191, kernel 0.0.112 vs 0.0.114) #2704 drift guard 注释里写明的机制)。rebase 提交同步了 kernel/package-lock.json(已是 0.0.158),漏了根 lockfile 里 packages["kernel"] 的版本字段。
  3. 影响范围与严重程度:只阻塞本 PR 合入(CI 红),无用户面影响、无运行时行为变化。
  4. 怎么修复的:在 PR 分支上跑 CI 同款命令 npm install --package-lock-only --ignore-scripts --no-fund --no-audit 重新生成根 lockfile——diff 恰好 1 行("version": "0.0.157" → "0.0.158",与 CI 日志里期望的 diff 逐字节一致);另跑了 kernel 侧同款幂等检查,零 diff 确认无需动。已推送 22cfc0a9b。
  5. 是否有回归问题:无——纯 lockfile 版本字段对齐,依赖解析、依赖版本、代码行为全部不变;lock-sync 两步(root + kernel)本地均已通过。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:没有。
  8. 是否引入了新的配置:没有。

收工判定:lock-sync 红已解,等新头 22cfc0a9b 上 CI 全绿后由人合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

review这个pr

ALWAYS_PROTECTED_TOOLS gave every compress exchange a permanent folding
exemption, so successful compression receipts accumulated on the wire
forever (148 receipts / ~170K chars measured on one session) and no range
could reclaim them — attempts died with 'Range would split N turn(s)'.

Only the newest COMPRESS_EXCHANGE_KEEP_RECENT=3 compress exchanges keep
the exemption; older exchanges fold like ordinary tool messages with
call+receipt+reasoning kept whole by the turn-integrity gate. Summaries
live on in block anchors, so decompress/search lose nothing. Host
protectedTools/isToolProtected patterns still win. acp_rule unchanged.

kernel-change: aged-compress exemption lift in protected.ts + unit-projection fold filter, 2 kernel tests
ranxianglei and others added 3 commits October 23, 2026 10:41
- drop the keepRecent=0 early return in collectAgedCompressCallIds: it inverted the parameter semantics (keepRecent=0 should age out EVERY exchange, not none); a no-op for every caller using >= 1
- correct the unit-projection comment: the lift overrides host protection patterns for recognized compress dialects by design (the MCP lane hard-protection hook would otherwise pin mcp__bili__compress forever); the old wording claimed host patterns still win
- correct the receipt-lace test header: it described apply-time turn-group snapping (pre-existing) instead of the actual fix (aged-exemption lift)
- fix one mis-indented if-block left by the conflict resolution
- kernel/package.json 0.0.158 -> 0.0.159 (+ both lockfiles), required by the kernel boundary for kernel/src changes
The entry said explicit host protectedTools/isToolProtected patterns
still win, but isExemptFold bypasses every layer of isMessageProtected
for aged compress-dialect exchanges (host message hook, protectedTools
patterns, isToolProtected hook alike). The override is mechanically
required: bili injects isToolProtected: isMcpBiliHardProtected into
every session's kernel config (src/config.ts), so MCP-lane compress
forms are pinned by the host hook and could never age out without it —
the same doctrine #4289 documented for failed exchanges. Reworded to
state the actual precedence; acp_rule and non-compress tools keep full
host priority.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重结论(先答)

查重: 不重复。 与 #4291/#4293/#4310(failed/refused 交换豁免回退)同属「compress 回执老化教义」家族,但本 PR 补的是 aged-successful 这一半——对象不同(成功回执 vs 失败/被拒交换),无重复 issue/PR。

标签

标签维持受理时判定:bug · scope/kernel · p1(缓存泄露类:成功回执只增不减、每轮原样重发、修复前无任何工具可回收)。


一、CI lock-sync 自动修复报告(本 lane)

两次 lock-sync 红,同一机制:分支重写把锁文件同步丢了。

  1. head a710d36:rebase 冲突解决把 kernel 版本提到 0.0.158,但根 package-lock.json 停在 0.0.157。我按 lock-sync job 同款命令(npm install --package-lock-only --ignore-scripts --no-fund --no-audit)重生成,diff 恰好一行,已推(后被分支重写丢弃)。
  2. head ca05ead(squash 重写后):同样漂移复现。我独立重生成后核对,与并发推上的 e73372734(chore: sync root package-lock)逐字节一致,故未重复推送。
  3. 随后 0fed81f6d(review fixes)把 kernel 提到 0.0.159 并同步了两侧锁文件——我已在我头上核验三处版本号全部对齐(根 lock/kernel package.json/kernel lock 均 0.0.159)。
  4. 我的推送 3959b1095 只含一个标注清晰的 docs commit:修正 CHANGELOG 条目里与代码事实不符的措辞(见下 F-A)。

lane 状态:2 次尝试已用完。 若 3959b1095 上 lock-sync 再红,本 lane 按规则停手只出判定,不再第三次修。


二、PR Review(owner 要求,独立审核)

审核基于最终树 3959b1095:diff 全窗通读 + 改动前代码的既有行为清点 + 定向测试本地实跑(16/16:#4348+#4289;更宽集 20/20:+#3236+protected-content)。与线程内早前分析收敛之处,均以我自己的证据链重新验证过;新增发现见 F-B/F-C/F-D。

① 方向关:对。 修的是碎屑的生产者(无年龄豁免策略本身),不是拦截症状(turn-split fatal)。独立核过:isMessageProtected(kernel/src/protected.ts:362)对任何 compress 交换无条件返回 true,#4289/#4307 之前唯一的口子是 failed/refused 前缀——issue 原假设(重锚失配)确实只是症状层。补齐教义家族最后一半,「疏不堵」成立。

② 回归关:既有行为逐项清点(改前代码为基准)

既有行为 结论
failed/refused 交换折叠(#4289/#4307) ✅ 保持——failed 集合与 aged 豁免在 filterProtectedToolMessages 里独立组合,互不干扰;定向套件 16/16 绿
acp_rule 永久豁免 ✅ 保持——bypass 只对 compress 方言(isCompressToolName)生效,acp_rule 永不进 aged 集合
宿主显式保护(非 compress 工具) ✅ 保持——非 compress 消息不进 aged 集合,host 模式全额生效
turn-integrity INV2(call/result 配对) ✅ 保持——两半随同一 toolCallId 原子地失去保护,不产生新的 split 类别;单半在范围内时整对 withdraw 的既有行为不变
ref 分配 / BLOCKED ref 机制 ✅ 不变——生产形态 = receipt 持数字 ref、call 无 ref(BLOCKED);范围按数组位置捕获无 ref 半,与事故形态(ref'd receipt m00097–m41896 + unref'd call)及 #4289 测试钉("successful stays BLOCKED")互相印证;assignRefsNode 不需要动,PR 没动是对的
单一投影契约(#3236) ✅ 保持——改动落在 projectUnit 咽喉点(filterProtectedToolMessages),apply / recommend 终筛 / acp_status 行视图共用同一判定;#3236 套件绿
golden 钉 / nudge 节奏 / 配置面 ✅ 不涉及——新常量为模块级,不在 defaultConfig 里;无 prompt/工具面变化
双压缩模式(§6) ✅ 模式无关——kernel 投影层改动,plugin/proxy 同路径受益,无 wire 注入变化

③④⑤ 代码/测试/diff 审查 — 四项发现:

diff 整洁度:小范围聚焦,每行都与本 PR 目的相关;版本纪律合规(kernel 0.0.159 + 双侧锁文件 + CHANGELOG 条目齐备)。


三、结构化八问

  1. 是什么问题:成功的 compress 回执从设计上被永久豁免折叠,长会话必然堆积几百条回执(事故会话 148 条/~170K 字符)每轮原样重发且无任何工具能回收;与重锚覆盖交错时表现为 Range would split N turn(s) 拒折——那是症状不是病因。
  2. 什么原因导致的:ALWAYS_PROTECTED_TOOLS 的硬编码豁免无年龄维度——它是堆积的生产者。failed(Failed compression invocations are never folded back for reclamation, and the context keeps ballooning under the weight | 失败的压缩调用永远不会被折叠回收,上下文被压缩残渣越撑越大 #4289)/refused(Compression calls with lost ACKs are permanently stuck in context: client-side message loss / model-generation switch re | 回执丢失的压缩调用永久滞留上下文:客户端丢消息/切模型世代重置产生孤儿,且无任何回收路径 #4307)已回退豁免,successful 是漏掉的最后一类。
  3. 影响范围与严重程度:所有发生过 ≥3 次压缩的长会话;触发即命中。严重度 p1(缓存泄露类:上下文只增不减、持续计费、修复前不可回收)。
  4. 怎么修复的:豁免从「永久」收窄为「仅最新 3 个交换」(工作记忆);更旧的按普通工具消息折叠,走单一投影咽喉点(projectUnit 内的 filterProtectedToolMessages),两半按 toolCallId 原子齐折,turn-integrity 门保证整组不劈;模型写的摘要逐字活在块锚里,decompress/search 零损失。
  5. 是否有回归问题:无——上表既有行为逐项过,唯一行为变更就是 PR 本意(旧=成功回执永久不可折 → 新=仅最新 3 个豁免),CHANGELOG 已披露。F-A 文档缺陷已在树内修掉(0fed81f + 3959b10);F-B recommend 侧保守分叉是有意的还是缺口,待你拍板(不拦合入)。
  6. 是否破坏缓存:否——折叠只减少出站字节,块锚稳定,latest-3 窗口内前缀不变;无 cache-key/去重键/游标变更。
  7. 是否改了设计方向:没有——延续并补完既有回执老化教义家族,刻意只动折叠路径,不扩散到 absorb/ccr/compress/recommend 其他判定站点。
  8. 是否引入了新的配置:没有——COMPRESS_EXCHANGE_KEEP_RECENT = 3 是内核内部常量,未进 config/env 面(配置面纪律守住)。

收工判定:建议合入——等 3959b1095 上 CI 全绿(lock-sync 在 e733727 已过,全套件在本头待跑),合并由你来(我不合并 PR)。后续两件等你拍板:F-B(recommend 侧要不要跟上老化)、F-C(rebuild 契约说明/replay 裸名缺口是否立项)。

@ranxianglei
ranxianglei merged commit 7681e2f into master Oct 23, 2026
41 of 45 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.746-master.7681e2f (rolling master tag).

npm install -g billion-context@0.0.746-master.7681e2f  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@pkuqiuning

Copy link
Copy Markdown

Follow-up: the aged exemption lands on the fold filter, but not on the gates that decide what can be picked — old compress exchanges still can't be selected

Context: #4354 exempts aged compress exchanges from filterProtectedToolMessages (via collectAgedCompressCallIds, COMPRESS_EXCHANGE_KEEP_RECENT=3), so folds that happen to span old exchanges can now absorb them. Verified on a master build (commit fba2d13), one long-lived pi session (187 compress-call messages in store, ~660 KB):

  • Manual folds over picker-eligible ranges: compress-call messages covered by active blocks went 0 → 81/187; the ~106 uncovered ones (≈101 of them BLOCKED, the rest inside the keep-recent age window) still ride the wire. Billed input for the following request dropped ~15% (297k → 252k tok) in one round of folds.
  • But the picker still does not offer the aged exchanges as range content, for two independent reasons: (a) their refs stay BLOCKED — the ref-allocation pass (assignRefs) has no aged exemption; and (b) buildCompressibleRanges still routes every message through the unprotected-path predicate, where ALWAYS_PROTECTED_TOOLS covers compress/acp_rule — no aged exemption there either. (In this build isExemptFold occurs only inside filterProtectedToolMessages; buildCompressibleRanges has no occurrence of it.) So the aged set only gets absorbed when a requested span happens to cover it.
  • Explicit ranges that would straddle the remaining blocked exchanges are rejected outright: refs … are known to this session but no longer back any visible or folded message … These refs are now recorded as DEAD: no range that includes them can compress now or later, whatever the neighbors. — so no fold can sweep them.

Net effect: compress call+receipt text still accounts for ~30% of outgoing tokens, and max compressible stays far under the nudge threshold (8790 < 50000 here).

Suggestion: age both gates together — e.g. refs for compress exchanges older than COMPRESS_EXCHANGE_KEEP_RECENT stop being BLOCKED and the range builder treats the isExemptFold set as viable content — so the normal nudge/preflight path can pick them up instead of relying on lucky span overlap. Lifting the ref-side BLOCKED alone still leaves every aged exchange un-selectable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

2 participants