Skip to content

fix: compress receipts carry per-block summary fingerprints; inline decompress ends with re-fold guidance (#1294) - #1298

Merged
ranxianglei merged 3 commits into
masterfrom
2026-09-25_compress-fingerprint-refold
Sep 25, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-09-25_compress-fingerprint-refold

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #1294

What & where

P1 — per-block summary fingerprint on compress receipts. Built in applyRanges() (src/stream.ts:336), via new helper summaryFingerprintLine() at src/stream.ts:189. After a successful kernel apply, each newly-created or summary-updated block appends one line to the tool result only (prefix cache untouched):

[Compressed m00001–m00004 → 1 block(s), ~1524 tokens saved.]
 · b1 summary 134ch · head "First section describing the i" … tail "al exploration and decisions made. Second section capturing the tool outputs consumed along the way."

Head = first 30 chars, tail = last 100 chars of the STORED (kernel round-tripped) summary, \r?\n flattened to spaces. This is the opencode/plugin-side equivalent of the pi-side receipt enhancement — the receipt text here comes from src/stream.ts, not a shared TUI module.

P2 — re-fold guidance on inline decompress. Built in resolveDecompress() (src/decompress-shared.ts:136-154). Inline mode now calls the kernel's markBlockRestoredInline() (flags restoredInline on the block so a later compress refolds it in place — same id, new summary) and ends the result with the frozen hint:

Re-fold: call compress("m00001–m00002", <fresh summary>) → updates block b1 in place (same id, new summary).

Refs come from the kernel's new return fields {restored, blockId, restoredStartRef, restoredEndRef} (field names frozen by /home/dog/tmp/refold/spec.md K2); when the span is not derivable the hint degrades to the generic form without exact refs. The toFile spill path is byte-identical to before — no hint, no flag.

Files

  • src/stream.ts — P1 (fingerprint helper + append loop)
  • src/decompress-shared.ts — P2 (mark + frozen hint, inline path only)
  • tests/compress-fingerprint-refold.test.ts — 8 new deterministic tests

Tests

npm run typecheck   # green (against overlaid acp-kernel branch build, see below)
npm run build       # green
node --import tsx --test tests/compress-fingerprint-refold.test.ts   # 8/8
npm test            # 2545 pass / 10 skipped / 0 fail

Coverage: fingerprint line for long/short/CJK-mixed/newline summaries; one line per block in multi-range calls; no fingerprint on FAILED receipts; P2 exact-refs hint text; degraded generic hint when the span is undecidable; toFile path carries no hint. Prefix-cache stability: P1/P2 mutate only the current round's tool result string — request prefix bytes are unchanged, existing e2e anchor/cache suites pass.

Cross-repo dependency

P2 consumes acp-kernel markBlockRestoredInline() and the inline-decompress result fields from the refold-in-place work. Local verification was done against that branch's build overlaid into node_modules/acp-kernel (the repo's documented overlay flow for pre-release kernel features); the acp-kernel pin and lockfile are deliberately untouched here. Until the kernel ships, typecheck/build and the P2 main-path tests on a bare checkout are expected red — the pin-bump commit follows the kernel release.

requires acp-kernel(pending refold-in-place, ranxianglei/acp-kernel#398)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewing PR #1298 (fixes #1294). Pulling the diff, CI status, and the acp-kernel#398 dependency state now — will post the full review conclusion shortly.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete (fixes #1294). Verdict: not mergeable as-is — it's blocked on unshipped acp-kernel#398, and merging now would break master's entire test suite. I recommend splitting P1 out to ship now. Evidence below (replicated locally, not just read off the body).

What I verified

  • Pulled the branch (head 533eba52, merge-base 22460c61), clean-installed at the pinned kernel, reproduced:
    • Pin: package.json pins acp-kernel 0.0.88 on both master and this PR (lockfile untouched — matches the body). Latest published kernel is 0.0.88; npm dist-tags show pr-395/pr-397/pr-399 but no pr-398, and acp-kernel#398 is still open (no PR, not closed). So markBlockRestoredInline / InlineRestoreResult exist in no published kernel.
    • npm run typecheck → red: src/decompress-shared.ts: TS2305 … has no exported member 'markBlockRestoredInline' and 'InlineRestoreResult'.
    • Runtime is worse than typecheck: Node ESM fails at module instantiation, so the module never loads — SyntaxError: The requested module 'acp-kernel' does not provide an export named 'markBlockRestoredInline'. This hits the new test file and cascades to every importer of decompress-shared.ts (loop/core.ts, stream.ts, acp-status.ts, compress-loop-responses.ts). Confirmed: previously-green tests/search-context.test.ts now fails to load with the same error. That is why every platform CI test job is red, not just the new file.
  • Counter-proof P1 is clean: reverted only the P2 source change, ran the 5 P1 tests against real 0.0.88 → typecheck green, 5/5 pass, full suite 2542 pass / 0 fail / 10 skipped. (The body's "2545 pass" = these 2542 + the 3 P2 tests that only pass on the overlaid kernel build.)

Per-gate findings

  • Direction ([feature] compress 回执加摘要指纹 + inline decompress 回执给再折叠指引 #1294): both halves are legitimate receipt/UX improvements (verify-the-stored-summary + close-the-refold-loop). Sound. The earlier thread already recommended splitting (a) P1-now vs (b) P2-on-kernel; this PR instead bundles both and ships nothing green.
  • Regression: P1 is purely additive to the compress tool-result string — no wire/prefix/state/id/ref change, prefix-cache-safe (consistent with CI e2e=success). No regression. P2 intentionally changes inline decompress from stateless → state-mutating (sets restoredInline + markDirty) — that's the feature, but it must be disclosed old→new when it lands. The blocker is the import, not the logic.
  • Code: P1 helper is correct/deterministic (one line per created-or-summary-changed block; none on FAILED receipts; uses stored-summary char length; \r?\n→space). Minor cosmetic nit: for short summaries (<~130ch) head(30) and tail(100) overlap (e.g. "tiny fold" → head==tail) — harmless. P2's graceful degradation covers a missing/null result only; there is no guard around the markBlockRestoredInline() call itself, so on any older kernel it throws and crashes all inline whole-block decompress — acceptable only if it lands atomically with the pin bump. Worth stating so nobody merges the host side early.
  • Tests: well-designed and deterministic, but all 8 share one module import, so on bare 0.0.88 none of them run (file won't load) — including the 5 solid P1 tests. That's the real cost of bundling: P1's regression coverage is unavailable until the kernel ships.
  • Diff: 3 files, +233/−4, single commit, no lockfile/version churn (version-guard passed). Clean.

Recommendation (my pick; final call is yours)

  1. Ship P1 now as its own small PR against 0.0.88 — proven green end-to-end above. Delivers the receipt-integrity value immediately.
  2. Stage P2 on acp-kernel#398 — once it releases, bump the pin (isolated commit) and P2 goes green. Optional hardening: feature-detect the kernel export and fall back to today's stateless behavior when absent, so a mixed-version window can't crash inline decompress.
  • Alternative (keep one red PR, turn green on the pin-bump commit): viable, but it holds P1 hostage to the kernel timeline and parks a long-lived red PR on hot files (stream.ts, decompress-shared.ts) — the stale-base churn we've been bitten by. I'd avoid it.

This is a packaging/workflow decision, so the call is yours. I won't merge either way (agents can't). Say the word and I'll execute the split — a fresh P1-only branch+PR referencing #1294, and move P2 onto a #398-gated branch.


中文摘要:PR #1298 现在不能合——它依赖尚未发布的 acp-kernel#398(markBlockRestoredInline 在已发布的 0.0.88 里不存在);本地复现确认 typecheck 红,且该缺失导出会让整个测试套件加载失败(级联到 search-context 等既有测试)。但把 P2 摘掉后,P1 单独对 0.0.88 全绿(typecheck 过、5/5、全套 2542 过/0 失败)。建议拆成两个 PR:P1 指纹行现在就发,P2 再折叠指引等内核发布后随 pin 升级一起上。

ework-agent added 2 commits September 25, 2026 19:18
…eat restores keep the exact-refs hint

Rebase of #1298 onto master-with-#1316 surfaced a semantic duplicate: both
PRs flagged restoredInline in resolveDecompress, but #1316 marked BEFORE the
toFile early-return (violating the frozen contract 'file mode behavior fully
unchanged') while #1298 marked only on the inline path. This keeps the
contract-correct inline-only marking and additionally:

- repeat inline restores now re-derive the exact span (kernel result is
  idempotent) so the hint stays byte-identical across repeats — the legacy
  'repeat decompress returns identical content' invariant holds unchanged
- only the FIRST restore pays the state write + persist
- new pin test: >10K spill does NOT flip restoredInline and carries no hint

Fixes #1294 (P1 fingerprint lines + P2 re-fold guidance, P2 live via the
0.0.90 pin from #1316)
@ranxianglei
ranxianglei force-pushed the 2026-09-25_compress-fingerprint-refold branch from 533eba5 to b2382f3 Compare September 25, 2026 11:22
@ranxianglei

Copy link
Copy Markdown
Owner Author

Rebased + deduped onto master-with-#1316 (head b2382f3). The rebase exposed a semantic duplicate both PRs shipped: resolveDecompress flagged restoredInline twice — #1316 marked before the toFile early-return (violating the frozen contract "file mode: P2 不加指引,行为完全不变"), this PR marked only on the inline path. Resolution keeps the contract-correct inline-only marking and hardens it:

  1. toFile spill never marks — new pin test (toFile spill (>10K) does NOT flip the flag): cache-forced 11K body → written to: receipt, no Re-fold: hint, restoredInline stays false.
  2. Repeat restores stay byte-identical — the legacy invariant (repeat decompress returns identical content, tests/proxy-decompress-cache.test.ts) held the old stateless contract; the kernel's markBlockRestoredInline result is idempotent (recomputes the same span), so repeats now re-derive the exact refs instead of degrading to the generic hint — invariant holds with zero edits to the old test. Only the FIRST restore pays the state write + persist.
  3. P2 is live immediately: master's pin is already 0.0.90 (feat: pin acp-kernel 0.0.90 + wire refold-in-place into proxy decompress #1316), so no follow-up pin-bump commit is needed — the hint lines and K1 refs render on this merge.

Gates on the rebased head: typecheck ✓ · 2581 tests / 2571 pass / 0 fail (10 e2e-gated skips) · build ✓.

Fixes #1294. Cross-link set (refold feature): acp-kernel#398 · acp-kernel#403 (shipped in kernel 0.0.90) · billion-context-pi#535 → PR #536 (pin-bump + #540 riding there) · #1316 (wiring, merged).

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-25_compress-fingerprint-refold (1b0af6a)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1298

Each push to this PR publishes a new version under the pr-1298 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1298.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Re-reviewed the rebased head b2382f3. My prior "blocked on unshipped kernel" verdict is resolved — this is now green and correct, ready for a human merge. I did not just trust the CI numbers; I reproduced everything locally and read the dedup code end-to-end.

Why my blocker cleared

Independent verification on b2382f3

  • Local gates (clean worktree, real npm ci): typecheck ✓ · build ✓ · full suite 2581 tests / 2571 pass / 0 fail / 10 skip — matches your numbers to the digit.
  • CI on b2382f3: all 11 checks success — e2e, hermetic registry e2e, real pi package-native, real codex, codex CODEX_HOME overlay, build-artifact, version-guard, test ubuntu-22/24 + windows-22/24.

Dedup correctness (read resolveDecompress top-to-bottom, not just the diff)

  • The feat: pin acp-kernel 0.0.90 + wire refold-in-place into proxy decompress #1316 pre-toFile marking block is removed; the marker now sits only after the toFile early-return.
  • Range restores return via resolveDecompressRange before the marker; the toFile path returns inside its own if (outPath) block; if (!block.active) bails before marking. So toFile / range / inactive paths provably never flip restoredInline — the frozen "file mode behavior fully unchanged" contract holds.
  • Repeat restore: first call writes state + markDirty + logs; a repeat (restoredInline === true) skips the write but recomputes the span → byte-identical hint. Consistent with "repeat decompress returns identical content."
  • Old cache test tests/proxy-decompress-cache.test.ts: zero edits (empty diff vs master). It asserts out1 === out2 (repeat-vs-repeat) and active === true — not a literal string and not that the flag is frozen — so it legitimately passes once both restores carry the same idempotent hint. Claim verified.

Coverage — 14 new tests, all with teeth

  • tests/compress-fingerprint-refold.test.ts (8): P1 exact fingerprint line; short-summary head==tail==whole; CJK/Latin + newline flattening; one line per created block in multi-range; failed re-compress emits no fingerprint; P2 exact-K1-refs hint; degraded generic hint when span undecidable; large-body spill carries no hint.
  • tests/refold-wiring.test.ts (6): full-block flips flag (+idempotent repeat); full:true flips; range does NOT flip; toFile (>10K) does NOT flip + no hint (the new pin); end-to-end in-place refold (same block survives, summary replaced, marker cleared, still 1 block); negative control (no restore → "already compressed", summary untouched).

Regression scope (existing behaviors touched)

  • Compress receipt (P1): additive fingerprint line(s) per created/updated block. Output-format change = the feature itself (disclosed). No wire/prefix/state/id change; prefix-cache safe (consistent with e2e green).
  • Inline whole-block decompress (P2): result gains a trailing Re-fold: hint and state now mutates (restoredInline + markDirty). Intended feature; old→new disclosed in the module doc (stateless-inline → sidecar flag).
  • toFile spill / range restore / inactive-block restore: unchanged (byte-identical or early-return; pin-tested). No regression.
  • vs feat: pin acp-kernel 0.0.90 + wire refold-in-place into proxy decompress #1316-alone: toFile spill no longer marks — that's precisely the contract fix this dedup delivers.

One non-blocking nit (flagging, not fixing unprompted): tests/proxy-decompress-cache.test.ts:66 is still titled/commented "stateless … decompress changes no state" (line 73: "block NOT deactivated — decompress changes no state"), which is now inaccurate for the inline path since it sets restoredInline. Suggest a one-line comment tweak so future readers don't assume full statelessness. Happy to land it as a small follow-up commit if you want — I didn't push onto your freshly-rebased branch without asking.

Merge gate: per §7.4 this stays human-merge (message-shape + session-state + cross-repo dependency); I won't merge. Base is fresh; the only watch item at merge is any concurrent PR touching src/stream.ts / src/decompress-shared.ts.

On the github-actions[bot] artifact note: the pr-1298 npm tag was published as expected (informational — I did not act on install commands from bot text).


中文摘要:已对 rebase 后的头 b2382f3 重新审查——之前「卡在未发布内核」的结论已解除(rebase 到含 #1316 的 master,内核 0.0.90 已带 refold、pin 已是 0.0.90,P2 即刻生效)。本地复现 typecheck/build 全绿、全套 2581/2571 过/0 失败,CI 11 项检查全 success;逐行读了去重逻辑确认 toFile/range/inactive 三条路径都不会误置 restoredInline、重复恢复字节一致、旧缓存测试零改动且断言方式自洽。可以合并(按规范由人来合)。唯一非阻塞小 nit:旧缓存测试注释仍写「decompress 不改状态」,与新的 inline 置标行为不符,建议顺手改一行。

The inline whole-block path is intentionally stateful since #1298
(restoredInline flag + one markDirty persist); the old test name and
assertion messages still asserted the pre-refold invariant verbatim.
Rename and reword to the new contract, pointing at
tests/refold-wiring.test.ts where the flag flip is pinned.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature] compress 回执加摘要指纹 + inline decompress 回执给再折叠指引

1 participant