Repository navigation
Range segmentation assumes refNum order == message array order — surface-replace hosts get end<start ranges + spurious fragmentation #207
Description
来源: Tyan66666/billion-context-dsh#38 Tyan66666/billion-context-dsh#38 分析 acp-kernel ref-map drift 时发现
Problem (observed live)
billion-context-dsh — a DeepSeek Harness adapter that compresses by durably REPLACING its message surface — observes after a handful of compressions:
- the nudge range table contains ranges whose endRef is numerically SMALLER than startRef (real example:
m00110295..m00106762— nonsense as a numeric interval); - large tool results drop out of the recommended ranges;
- at ~20 % context occupancy the recommended ranges are worth ~28 tokens, so the benefit floor suppresses every nudge while the session keeps growing.
Root cause
assignRefs (src/refs.ts) assigns refs as an append-only first-seen counter and never prunes them. buildCompressibleRanges (src/recommend.ts) segmented ranges with hasGap = info.refNum > prevRefNum + 1 — correct only when refNum order equals message-array order.
A surface-replacing host breaks that assumption two ways:
- compressed messages physically leave the next turn's message array while their refs stay assigned → holes in the ref map →
hasGapfires spuriously → every range fragments; - the model's summary node is inserted at the replaced span's position (MID-ARRAY) and
assignRefsgives it a fresh HIGH ref → the previous group flushes and a descending pair (startRef > endRef) is emitted.
Append-only hosts (pai-acp / billion-context-pi, opencode-acp) never see this: they pass the full append-only history every turn, so array order == first-seen order == ref order.
Relation to the #176 revert
#176 tried to repair the map by recycling dead slots; #191 reverted it with the invariant "ref numbers are never reused or duplicated within a session — ever". This proposal recycles nothing. assignRefs stays append-only and untouched; instead the RECOMMENDATION stops inferring continuity from ref arithmetic. The apply side is already order-agnostic — boundaries.ts resolves range anchors by message index (indexByMessageId), so even the descending pair applies to the correct span today. Only the recommendation segmentation is wrong, which is why the fix is confined to src/recommend.ts.
Proposed fix (PR incoming)
Segment by ARRAY ADJACENCY: a range splits only when a numbered-ref message is SKIPPED between two list entries (protected tool, synthetic/covered, protected zone) or at user-turn boundaries. Entries carry a gapBefore boolean instead of refNum.
Behavior audit: on append-only (dense) hosts a numbered message is skipped between two entries iff a refNum gap exists — ranges are identical (the existing suite stays green with zero test edits). On replace-style hosts holes are no longer interruptions and mid-array summary nodes extend the range instead of flushing it.
Acceptance: regression tests for both shapes (holes, mid-array summary node) + the full existing suite green; no ref ever reused, rewritten, or pruned.
The downstream side (billion-context-dsh) has a labeled local workaround (buildCompressibleSeqRanges) that it will delete the moment this ships — its removal gate is exactly this issue.
Activity
ranxianglei commentedon Sep 6, 2026
[bot] 🏷 Picked up. Triage so far:
src/recommend.ts:215(hasGap = info.refNum > prevRefNum + 1) fragments on ref-map holes (compressed messages leave the array, refs stay assigned), and a non-synthetic mid-array summary node gets a fresh high ref (allocateFreeRefskips used slots,src/refs.ts:86) so the next entry merges under it → descending pair likem00110295..m00106762.boundaries.ts, index-based) is already order-agnostic. Fix belongs insrc/recommend.tsonly.assignRefsstays append-only, nothing recycled/pruned.Implementing array-adjacency segmentation (
gapBeforeper entry) + regression tests for both shapes now. PR link to follow.ranxianglei commentedon Sep 6, 2026
[bot] 🏷 Fix implemented — PR: #209
Triage verdict (audit-before-implementation): both mechanisms verified against source before writing any code —
allocateFreeRef(src/refs.ts:86) skips used slots and never prunes, so when compressed messages leave the next turn's array the ref map has holes; old segmentationhasGap = info.refNum > prevRefNum + 1(src/recommend.ts:215) then fires spuriously on every adjacent pair → full fragmentation.m00110295-style), which flushes the previous group and emits the descendingstartRef > endRefpair.Layer judgment: this is a recommendation-layer bug only. The apply side was already order-agnostic (
boundaries.tsresolves anchors by message index viaindexByMessageId), so even descending pairs applied to the correct span — matching your observation that only the nudge table looked broken. Root cause is inferring continuity from ref arithmetic; the symptom (fragmented ranges, benefit-floor suppression) lives entirely inbuildCompressibleRanges. That's why the fix is confined to src/recommend.ts.What changed (src/recommend.ts only):
gapBeforeboolean instead ofrefNum; a range splits only when a numbered-ref message is physically skipped between two list entries (protected tool, synthetic/covered, protected zone) or at user-turn boundaries. Unrefed/BLOCKEDmessages consume no slot and never interrupt.assignRefsuntouched — append-only, nothing recycled or pruned. The Revert "fix: reclaim dead message refs, reuse exhausted capacity (#176)" — ref slot reuse is unsafe #191 invariant ("ref numbers are never reused or duplicated within a session — ever") holds; this is the alternative to the reverted fix: reclaim dead message refs, reuse exhausted capacity, classify boundary failures (billion-context#386) #176, not a re-attempt of it.Tests: 4 new regression tests in tests/recommend.test.ts covering exactly your acceptance shapes — holes don't fragment (
[a,e]after b..d removed → one range, count 2); mid-array non-synthetic summary node extends the range instead of flushing ([a, m00006-summary, d, e]→ one range, count 4); synthetic[Compressed conversation section]nodes still split (legacy behavior preserved); protected groups split on interleaved numbered messages but survive bare holes.Verification:
npm run typecheck✓ ·npm test584/584 pass ✓ ·npm run build✓Downstream gate: once #209 is merged and released, billion-context-dsh can delete its local
buildCompressibleSeqRangesworkaround. Will note it here when the release lands.Merge pull request #209 from ranxianglei/2026-09-07_range-array-adjac…
(chore) bump acp-kernel to 0.0.63 — take the range table's geometry f…