Skip to content

[PR] fix: accept merged flat compress form, recognize U+2212/FF0D dashes, report honest rejection reasons (#4252) - #4255

Closed
ranxianglei wants to merge 4 commits into
masterfrom
2026-10-22_4252-flat-compress-salvage
Closed

ranxianglei wants to merge 4 commits into
masterfrom
2026-10-22_4252-flat-compress-salvage

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4252

kernel-change: model-facing compress parser — accepts one more documented shape (summary text in content beside top-level refs) and removes two silent-misfold paths observed live in v0.1.204 (U+2212 dash narrowing a 19-message range to 1; masked rejection reasons). Owner-approved by the issue's fix plan. Kernel version 0.0.151 → 0.0.152.

What changes

  1. RC1 — merged flat single-range form accepted. {content: "<summary text>", startId, endId, topic} (the shape the schema's two separately-documented forms invite) now parses as ONE range: plain prose or a JSON-encoded {summary,topic} object inside the string; top-level summary wins over content text; all ref spellings (startRef/endRef/messageId). Guard rails: line-form strings whose first line carries refs headers are NEVER hijacked (every input that parses today is untouched); broken JSON shells, non-object structures, and summary-less JSON objects beside refs stay rejected. diagnostics.flatSummarySalvage marks the recovery.
  2. RC2 — U+2212/U+FF0D recognized; unrecognized separators rejected. The refs separator class gains the Unicode minus sign and fullwidth hyphen-minus (models copy dashes from rendered summaries, where ASCII - is rare). A header with two refs separated by an unrecognized divider (m00150 => m00168, adjacent refs) is now REJECTED naming the separator instead of silently folding ONE message instead of N. A wordy gap (m00150 see also m00168) stays a topic citation with the single-message read unchanged.
  3. RC3 — rejection reasons unmasked. String content whose entries carry refs headers but validate to zero ranges reports kind=no-valid-ranges with per-entry reasons (e.g. missing summary) instead of a blanket content-not-array that dropped them ([bug] 0.1.63 起 Windows persist EPERM 风暴:会话持久化静默丢数据(4.9k 次/117 会话)+ compress no-valid-ranges 拒绝 #362 visibility regression). Prose without any refs header keeps content-not-array (ReDoS regression tests assert exactly this).
  4. Schema text + proxy log line. Parameter/tool descriptions document the accepted merged shape (goldens regenerated — description-text-only, verified every diff line is a description field); parseCompressInput emits a flat-summary salvage warn same-style as the compress: literal backslash-n in string content still fails as kind=content-not-array, hiding the real missing-summary reason #3780/lenient compress-arg parsing: salvage salvageable model output #603 salvages.

Behavior changes (old → new)

Shape Old New
content prose + top-level refs rejected content-not-array (retry loop, ~2.6M tok refold churn in the live session) accepted as one flat range + flatSummarySalvage flag + proxy warn
m00150−m00168 / m00150-m00168 header silently folded m00150–m00150 (1 msg) folded m00150–m00168 (the intended range)
m00150 => m00168 header (unrecognized divider, non-wordy gap) silently folded m00150–m00150 rejected, reason names the separator
header-only string entries (no summary under header) content-not-array, no per-entry reasons no-valid-ranges + per-entry reasons

No wire-format change (schema structure/types untouched), no persistence/config change, no threshold change.

Tests

awork added 3 commits October 22, 2026 11:26
…st rejection reasons (#4252)

- RC1: a string content holding the summary text beside top-level refs
  (startId/startRef/messageId x endId/endRef) is recovered as ONE flat
  range — plain prose or a JSON-encoded {summary,topic} object; top-level
  summary wins over content text. Line-form strings whose first line
  carries refs headers are never hijacked, so every input that parses
  today is untouched. Broken JSON shells / non-object structures beside
  refs stay rejected. Diagnostics gain flatSummarySalvage.
- RC2: the refs separator class gains U+2212 (minus sign) and U+FF0D
  (fullwidth hyphen-minus); a header with two refs separated by an
  unrecognized non-wordy divider is now REJECTED naming the separator
  instead of silently narrowing to the first ref. A wordy gap stays a
  topic citation (single-message read unchanged).
- RC3: string content whose entries carry refs headers but validate to
  zero ranges reports kind=no-valid-ranges with the per-entry reasons
  (missing summary etc.) instead of a blanket content-not-array that
  dropped them (#362 visibility). Prose without refs headers keeps
  content-not-array.
- Schema descriptions document the accepted merged shape (goldens follow
  in the host commit).

kernel-change: model-facing compress parser accepts one more documented
shape and stops two silent-misfold paths observed live in v0.1.204
(#4252); owner-approved by the issue's fix plan.
…text

Description-text-only changes (verified: every diff line is a
description field, 78 symmetric pairs across the four goldens): the
content/startId/endId/summary parameter descriptions and the tool
description now document the accepted merged flat single-range shape
(summary text in content beside top-level startId/endId). No schema
structure, type, or requirement change.
…4252)

parseCompressInput emits a flat-summary salvage warn (same style as the
#3780 double-escape and #603 quote salvages) when the kernel recovers a
range from summary text riding in content beside top-level refs.
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-22_4252-flat-compress-salvage (a107517)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5055-pr.4255.07ceb44

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4255.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: accept merged flat compress form, recognize U+2212/FF0D dashes, report honest rejection reasons (#4252) [PR] fix: accept merged flat compress form, recognize U+2212/FF0D dashes, report honest rejection reasons (#4252) Oct 22, 2026
The lock-sync CI lane (#3782) also enforces the kernel's own lockfile;
the 0.0.151 -> 0.0.152 bump in kernel/package.json left it stale.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

接单:PR review(#4255 → Closes #4252)

查重: 不重复 —— 本单是 [PR] review 工单(对应 GitHub PR #4255,源头 issue #4252),不是新缺陷报告。按失败签名(compress parser / content-not-array / U+2212 separator / flat single-range)检索 open+closed issue 与近期 PR,无同机制并列条目;#3780/#603/#470 是相邻 salvage 先例,非同一缺陷。

标签: bug, pi, p1, scope/kernel(线上会话是 pi 车道;缺陷是缓存命中崩坏导致 ~2.6M token 回补风暴,按 owner 修订 2026-10-19 归 p1;修复主体在 kernel/)。

CI 现状(已查明,非代码问题):当前唯一红腿是 lock-sync,它把整个 test/static-gates 矩阵级联 skip 了(fail-fast tiering 设计行为,所以 check 名未展开):

  • PR 把 kernel/package.json 从 0.0.151 提到 0.0.152,根 package-lock.json 已同步更新,但 kernel/package-lock.json 漏了再生成——仍记录 acp-kernel 0.0.151,与 0.0.152 漂移;
  • lock-sync 的第二步(cd kernel && npm install --package-lock-only 后 git diff --exit-code)正是为这个模式设的([PR] kernel: sync package-lock.json to 0.0.136 (lock left at 0.0.133 by the bump chain) #3782 先例:0.0.134→0.0.136 连续三个版本漂在 0.0.133)。

处置:这是几行级小修,按「小问题直接修」直接在 PR 分支上补一个独立标注的 commit(只动 kernel/package-lock.json)再 push,让全量 test 矩阵真正跑起来;不另开 issue。同时我这边开始独立审 diff(kernel 变更,人工 review 档,auto-merge 不适用)。审完给八问结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):目标 issue #4252 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR p1 pi scope/kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compress calls rejected when the summary text rides in content next to top-level startId/endId — plus U+2212 dash silently folds the wrong range

1 participant