Skip to content

[PR] fix: salvage double-escaped newline in compress line-form string content - #3785

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-19_compress-double-escape-salvage
Oct 19, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-19_compress-double-escape-salvage

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What this fixes

A model that serializes its compress args as text escapes the header/summary separator twice, so the documented line-form content string carries a literal backslash+n (two characters) instead of a real newline. Undecoded, the kernel finds no newline → empty summary → rejects with kind=content-not-array, which hides the real reason (missing summary after the refs header line) and sends the model to retry the same broken shape. This is the next rung of the compress-arg shape-drift ladder (#1494 / #1559).

Repro from the issue: {"content": "m00003–m00107 example topic\\nTASK AS OF THIS BLOCK: ..."} fails, while the identical payload with a real newline succeeds.

Root cause

parseCompressInput → kernel parseObjectValue: a content string with no real newline goes through splitLineEntries (single chunk = whole string) → parseLineEntry (nl === -1) → summary === "" → rejected. The bare branch then sets diag.kind = "content-not-array" and returns without surfacing the per-entry reasons it did collect.

Fix (host-side, no kernel change)

In src/compress-tool.ts parseCompressInput, when the first parse is not ok, decode \n / \r\n once from a double-escaped content string and retry, adopting the result only if it yields strictly more ranges. Same pre-normalization discipline as #1494.

Safety invariant: a string with no real newline always parses to zero ranges undecoded, so adopting only on strictly-more-ranges can never degrade a working parse. A payload that already carries a real newline is left untouched (a literal backslash-n inside its summary body survives verbatim).

Boundary note: the shared home for any future multi-lane fix is the kernel parser (kernel/src/parse-compress-input.ts parseLineEntry). Kept host-side here on purpose — no kernel release cycle is needed for a proxy-lane arg-shape salvage (#1494 precedent), and every execution lane already converges on this single entry point.

Behavior change (disclosed)

  • Before: a double-escaped line-form string was rejected with [Compression FAILED: ... kind=content-not-array ...]; the model retried the same shape.
  • After: it is decoded once and recovered as a valid range, then folded normally (when it decodes into refs + a non-empty summary); it is still rejected unchanged when it does not decode into that form (e.g. no refs header).

No wire-shape change; both compression modes are unaffected (arg parsing precedes mode-specific handling); no message-ref/id reissue.

Config surface

None. No new env var, config key, or CLI flag.

Tests

tests/issue1494-compress-params.test.ts (+4):

  • positive — double-escaped string recovered at parse level (startRef/endRef/topic/summary correct)
  • positive — folds end-to-end through applyRanges (no FAILED, no content-not-array)
  • negative — a payload with a real newline is NOT re-decoded (literal backslash-n in the body preserved)
  • negative — a ref-less escaped string is NOT salvaged into a fake range

Targeted suite green locally (13/13 in the file); the full-suite verdict is remote CI.

Closes #3780

@ranxianglei ranxianglei changed the title fix: salvage double-escaped newline in compress line-form string content [PR] fix: salvage double-escaped newline in compress line-form string content Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-19_compress-double-escape-salvage (c824436)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4499-pr.3785.4a7aade

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3785.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复——按失败签名(kind=content-not-array / double-escaped)检索 open+closed issue 与 PR:仅本 PR #3785 与其原始 issue #3780 在途;同家族 #1494/#1495/#1497/#1559 均已关闭,无并行重复修复。

标签: bug + dsh + scope/kernel + p2(缺陷修复;复现环境为 dsh desktop 插件 lane,机制本身是模型参数序列化漂移、非客户端专属;属压缩管线参数解析;能用但特定条件下有明显问题→p2)。

审核进行中(agent 提交的 PR,按规则不重做、只核对代码与 CI):

  • 当前 head (123301b) 的 CI 全绿(test ubuntu/windows × node 22/24 各 shard + static gates 全部 success);
  • 安全不变量已对 master 上的内核解析器源码逐条核验:kernel/src/parse-compress-input.ts 自 PR base 以来零改动,且「无真实换行的 content 字符串在未解码时必解析为 0 ranges」在 master 代码上成立(line-form 回退路径 parseLineEntry nl===-1 → summary 为空 → 拒绝;JSON-array 路径若成功则 ok、根本不触发 salvage),strictly-more-ranges 采纳条件不可能劣化任何可工作解析;
  • master 自 base 前进 99 个文件,与本 PR 触碰的 2 个文件零重叠;
  • 正在 rebase 到最新 master 并跑本地定向测试,rebased head 的 CI 绿后按结构化八问给最终审核结论。

…ent (#3780)

A model serializing its compress args as text escapes the header/summary
separator twice, so the documented line-form 'content' string carries a
literal backslash+n instead of a real break. Undecoded, the kernel finds no
newline, yields an empty summary, and rejects with kind=content-not-array —
hiding the real missing-summary reason and sending the model to retry the same
broken shape.

Host-side pre-normalization in parseCompressInput (same discipline as #1494):
when the first parse is not ok, decode backslash-n / backslash-r-backslash-n
once and retry, adopting the result only if it yields strictly more ranges. A
string with no real newline always parses to zero ranges undecoded, so a
working parse can never be degraded; payloads that already carry a real newline
are left untouched. The kernel parser stays the shared home for any later
multi-lane fix.
@ranxianglei ranxianglei added bug Something isn't working dsh p2 scope/kernel labels Oct 19, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-19_compress-double-escape-salvage branch from 123301b to c824436 Compare October 19, 2026 08:09
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #3785 审核结论(独立重审)

已执行:rebase 到最新 master(123301b → c82443600,零冲突——master 自 base 前进 99 文件,与本 PR 的 2 个文件零重叠;kernel/src/parse-compress-input.ts 自 base 以来零改动)→ 本地 build + typecheck + 定向测试 13/13 全过 → rebased head 的远程 CI 全绿(18 success / 12 skipped 为门控 e2e lane / 0 failure)。以下结论基于我自己的 diff 与源码核验,未沿用 issue 侧分析。

八问:

  1. 是什么问题:模型把 compress 参数当文本序列化时,行形式的头/摘要分隔符被转义两次,content 里是字面 backslash+n(两个字符)而非真换行;内核找不到换行 → summary 为空 → 拒绝,且报出的 kind=content-not-array 掩盖了真实原因(refs 头行后缺 summary),模型因此重试同一个坏形状。
  2. 什么原因导致的:内核 parseLineEntry 以真实 \n 为头/摘要边界,无真实换行的单 chunk 必得空 summary 被拒;bare 分支只置 kind=content-not-array,不暴露已收集的 per-entry 原因。属 compress 参数形状漂移梯度的下一阶(compress failed may be caused by params #1494/fix: surface parse-dropped compress entries; normalize single-object content (#1494) #1497/[kernel] parse ladder: string in the content slot ({"content": "m1–m5 …"}) is rejected as content-not-array — next salvage candidate for weak-model drift #1559 同族)。
  3. 影响范围与严重程度:任何「模型输出双转义行形式」的场景都会触发(复现环境 dsh desktop lane + DeepSeek;机制是模型序列化行为,不限客户端)。p2:该次压缩调用失败 + 浪费一轮重试,会话本身可用、可自愈。
  4. 怎么修复的:host 侧预归一化(compress failed may be caused by params #1494 同一纪律)——首次解析失败且 content 是「无真实换行但含字面 \n」的字符串时,解码一次后重试,仅当 range 数严格更多才采纳。我逐条对 master 内核源码核验了安全不变量:无真实换行的字符串未解码时必解析为 0 ranges(line-form 回退路径 nl===-1 → 空 summary → 拒;JSON-array 路径若成功则 ok、根本不进 salvage),所以采纳条件不可能劣化任何可工作解析;含真实换行的 payload 提前 return 不碰,正文里的字面 backslash-n 逐字保留。多 lane 共享归宿(内核 parseLineEntry)以注释 boundary note 留档,本次刻意不动内核——省一个 kernel release 周期,且所有执行 lane 都收敛在这个入口。
  5. 是否有回归问题:无。触碰路径既有行为逐项清点:①所有成功解析不受影响(salvage 由 !ok 门控);②失败解析中仅该形状改变结局,其余拒因/日志不变;③日志仅新增一条 rescue warn,其余行原样;④wire 形状不变(参数解析先于模式处理,plugin/proxy 两模式共用此入口);⑤无阈值/时序/输出格式漂移,无 fail-fast→吞错(解不出来的 payload 仍按原样拒绝)。rebase 后在 master 上重新验证成立(见首段)。
  6. 是否破坏缓存:不涉及——未触碰缓存/重试/增量游标/去重键;无 message-ref/id 再发号。
  7. 是否改了设计方向:没有——延续 compress failed may be caused by params #1494 的 host 侧预归一化路线,边界注释明确未来多 lane 修复的内核归宿(届时走 human-gated kernel 变更)。
  8. 是否引入了新的配置:无(与 PR 声明一致)。

测试有效性(④):4 个新测试有牙——正例钉死 startRef/endRef/topic/summary 映射 + applyRanges 端到端 receipt([Compressed m00001–m00002 → 1 block(s)、无 FAILED/content-not-array、active blocks==1);两个负例分别钉死「含真实换行不再解码」与「无 refs 不救成假 range」,正负成对符合 gate 纪律(#3617)。本 bug 回归时 test 10/11 会抓住。

已知残留(不阻塞):同一字符串内真实换行与字面 \n 混合转义的 payload 不救(设计如此,避免动到有效载荷);OP 建议 2(model-facing FAILED 消息暴露 invalidReasons / 独立 kind)属内核诊断面,留待另行决策——rescue 落地后该误导 kind 场景已大幅收窄。

§7.4 auto-merge 判定:全部满足(单模块、回归测试复现原 bug、rebased head 绿、无 config/persistence/wire/kernel 触碰、纯 fix:、diff 干净、Fixes #3780、不涉 load-bearing infra)→ 可合入,等待人工合并。

@ranxianglei
ranxianglei merged commit f74f266 into master Oct 19, 2026
40 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.622-master.f74f266 (rolling master tag).

npm install -g billion-context@0.0.622-master.f74f266  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compress: literal backslash-n in string content still fails as kind=content-not-array, hiding the real missing-summary reason

1 participant