Skip to content

[PR] test(preflight): cover calibrated exhaustion on the DSH Responses lane - #3968

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
lisihao:2026-10-20_responses-calibration-regression
Oct 20, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
lisihao:2026-10-20_responses-calibration-regression

Conversation

@lisihao

@lisihao lisihao commented Oct 20, 2026

Copy link
Copy Markdown

The calibrated preflight regression covers Chat Completions, while DSH's plugin lane can send Responses requests. This adds a Responses SSE fixture and a x-bili-plugin: dsh case that exercises actual summary folding, then checks that calibrated exhaustion refuses the main request and reports a context figure above the tested input budget.

The runtime fixes are already merged in #3650 and #3649. This PR adds the missing wire/lane regression; it does not resubmit those changes or alter overflow policy. Refs #3645.

Validation:

  • The new case fails against the unmodified v0.1.202 preflight/server sources on its diagnostic assertion (the quoted raw size is below the test window).
  • All four cases in tests/issue3645-calibrated-futility.test.ts pass against the corrected sources.
  • TypeScript checking on base 440c5114 and on this test-only head reports the same existing TS2307 error for mysql2/promise in src/storage/mysql.ts; this PR leaves that unrelated dependency change untouched. The four targeted cases pass on the current base plus this change.

The test uses a loopback fake upstream and synthetic input. It requires no credentials, real model request or private diagnostic material.

@ranxianglei ranxianglei changed the title test(preflight): cover calibrated exhaustion on the DSH Responses lane [PR] test(preflight): cover calibrated exhaustion on the DSH Responses lane Oct 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: qwen/qwen3.8-27b)

查重: 不重复。 检索 open+closed 的 issue/PR(关键词 calibrated futility / responses lane / preflight + 失败签名 tests/issue3645-calibrated-futility.test.ts):唯一同主题条目是本 PR;上游 #3645 已关闭、修复 #3650/#3649 已合入 master。#3663(失败文案混用 calibrated/raw)是不同症状(报告措辞),与本 PR 的回归测试缺口不重叠。

标签: pr(既有)+ dsh(DSH 插件车道)+ scope/ci(纯回归测试补强)+ p3(测试覆盖缺口,无用户可见行为变更)。

状态: PR 为 draft、head a15f500,base 已落后 master(#3966 mysql2 依赖修复已合入,PR 描述里提到的 TS2307 在现 master 上应已消失)。正在做:拉取分支 → rebase 到当前 master → 定向跑 tests/issue3645-calibrated-futility.test.ts → 核对「对未修复源码该 case 必须失败」的声明 → 按八问出完整 review 结论。稍后更新。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: qwen/qwen3.8-27b)

PR 审核结论(独立重审,非复读 issue)

已完整走查 diff(+95/-5,仅 tests/issue3645-calibrated-futility.test.ts)并双向实测。先说验证证据,再按八问收口:

正向验证:把该 commit cherry-pick 到当前 master(74b894d66,与 #3963 合入后一致)——零冲突;npm run typecheck 干净(描述里提到的 TS2307/mysql2 在 master 上已由 #3966 修复,rebase 后不复存在);npm run build 绿;node --import tsx --test tests/issue3645-calibrated-futility.test.ts 4/4 通过。

负向验证(测试有牙):在 rebased head 上精确回退 #3650 的修复 hunk(preflight.ts 的校准比较全部还原 + server.ts fail-fast 引用还原为 session.stats.lastInputTokens),新 case 恰好死在诊断断言上:

quoted Responses figure must be calibrated and over-window (got ~18160 vs 30000)

即引用了未校准的 raw 尺寸(≈18160 < 窗口 30000)——与作者声明的 v0.1.202 失败模式一字不差。另有一个值得写进记录的事实:pre-fix 代码下本 case 的折叠并未被跳过(summaries≥1 仍过),说明这个 Responses case 钉的是「fail-fast 引用尺度」这一半(server.ts 侧),与原 chat case 钉的「futility gate 误判」一半互补,两条腿各管一个回归面。

八问

  1. 是什么问题:[Bug][preflight] Calibrated deficit vs uncalibrated foldable mass prematurely skips raw folds (0.1.199) #3645 的回归防护只覆盖了 Chat Completions wire,而 DSH 插件车道实际会发 Responses 请求(/v1/responses),该 wire/lane 组合没有护栏。本 PR 纯测试补齐这个缺口。
  2. 什么原因导致的:覆盖缺口,不是新 bug。根因([Bug][preflight] Calibrated deficit vs uncalibrated foldable mass prematurely skips raw folds (0.1.199) #3645:未校准可折叠质量 vs 校准后缺口、两种尺度互比)已由 [PR] fix(preflight): compare foldable mass to deficit on one calibrated scale (#3645) #3650/[PR] fix: tier-recovery candidate search slides past over-budget pairs instead of declaring all tiers exhausted #3649 修掉并合并;原回归只用 chat wire,Responses 车道无人看守。
  3. 影响范围与严重程度:零运行时行为变更(不碰任何 src/);若未来有人把校准尺度修复打回去,DSH Responses 车道的校准耗尽场景会再次引用 sub-window 的 raw 数字当「超窗」依据——目前无用户可见故障,属测试层缺口,p3 合理。
  4. 怎么修复的:加一个符合真实 API 事件名的 Responses SSE fixture(response.created → … → response.completed),mockUpstream/boot 按 wire 参数化(默认 openai,旧调用点不受影响),新增 dsh 插件车道 case:k̂=2 种子会话 + 「摘要比被折 span 更大、折叠永远关不上缺口」的构造,断言四件事——真发生了摘要折叠(summaries≥1)、主请求未被转发(forwarded==0)、确定性 413 + preflight_compress_failed、引用的 context 数字是校准后且超窗。所有 fixture 尺寸都带显式 sizing 守卫断言,意图自文档化。
  5. 是否有回归问题:无。逐项清点既有行为:mockUpstream()/boot() 签名各加一个默认参数,openai wire 下输出逐字节不变;原有三个 case 的断言一行未动;无任何源码改动 → 生产行为面为零。高危模式(语义漂移/吞错/阈值时序漂移)逐项不适用。
  6. 是否破坏缓存:不涉及(纯测试,不触碰缓存/重试/增量游标/去重键)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

代码质量:命名达意(responsesSse/responsesPlain/responsesBashPair/postResponses/MockWire)、风格与文件既有惯例一致、无死代码、无新依赖;fixture 的事件序列与 OpenAI Responses SSE 实际形状吻合。达标。

唯一程序性问题:分支基线 440c51146(mysql-backend 那条合并线)不在 master 主线上——mysql 后端后来经 #3955/#3966 重新落地,所以 GitHub 显示 behind、head 上 103 个 check-run 全 skipped(draft 未触发 CI)。我已验证:cherry-pick 到 74b894d66 零冲突且全绿。请 rebase 到当前 master 并取消 draft,CI 跑完即可合并;rebase 是单 commit 平移,无实质工作。若需要我代为推送到你的 fork 分支,说一声即可。

判定:审核通过,待 rebase + un-draft 后可合并。

@ranxianglei
ranxianglei marked this pull request as ready for review October 20, 2026 16:42
@ranxianglei
ranxianglei merged commit 5d2ea3c into ranxianglei:master Oct 20, 2026
67 of 103 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.691-master.5d2ea3c (rolling master tag).

npm install -g billion-context@0.0.691-master.5d2ea3c  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants