Skip to content

fix #589: forward trailing usage-only frame on raw tool-call rounds - #593

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-07_openai-usage-frame
Sep 7, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-07_openai-usage-frame

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Root cause

dsh sends stream_options: {include_usage: true}. SGLang/vLLM deliver usage in a trailing empty-choices frame (choices:[] + usage), sent AFTER the finish chunk and BEFORE [DONE].

In the OpenAI loop adapter, tool-call rounds take the raw-passthrough path (sawRealToolCall): the finish chunk forwards verbatim (SGLang puts no usage there), but the trailing usage frame hit the !choice branch of parseStream, was parsed into the internal usage event, then continue — never forwarded to the client. Result: every tool-call round reported 0/0 usage; dsh's context circle (last-wins pressure from usage chunks) collapsed to a few hundred tokens (surface-only growth). Rebuilt rounds were unaffected (usage embedded via emitCompletion), which is why only tool-call STEPS in the user's session jsonl carried inputTokens:0.

Not #590-class (that was pi over-reporting via the #408 backfill); this is under-reporting via a dropped frame. Affects all include_usage clients, not just dsh.

Repro (before): direct upstream sends finish → choices:[] usage:{...} → [DONE]; via bili the usage frame vanished.

Fix

src/loop/adapter-openai.ts:

Verification

  • new tests/issue589-usage-frame.test.ts (4 tests): raw round forwards the frame patched + ordered finish→usage→[DONE]; verbatim when credit=0; non-tool rounds keep swallowing it (no dup); frames without usage not forwarded
  • full suite: only the 2 known plugin-agent env fails; typecheck + build clean
  • live: replayed a dsh-shaped tool-call request through a local build — client now receives the trailing usage frame before [DONE]

Fixes #589

OpenAI stream_options.include_usage clients (dsh, OpenAI SDK) read usage
from the trailing empty-choices frame. The loop adapter parsed it into
the internal ledger then dropped it, so raw tool-call rounds reported
0/0 to the host. Forward it verbatim on sawRealToolCall rounds, patched
with the #408 prepare-time credit via the extracted patchUsageChunk
helper (also reused by the finish-chunk raw branch). Rebuilt rounds
stay unchanged (emitCompletion embeds usage; forwarding the uncredited
raw frame there would duplicate and undercut the backfill).
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-07_openai-usage-frame (2ed7c59)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-593

Each push to this PR publishes a new version under the pr-593 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr593.tgz
npm install -g package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · qwen3.8-27b

[bot] 🏷 Review 完成,结论:建议合并。我没有只读 diff,把根因、修复、影响面都独立复验了一遍。

1. 根因复核(代码级确认,与你描述一致)
旧代码 src/loop/adapter-openai.ts 的 !choice 分支:尾帧(choices:[] + usage)被解析进内部 usage 事件后直接 continue,从不回传客户端;raw 工具轮(sawRealToolCall)的 finish 帧原样转发,而 SGLang 不在 finish 帧放 usage → include_usage 客户端每个工具轮看到 0/0。链路成立。

2. 红绿验证(新测试文件在修复前后各跑一遍)

  • PR 父提交 01fd5de(修复前):tests/issue589-usage-frame.test.ts case 1/2 失败(客户端流里没有尾帧),case 3/4 通过(非工具轮、无 usage 空帧本来就吞——行为不变)
  • PR head 2ed7c59:4/4 通过

3. 全链路实测(不只单元层)
起了一个仿 SGLang 帧序的假上游(tool delta → finish(tool_calls) → choices:[]+usage{prompt_tokens:50000} → [DONE]),本地构建 bili 代理,发 dsh 形状请求(stream_options:{include_usage:true} + tools):

  • 旧构建(01fd5de):客户端收到 finish → [DONE],尾帧丢失(usage frame 未出现)→ 复现 bug
  • 新构建(2ed7c59):客户端收到 finish → usage 帧(prompt_tokens:50000,顺序正确)→ [DONE] → 修复生效

4. 代码审查要点

5. 验证结果(pr-593-head @ 2ed7c59)

  • typecheck ✅ / build ✅
  • 全量测试:1146 个,1145 过;唯一失败是 tests/launcher.test.ts "resolveClientCommand: codex/claude resolve to themselves" —— 本沙箱恰好装了 /usr/bin/codex,命令路径解析的环境差异,与本 PR 无关(非 usage 路径)
  • 真实 codex 的 e2e 需要本地 upstream(默认 127.0.0.1:8199),本环境不可达没跑。这是 request-pipeline 改动,建议合并前手动触发一次 ci-e2e workflow(或接受上述单元红绿 + 全链路 SSE 实测的覆盖)

6. 查重
全库检索无重复项。关联关系:#589(本 PR 目标 issue);#590 同链路反方向(那个是 pi 过量回填,这个是少报丢帧,机制不冲突);#425/#408 是被复用的回填机制;#591(pi 排除)已随 v0.1.88 进 master。互不冲突。

结论:根因分析准确,方案打到核心(补转发丢掉的帧,而不是在 dsh 侧采样或阈值上绕),改动最小、证据链完整(单元红绿 + 全链路前后对比)。可以合并——合并操作留给你(系统规则禁止我 merge):#593

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bili dsh 原版那个 圆圈, 显示当前上下文大小的. 经常显示只有几百token

1 participant