Repository navigation
fix #589: forward trailing usage-only frame on raw tool-call rounds - #593
Conversation
OpenAI stream_options.include_usage clients (dsh, OpenAI SDK) read usage from the trailing empty-choices frame. The loop adapter parsed it into the internal ledger then dropped it, so raw tool-call rounds reported 0/0 to the host. Forward it verbatim on sawRealToolCall rounds, patched with the #408 prepare-time credit via the extracted patchUsageChunk helper (also reused by the finish-chunk raw branch). Rebuilt rounds stay unchanged (emitCompletion embeds usage; forwarding the uncredited raw frame there would duplicate and undercut the backfill).
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-593Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr593.tgz
npm install -g packageThis comment is automatically updated on each push. |
[bot] 🏷 Review 完成,结论:建议合并。我没有只读 diff,把根因、修复、影响面都独立复验了一遍。 1. 根因复核(代码级确认,与你描述一致) 2. 红绿验证(新测试文件在修复前后各跑一遍)
3. 全链路实测(不只单元层)
4. 代码审查要点
5. 验证结果(pr-593-head @ 2ed7c59)
6. 查重 结论:根因分析准确,方案打到核心(补转发丢掉的帧,而不是在 dsh 侧采样或阈值上绕),改动最小、证据链完整(单元红绿 + 全链路前后对比)。可以合并——合并操作留给你(系统规则禁止我 merge):#593 |
Root cause
dsh sends
stream_options: {include_usage: true}. SGLang/vLLM deliver usage in a trailing empty-choices frame (choices:[]+usage), sent AFTER the finish chunk and BEFORE[DONE].In the OpenAI loop adapter, tool-call rounds take the raw-passthrough path (
sawRealToolCall): the finish chunk forwards verbatim (SGLang puts no usage there), but the trailing usage frame hit the!choicebranch ofparseStream, was parsed into the internal usage event, thencontinue— never forwarded to the client. Result: every tool-call round reported 0/0 usage; dsh's context circle (last-wins pressure from usage chunks) collapsed to a few hundred tokens (surface-only growth). Rebuilt rounds were unaffected (usage embedded viaemitCompletion), which is why only tool-call STEPS in the user's session jsonl carriedinputTokens:0.Not #590-class (that was pi over-reporting via the #408 backfill); this is under-reporting via a dropped frame. Affects all include_usage clients, not just dsh.
Repro (before): direct upstream sends finish →
choices:[] usage:{...}→[DONE]; via bili the usage frame vanished.Fix
src/loop/adapter-openai.ts:patchUsageChunk(eventStr, parsed, u, hostCredit)— the 代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408 finish-chunk credit patch, now reused by both call sites!choicebranch: whensawRealToolCall, additionallyyield {kind:"meta", chunk: patchUsageChunk(...)}so the client receives the trailing usage frame, credited like the finish chunk. Rebuilt rounds stay unchanged (emitCompletion already embeds usage; forwarding the raw frame there would duplicate it AND undercut the backfill for last-wins clients).Verification
tests/issue589-usage-frame.test.ts(4 tests): raw round forwards the frame patched + ordered finish→usage→[DONE]; verbatim when credit=0; non-tool rounds keep swallowing it (no dup); frames without usage not forwarded[DONE]Fixes #589