Repository navigation
fix: count custom-shape media parts in context-size decisions (images no longer invisible to folding) - #3412
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4321-pr.3412.1b838a9This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3412.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…3412 review) Review of #3398's opaque-media metering found pixels billing (the default auto route) billed EVERY gated non-text part at the flat PIXEL_IMAGE_FALLBACK_TOKENS (16K) regardless of content. Codex echoes its own {type:"compaction"} handoff items back on every post-compaction turn; each billed 16K phantom tokens in the preflight/nudge/side-guard meters, and the side-request guard hard-blocked (413) payloads that fit the window — tests/codex-compact-e2e.test.ts failed on the PR head. opaquePartCost (pixels) now prices by what the part carries: an embedded base64 image keeps the dimension-based tile cost (flat fallback only for long unparseable blobs >= 4KiB decoded), a remote ref keeps REMOTE_IMAGE_TOKENS, and carrier-less structured text stays at serialized bytes/4. The count walkers gate their #3398 increments on the same carrier predicate so non-image mass never rides the #1843 per-image multiplier or poisons the learned-cost EMA denominator. Bytes-mode pricing and known-shape counting are unchanged. Regression proof: codex-compact-e2e test 1 + two new unit tests in tests/issue3398-opaque-media-reserve.test.ts.
(ai: vllm-qwen/qwen3.8-27b) 审核中发现一个回归,已在 PR 分支上修复(commit 问题:CI 三个 shard-1 job(ubuntu×22/24、windows×22)确定性失败于 根因:pixels 计费(默认 auto 路由即 pixels)下, 修法(单一收口点 验证:本地定向测试全绿—— |
(ai: vllm-qwen/qwen3.8-27b) 审核结论(结构化八问)CI 已全绿(新 head
行为变更披露(旧→新+原因):
附带核实:PR 描述里"anthropic codec 顶层未知 block(含 document)被静默丢弃、另立 #3410"的观察与 判定:回归已修复并钉住测试,CI 全绿,PR 可以合并(请人工合并)。 |
d727d6e to
4430a2a
Compare
Custom-shape media parts that bili forwards verbatim — dsh read_image
results {type:"image",attachment:{...}} and same-class carriers on the
other wires — were priced 0 by every meter: not the kernel text model
(#488), not imageReserve (known shapes only), not the raw probe. A
session slicing one image into 58 read_image calls accumulated ~7MiB of
unpriced payload; preflight never fired and nothing ever folded.
Price every non-text part the codecs carry verbatim at serialized
bytes/4 (bytes mode) or PIXEL_IMAGE_FALLBACK_TOKENS (pixels mode),
capped: openai user-role parts (allNonTextParts sidecar), anthropic
nested tool_result media + redacted_thinking payloads, google unknown
parts, responses user-item custom parts + unknown item types. Charges
are gated exactly to what each codec forwards — roles whose content
reduces to text-only stay untouched. The probe gains structured-content
markers so such bodies reach the parser; count walkers stay in lockstep
for the #1843 learned-cost EMA. Folded ranges render distinct
[media: type ~KB] placeholders for opaque payloads.
No config surface added. Behavior change: sessions carrying
unrecognized non-text parts now count their true mass in preflight/nudge
decisions (was silent 0) — compression triggers where previously idle;
standard-shape image sessions make byte-identical decisions.
…p pricing responses control items (#3398) CI caught two defects in the first revision of the opaque-media metering: 1. The responses walker priced UNRECOGNIZED item types as opaque media. The codex lane re-prepends its control carriers (compaction handoffs, compaction_trigger) verbatim via the preamble path — they carry no media at all. A 304-byte side request was charged 2 x 16384 = 32768 tokens, tripping the fail-closed side-request guard (estimate >= limit x 1.15) into a deterministic 413 on a legitimate request. 2. opaquePartCost returned the flat PIXEL_IMAGE_FALLBACK_TOKENS in pixels mode. 'auto' billing resolves to pixels for every host that has not opted into bytes (#1843 L2), so every unrecognized carrier paid the unparsable-container flat cost regardless of actual size — orders of magnitude above reality for small payloads. Fix: opaquePartCost is now serialized-bytes/4 in BOTH modes (proportional lower bound: overcharging hard-blocks legitimate requests in fail-closed guards, availability #3128; undershoot only delays folding, the safe direction), and the responses walkers no longer price unknown item types at all — media on the responses wire rides input_image or user-item content parts, both already priced.
4430a2a to
f8b2a97
Compare
|
重新review |
(ai: vllm-qwen/qwen3.8-27b) 重新 review 完成(针对当前 head 结构化八问1. 是什么问题:宿主自定义形状的媒体部件(如 dsh 2. 什么原因导致的:三层计量全部漏掉同一批字节——内核文本模型只数 3. 影响范围与严重程度:发自定义形状媒体的宿主(dsh read_image 类)——核心承诺失灵(上下文永不折叠、触发代数读小文本通道对满窗口),属 p1 级;标准形状图片会话与纯文本会话完全不受影响(有回归钉测试)。 4. 怎么修复的:单一原则——codec 实际原样转发的、形状走认器不认识的部件,一律按序列化字节/4 计价,且在两种计费模式下行为一致;每条线路的门禁严格对齐该线路 codec 的真实保留规则(openai 只计 user 角色——其他角色在 codec 里降为纯文本;anthropic 计 redacted_thinking.data + tool_result 嵌套非文本块;google 计未知 part;responses 只计 user item 的自定义 part)。第二提交( 5. 是否有回归问题——逐项过完,无无意回归:
6. 是否破坏缓存:不涉及。计量只影响折叠决策,不改系统消息、不改前缀、prompt-cache 路由未触碰。 7. 是否改了设计方向:无转向。所有窗口门仍走单一咽喉点( 8. 是否引入新配置:无。零新增 env/config/默认值。 行为变更披露(旧值→新值)
披露的残留边界(均非阻塞,方向安全=只会晚折叠不会误杀):① 假设存在顶层自定义 item 携带 base64 媒体(responses 线),当前不计价——已知宿主均不这么发,媒体在该线上实际都走 input_image/user-item part;② google 线的探针只认 inlineData/fileData 字面量,完全无这些字段的未知 part 不进解析器;③ anthropic tool_result 内嵌未知类型块同理受探针限制。三者共同特征:只影响「假想载荷」,不影响任何现有宿主的真实流量。 验证记录(本轮独立执行)
判定可合。 当前 head |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.574-master.cda8c83 # this exact build
npm install -g billion-context@master # always the newest merged build
|
1 participant
Closes #3398
What was broken
dsh-style custom media parts —
read_imageresults shaped[{"type":"text",…},{"type":"image","attachment":{…}}]— were priced 0 by every size meter in bili:imageTokensInParsedBody) prices known shapes only (image_url,file, anthropicimage, responsesinput_image, googleinlineData/fileData) — a customattachmentpart falls through to 0;bodyHasImagesProbe) does not recognize such bodies at all and skips parsing.So a session slicing one image into 58
read_imagecalls accumulated ~7 MiB of payload that every decision treated as free: preflight never fired (small text channel vs full-window budget), the nudge gate stayed idle, nothing ever folded — exactly the log evidence in #3398 (session-f0bdca42: inbound flat at 7.2 MiB over ~25 requests, zero fold events, billed input only 14K→118K).Fix
Price every non-text part that bili's codecs forward verbatim at its serialized mass (serialized-bytes/4 in bytes billing mode,
PIXEL_IMAGE_FALLBACK_TOKENSfallback in pixels mode, subject to the existing cap), gated exactly to what each codec preserves:role:"user"messages (ridesrawOpenaiContentParts, #1205)tool_result(base64 / url / other) +redacted_thinking.datarawGoogleParts(text / functionCall / functionResponse / videoMetadata skipped)Roles whose content reduces to text-only in the codec are never charged (those bytes do not ride the wire). The raw probe gains structured-content markers so such bodies reach the parser; the count walkers stay in lockstep so the #1843 learned-cost EMA calibrates against real relay billing for these carriers too. Folded ranges render a distinct
[media: <type> ~<KB>]placeholder in summaries for opaque payloads (honest note; unlike standard images they have no stored pixels to restore).Behavior change (old → new)
[media: …]notes when folded out. Standard-shape image sessions make byte-identical decisions (all-text array bodies stay free — pinned by test).Config surface
None added. No new config fields, env vars, or CLI flags — a pure metering change under the existing imageReserve math.
Tests
tests/issue3398-opaque-media-reserve.test.ts: 9 unit tests (per-wire pricing, role gates, regression pins: all-text array stays free, standard image_url unchanged, top-level anthropic document stays unpriced because the codec drops it) + 1 hermetic integration test (200K window, bytes-billing route, 10×72 KB attachment slices + filler history → asserts preflight fires, filler folds off the wire, all slices stay resident). Verified against the bared source: the integration test fails pre-fix withgot 0 summarization calls— the exact symptom from the issue log.Observation (out of scope)
While verifying carrier preservation per wire, noted that the anthropic codec's block switch has no default case: unknown top-level blocks (including
document) are silently dropped from rebuilt history with no warning, while every other wire preserves unknown carriers verbatim. Not touched here (different failure domain — silent data drop, not metering); filed separately as #3410 if you want it tracked.