Repository navigation
fix: route Responses reasoning through compression pipeline (Codex cache explosion) - #71
Merged
Merged
Conversation
Responses API `reasoning` items were treated as opaque preamble and re-sent verbatim every turn, accumulating without bound (broke Codex's prompt-cache prefix and blew up context). Anthropic `thinking` already flowed through the compression pipeline as a tracked message and was bounded; only the Responses path was broken. Route `reasoning` the same way: convert each item to a tracked BiliMessage (contentType "reasoning") that carries the original item verbatim in rawResponsesItem. The kernel hides the message once its range is compressed, so old reasoning drops automatically and never needs to be merged into a text summary (which would corrupt the opaque encrypted_content). Escape hatch: ACP_REASONING_KEEP=none drops all reasoning items. Refs: dog/billion-context-pi#15
Addresses oracle review of PR #71 (blocking B1 + follow-ups): - reasoning vanishes from output once its turn is covered by a block (the central guarantee of the fix, previously untested) - reasoning id is stable across turns (Codex re-sends same items) - multiple reasoning items keep distinct ids and order - reasoning without encrypted_content (older API) round-trips Refs: dog/billion-context-pi#15
This was referenced Aug 9, 2026
Merged
ranxianglei
added a commit
that referenced
this pull request
Aug 9, 2026
Ported master's reasoning-through-compression (#71) and count_tokens (#72) features onto PR#73's layout-based patching architecture: responses.ts: - Removed 'reasoning' from OPAQUE_ITEM_TYPES (master: reasoning is conversation history, routed through compression, not preserved verbatim) - Added droppedReasoning counter + shouldDropAllReasoning() escape hatch (ACP_REASONING_KEEP=none) - Added 'reasoning' case to responsesToCore switch: creates a tracked BiliMessage (contentType 'reasoning', rawResponsesItem carried) so the kernel hides it once its turn is summarized; dropped reasoning uses 'continue' to skip layout.push so it is fully removed - Added reasoning re-emit to coreToResponses (assistant contentType 'reasoning' -> push raw verbatim) - patchOriginalItem needs no change: reasoning items are preserved via slot.original when their turn is live, dropped (nextById miss) when compressed -- correct by construction server.ts: - Added countTokens detection (isCountTokensRequest) + prepareCountTokens (read-only processTurn mirror so count_tokens reports post-compression count; ACP_COUNT_TOKENS_PASSTHROUGH=1 escape hatch) - Merged prepare ternary: countTokens -> anthropic -> openai -> responsesCompact -> responses Validation: typecheck clean, 216 tests pass (incl. 9 reasoning + 3 count_tokens + custom_tool_call fix), build ok.
This was referenced Aug 9, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Closes the root cause of dog/billion-context-pi#15 — the universal
biliproxy made Codex/Responses-APIreasoningitems explode context and break Codex's prompt-cache prefix.Root cause
reasoningitems were treated as opaque preamble (OPAQUE_ITEM_TYPESinsrc/responses.ts) and re-sent verbatim every turn, accumulating without bound:bili (3).log): preserved preamble items grew 1 → 2 → 3 → 9 → 11 → 50 (reasoning×49); input grew 42k → 78k tokens whilecachedstayed pinned at 14080 (theadditional_toolsprefix); 3×responses stream ended without completion(relay dropping oversized streams).Only the Responses path was broken. Anthropic
thinkingalready flows through the compression pipeline as a tracked message (src/anthropic.ts) and is bounded — which is why other clients are fine.Fix
Route
reasoningthe same waythinkingalready goes: convert each item to a trackedBiliMessage(contentType: "reasoning") that carries the original item verbatim inrawResponsesItem. The kernel hides the message once its range is compressed, so old reasoning drops automatically and never needs to be merged into a text summary (which would corrupt the opaqueencrypted_content).reasoningfromOPAQUE_ITEM_TYPES(keptadditional_toolsthere — Codex requires it atinput[0]).additional_toolsstill preserved verbatim at the front of every request.ACP_REASONING_KEEP=nonedrops all reasoning items.Why this is better than the closed #70
#70 was a positional band-aid (keep only the most-recent-response reasoning by index). It solved size but not cache. This PR routes reasoning through the kernel like every other message, so it is hidden exactly when its turn is summarized — uniform with the Anthropic path and no hand-rolled index logic.
Verification
npm run typecheck✅npm test→ 172/172 pass (169 + 3 new round-trip tests)npm run build✅as any/@ts-ignore.New env var
ACP_REASONING_KEEP— documented in both READMEs.