Repository navigation
feat: OpenAI Responses API support (/v1/responses) - #4
Merged
Merged
Conversation
Third protocol alongside anthropic + openai chat completions. This lets
billion-context serve clients that only speak the Responses API (notably
Codex 0.147, which dropped wire_api='chat' and forces 'responses').
New files:
- src/responses.ts: ResponsesRequestBody (input/items) <-> CoreMessage.
input items are typed (message/function_call/function_call_output) and
map closer to core messages than chat completions does.
- src/stream-responses.ts: Responses SSE rewriter. Responses carries the
event type in the SSE event: line (not in JSON data, unlike chat).
Suppresses compress function_call arguments, emits result as a text
delta + clean response.completed at the end.
- tests/proxy-responses-stream.test.ts: passthrough + compress-suppress.
server.ts wiring:
- detectProtocol: /responses -> 'responses'
- prepareResponses: responsesToCore -> processTurn(text-only) ->
coreToResponses -> inject instructions (system) + nudge (trailing
user item) + injectResponsesTool (flat Responses tool format).
- forward: responses branch uses rewriteResponsesSseStream (single-round;
multi-round compress loop is chat-completions-only for now).
- non-stream: rewriteResponsesJsonResponse.
compress-tool.ts: COMPRESS_TOOL_RESPONSES (flat {type:function,name,
parameters} form, not nested function:{...}).
compressLoopResponsesStream mirrors compressLoopStream but for the Responses API: accumulates function_call items (output_item.added + function_call_arguments.delta), executes proxy-only calls, appends function_call + function_call_output items to input[], and re-requests (up to 5 rounds). Real function calls and plain text pass through. This fixes the single-turn bug where a compress call ended the Codex turn, leaving the model unable to continue its work. - src/compress-loop-responses.ts (new): the loop + SSE event classifier - src/compress-loop.ts: export buildVisibilityMarker (reused) - src/server.ts: Responses branch now uses the loop instead of the single-turn rewriter - tests/proxy-responses-stream.test.ts: multi-round test with mocked fetch (compress -> re-request -> continue)
…t.delta) Fixes the 'OutputTextDelta without active item' error from Codex. The Responses API requires a message item to be declared via response.output_item.added + response.content_part.added BEFORE any response.output_text.delta for that item. Previously the proxy emitted bare output_text.delta with an undeclared fixed item id. Changes: - classifyResponsesSseEvent: mark response.created/in_progress/completed as isMeta so round 2+ suppresses them (merged stream = single open) - buildMessageItemSequence replaces buildOutputTextDelta: emits the full added+content_part.added+delta+done+content_part.done+item.done sequence - Unified passthrough: round 2+ now passes upstream content events as-is (preserving their own output_item.added declarations) instead of re-wrapping deltas with a synthetic item id - nextOutputIndex counter tracks emitted items across rounds
…esponses_lite) Three layered bugs prevented Codex code_mode from working through the proxy. All verified with real Codex CLI + comfly relay (gpt-5.6-sol, responses_lite). 1. additional_tools dropped (src/responses.ts) responsesToCore's default branch discarded all unknown item types, including `additional_tools` which carries the code_mode exec/wait tool definitions and must stay at input[0] (openai/codex client.rs splice(0..0)). Fix: OPAQUE_ITEM_TYPES set + preamble capture; prepareResponses re-prepends them verbatim so code_mode tool exposure is preserved. 2. responses_lite contract broken (src/server.ts) injectResponsesInstructions wrote the compress prompt into the top-level `instructions` field, but responses_lite requires instructions="" (the real prompt lives in input[1] developer message). A non-empty instructions field took the response off the lite processing path. Fix: inject compress prompt + base system as a developer message inside input (after additional_tools, before conversation); top-level instructions is never touched. 3. exec results dropped -> infinite loop (src/responses.ts) custom_tool_call / custom_tool_call_output (code_mode exec call + result) hit the default branch and were discarded. The model never saw its own tool results, so it re-issued the same exec call every round. Fix: add explicit cases in responsesToCore (-> assistant tool-call / tool tool-result) and coreToResponses (-> custom_tool_call / custom_tool_call_output), so exec history round-trips through compression. Also: - ACP_DUMP_REQ=1 gates request body dumps (avoid concurrent file corruption) - ACP_DEBUG logs input item types + rebuilt conversion stats - ACP_NO_COMPRESS_PROMPT / ACP_RENDER_NONE are debug toggles - Text-protocol scaffolding (stream-responses.ts, buildCompressTextSystemPrompt) remains in tree but is inert under the default function protocol.
Addresses review findings on the HTTP layer before public/long-running use: - readBody: cap at 100 MB (reject oversized bodies with 413) to avoid memory exhaustion; previously unbounded. - All three upstream fetch sites (server.ts forward, compress-loop, compress-loop-responses) now use fetchWithTimeout (10 min AbortController) so a stuck upstream cannot hold a client connection open forever. - config.ts safeReadJson no longer silently swallows parse errors (only ENOENT is quiet); a malformed providers file now logs a warning. - server.ts catch returns 413 for body-size rejections (vs generic 502). - session.ts: documents the known limitation that state is in-process only (restart drops it; multi-instance does not share) and that session id defaults to a hash of the first user message (multi-tenant must send x-acp-session). Persistence tracked as TODO. New file: src/fetch-util.ts (fetchWithTimeout + MAX_REQUEST_BYTES).
…ests
Addresses four protocol-correctness bugs found by reviewing the Responses
implementation. All covered by new tests (tests/proxy-responses-review.test.ts).
1. Standard function tools were rewritten as custom tools (responses.ts)
coreToResponses used PROXY_TOOL_NAMES.has(name) to pick function_call vs
custom_tool_call. A host's normal function tool (e.g. get_weather) hit the
else branch and was emitted as custom_tool_call — changing Responses API
semantics and risking upstream rejection.
Fix: track call_ids that entered as custom_tool_call/custom_tool_call_output
in a Set, and emit the original item type by call_id membership.
2. Fabricated success after a failed terminal (compress-loop-responses.ts)
classify only treated response.completed as terminal. response.failed and
response.incomplete passed through (default), then buildCompleted emitted a
synthetic response.completed — giving the client a contradictory state
machine (failed AND completed).
Fix: classify failed/incomplete/error as terminal (isMeta, no auto-yield),
capture the raw event, and at stream end replay it verbatim instead of
synthesizing completion.
3. Valid CRLF SSE streams were swallowed (compress-loop-responses.ts)
The SSE loop split on indexOf("\n\n"). A service emitting \r\n\r\n event
separators never matched, so events (including response.created/completed)
vanished and the proxy synthesized a bogus response id.
Fix: normalize CRLF / lone CR to LF after each decode; join multi-line
data: fields per SSE spec (previously only the first data: line was kept).
4. Unknown Responses input items silently dropped (responses.ts)
The default switch branch dropped anything outside the fixed type list, so
OpenAI adding new item types (or provider extension types) would lose data.
Fix: preserve unknown items verbatim in the preamble (forward-compat).
Also:
- ACP_DUMP_REQ=1 gates request body dumps (avoid concurrent file corruption)
- ACP_DEBUG logs input item types + rebuilt conversion stats
- [acp-usage] logs per-round cache hit ratio from response.completed usage
- [acp-compress-args]/[acp-compress-input] diagnostics for compress parse failures
…ss/acp_status) injectResponsesTool only injected COMPRESS_TOOL_RESPONSES, but the compress loop (compress-loop-responses.ts executeProxyTool) dispatches all 4 tool names in PROXY_TOOL_NAMES, and the injected compress prompt documents search_context / decompress usage. The model could read the instructions, try to call search_context to locate a block, then decompress it — but those tools were absent from the tools array, so the model reported 'cannot call search_context'. This was a declaration/implementation mismatch: the loop was ready to execute decompress/search_context/acp_status, but server.ts never declared them. Fix: add DECOMPRESS_TOOL_RESPONSES, SEARCH_CONTEXT_TOOL_RESPONSES, ACP_STATUS_TOOL_RESPONSES (Responses flat format), group them as ACP_TOOLS_RESPONSES, and injectResponsesTool now appends all four (idempotent on tool name, matching the OpenAI chat path which already injects all four via ACP_TOOLS_OPENAI).
Four protocol-correctness issues found by review.
1. Query string broke protocol detection (server.ts)
detectProtocol used url.endsWith("/responses") etc. A request like
/v1/responses?foo=1 did not match, so compression/injection silently
disabled. Strip the query string (url.split("?")[0]) before matching.
2. CRLF SSE events lost across ALL paths (not just responses)
The SSE splitter only matched \n\n. CRLF-emitting upstreams (\r\n\r\n)
dropped every event and forced synthesized response ids. Previously only
the responses path was fixed; now a shared normalizeSseLineEndings helper
(sse-util.ts) is applied at every split site: compress-loop.ts (chat),
compress-loop-responses.ts, stream.ts, stream-openai.ts, stream-responses.ts.
Also drains the residual SSE buffer at stream end in compress-loop.ts
(previously the last event was dropped if the stream lacked a trailing
blank line).
3. 10-min timeout did not cover the response body (fetch-util.ts)
fetchWithTimeout cleared the timer when fetch() resolved (headers only), so
a stalled SSE body was never aborted — contradicting the docstring.
Changed the API to return { response, clearTimer }; callers clear the
timer only after the body stream is fully consumed, so the timeout now
guards the full headers+body lifecycle. All three call sites updated
(compress-loop.ts, compress-loop-responses.ts multi-round loops use an
activeClearTimer cleared in the reader finally; server.ts passthrough
clears at every stream-consumed exit point).
…ess time)
decompress used collectBlockContent(state, block, ctx.messages) which matches
block.effectiveMessageIds against CoreMessage.id in ctx.messages. But
ctx.messages is the POST-FOLD view (prepared.processedMessages): messages
already compressed in an earlier round are replaced by their summary and are
absent, so collectBlockContent returns 0 items. The block was then
deactivated unconditionally, discarding it and making later retries fail.
A second, deeper issue: message ids are raw-${idx}, regenerated every round
by responsesToCore, so ids are not stable across rounds — even keeping the
pre-fold messages would not guarantee a match.
Fix: cache the original content at compress time, when the source messages
are still present in this round's ctx.messages (a freshly-compressed block
has not been folded yet — folding happens in the next processTurn). The
cache lives on the session (session.blockContents: Map<blockId, {text,count}>).
decompress reads from the cache first and only falls back to
collectBlockContent. deactivateBlock now runs only when content was actually
recovered (count>0 or a cache hit), so a 0-result no longer poisons the block.
Applied to both the chat path (compress-loop.ts) and the responses path
(compress-loop-responses.ts). Originals are captured in stream.ts applyRanges
right after applyCompression, by diffing block ids before/after and calling
collectBlockContent on the new blocks with full:true.
Codex review found two issues with the decompress cache added in d75326d: 1. full:false semantics broken — cache was populated with full:true unconditionally, so a decompress hit returned the complete original even when the caller did not pass full:true (default is one-level: direct messages + nested child summaries). 2. cache never cleared — successful decompress deactivated the block in state but left the cached originals in session.blockContents, causing unbounded memory growth and retained sensitive content over a long session. Fix: - session.blockContents now stores {one, full} views; applyRanges caches both at compress time. - resolveDecompress (extracted to src/decompress-shared.ts, shared by both compress-loop files) selects the view by args.full, deletes the cache after a successful decompress, and only deactivates the block when content was actually recovered. Tests: 4 new (proxy-decompress-cache.test.ts) covering one vs full views, cache deletion, and missing-cache fallback. 89 pass.
PROBLEM: the three wire converters (anthropic.ts/openai.ts/responses.ts)
assigned message ids as `raw-${idx}` — a *position* index, not an
identity. None of the three protocols attach a stable id to request-side
message items, so ids were derived from array position. As soon as a
client (or another plugin) deletes/reorders messages:
Turn1 [A,B,C,D,E,F] → compress B,C → effectiveMessageIds=[raw-1,raw-2]
Turn2 client drops B,C → sends [A,D,E,F] → D shifts to raw-1, E to raw-2
covered={raw-1,raw-2} now matches D and E → they get swallowed under
the B+C summary. Live content silently disappears.
FIX 1 — content fingerprints (src/message-id.ts):
id = h_${sha256(role|contentType|toolCallId|toolName|text)}
Two identical messages collide; a ClusterCounter appends _${N} so the
Nth duplicate gets a distinct id (prevents `covered` sets from
collapsing two unrelated turns onto one id and losing recent content).
Trade-off vs a real client id: deleting a duplicated message only
disturbs the cluster of that identity (local damage), never the whole
downstream (global damage like position). Fully distinct messages are
completely immune to reordering/deletion.
tool_call/tool_result bind to the protocol's call_id (tool_use.id /
tool_call_id / call_id), so two same-name same-args calls stay distinct.
FIX 2 — orphan block GC (src/orphan-gc.ts):
With fingerprints, a block whose effective ids the client has deleted
becomes an orphan: no anchor in the visible context, so its summary
floats to the top (insertAt ?? 0), polluting the model's view. After
each processTurn we count consecutive turns a block has had zero id
hits; at ORPHAN_THRESHOLD=3 it is deactivated and its blockContents
cache entry is dropped. Partial orphans (some ids present) are left
alone — deactivating on partial matches would destroy blocks during
normal incremental edits.
Tests (10 new, 99 pass):
- proxy-message-id: stability, reordering immunity, duplicate distinctness,
call_id binding, prefix-vs-mNNNNN non-collision (anthropic/openai/responses).
- proxy-orphan-gc: hit-keeps-active, threshold-reap, streak-reset, cache-clear.
- basic/proxy-basic: updated h_* prefix assertions.
1. reapOrphanBlocks now checks the client's raw input (msgs) instead of
the kernel's folded output (turn.messages). The folded output never
contains the original ids of compressed messages, so the old param
would eventually count every block as orphaned. (In practice h_msg0
survives in the fold so it didn't reap, but msgs is the correct
semantic — it reflects what the client actually sent.)
2. prepareResponses logs the final tools[] forwarded upstream when
ACP_DEBUG is set, showing each tool's type:name (function vs
namespace vs custom) so we can confirm ACP tools are present.
3. forward() tool-name diagnostic fixed: Responses API uses flat
{name} not nested {function:{name}}, so it always showed '?'.
ranxianglei
force-pushed
the
2026-08-07_responses-api
branch
from
August 8, 2026 01:02
d0d9b31 to
95354e9
Compare
Node's fetch (undici) transparently decodes gzip/br responses, so response.body is plaintext. But response.headers still carries the upstream 'content-encoding: gzip' marker. The proxy copied upstream headers verbatim to the client (server.ts:520), so clients received 'content-encoding: gzip' over plaintext bytes and tried to decompress them a second time — corrupting JSON/SSE parsing. Add 'content-encoding' to UPSTREAM_HOP_HEADERS so it is dropped on both sides: the request forward (client→upstream, line 508) and the response copy-back (upstream→client, line 520). This is the standard reverse- proxy fix for the fetch-auto-decompress trap. Found by codex code review.
The proxy injects <acp tokens=".." type="..">mNNNNN</acp> tags into INPUT messages so the model can reference them. When the model echoes a tag in its OUTPUT (in-context learning), the metadata leaks to the client as visible noise — most visible in Codex, which renders raw output text. All three protocols had tag-echo DETECTION (warn logs) but none STRIPPED. This adds a streaming-safe stripper to the responses path (stream + non-stream): Stream (stream-responses.ts): - stripInjectedTagsFeed/Flush: incremental buffer that drops complete echoed tags, holds back a potential partial open marker at delta boundaries, and drops unclosed tag fragments at stream end. - routeResponsesEvent intercepts response.output_text.delta (both modes), runs the delta through the stripper, emits the safe portion (or suppresses). - Generator flushes the stripper buffer before the compress tail. - Open marker <acp (space) and close </acp> are distinct from the text- protocol trigger <acp_compress>/</acp_compress>, so they never interfere. Non-stream (rewriteResponsesJsonResponse): - Regex strip of <acp [^>]*>...</acp> from all output_text parts, regardless of whether compress ran. Tests (6 new, 105 pass): single-delta strip, split-delta strip, partial-open hold+release, <acp_compress> not stripped, non-stream strip, no-tag unchanged.
This reverts commit e711d4e.
ranxianglei
added a commit
that referenced
this pull request
Aug 8, 2026
Two bugs from the latest proxy review: 1. readBody() called req.destroy() after rejecting — the client connection was cut off with no HTTP response, leaving handle() unable to write 413. Fix: reject with a typed BodyTooLargeError WITHOUT destroying; handle() catches it and writes a clean 413 JSON response. 2. TEXT_PROTOCOL suppressed ALL output_text.delta/done and forged a response.completed with output:[] even when NO compress trigger was present, corrupting normal Responses output. Fix: only rebuild when a trigger was actually caught; otherwise pass every event through verbatim. Added 4 regression tests (3 readBody + 1 responses no-trigger passthrough).
LX666-666
pushed a commit
to LX666-666/billion-context
that referenced
this pull request
Aug 9, 2026
…ianglei#2,ranxianglei#3,ranxianglei#4,ranxianglei#6) ranxianglei#1 (CRITICAL) protocol conversion was lossy: image->[image] placeholder, thinking signature dropped (Anthropic rejects unsigned thinking), tool_result .is_error dropped, OpenAI developer->system, image_url/input_image deleted. Fix: BiliMessage sidecar (src/bili-message.ts) extends CoreMessage with optional raw*/signature/isError/originalRole/image* fields. The kernel only does {...msg} spread copies, so sidecar survives the pipeline to coreToX, which replays the original block losslessly. 8 round-trip tests added. #2b (MAJOR) session-id extractKey() lowercased the API key before hashing — two distinct valid keys differing only in case collided into one hash bucket. Remove toLowerCase; keep raw (trim only). ranxianglei#3 (MAJOR) auto-update stale-lock takeover never deleted the stale lock file, so writeFile({flag:'wx'}) always failed EEXIST -> catch returned null -> a single crash during update permanently blocked all future auto-updates. Add unlink(LOCK_FILE) before the wx write, with ENOENT tolerated. ranxianglei#4 (MAJOR, security) /__bili/ management endpoints (config read/write, reload, session stats) had no auth. With --host 0.0.0.0 a remote caller could rewrite upstream routing to exfiltrate API keys. Gate all /__bili/ + /__acp/ paths to loopback connections only (127.x / ::1); model-traffic endpoints stay open. ranxianglei#6 (MINOR) build artifact was not self-contained: acp-kernel stayed as a runtime dependency (dist had 'from "acp-kernel"'). tsup now sets noExternal:[acp-kernel,node-forge] and acp-kernel moves to devDependencies, keeping the exact pin (0.0.17). dist verified: 0 'from acp-kernel' refs.
This was referenced Aug 10, 2026
ranxianglei
added a commit
to LX666-666/billion-context
that referenced
this pull request
Aug 10, 2026
…eload Review feedback from ranxianglei (PR#73 comments ranxianglei#3-ranxianglei#4): 1. [ranxianglei#9 critical] Restore Responses session-identity fallback chain (responses.ts): header → body.session_id → metadata.session_id → previous_response_id → content-fingerprint(hashId). The previous generated-randomUUID() broke compression for headerless clients (Pi, plain OpenAI SDK) — every request got a new session so compression state never accumulated. 2. [ranxianglei#1+ranxianglei#2 critical] parseState robustness (codex-takeover.ts): corrupted state file now returns undefined (treat as stale/recoverable) instead of throwing, which would leave Codex CLI unusable. Added sectionPrefixLength type validation (non-negative number) to prevent NaN slicing that could wipe the entire config.toml. 3. [Comment#4 ranxianglei#2] PUT /__bili/config onChanged now also refreshes routes (server.ts): previously only proxy fields were refreshed, so pure API callers (non-Web-UI) had to also POST /reload for provider config to take effect. Now PUT guarantees save-then-apply in one call. 4. Fix misleading context-window hint on routing page (page.ts).
This was referenced Aug 29, 2026
Closed
Closed
This was referenced Oct 12, 2026
Merged
Closed
ranxianglei
pushed a commit
that referenced
this pull request
Oct 14, 2026
…mum (#3071) #3071: on a model with NO declared output ceiling, the proxy's starved-budget restore used the client's own last non-starved high-water as its only lower bound. That estimate decays unbounded from the client's raw-history size (..., 3071, 2061, 252, <=200), so once starved the session pins at a death rattle and every turn truncates at max-tokens (finish=length, 0 visible chars); a fresh fork child inherits the rattle and forwards max_tokens=1 verbatim. On v0.1.195 this blanked 9 turns across 3 DSH sessions in one day. Fix (minimal — matches the issue's suggestion #1 literally): when no source knows a ceiling, the STARVED / born-starved / fork-inherited restore target is floored at OUTPUT_RESTORE_ABSOLUTE_FLOOR (8192). A non-starved value is still trusted verbatim (client's explicit choice; far smaller blast radius), and a known-ceiling model is untouched. The no-ceiling WARN dedup moves from once-per-model (global) to once-per-session+model so each affected session logs its own WARN instead of staying silent behind another session's first hit. Paths fixed: 1 (no-ceiling restore pinned at poisoned high-water), 2a (killed session got zero warns), 2b (fork child forwarded the starved budget verbatim). Path 3 (headroom reserving the shrunken budget) needs no change: the restore runs before the headroom reservation reads max_tokens, so the floored value flows in automatically. Deferred to owner sign-off (product decisions / pinned fail-closed behavior, flagged in-thread): #4 invalidating the #2490 dsh planning floor after a refused replay (contradicts its sticky-by-design overflow guarantee), and #5 retrying a starved finish=length turn (reverses #2756's deliberate exclusion; the root-cause floor fix should remove most of those empty turns first). Closes #3071
This was referenced Oct 15, 2026
Closed
Closed
ranxianglei
pushed a commit
that referenced
this pull request
Oct 21, 2026
… fold-reconcile fully re-anchors (#4131) The #1001 unannounced-rewrite detector judged history rewrites on raw-id hash identity alone, outside the order-centric reconcile ladder. A bili upgrade across v0.1.201 re-mints non-text message ids by design (#3642 D1 accepted this one-time churn and promised the reconcile ladder would absorb it), so on the first turn after a cross-version restart the detector saw the collapse, marked a compaction boundary anyway, and applyCompactionArchive pruned the ref map + tombstoned hundreds of still-cited refs — destroying fold coverage for a pure transform (incident: 212 covered refs re-anchored with 0 unmatched, then pruned to 392 live ids). detectUnannouncedHistoryRewrite now takes this pass's fold-reconcile verdict: when kind is reanchored/resend with unmatched === 0, the collapse is explained churn — the destructive boundary is suppressed and an id-churn-adoption conflict event + info log are recorded instead. Off/noop verdicts carry no evidence and keep legacy behavior; partial claims (unmatched > 0) keep the legacy escalation path unchanged. Second facet (issue expectation #4): applyCompactionArchive now names in its log line how many refs the prune drops and which block coverages they sit under, so any remaining destructive prune is auditable instead of silent. Tests: tests/issue4131-cross-version-churn-gate.test.ts pins both directions (explained churn suppressed; genuine rewrite still escalates; off/noop verdicts do not suppress; legacy callers unchanged) plus the log facet.
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third protocol alongside anthropic + openai chat completions.
Why
Codex 0.147 dropped `wire_api="chat"` and forces the Responses API. Without this, billion-context can only transparently-proxy Codex (no ACP compression) — defeating the value prop for token-billed official OpenAI users.
What
Core Responses API support
Codex code_mode integration fixes
Robustness fixes (3 rounds of codex review)
Message id stability (critical correctness fix)
Decompress correctness across rounds
Reverse-proxy fixes
Verification
Known limitations (follow-up PRs)
acp-kernel pinned to exact 0.0.17 (for `renderTags: "text-only"`).