WS client lane: fall back to HTTP/SSE egress when the upstream refuses the upgrade by ranxianglei · Pull Request #3447 · ranxianglei/billion-context

ranxianglei · 2026-10-16T19:34:42Z

What & why

A Codex/OpenCode Responses WebSocket client pointed at an HTTP/SSE-only supplier now works through bili. When the supplier refuses the WS upgrade before any generation request is sent, bili's upstream leg re-issues the same turn over plain POST /responses + SSE instead of failing the turn.

The client leg is untouched — WS deltas, local checkpoints, and ACP behave exactly as before. Only the bili→supplier egress gains an HTTP/SSE path. Suppliers that do support WS are unaffected: the WS leg still wins whenever its handshake succeeds.

⚠️ Behavior change (old → new)

Scenario	Before	After
Upstream refuses the WS upgrade (pre-generation)	Turn fails with `Responses WebSocket handshake failed`	Falls back to HTTP/SSE egress on the same URL; turn succeeds if the supplier serves `/responses`
Abort / mid-stream error after generation started	Hard fail	Hard fail (unchanged — a started generation is never replayed)

Why: the point of the issue — WS clients should reach suppliers that only speak HTTP/SSE. A pre-generation refusal means the supplier simply isn't offering WS, so retrying over its HTTP endpoint is correct. Once any bytes of a generation have been sent, replaying over a different transport would corrupt the exchange, so those still hard-fail.

How it works

ResponsesWsUpstream.fetch() tries the WS handshake; on a genuine pre-generation refusal (isHandshakeRefusal) it calls the new egressHttp().
egressHttp() sends the full expanded input (no previous_response_id, no WS-only type field — a fresh connection shares no checkpoint), forces stream:true, and returns the supplier response unchanged, so the existing relay forwards it exactly like the proven non-WS codex lane (2xx → SSE→WS frames, non-2xx verbatim with status preserved).
Empty and nonempty generate:false warmups establish a usable client checkpoint (a later previous_response_id continuation resolves instead of throwing not_found) and surface incomplete streams explicitly rather than hanging.
fetchWithTimeout(..., forceDirect) skips the ALS transport lookup so the fallback doesn't recurse back into the intercepted WS transport; the idle watchdog is released on body settle so a fallback turn doesn't pin the event loop.

Scope / non-goals

Client-facing wire shape: unchanged.
Kernel: unchanged (no kernel/** changes).
Config surface: no new config/env/CLI added — the fallback is automatic and knob-less by design (a forced-HTTP opt would be a separate, owner-gated change if ever wanted).
Missing-checkpoint / reconnect already resolve to full-history recovery (covered by tests).

Tests

Targeted suites green locally (full regression = remote CI on the pushed head):

tests/responses-ws-http-fallback.test.ts (new): hermetic upgrade-refusing HTTP/SSE supplier covering probe+turn, tool-call byte-exact args/order/usage, warmup checkpoint + continuation, nonempty generate:false, upstream-400 passthrough, unterminated stream, missing-checkpoint recovery.
tests/responses-websocket.test.ts: the refused-handshake fault test updated to assert the new fallback while preserving its diagnostic-hygiene and clean-reconnect guarantees.
codex-responses-ws, responses-prewarm-generate, ws-bridge: unchanged, still green.
tests/e2e/e2e-pi-codex-ws.test.ts scenario C: expectation inverted from “refusal surfaces as failure” to “refusal falls back to HTTP egress and the turn completes.” Reasoned against the mock contract (refuses the upgrade, serves POST/SSE); verified by the ci-e2e-pi-ws lane — not run locally, per the E2E-is-CI test discipline.

Closes #3342

github-actions · 2026-10-16T19:35:50Z

📦 Built Package Artifact

Branch: wt-3342-ework-daemon (3dcbe3e)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4064-pr.3447.3a4ee92

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

Download the artifact from the Actions run
Extract the tarball and install:

tar xzf billion-context-pr3447.tgz
npm install -g package

Install instructions are refreshed on each push.

ranxianglei · 2026-10-16T19:36:56Z

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重(前置):按失败签名/主题检索 open+closed issue 与近期 PR(「WS 上游 / HTTP fallback / handshake refused / transport decouple」,不只标题)。相关但不重复:#2121 是客户端腿(pi native 拦截 codex WS 流量,记录"上游拒绝 WS 表现为 mid-stream 失败、无同轮 fallback");#1859 提供独立 client/upstream checkpoint + ACP-aware WS 但明确无网络 HTTP bridge;#1866 是 sparse terminal/tool 保真目标。没有任何其他条目实现 bili→supplier 的 WS→HTTP/SSE 出口回退。查重:不重复——本 PR 即 #3342(owner 已拍板走 B:agent 独立实现)的实现交付。

这是带 ework-agent-pr 标记的 [PR] issue,我不重做,改为:拉分支核对代码、独立回归审查、确认 CI、给结论。审查进行中,结论稍后按八问回帖。

…refuses the upgrade A Codex/OpenCode Responses WebSocket client pointed at an HTTP/SSE-only supplier now works through bili: when the supplier refuses the WS upgrade before any generation request is sent, bili's upstream leg re-issues the same turn over plain HTTP /responses+SSE instead of failing. The client leg (WS deltas, local checkpoints, ACP) is untouched; only the bili-to-supplier egress changes. egressHttp sends the full expanded input (no previous_response_id, no WS-only type field), forces stream:true, and returns the supplier response unchanged so the relay forwards it exactly like the non-WS codex lane. Fallback fires only on a genuine pre-generation WS-upgrade refusal; abort and mid-stream errors still hard-fail so a started generation is never replayed. Empty/nonempty generate:false warmups establish a usable client checkpoint and surface incomplete streams explicitly. forceDirect on fetchWithTimeout breaks the ALS transport recursion, and the idle watchdog is released on body settle so a fallback turn does not pin the event loop. Closes #3342

ranxianglei · 2026-10-16T19:46:41Z

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review 进度(独立审核,未照抄 #3342 线程结论):

代码与行为已审完。逐行读了 src/responses-ws.ts(+113)、src/fetch-util.ts(+4/-1)及三个测试文件,对照改动前的既有行为逐项清点。核心判定:
- fallback 门精确:isHandshakeRefusal 只匹配 Responses WebSocket handshake failed,该字符串全仓库仅一处抛出点(connect() 握手 promise,生成开始之前);abort 走 DOMException AbortError 不匹配 ⇒ 「生成已开始绝不重放」的边界成立。
- egress 形状正确:回退请求 = 客户端帧去掉 stream/stream_options/background/previous_response_id/type(两条 client lane 在此层都已是全量历史),强制 stream:true,响应原样交回既有 relay(SSE→WS 重构帧路径与非 WS codex lane 完全复用);forceDirect 跳过 ALS transport 查找,不会递归回被拦截的 WS transport。
- watchdog 不泄漏:releaseUpstreamWatchdog 在 body settle(done/error/cancel)时恰好释放一次 idle timer,空 body 立即释放。
- 行为变更均已披露:① pre-generation 拒绝从硬失败变回退(PR 表格里列了);② generate:false warmup 现在建立可用的 client checkpoint——这条对 WS-capable supplier 同样生效(不只回退路径),PR 正文已写明,我核实了它不破坏既有续接语义(后续 previous_response_id 解析成功而非 not_found)。
- 测试有牙:新 suite 7 个场景断言的是线上可观测行为(egress 请求形状、tool-call 字节级 args/顺序、checkpoint 续接、400 透传、未终结流的显式错误帧),不是恒真断言;e2e scenario C 的反转有 mock 契约支撑(拒升级但同端口服务 POST/SSE)。
发现 stale base,已处理:分支落后 master 263 个提交。PR 触碰的 4 个文件在窗口内 master 零改动(所以 GitHub 报无冲突、diff 逐字节可应用),但依赖模块 config.ts/persist.ts/server.ts 窗口内有 50 个提交(+735/-249)——合并树 ≠ CI 测过的树。按 §7.2 纪律把分支 rebase 到当前 master 尖(ce9bc90a3),新 head 4b0795ef4(force-with-lease 推回 PR 分支,内容零变化,纯换基底)。
rebase 后本地定向验证已过:typecheck 干净 + 五个定向套件全绿(responses-ws-http-fallback 8/8、responses-websocket 41/41、codex-responses-ws 9/9、responses-prewarm-generate 5/5、ws-bridge 3/3)。

完整 CI 正在新 head 上跑(静态门禁 + test 分片 + e2e lane),绿后我给最终八问结论。

ranxianglei · 2026-10-16T20:11:13Z

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review 完成(独立审核:diff、既有行为、测试均自己重读重判,未照抄 #3342 线程结论)。判定:可合并。

先说一件我动过的事:分支原 head 78a70c384 落后 master 263 个提交。PR 触碰的 4 个文件在窗口内 master 零改动(无冲突),但依赖模块 config.ts/persist.ts/server.ts 有 50 个提交(+735/-249),合并树 ≠ CI 测过的树。我已把分支 rebase 到当前 master 尖 ce9bc90a3,新 head 4b0795ef4(内容零变化,纯换基底,force-with-lease 推回 PR 分支)。rebase 后本地 typecheck 干净 + 五个定向套件 66/66 全绿;新 head 上完整 CI 已全绿:主 CI(静态门禁 + ubuntu/windows × node22/24 六个 test 分片)、e2e-pi-codex-ws(即 scenario C「拒绝→回退→turn 完成」反转断言所在 lane)、以及 real codex / dsh / pi / opencode / registry / persona / fake-codex / Windows overlay 各 e2e lane 全部 success,零失败。

结构化八问:

是什么问题:用 Responses WebSocket 的 Codex/OpenCode 客户端连一个只会 HTTP/SSE 的供应商时,整个 turn 直接挂掉(Responses WebSocket handshake failed)——WS 客户端无法触达非 WS 供应商。
什么原因导致的:能力缺口而非缺陷:bili→supplier 的出向腿把传输选择做成了二选一(WS 握手成功才干活,失败就报错),没有「上游不提供 WS 就走它的 HTTP 端点」这条路。Support Codex Responses WS clients with HTTP/SSE upstreams, preserving local delta checkpoints #3342 owner 拍板的 B 方向就是拆开 client-facing 传输与 upstream 传输。
影响范围与严重程度:修复前——该组合下完全不可用(每次 turn 必败);触发面限于「WS 模式客户端 + 非 WS 供应商」。支持 WS 的供应商不受任何影响。
怎么修复的:出向腿加一条 egress 分支,决策点放在 fetch() 的 catch:isHandshakeRefusal 精确匹配握手失败字符串(全仓库唯一抛出点在生成开始前的 connect() 握手 promise;abort 走 AbortError 不匹配 ⇒ 「已开始生成的 turn 绝不换传输重放」这条边界由匹配面的唯一性保证)。命中后清 transport 状态,egressHttp() 发全量输入(去 previous_response_id/type、强制 stream:true)到同一 URL,响应原样交回既有 SSE→WS relay——与非 WS codex lane 复用同一条重构帧路径,不新造转发逻辑。两个配套:fetchWithTimeout(forceDirect) 跳过 ALS transport 查找防递归回被拦截的 WS transport;watchdog 在 body settle 时恰好释放一次,回退 turn 不 pin event loop。
是否有回归问题:逐项清点过改动前代码承担的既有行为——①客户端 wire(deltas/checkpoint/ACP):零 diff,保持;②WS-capable 供应商:握手成功即走原路径,egressHttp 不可达,保持(codex-responses-ws 套件钉住);③abort/流中错误:仍硬失败不重放,保持;④非 2xx 透传:status+body 原样(测试钉住 400 透传不被误标 502),保持;⑤fault 路径诊断卫生(日志不泄 secret、clean reconnect):更新后的 fault 测试保留原断言,保持。有意变更两条,均已披露(PR 表格):pre-generation 拒绝从硬失败→HTTP/SSE 回退(turn 成功);generate:false warmup 现在建立可用 client checkpoint——注意这条对 WS-capable 供应商同样生效(不只回退路径),我核过续接语义:后续 previous_response_id 解析成功而非 not_found,无破坏。Kernel 零 diff;config surface 零新增(config-surface-guard 绿)。结论:未发现无意回归。
是否破坏缓存:不影响。client 侧 prefix cache 前提未变(client leg 零改动);HTTP egress 是全新连接、无 previous_response_id,不对供应商侧缓存做任何假设;回退后 transport history 清空、下一 turn 重新全量发送——正确语义,略有冗余无正确性问题。重试/去重键均未触碰。
是否改了设计方向:没有转向,是既有边界的自然延伸:client leg 与 egress leg 解耦正是 Support Codex Responses WS clients with HTTP/SSE upstreams, preserving local delta checkpoints #3342 已拍板的方向 B,本 PR 是它的实现;两 lane 架构、kernel 边界、单写者契约均未动。
是否引入新的配置:无。回退自动、无 knob(与 PR 声明一致;将来若要 forced-HTTP opt-in 属 owner-gated 配置变更,另立)。

测试有效性:新 suite 7 个场景断言的都是线上可观测行为(egress 请求形状/auth 随行、tool-call 字节级 args 与顺序、warmup checkpoint 续接、400 透传、未终结流显式错误帧而非挂死、跨连接丢 checkpoint 的全量恢复),不是恒真断言;e2e scenario C 的反转有 mock 契约支撑(拒升级但同端口服务 POST/SSE)且已在 ci-e2e-pi-ws lane 实测绿。残留小缺口(不阻塞):abort-during-HTTP-fallback 无专门测试(signal 管道与既有 lane 相同,风险低),建议后续补。

收工判定:可合并 —— 独立审核通过、行为变更已披露、rebase 后合并树完整 CI 全绿。请 owner 自行合并(PR 合并为人工操作)。

Review follow-up for #3447 (Closes #3342): the review flagged that no test covered a client disconnect while the HTTP/SSE fallback egress is in flight. The fixture gains a stall mode holding the supplier request open without responding; the new test starts a turn, waits for the egress to land, terminates the WS client, and asserts the turn rejects with the client-close error, the server-side request observes the abort, and no idle-watchdog timer is left behind by the aborted leg.

ranxianglei · 2026-10-16T20:45:08Z

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 收尾补充:此前结论中指出的唯一缺口——客户端在 HTTP fallback egress 飞行中断开时,abort 链路没有测试钉住——已按「小问题直接修」直接补进 PR 分支。

新增 commit 3dcbe3ed8(test-only,+41 行,仅 tests/responses-ws-http-fallback.test.ts):

fixture 新增 stall 模式:supplier 收到 POST 后持有请求不响应(不产生任何帧);
新测试:发起 turn → 等 egress 落地 → terminate WS 客户端 → 断言 ① turn 以 client-close 错误拒绝 ② 服务端侧请求确实观察到 abort ③ 被中断的 egress 不残留 idle-watchdog 定时器(相对基线断言,避免跨测试状态耦合)。若 abort 传播断裂,持有的 socket 会卡住 upstream.close() 触发超时——失败是响亮的,不是静默泄漏。

验证:

本地定向:responses-ws-http-fallback 9/9(含新测试)+ npm run typecheck 干净;
远程 CI(head 3dcbe3ed8):主 CI 28 success / 13 skipped / 0 fail;独立 e2e workflow runs 11/11 success(含场景 C 期望反转后的 e2e-pi-codex-ws、real codex、fake-codex、codex-persona、native dsh/pi/opencode、local-registry、Windows codex overlay、PR Build Artifact)。

行为变更:本 commit 纯测试,零行为/配置/wire/kernel 面变化;PR 整体披露的行为变更维持原回复所列两条(pre-generation WS 拒绝 → 同 URL HTTP/SSE 回退;generate:false warmup 建立可用 checkpoint)。

八问速览(相对本轮补充):① 什么:abort 覆盖缺口;② 原因:原 PR 未覆盖该路径(非缺陷,是测试面遗漏);③ 影响:仅测试面,无运行时影响;④ 修法:stall fixture + 端到端断言 abort 传播与计时器回收;⑤ 回归:无(只加不改);⑥ 缓存:不涉及;⑦ 设计方向:不涉及;⑧ 配置:不涉及。

判定不变:可合并。合并由人来执行:#3447

github-actions · 2026-10-18T06:05:46Z

📦 Merged Build Published

This merge to master was published as 0.0.564-master.6ef1928 (rolling master tag).

npm install -g billion-context@0.0.564-master.6ef1928  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei mentioned this pull request Oct 16, 2026

Support Codex Responses WS clients with HTTP/SSE upstreams, preserving local delta checkpoints #3342

Closed

ranxianglei force-pushed the wt-3342-ework-daemon branch from 78a70c3 to 4b0795e Compare October 16, 2026 19:46

ranxianglei merged commit 6ef1928 into master Oct 18, 2026
41 checks passed

Navigation Menu

Repository navigation

WS client lane: fall back to HTTP/SSE egress when the upstream refuses the upgrade - #3447

WS client lane: fall back to HTTP/SSE egress when the upstream refuses the upgrade#3447
ranxianglei merged 2 commits into
masterfrom
wt-3342-ework-daemon

ranxianglei commented Oct 16, 2026

Uh oh!

github-actions Bot commented Oct 16, 2026 •

edited

Loading

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ranxianglei commented Oct 16, 2026

What & why

⚠️ Behavior change (old → new)

How it works

Scope / non-goals

Tests

Uh oh!

github-actions Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Built Package Artifact

Option A — Install this exact build (recommended)

Option B — Download artifact

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

ranxianglei commented Oct 16, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 18, 2026

📦 Merged Build Published

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Oct 16, 2026 •

edited

Loading