[Bug]: Failover combo does not move on when the native Codex upstream declines with server_is_overloaded before any output #8107

Activity

1. github-actions commented on Nov 2, 2026

   

   github-actionsBot

   on Nov 2, 2026 – with GitHub Actions

   Last edited by github-actions

   Contributor

   More actions

   _{Automated translation bookkeeping — detected language: English.}
2. coderabbitai commented on Nov 2, 2026

   

   coderabbitaiBot

   on Nov 2, 2026 – with coderabbitai

   Contributor

   More actions

   🔗 Related PRs

   #7482 - feat(fallback): OCX-11 fallback off per scope, with home_unreachable on Children [merged]

   ---

   📝 Issue Planner

   _{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

   • Create Plan

   ---

   🧪 Issue enrichment is currently in open beta.

   To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

   issue_enrichment:
     auto_enrich:
       enabled: false

   💬 Have feedback or questions? Drop into our discord!
3. github-actions

   added

   bugSomething isn't workingSomething isn't working

   account-poolOAuth, credentials, Codex pool, quota, failover, plansOAuth, credentials, Codex pool, quota, failover, plans

   on Nov 2, 2026
4. Ingwannu commented on Nov 4, 2026

   

   Ingwannu

   on Nov 4, 2026

   Owner

   More actions

   Thanks for the detailed attempt record. I traced this against current dev (72e80d280d). The relevant code is unchanged since 2.95.0, and the stop happens for a different reason than the one in the report.

   • Combo children skip the [Bug]: Pooled native Codex requests do not retry or fail over on upstream server_is_overloaded / upstream_server_error before output #6969 hold on purpose (core-codex-account.ts:163-177). Even with the hold enabled, an unrecovered projection is restored to the original 200 stream in passthrough-dispatch.ts:1711 before the combo sees it. So enabling the hold for combo attempts would not change the result.
   • Without the hold, the 200 SSE is committed at response.created and the overload error frame is relayed in-band. That body then goes through the combo stream preflight (core-combo.ts:1255). For a zero-output 503 server_is_overloaded, the preflight would normally return failed, without the non-replayable marker, and comboFailureDecision would hop.
   • The likely defect: the Codex WS metadata channel writes codex.rate_limits / codex.response.metadata control frames into the same SSE body (codex-ws-exchange.ts:626-716, codex-ws-metadata.ts:242). The combo preflight only treats response.created/in_progress/queued/heartbeat as pre-output control events (combo-stream-preflight.ts:24). Every other type commits the child (comboStreamPayloadCommitsOutput, :148-162, :336). So a rate-limit frame ahead of response.created commits the native target before the overload arrives, and the combo never reaches its next target. That matches your one attempt and candidates[1]: not-selected.

   Proposed fix: treat the two Codex metadata frame types as non-committing control events in the combo preflight. Add a unit test and a fake-WebSocket combo failover regression test (rate_limits → created → in_progress → error 503). Making the hold apply to the plain main context outside combos is a separate change and is not needed for the combo case.

   One thing to confirm: your stage record shows relayedEvents: 4, one more than the created → in_progress → error sequence we test with. If you can share only the ordered type values of the frames before the error (no payloads), we can check whether a new Responses event type also needs to be treated as pre-output.