[Soul4723]

Why

When every credential for a model is out of quota, CLIProxyAPI answers with a retryable 429 and Retry-After. Codex CLI treats any 429 that is not error.type: usage_limit_reached as retryable: it sleeps on Retry-After and tries again, so a client like T3 Code shows the thread as "working" for hours. T3 Code pauses a thread and resumes it at the reset only when the run fails as a usage limit with a known reset time. Upstream PRs disagree on whether the proxy should send a final usage limit or a retryable 429, so this adds a config toggle and keeps today's behavior as the default.

What changed

• New setting routing.cooldown.quota-response-mode: retryable | usage-limit (legacy flat key quota-cooldown-response-mode). Empty means retryable, and unknown values fail config loading. It is applied at startup and on hot reload, appears in the config change summary, and is documented in config.example.yaml.
• Executors now mark a 429 as explicit quota exhaustion for:
  • Codex usage_limit_reached (HTTP and WebSocket forms)
  • OpenAI insufficient_quota
  • Google RESOURCE_EXHAUSTED with ErrorInfo reason QUOTA_EXHAUSTED, from Gemini, Vertex (including Vertex OpenAPI, which is served by the OpenAI-compatible executor) and Antigravity. A shared helper in helps/google_quota.go reads the reset from RetryInfo.retryDelay, or else from a positive Retry-After.
  • an Anthropic unified 5h/7d rejection. The window reported is the rejected window whose reset equals the deadline the cooldown keeps: 300 for 5h, 10080 for weekly. If the kept deadline matches neither, the window is left unknown.
• The cooldown records this evidence only when the upstream also gave a reset time. The evidence stays attached to the deadline it set, is saved with cooldown state (.cds), and is cleared with that cooldown.
• In usage-limit mode, the final error of Execute and ExecuteStream is checked against the whole eligible pool, ignoring which credentials this request already tried. The pool uses the same filters as selection:
  • canonical provider names
  • disabled credentials skipped, but status-disabled ones kept, as selection does
  • a registered executor for the credential's provider
  • support for the route model, plus pinning
• For a pooled OpenAI-compatible alias, cooldowns are recorded under each backend model. Each credential is checked against every backend behind the alias, using the state model that blocks that backend; listing the backends does not advance the pool rotation. The credential counts as blocked only when selection blocks it with evidence, or when every backend is blocked by a cooldown with evidence.
• The error becomes a final usage limit only when every candidate is blocked by evidence that covers its blocking deadline and the earliest reset is more than 5 minutes away. The check runs on a 429, and also on auth_not_found (no executable credential), which is how an all-cooling pooled alias ends. Mixed pools, unclassified 429s, exponential backoff and resets of 5 minutes or less keep the retryable answer.

Wire output

retryable (default): unchanged on every path. Existing tests pass unchanged.

usage-limit, both when every credential is already cooling at selection (path a) and when the last credential returns 429 during this request (path b), before any response bytes are written:

• Status 429, X-Should-Retry: false, and no Retry-After. An upstream Retry-After from passthrough headers is removed.
• X-Codex-Primary-{Used-Percent,Window-Minutes,Reset-At} when the exhausted window is shorter than a day, and X-Codex-Secondary-* for the weekly window. Used percent is 100, taken from the credential that reopens first. No Codex headers are sent when the window is unknown.
• CORS: in usage-limit mode, Access-Control-Expose-Headers adds X-Should-Retry and the six X-Codex-* headers after the base list. In retryable mode the list is byte for byte the one on contrib.
• /v1/responses and /v1/chat/completions keep the existing cooldown body and add four fields:

{"error":{"code":"model_cooldown","type":"usage_limit_reached","resets_at":1793593893,"retryable":false,"limit_window_minutes":300,
  "message":"All credentials for model M are cooling down via provider P (last error: usage_limit_reached: The usage limit has been reached)",
  "model":"M","provider":"P","reset_seconds":7200,"reset_time":"2h0m0s","last_upstream_error":"usage_limit_reached: The usage limit has been reached"}}

• /v1/messages keeps rate_limit_error. The message ends with the suffix that T3 Code parses (fix(server): keep a gateway's Claude reset time on a direct 429 pingdotgg/t3code#25438):

{"type":"error","error":{"type":"rate_limit_error","message":"All credentials for model M are cooling down via provider P (last error: ...) (reset after 2h0m0s)","resets_at":1793593893,"retryable":false}}

The response is always 429. The status-code setting from router-for-me#6245 is not on dev, so there is nothing to override.

Scope

Covers config, evidence, and paths (a) and (b) on Responses, Chat Completions and Anthropic Messages, streaming or not, as long as nothing has been written yet. Not covered, left for a follow-up PR:

• Errors after the first byte of an SSE stream. HTTP stays 200 there, so the metadata has to go into the terminal error event.
• The Responses WebSocket path, which still closes without an error event on quota failures.

ExecuteCount, Home mode (Home owns cooldowns) and the /v1/live and alpha-search routes are unchanged.

Tradeoffs

• The Error struct can't gain fields because TestErrorLegacyUnkeyedLiteralCompatibility pins its unkeyed literal. Setting Error.Code would change Error() text in retryable mode. So the evidence travels on new Result fields, QuotaExhausted, QuotaLimitWindowMinutes and QuotaResetAfter, set where each result is built.
• Vertex OpenAPI errors keep the OpenAI-compatible error text and retry hint (Retry-After); only the evidence and QuotaResetAfter are added.
• Gemini and Vertex native errors keep RetryAfter() nil and the same error text. Their quota reset travels as QuotaResetAfter (new QuotaResetError interface), and only usage-limit mode uses it as the cooldown deadline when the result has no retry hint. So retryable mode keeps the exact cooldown and response it had for these errors. One effect: a Google credential that cooled down in retryable mode has no recorded evidence, so switching to usage-limit mid-cooldown keeps it retryable until its next failure.
• The decision is made once, at the end of execution, instead of in each of the four places that build a cooldown error. This covers path (a) and path (b) with one pool check.
• One global policy with a fixed 5-minute threshold, matching fix(auth): report explicit quota exhaustion as a final usage limit router-for-me/CLIProxyAPI#6647 and fix(auth): report exhausted Codex quota as a usage limit Codex clients stop on router-for-me/CLIProxyAPI#7184.

Evidence

All commands below ran on this branch at 3f7a4f1e on Go 1.27.3. The Vertex OpenAPI fix (e6987148) was checked afterwards; see the end of this list.

• go build ./...: OK.

• go vet on every changed package: clean.

• go test ./...: all 102 packages OK.

• go test -race on sdk/cliproxy/auth/..., sdk/api/handlers/..., internal/config/..., internal/api/... and internal/runtime/executor/...: all OK except two tests, and both fail the same way on untouched contrib (7cad1cb7):

  • internal/api/middleware TestRequestLoggingMiddleware_StreamingPreservesFullUUIDForLogger: a data race between processStreamingChunks and Finalize in response_writer.go. It failed 40 of 40 -race runs on both this branch and contrib. This branch does not touch that package.
  • TestResponsesInterruptStopsHTTPUpstream: failed 11 of 40 -race runs here and 14 of 40 on contrib.

• TestServiceHeartbeatFailoverDoesNotDrainBlockingScope (sdk/cliproxy) is flaky on contrib too. It failed with new subscription did not start while the old scope remained active:

  Checkout	Without -race	With -race
  contrib	2 of 40	2 of 40
  this branch	2 of 40	3 of 40

  Both checkouts ran at the same time on one machine.

• Vertex OpenAPI fix, on e6987148: go build ./... OK. go vet ./internal/runtime/executor/ clean. go test on the changed package OK. go test ./...: 101 of 102 packages OK; the exception was TestResponsesInterruptStopsHTTPUpstream (flaky on contrib, above), and that package then passed 3 of 3 reruns. go test -race ./internal/runtime/executor/... ./sdk/cliproxy/auth/...: OK.

Each review fix has a regression test that was run and seen failing before the fix:

• sdk/cliproxy/auth/usage_limit_alias_pool_test.go:
  • A pooled alias whose backends are all quota-exhausted gets the usage limit, for both execute and stream and for paths (a) and (b). Before the fix it got the raw insufficient_quota error, then auth_not_found.
  • A pooled alias with one available backend stays retryable.
  • A credential whose provider has no registered executor does not block the usage limit.
• internal/runtime/executor/vertex_openapi_quota_evidence_test.go: through the manager, routes deepseek-ai/deepseek-v4.1-flash-maas via Vertex OpenAPI, and the upstream returns a 429 with QUOTA_EXHAUSTED, retryDelay 7200s and Retry-After: 7200. For execute and stream:
  • Default mode returns the raw error with unchanged text and retry hint, plus the evidence and a 2h reset.
  • usage-limit mode then returns a usage limit about 2h out.
  • Before the fix the evidence was missing and usage-limit mode returned a plain retryable model_cooldown.
• internal/runtime/executor/google_quota_evidence_test.go: Gemini and Vertex, execute and stream, report evidence and the reset for QUOTA_EXHAUSTED and keep RetryAfter() and the error text unchanged. RATE_LIMIT_EXCEEDED does not count as evidence.
• sdk/cliproxy/auth/usage_limit_refusal_test.go and sdk/api/handlers/quota_usage_limit_response_test.go: a quota reset without a retry hint becomes a usage limit only in usage-limit mode. retryable mode keeps the old response.
• internal/runtime/executor/quota_exhausted_evidence_test.go: TestClaudeQuotaWindowMatchesRetainedReset pairs the window with the kept reset, and leaves it unknown when it does not match.
• internal/api/cors_usage_limit_test.go: in retryable mode (empty or explicit), Access-Control-Expose-Headers equals the base list joined with ", ". In usage-limit mode it adds the seven headers.

Earlier tests:

• sdk/api/handlers/quota_usage_limit_response_test.go runs the real manager, a fake executor and the real handlers. It covers both modes × paths (a) and (b) × the three surfaces × stream on/off (24 cases), and asserts status, headers, resets_at, the window and the Claude suffix. It also covers the mixed pool, a generic long 429, a reset under 5 minutes and a missing reset across all surfaces, plus switching the mode while a cooldown is active.
• sdk/cliproxy/auth/usage_limit_refusal_test.go covers the boundary at exactly 5 minutes (retryable) and at 5 minutes plus 1 second (usage limit). It also checks that evidence is recorded only with a provider reset, that it must cover the model's blocking deadline, that the weekly window goes to the secondary headers and an unknown window gets none, and that evidence survives a restart through FileCooldownStateStore for credential-scoped and model-scoped cooldowns.
• internal/runtime/executor/quota_exhausted_evidence_test.go checks which upstream 429s count as evidence and the window length taken from the Codex and Anthropic headers.
• internal/config/quota_cooldown_response_mode_test.go round-trips the v8 key and the legacy key through load, save and reload, and checks that unknown values are rejected. internal/api/quota_cooldown_response_mode_test.go checks startup, then reloads a config file through LoadConfig and UpdateClients and sees the mode change. internal/watcher/diff checks the change summary line.

Revert check, run on a clean contrib checkout against the first version of this PR (18eaa8ed):

• With the new test files only, every new test file fails to compile.
• With the change applied but its decision points stubbed out (the pool check, evidence recording, the mode setter, executor evidence, value parsing, the v8 key, the handler headers and the Claude suffix), 41 subtests fail. That is every usage-limit assertion. The 43 that still pass are the retryable-mode and "stays retryable" checks, which should pass either way, plus two helper tests whose functions were not stubbed.

Credit

The evidence and deadline tracking is ported from router-for-me#6647. The Codex rate-limit headers, the withUsageLimit cooldown shape and the wrapped-error handling are ported from router-for-me#7184. Both are adapted here behind the toggle.

Merge danger

Two-way door. The default retryable keeps today's responses. usage-limit is opt-in and can be switched off by hot reload. Cooldown state files gain two optional fields (exhausted, limit_window_minutes), which older builds ignore.