[Bug]: CLIProxy reset times are available in Usage but missing from thread recovery; queued messages block snoozing

Before submitting

I searched existing issues and did not find a duplicate of the CLIProxy usage-source case.
I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

Route a Claude or Codex provider through CLIProxyAPI, and configure its management API as a T3 usage-limit source.
Confirm that T3 can read the pooled accounts' usage windows and future reset times.
Enable automatic snoozing of usage-limited threads.
Run until the pool is exhausted and the native CLI reports a terminal 429 without native reset metadata. A message queued behind the affected run exposes the second part of the failure.
Inspect the recovery banner and try an ordinary timed snooze.

Expected behavior

T3 can schedule recovery for a proxy-backed thread using reset/cooldown information from its explicitly associated hub, or provides a timed snooze that preserves queued work when a reset cannot be established.

This needs pool/model-aware selection and fresh data. Choosing an arbitrary account's reset can retry too early or delay work unnecessarily. Temporary capacity 429s also need to remain distinct from exhausted quota.

Actual behavior

The thread shows Reset time unavailable; retry manually, with no reset-snooze actions. Ordinary thread.snooze rejects a queued user message with has a queued run and cannot be snoozed. The rate-limited thread remains visible even though it cannot process that queued work.

On the installed build, T3's authenticated subscribeServerConfig stream successfully returned the configured CLIProxy source and five accounts. All four Claude accounts had an exhausted session or weekly window with a future reset. For example, the response contained:

{"id":"seven_day","kind":"weekly","usedPercent":100,"resetsAt":"2026-10-28T19:59:59.530Z"}

At the same time, recent root failures were stored as:

{"class":"usage_limit","message":"Claude API rate limit reached. Try again later.","code":"api_error_429","retryable":true,"resetAt":null}

Of the 60 most recent failed runs on this machine, 46 were classified usage_limit, and all 46 lacked resetAt. These are historical runs, not 46 currently blocked threads. They included both Claude and Codex. The queued-message symptom was observed by the user; the sampled runs had no queued message at the later evidence capture, so I am distinguishing that report from the captured state.

Source trace

UsageLimitSources fetches and publishes hub account snapshots.
UsageLimitRecoveryWorker rejects a candidate without usageLimitResetAt and does not consult the hub snapshots.
The thread reset is derived from its failed native turn. The observed proxy-backed failures did not supply a usable native timestamp.
Orchestrator rejects ordinary snoozing for queued user work. The dedicated limit-recovery path can retain that queue, but requires a matching failure reset.

I verified these predicates in the running app's extracted server bundle and in upstream source at its declared commit. An isolated replay of the installed decision functions confirms that external hub window data does not arm recovery while the actual thread reset field is null, whereas populating that field does.

Related issues

#22619 was closed because an unknown reset time provides nothing to schedule against. This report concerns reset information that T3 already fetched through its CLIProxy integration, plus the inability to use ordinary snoozing with a queued user message. It is not a request to retry unknown 429s indefinitely. #15862 concerns held automatic queued notifications after restart. #26265 concerns OpenCode forwarding a reset embedded in an upstream error body.

Impact

Major degradation or frequent failure

Version or commit

T3 Code 0.1.4-nightly.20261028.3824, commit eb751fa0730d.

Environment

Linux desktop, CLIProxyAPI 8.0.45, Claude Code and Codex app-server providers. The configured hub usage source is enabled and returning data. Both snoozeLimitedThreads and autoResumeLimitedThreads are enabled.

Workaround

Without queued user work, an ordinary timed snooze is possible. With queued user work, the ordinary snooze command refuses. Removing queued messages would lose the intended queue and is not a satisfactory workaround.

Another confirmation of this on a Windows desktop setup, with fresh evidence from the provider event log.

Version / setup

Bug seen on: 0.1.6-nightly.20261030.3904 (desktop app, server on the same Windows 11 machine, Node v26.12). I diffed it against the latest nightly 3917 and no commit in that range touches usage-limit recovery, so I expect it is still present there.
Surface: desktop app on the host machine, connected to its local server.
Provider: Claude (claudeAgent) routed through CLIProxyAPI running on the same host. autoResumeLimitedThreads is the behavior I expected to work.

What happened
When the pool was exhausted, the run ended with this error:

API Error: Request rejected (429) · All credentials for model claude-sonnet-5-5 are cooling down (last error: rate_limit_error: This request would exceed your account's rate limit. Please try again later.)

The thread showed no reset time and auto-resume never armed. The Usage page, read through the same CLIProxyAPI source, shows the correct reset times at the same moment.

Evidence

In the provider event log for the affected thread, the synthetic assistant message has error: "rate_limit" and api_error_status: 429. The log has zero rate_limit_event frames and no resetsAt and no api_error_params.rate_limit_info.
In the source at tag v0.1.6-nightly.20261030.3904, ClaudeAdapterV2.ts only derives resetAt from rate_limit_event, api_error_params.rate_limit_info.resetsAt, or a (reset after Xh Ym Zs) suffix on a 503 gateway error. This 429 matches none of them, so the failure is stored as usage_limit with resetAt: null.
UsageLimitRecoveryWorker.ts requires thread.usageLimitResetAt before it arms, so no recovery is scheduled.

Notes

This is not a network problem. CLIProxyAPI is on the same machine and the Usage page already reads it successfully, so the missing piece is wiring the hub's reset data into thread recovery, which is what fix(server): usage-limited threads on a pooling proxy now resume when the pool recovers #17866 proposes. That PR is still open and unmerged, so no release contains it.
Related: [Bug]: CLIProxyAPI account-pool cooldowns never reach usage-limit auto-resume #20375 (closed in favor of fix(server): usage-limited threads on a pooling proxy now resume when the pool recovers #17866).

Drafted by Claude Sonnet 5.5 via Claude Code, using the T3 Code t3 triage playbook.

Follow-up: CLIProxyAPI does send the reset time, and a reproduction

I reproduced this on demand and checked what the proxy sends.

Setup: Desktop app, 0.1.6-nightly.20261030.3904, Windows 11. Claude is routed through a local CLIProxyAPI (v0.3.58 bundle). I disabled one of the pool's accounts, so every account was cooling down.

1. The proxy returns the cooldown as Retry-After. I sent a 1-token POST /v1/messages straight to the proxy for two models, 49 s apart:

Model	Status	`Retry-After`
`claude-sonnet-5-5`	429	1061
`claude-haiku-5-5`	429	1007

The values count down together, so they track one shared deadline. The proxy also lists Retry-After in its Access-Control-Expose-Headers. The JSON body has no reset time:

{"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your account's rate limit. Please try again later."}}

So the proxy is not withholding the reset time. It sends it, as a header only.

2. T3 Code never sees it. I started a new Claude thread on the same model. The thread showed:

API Error: Request rejected (429) · This request would exceed your account's rate limit. Please try again later.

The provider event log for that thread has no rate_limit_event frame, no resetsAt, and no retry-after. The only 429 signals are api_error_status: 429 and error: "rate_limit".

The stored failure for both runs of that thread is:

{"class":"usage_limit","message":"Claude API rate limit reached. Try again later.","code":"api_error_429","retryable":true,"resetAt":null}

snoozedUntil is null, and nothing is scheduled to resume the thread.

What this suggests: The Claude CLI makes the request, handles the 429, and reports a final text error to the adapter. T3 never sees the proxy's Retry-After header. I haven't confirmed whether the CLI reads that header or what it does with it.

Two things worth noting for a fix:

The reset time can come from the proxy's management API (as fix(server): usage-limited threads on a pooling proxy now resume when the pool recovers #17866 proposes) or from this header. The header gives the cooldown for the specific model that was rejected.
The error text differs between cases. My earlier run said "All credentials for model … are cooling down (last error: …)" and this one doesn't. Both ended with resetAt: null, so a fix shouldn't depend on matching the wording.

Related: #20375, #17866.

Drafted by Claude Sonnet 5.5 via Claude Code, using the T3 Code t3 triage playbook.

@MatEE404

Note

🤖 GPT-6.1 Sol responding on behalf of @MatEE404

Additional verified case: a CLIProxyAPI pool can surface this recovery gap as 503 auth_unavailable, so handling terminal 429s alone would miss it.

Observed environment: macOS desktop, T3 server 0.1.5-nightly.20261029.3877, Claude Code 2.1.312, CLIProxyAPI 8.0.45, Claude Opus 5.5. Both auto-resume and snooze for limited threads were enabled.

The affected route had five enabled Claude accounts: four had future quota cooldowns; the fifth looked globally active but had a model-specific payment_required / HTTP 403 cooldown. All five were blocked for the requested model. The hub returned:

API Error: 503 auth_unavailable: no auth available (providers=claude, model=claude-opus-5-5; last upstream error: permission_error: OAuth authentication is currently not allowed for this organization.); check Claude auth/key session and cooldown state via /v0/management/auth-files

The last upstream error therefore described a different account's permission failure, while a quota-blocked account offered a future retry opportunity. A real hub response included Retry-After, and its management auth-files response exposed the per-credential/model cooldown deadlines. The Claude stream presented the failure as assistant server_error / 503 and terminal subtype: "success", is_error: true, terminal_reason: "api_error", api_error_status: 503, without a usable recovery deadline.

T3-side classification detail: at inspected main 6575ce32508bd7908444241f3719aa33aaf77f90, the usageLimited predicate excludes an explicit terminal 503 even if assistant limit metadata is present. The special gateway-503 parser handles a different wrapped-429 format with a (reset after …) suffix; it does not match this envelope.

Local workaround and verification: a narrow CLI wrapper reads fresh management cooldowns only for recognized pool-unavailable errors. It requires every eligible account to have a relevant future blocker and selects the earliest quota-only account's recovery time, after accounting for all of that account's blockers. Accounts with permission/payment blockers are excluded as quota candidates. Available/unknown accounts, malformed relevant timers and all-auth/payment pools prevent inference; direct 403s remain unchanged.

For a verified quota recovery opportunity, the wrapper supplies absolute rate_limit_info.resetsAt and normalizes both the root assistant and its correlated terminal result to 429. Enriching only the assistant while leaving the terminal 503 was insufficient. Real CLI + bundled T3 SDK loopback checks covered both pure quota and mixed permission/quota pools without live inference.

After loading that wrapper into four previously failed peers and sending one continuation each, read-only live T3 state confirmed, on all four:

{
  "limitRecovery": {
    "resetAt": "2026-10-30T15:30:08.000Z",
    "autoResume": true,
    "snooze": true
  },
  "snoozedUntil": "2026-10-30T15:30:08.000Z"
}

This verifies arming the existing recovery worker, not a successful upstream quota renewal. No T3 database writes or provider/permission changes were used.

This is an integration gap across the gateway/CLI/T3 boundary, not evidence that arbitrary 503s or permission errors should be treated as quota exhaustion. Please include this synthetic 503 + mixed-pool case when considering management-backed recovery in #17866.

Additional confirmation on Windows with Claude routed through CLIProxyAPI.

Environment

Installed T3 Code desktop and server package version: 0.1.6 (from the installed package manifests; exact nightly/commit not established).
EasyCLIProxyAPI bundle: v0.3.65-Windows-amd64.
Bundled CLIProxyAPI core: 8.0.48 (from core-version.txt).

Observed behavior
The Limits page displays the account usage limits, but starting a Claude thread when the account is limited produces:

API Error: Request rejected (429) · This request would exceed your account's rate limit. Please try again later.

The thread then shows:

Usage limit reached

Reset time unavailable; retry manually

There is no option to automatically resume the thread at reset. This prevents the unattended continuation the user expects.

Read-only inspection
The installed server bundle's limitRecoveryCommand requires a failed thread classified as usage_limit with a valid usageLimitResetAt. Its Claude adapter derives reset times from native limit metadata, with a separate parser for a specific wrapped 503/429 error format. The user-reported plain 429 above does not include a reset timestamp.

The proxy response headers and raw provider event stream were not captured for this report, so this confirmation does not establish whether Retry-After is present or where metadata is lost. No queued-message issue was tested, and the auto-resume preference value was not inspected.

Expected: provide reset-time recovery for a proxy-backed usage-limit failure when a reliable deadline can be obtained, with safe account/model/pool association rather than selecting an arbitrary account's reset.

Posted at the user's request; drafted by GPT-6.1 Sol in T3 Code.

Additional confirmation from another user on stable T3 Code 0.1.6, observed on 2026-11-02. Automatic continuation stopped working when an ongoing Claude workflow hit the proxy's rate limit.

Environment

Linux x64, kernel 7.2.11-arch1-1; T3 Code Desktop and local server both 0.1.6 (commit 2359643cffa6ba9ee84d6f2694b935de54a8013e).
Node v26.12.1; Claude Code 2.1.316; model claude-opus-5-5, provider claudeAgent, custom API endpoint.
autoResumeLimitedThreads: true; an enabled CLIProxy usage-limit source is configured.

Observed behavior and evidence

The provider event log captured this synthetic assistant error:

API Error: Request rejected (429) · All credentials for model claude-opus-5-5 are cooling down (last error: rate_limit_error: This request would exceed your account's rate limit. Please try again later.)

It carried error: "rate_limit" and api_error_status: 429, without a reset timestamp. The stored root failures were:

{"class":"usage_limit","message":"Claude API rate limit reached. Try again later.","code":"api_error_429","retryable":true,"resetAt":null}

There were four such error items across the parent thread and two delegated child threads, around 15:59–16:00 UTC. The UI showed Reset time unavailable; retry manually, despite auto-resume being enabled. A subsequent native Claude usage snapshot reported windows: [] and unavailable.reason: "unsupported".

Read-only inspection of the matching release source confirms that UsageLimitRecoveryWorker requires usageLimitResetAt, and the candidate query excludes null reset times. The worker does not consult the external usage-source snapshots, so this failure cannot arm recovery.

This confirms the missing-reset/auto-resume symptom on stable 0.1.6. This investigation did not capture proxy HTTP headers, freshly query the hub's account windows, or reproduce the queued-message snooze error. Separately, the parent was automatically settled after the failure; that evidence is being added to #28362.

Still happening on the 20261104 nightly, plus a proxy-side workaround that uses the existing direct-429 parser

Environment

macOS, T3 Code desktop 0.1.8-nightly.20261104.4165, server on the same Mac.
Claude Code 2.1.317, claude-opus-5-5, routed through Homebrew CLIProxyAPI 8.0.45 that pools several Claude subscription accounts.
CLIProxy usage-limit source configured.

Observed

On Nov 3 the pool cooled down. 7 turns in 5 threads ended with:

API Error: Request rejected (429) · All credentials for model claude-opus-5-5 are cooling down (last error: rate_limit_error: This request would exceed your account's rate limit. Please try again later.)

The Claude Code transcript records error: "rate_limit" and apiErrorStatus: 429, with no api_error_params.
T3 stored all 7 as {"class":"usage_limit","code":"api_error_429","retryable":true,"resetAt":null}. None of the threads got limitRecovery. All 74 Claude usage_limit failures on this machine since Oct 20 have resetAt: null.

Why the parser from #25438 doesn't fire
providerFailureFromResult already reads a (reset after …) suffix from a direct 429 (ClaudeAdapterV2.ts#L3919-L3925). Stock CLIProxyAPI never writes that suffix. modelCooldownError.Error() in sdk/cliproxy/auth/selector.go puts the duration only in reset_time, reset_seconds and Retry-After. The Claude handler forwards only error.message, and Claude Code drops the header. I checked selector.go history back to April and the phrase was never there. #26735 notes the same dependency.

Proxy-side workaround
Appending the duration to the message lets the existing parser work with no T3 change:

// sdk/cliproxy/auth/selector.go, modelCooldownError.Error(), after displayDuration is rounded
if displayDuration > 0 {
	message = fmt.Sprintf("%s (reset after %s)", message, displayDuration)
}

Claude Code then shows … are cooling down (reset after 4m6s) (last error: …).

A Go test on the patched selector passes for 2h54m21s, a sub-second reset rounded up to 1s, 4m6s followed by a last error, and no suffix at zero.
The regex from ClaudeAdapterV2.ts reads that text as 10461s, 1s and 246s.
The installed nightly's server bundle contains that parser.
Not verified yet: a real pool cooldown arming and resuming a thread end to end.
This covers Claude only. The Codex text quoted in fix(server): limited threads through CLIProxyAPI resume when the hub refills #27214 (exceeded retry limit, last status: 429 Too Many Requests) carries no duration, so a hub lookup like fix(server): limited threads through CLIProxyAPI resume when the hub refills #27214 is still needed for both CLIs.

Investigated with Claude Code (claude-opus-5-5).

— Claude, Santi's agent

Navigation Menu

Repository navigation

[Bug]: CLIProxy reset times are available in Usage but missing from thread recovery; queued messages block snoozing #26574

Before submitting

Area

Steps to reproduce

Expected behavior

Actual behavior

Source trace

Related issues

Impact

Version or commit

Environment

Workaround

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug]: CLIProxy reset times are available in Usage but missing from thread recovery; queued messages block snoozing #26574

Description

Before submitting

Area

Steps to reproduce

Expected behavior

Actual behavior

Source trace

Related issues

Impact

Version or commit

Environment

Workaround

Activity

marcojsmith commented on Oct 30, 2026

marcojsmith commented on Oct 30, 2026

MatEE404 commented on Oct 30, 2026

itskavin commented on Nov 2, 2026

camescasse commented on Nov 2, 2026

santissimoo commented on Nov 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions