Skip to content

[Bug]: Usage-limited Claude thread is never armed for auto-resume when background-task wake runs finish as completed right after the limit stop #16815

Open
Open
[Bug]: Usage-limited Claude thread is never armed for auto-resume when background-task wake runs finish as completed right after the limit stop#16815
Labels
bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

Description

@LukeDolce
LukeDolce
opened on Oct 7, 2026

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Related but different: #15665 (a follow-up turn fails with resetAt: null) and #16233 (a queued background notification blocks snooze). Here the follow-up turns are not failures at all: they finish as completed, so the thread is never armed for auto-resume.

Filed by the user's Claude Code agent at their request. Everything below is sanitized: no conversation content, project names, paths or account details. Thread labels A and B are arbitrary.

Area

apps/server

Summary

With Auto-resume limited threads and Snooze limited threads both on, a Claude thread that hit its five-hour limit while background subagents were running was never scheduled to resume. Its limited run stored a valid future reset, but about 2 seconds later a background-task notification started a new root run that finished as completed in under 2 seconds. Five more such runs followed. The thread's latest executed run was therefore completed, limitRecovery stayed null, no limit-resume: message was ever dispatched, and the thread sat idle for 4 h 42 m after the reset until the user sent a message.

A second Claude thread on the same account, where no further run followed the limited one, hit the same limit window about 73 seconds later. It was armed and resumed 1 second after the reset.

Steps to reproduce

Observed incident on an unmodified official nightly. Not a minimized reproduction, and no fresh quota-consuming run was made.

  1. Settings: autoResumeLimitedThreads: true, snoozeLimitedThreads: true.
  2. In a Claude (claudeAgent, Orchestrator V2) thread, start several background subagents (11 in this incident) and let the root turn end normally while they work.
  3. Exhaust the five-hour usage window. A background-task notification starts a root run that fails with usage_limit.
  4. Other background subagents die on the same limit. Each one's completion notification starts another root run within seconds.
  5. Wait for the reported reset time.

The reproduction in #15665 (real CLI against a mock Messages API) should be adaptable: after the first limited turn, let a background task complete so its wake turn starts within a few seconds.

Expected behavior

The limited run is armed for recovery and the thread resumes at the reset time, exactly as the comparison thread did. A wake turn that makes no progress because the account is limited should not cancel or hide a pending recovery.

Actual behavior

Thread A, read-only from the V2 projection tables. t0 is the moment the limited run failed. R is the stored reset, t0 + 4 h 20 m.

Run Started by Ended Ran for Status Turn items
2 background-task notification t0 30.5 s failed ..., assistant_message (the provider's limit text), error with failure.class: usage_limit, failure.code: api_error_429, failure.resetAt: R
3 background-task notification t0 + 2.1 s 1.8 s completed notification, checkpoint
4 background-task notification t0 + 6.8 s 0.6 s completed notification, checkpoint
5 background-task notification t0 + 9.8 s 0.8 s completed notification, checkpoint
6 background-task notification t0 + 34.3 s 0.7 s completed notification, system_notice, checkpoint
7 background-task notification t0 + 41.0 s 1.3 s completed notification, assistant_message (61 characters: You've hit your session limit · resets 3am (America/New_York)), checkpoint
8 background-task notification t0 + 5 m 14 s 0.6 s completed notification, system_notice, checkpoint
  • Thread A limitRecovery: null. It was never armed.
  • Thread A turn items matching limit-resume: none.
  • Thread A next activity: a user message at R + 4 h 42 m.
  • Runs 3 to 8 carry no error item. Run 7 stores the provider's limit message as an ordinary completed assistant message.

Thread B, same account and same limit window, no run after the limited one:

Event Time
Limited run failed (usage_limit, resetAt: R) about t0 + 73 s
limitRecovery armed { autoResume: true, snooze: true, resetAt: R }
limit-resume run requested R + 1 s
That run completed, and the thread kept working

Analysis

Read from the installed server bundle of this build, not from a source checkout.

  • ProjectionStore.getLimitRecoveryCandidates selects each thread's latest executed run (the run that ended last) and requires r.status = 'failed' plus a usage_limit error item with a non-null resetAt on that same run.
  • limitRecoveryCommand in the usage-limit recovery worker returns null unless thread.status === "failed" and thread.lastErrorClass === "usage_limit".
  • So a wake run that ends as completed becomes the latest executed run and removes the thread from the candidate set. In this incident that happened 2.1 s after the limit stop, before the limited run had been armed.
  • Separately, a turn that could not do any work because the account was limited was classified completed. In run 7 the only output is the provider's limit text.

Why this is not covered by a fix for #15665: carrying the reset onto follow-up failures does not help when the follow-up turns are not recorded as failures.

Possible directions, for the maintainers to judge:

  • Classify a wake turn whose provider result is the limit message or a 429 as a usage_limit failure, including when a background-task notification started it.
  • Let candidate selection use the most recent usage_limit failure whose reset is still in the future, ignoring later notification-only runs that produced no work.
  • Hold background-task wake turns while the thread is limited, and release them with the resume.

Impact

Major degradation or frequent failure.

Auto-resume silently does nothing for threads that have background agents running when the limit hits, which is the pattern most likely to exhaust a limit. Nothing resumed the thread and its work idled overnight. I did not capture how the thread was presented in the UI at the time.

Version or commit

T3 Code (Nightly) 0.0.46-nightly.20261005.2702, official Windows build. Claude Code CLI 2.1.292.

Environment

  • Windows 11 Pro 10.0.26200, desktop app with its local server.
  • Provider claudeAgent, model claude-opus-5-5, Claude subscription login, Orchestrator V2.
  • The machine did not sleep during the window (no Kernel-Power sleep or wake events), and the same server process was running before the limit and after the reset.

Logs or stack traces

None retained. server.trace.ndjson rotates at 10 MB roughly every three minutes here, so the files covering the limit stop and the reset were gone by morning. The evidence above comes from orchestration_v2_projection_runs, orchestration_v2_projection_turn_items and orchestration_v2_projection_threads, opened read-only.

Workaround

Send a message to the thread after the reset. Avoid launching many background agents when the usage window is nearly spent.

Activity

  1. juliusmarminge commented on Oct 7, 2026

    @juliusmarminge
    juliusmarminge
    on Oct 7, 2026
    Member

    Note

    Grok responding on behalf of Julius.

    Thanks for the detailed write-up. I checked it against main and it's a real bug.

    Root cause

    • ProjectionStore.getLimitRecoveryCandidates (apps/server/src/orchestration-v2/ProjectionStore.ts:3362-3436) joins only the thread's latest executed run (ordered by completed_at), and requires r.status = 'failed' plus a root error item on that same run with failure.class = 'usage_limit' and a non-null failure.resetAt that's later than the run's completed_at.
    • limitRecoveryCommand (apps/server/src/orchestration-v2/UsageLimitRecoveryWorker.ts:11-25) returns null unless thread.status === "failed" and lastErrorClass === "usage_limit". Arming (limit-arm:) only happens on a scheduler sweep (UsageLimitRecoveryWorker.ts:78-113), so it isn't immediate.
    • Background-task wake runs aren't gated on limit state. ClaudeAdapterV2 (Adapters/ClaudeAdapterV2.ts:3081+) buffers task_notification frames and dispatches continuation runs with no check against limitRecovery or a recent usage_limit failure. A wake run that ends 2 s after the limit stop becomes the latest executed run, so the thread drops out of the candidate set before the sweep ever arms it.
    • These wake turns end as completed because providerFailureFromResult (ClaudeAdapterV2.ts:2517-2555) returns null for a success result with is_error: false. The usageLimited heuristic (ClaudeAdapterV2.ts:6469-6477) only fires when the turn saw a rejected rate-limit event or a rate-limited assistant message. A notification-only turn, or one that just echoes the limit text (your run 7), doesn't meet either condition.

    Fix options

    1. Classify limit-text/429 wake turns as usage_limit. This is worth doing, but it isn't enough alone. Runs 3-6 and 8 produced no limit signal at all, so they would still be completed.
    2. Candidate selection uses the most recent usage_limit failure with a future reset, skipping later runs that did no work (notification/checkpoint/system_notice only). Recommended. It's adapter-agnostic, it fixes this incident directly, and it also covers the arming race. Both the SQL and limitRecoveryCommand need to stop relying on thread.status === "failed" for this case, keyed off the limited run's id.
    3. Hold background wake turns while limited and release them on resume. This is the cleanest behaviour, but it's the biggest change. It also touches the queued-notification/snooze interaction in [Bug]: Usage-limited Claude thread cannot be snoozed because of an unheld automatic queued notification #16233, so it's better as a follow-up.

    Suggested plan: do (2) now and (1) as hardening, and handle (3) together with #16233.

    Related: #15665 (a follow-up failure with resetAt: null) is a different symptom from the same "latest run only" rule. #16233, #15130 and #14169 are adjacent but not duplicates. PR #15553 is UI-only and doesn't change arming.

    Workaround: after the reset, send any message to the thread. Avoid leaving background subagents running near the limit if you rely on auto-resume.

  2. juliusmarminge
    added
    bugSomething is broken or behaving incorrectly.
    via-triageFiled through npx t3 triage
    on Oct 7, 2026

  3. Neonsy commented on Oct 9, 2026

    @Neonsy
    Neonsy
    on Oct 9, 2026

    Still reproduces on 0.0.46-nightly.20261009.2861 (3b6af0bd1600), Windows 10 Pro 10.0.19045, Claude Code CLI 2.1.295, autoResumeLimitedThreads: true. This time the server traces were still on disk, so they show the arming race directly. Times are UTC

    Time Event
    12:21:04.774 Provider rate_limit_event: rejected, five_hour, reset 12:40:00
    12:21:05.595 Recovery sweep. Nothing to arm yet
    12:21:06.365 Run 49, started by a "7 background tasks failed" notification, fails with usage_limit and resetAt 12:40:00
    12:21:06.880 First wake run dispatched (message.dispatch with a provider-continuation: command), 515 ms after the failure
    12:21:08.332 Run 50 completed. Provider result success, is_error: false, num_turns: 0, 14 ms
    12:21:10.611 Next sweep. The latest executed run is now run 50 (completed), so nothing is armed
    12:21:18.605 and 12:21:25.536 Runs 51 and 52 complete the same way
    12:40:00 onward No limit-arm: or limit-resume: command and no new run (checked through 12:41)
    • The thread has no limitRecovery. Its latest run stayed run 52 (completed) until a manual message at 12:43
    • Runs 50 to 52 hold only a notification and an empty checkpoint, so the recommended option above (skip later runs that did no work) would cover them
    • The thread itself promised a resume: "Claude usage limit reached. This turn is paused until the 5-hour limit resets in 19m."

    Client side, from the installed bundle. The composer banner reads the same latest-run status (Pi.status === "failed" && Pi.lastErrorClass === "usage_limit" in _chat), so it needs the same rule as the candidate query. Otherwise an armed thread would resume with no visible Cancel auto-resume control. In this incident the banner condition held for about 0.5 s, from the push after the failure (12:21:06.372) to the run 50 update (about 12:21:07.10). The client did process that state, since it sent a thread.visit for this thread at 12:21:06.590. In that window the banner could only offer Resume at reset, because Cancel auto-resume requires an armed recovery

    # Provider events log (trimmed)
[12:21:04.774Z] rate_limit_event {"status":"rejected","rateLimitType":"five_hour","resetsAt":1791549600}
[12:21:07.458Z] result {"subtype":"success","is_error":false,"num_turns":0,"result":"","duration_ms":14}
[12:21:18.486Z] result {"subtype":"success","is_error":false,"num_turns":0,"result":"","duration_ms":14}
[12:21:25.484Z] result {"subtype":"success","is_error":false,"num_turns":0,"result":"","duration_ms":12}

# server.trace.ndjson (start time, span, command)
12:21:05.595Z UsageLimitRecoveryWorker.sweep
12:21:06.372Z ws.orchestrationV2.projectShellItems
12:21:06.590Z orchestrationV2.dispatch.once  thread.visit
12:21:06.880Z orchestrationV2.dispatch.once  message.dispatch  provider-continuation:message:...
12:21:10.611Z UsageLimitRecoveryWorker.sweep
# No limit-arm: or limit-resume: command in any retained trace

    After the reset the subagents were resumed by hand, and their failure notification rows kept a red Failed badge while they ran. That part is filed separately as #17486

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions