[Bug]: Input sent to a Claude thread around a provider crash is never delivered (steered message lost; queued message held and overtaken) #22423

Activity

1. dpearson2699 commented on Oct 21, 2026

   

   dpearson2699

   on Oct 21, 2026

   More actions

   Another case of part B (a held queued message overtaken by later work), with the overtaking paths traced. Here the hold came from a server restart, not a provider crash. Messages sent over MCP can't release it, and nothing tells the sender it exists.

   Version: T3 Code (Nightly) 0.1.1-nightly.20261020.3438, desktop app on macOS, local server. The relevant code is unchanged on main at 448b8a9 (apps/server/src/orchestration-v2/Orchestrator.ts L1404, L1474, L5264-5281, L5395-5397; ProviderRuntimeRecoveryService.ts L326-334).

   What happened

   A coordinator thread drives a long-running delegate_task child (Claude) entirely over MCP. Nobody types into the child.

   1. A scheduled-task message was queued behind an active run on the child (queued, not held).
   2. The desktop app updated and restarted the server. runtime-reconcile:shutdown cancelled the active run and set queueHeld: true on the queued run.
   3. For the next ~13 h the hold was never released, and the child kept working normally:
      • Delegated-completion wakes (fixture:delegated-completion:dispatch:*) started new runs whenever the thread was idle: 20 runs.
      • t3_thread_send messages that arrived while the thread was idle started immediately. The repro below shows this happens even with mode: "queue".
      • Every t3_thread_send with mode: "queue" that arrived while a run was active was created already queueHeld: true (queuePosition 2, 3, 4, then 5, 6) and never started. When the active run ended, a later send and four completion wakes started ahead of them.
   4. Six queued messages sat there with startedAt: null until the coordinator noticed and t3_queue_canceled them, then resent them with mode: "auto" (which started at once).

   Nothing failed. t3_thread_send returned {status: "queued", delivery: "queued"}. task_status on the queued send's task said status: "queued", workState: "working", and t3_queue_list listed the items with no held flag. From the sender's side, nothing distinguishes this from a normal queue that will drain.

   Why (traced)

   • Shutdown reconcile and provider recovery mark every queued run queueHeld: true.
   • startNextQueuedRun refuses to drain while any queued run is held.
   • The only releases are queue.resume and heldRunsToRelease (added by fix(server,mobile): a message you send resumes a held queue, and mobile can resume one #21473). heldRunsToRelease applies only when createdBy === "user" and creationSource is web or mobile. MCP sends (createdBy: "agent", creationSource: "mcp"), delegated-completion wakes, and scheduled tasks never release. No MCP tool exposes queue.resume.
   • A new queued run inherits queueHeld: true whenever a held run exists and nothing was released.
   • Dispatch to an idle thread doesn't look at the queue at all, so later sends and wakes start ahead of the held runs. That answers the "which path started them" question above.

   So #21473 fixed this for people typing in the app, but any thread driven by an agent over MCP (delegated children especially) gets stuck behind one restart or Stop until a human resumes it. The day before, on another delegated child, a cancelled run left the next queued run held for ~37 h while a restart continuation started ahead of it.

   Minimal repro (scratch, Haiku, ~2 min)

   1. From a parent thread, delegate_task a child that delegates two async grandchildren and ends its turn.
   2. t3_thread_send the child a message with mode: "auto" (it starts), then two more with mode: "queue" (runs N+1, N+2 queued).
   3. task_cancel the task for the running message. That stops the thread like a user Stop: N is interrupted, and N+1 and N+2 become queueHeld: true.
   4. t3_thread_send the now-idle child a message with mode: "queue". It starts immediately (delivery: "started").
   5. While it runs, send another with mode: "queue". It is created already held.
   6. The grandchildren finish, and their completion wakes start new runs. The three held runs never start, and nothing reports that they are held.

   Control: without step 3 the two queued sends drain in order right after the active run, ahead of the completion wakes, so the hold is what causes this.

   Expected

   Any one of these would do:

   • Never start newer work, a send or a wake, ahead of an older held queued message on the same thread. Or release the hold when an agent sends a message, as a web/mobile user message does.
   • Expose queue.resume (or a held flag plus resume) to MCP callers. Make t3_thread_send / task_status / t3_queue_list say when a message is queued behind a held queue.

   Related: #19233 (restart hold behind a cut /compact), #22606 (held queue with no explanation after a failed run), #21473 / #18792 (the user-message release).
2. nkoynov commented on Oct 25, 2026

   

   nkoynov

   on Oct 25, 2026

   Contributor

   More actions

   Another restart hold of part B, this time after an unclean stop (power loss), where the server's own restart continuation ran without asking while the held queue waited three and a half hours. 0.1.3-nightly.20261024.3617 on Linux, OpenCode provider, restart continuations on. Times are UTC, from orchestration_v2_projection_runs.

   • A scheduled task bound to the thread (every 30 minutes) queued a run at 08:24 behind the active run.
   • The machine lost power. At 09:13:14 startup recovery cancelled the active run and set queueHeld: true on the queued one. The restart continuation started at 09:13:20 and finished normally.
   • The next two occurrences of the same task, at 09:13:19 and 09:43, were created already held.
   • Over the next 3.5 hours about 20 other runs started right away on the idle thread, later occurrences of the same task among them, overtaking the three held ones.
   • At 12:43:29 a message I sent from the web app released the hold, and the three stale occurrences ran back to back at 12:44.

   So the consent the hold waits for wasn't asked anywhere else: the server continued the cut turn on its own, and the task kept running through its newer occurrences. The held ones only came back hours late and out of order. The hold is set for every queued run in ProviderRuntimeRecoveryService.ts (L317-332 on main 3d00499), and heldRunsToRelease in Orchestrator.ts excludes restart continuations and scheduled tasks on purpose (#21473).

   One way out: when startup recovery continues the cut run, don't hold the queue behind it (or release the hold once the continuation settles). A held scheduled-task occurrence that a newer occurrence of the same task has overtaken could be dropped rather than delivered late (related: #16818).

   Model and harness: Claude Opus 5.5 via OpenCode 2 in T3 Code.