[Bug]: Delegated-task completion notice stays queued when the child finishes during the parent's turn #18859

Activity

1. juliusmarminge commented on Oct 12, 2026

   

   juliusmarminge

   on Oct 12, 2026

   Member

   More actions

   Note

   Grok responding on behalf of Julius.

   Thanks for the report. I don't think the open delegated-delivery PRs cover what you're seeing:

   • fix(server): a delegated result whose steer never landed wakes the parent after it settles #14732 fixes a delivery that was reserved as a steer and then rejected while the parent turn ended. That delivery never becomes a queued run. Your notices are queued runs (ordinal:2551, ordinal:2567) that stay queued while later runs start and finish.
   • fix(server): deliver run-specific delegated results exactly once #18649 (deliver per-run results exactly once) and fix(server): bound delegated completion delivery retries #15785 (bound delivery retries) are about which result gets delivered and retry limits. Neither changes when a queued run gets promoted.
   • delegate_task async completions are silently starved after two wake deliveries per cohort — parent never wakes for third-wave children (V2 branch) #12285 (wake cap per cohort) left results pending and never queued, so it's a different symptom.

   The closest match is #15862: after a restart, queued runs get queueHeld: true. Later runs then start directly while the held ones never drain until you resume or cancel them. That matches "queued for hours while 50+ later runs complete." Before I mark this as a duplicate, could you check the following?

   1. Does t3_queue_read (or the run payload in orchestration_v2_projection_runs) show queueHeld: true on runs 2551 and 2567?
   2. Did the server or app restart, or did you hit Stop, between those children finishing and the later turns running?
   3. Your T3 Code version / commit, and whether this is main or a fork of orchestration V2.
   4. Do the later turns that ran start from your own messages, or from other delegated wakes?

   If the runs aren't held, then a queued run is being skipped at settlement on a normal queue, and that's a separate bug we'd want to fix.

   👀React with 👀1Reacted by nitedani
2. juliusmarminge

   added

   needs more infoInitial triage showed no bug. Awaiting more infoInitial triage showed no bug. Awaiting more info

   via-triageFiled through npx t3 triageFiled through npx t3 triage

   on Oct 12, 2026
3. nitedani commented on Oct 13, 2026

   

   nitedani

   on Oct 13, 2026

   Last edited by nitedani

   Author

   More actions

   Opus 5.5 (medium effort) wrote this; GPT-6.1-Sol (Codex, medium effort) reviewed it.

   Thanks for the quick triage! It's #15862. The hold came from a server restart, not from run 2550's failure at 20:07:48Z. In orchestration_events, run 2551 (the completion notice of a child that finished during a turn) got held only by the startup reconcile:

   run.created 2026-10-12T20:07:19.422Z queued queueHeld=None command:fixture:delegated-completion:dispatch:…
   run.updated 2026-10-12T20:14:33.077Z queued queueHeld=True command:runtime-reconcile:startup:…
   run.updated 2026-10-13T00:03:22.423Z cancelled (by me)
   

   Every run that had to queue after that inherited the hold (Orchestrator.ts#L5035-L5039): 2555, 2567, 2586 (delegated completion notices) and 2573 (a background-command notice). Runs that started directly kept working, from my messages and from automatic delegated wakes, which is why the thread looked healthy while those notices sat for up to about four hours.

   I don't know whether Stop was pressed in between. The app is the official nightly t3 v0.0.46-nightly.20261012.3021, which is tag v0.0.46-nightly.20261012.3021 at 7b6d907, on main.

   #18792 looks like it would have kept the later notices out of the hold. 2551 itself would still have waited for a manual Resume, and a delegated completion notice isn't something a person is watching for, so I'd suggest restart holds skip automatic wakes. Cancelling the five held runs emptied the queue, which unblocks the thread for now.
4. JorgeMenaDev commented on Oct 16, 2026

   

   JorgeMenaDev

   on Oct 16, 2026

   More actions

   Another case, on t3 v0.0.46-nightly.20261008.2801 (macOS, Claude parent thread). This one happened while the parent was idle, not mid-turn.

   1. 08:56:54 UTC: the parent calls t3_thread_send to an already-finished delegate_task child (clientRequestId acr-1666-fix-r2). The parent ends its turn at 08:56:57.
   2. About 08:59: the child finishes. task_status later shows status: completed, workState: result_available.
   3. No "reached a terminal state" notice ever arrives in the parent. The parent then runs two more turns (a scheduled-task wake at 11:10 and a user message at 12:00), and the notice is not delivered with either.
   4. 13:18: I read task_status by hand. t3_queue_list on the parent is empty afterwards (reading the status may have acknowledged a queued notice).

   Two earlier t3_thread_send continuations to the same child in the same parent (08:49 and 08:50) did wake the parent normally. Workaround on our side: call watch_pull_request after every push, so a CI result wakes the parent even when the notice never arrives.

   👀React with 👀1Reacted by nitedani
5. nitedani commented on Oct 16, 2026

   

   nitedani

   on Oct 16, 2026

   Author

   More actions

   Opus 5.5 (high effort) wrote this; Opus 5.5 (medium effort) reviewed it.

   Thanks for the second case! It matches #17696, merged on 2026-10-10, which gives each message sent to a finished child its own task and completion notice. Before it, a follow-up could reuse the original task, so its result might get no completion notice of its own. Your v0.0.46-nightly.20261008.2801 build predates that fix. It's a different path from #18792, which is about a held queue. v0.0.46-nightly.20261010.2922 or later should deliver the notice for a continuation; if it still doesn't there, that's a new bug worth its own issue.
6. EtanHey commented on Oct 20, 2026

   

   EtanHey

   on Oct 20, 2026

   Contributor

   More actions

   Additional triage evidence for #18859. This initially looked like the same missed-wake symptom, but the captured cases are explained by queued delivery followed by result acknowledgement. This is not a confirmed reproduction of the queue-held defect in this issue.

   What happened

   On October 20, 2026, a Claude parent delegated 60 Cursor tasks with mode=async, in waves of at most eight. The parent reported that some terminal children never woke it. It discovered their results through task_status after their output files appeared.

   The actual P3 activity occurred around 01:03–01:20 UTC, or 04:03–04:20 Asia/Jerusalem.

   Diagnosis

   All 60 children reached a terminal state and published results. Provider completion prompts named 42 children. The remaining 18 were all acknowledged through task_status before their own provider completion prompt was delivered:

   • Fifteen had queued wake runs. Reading their results acknowledged delivery and cancelled those queued runs before they started.
   • One was added to a batched queued notice, then removed after acknowledgement.
   • Two were acknowledged while another delivery for their originating parent run was active, before a further notice was created.

   All 18 completed while the database recorded a parent run as running. Nine also had previously queued work. Each acknowledgement already had a nonempty result and an earlier result context transfer. This does not reproduce acknowledgement before result publication.

   The checked source documents terminal task_status reads as acknowledgement and explicitly cancels an empty queued wake after acknowledgement. That source is from the later installed build, not a verified copy of the incident binary.

   A further database audit covered 104 terminal tasks after the current build started: 47 had completed parent wake runs and replies, 20 had acknowledged published results, and 37 had explicit task-delivery cancellation commands. No terminal task remained pending or claimed in that snapshot. This does not rule out transient or unretained failures.

   Steps to reproduce

   Observed workflow, not a deterministic reproduction of lost delivery:

   1. Use a Claude parent to delegate independent Cursor tasks asynchronously, with up to eight running at once.
   2. Let child completions queue while the parent handles an earlier completion.
   3. During that active turn, read another terminal child's result with task_status.
   4. Observe acknowledgement and cancellation or removal of its queued completion notice. No separate provider prompt for that child then arrives.

   An irretrievably lost wake without an intervening acknowledgement or cancellation has not been reproduced.

   Version

   Incident version: unknown.

   Current desktop and server: 0.1.1-nightly.20261020.3386. Desktop traces record update installation at 02:12:28–02:12:33 UTC, after P3. The release was published at 01:46:21 UTC, also after P3. These observations prevent assigning the current version to that incident. The later audit covers activity after this build started.

   Triage CLI: 0.1.0, launched with npx t3 triage.

   Environment

   MacBook Pro, Darwin 27.0.0, arm64. T3 Code Nightly desktop app with its local server. Triage Node version: v26.8.2. P3 parent provider: Claude Agent, model claude-opus-5-5. Child provider: Cursor, model default/Auto. Incident agent CLI versions were not established.

   Evidence

   One working wake and one apparently missing wake, using database event sequence numbers:

   2026-10-20 UTC  task          event
   01:10:21.328   09-28-B       completed/claimed       seq 1365413
   01:10:21.411   09-28-B       wake run 16 running     seq 1365428
   01:10:33.327   09-28-B       delivered               seq 1365592
   
   01:10:27.313   09-28-A       completed/claimed       seq 1365518
   01:10:27.328   09-28-A       wake run 18 queued #2   seq 1365524
   01:10:45.794   09-28-A       acknowledged by run 17  seq 1365726
   01:10:45.794   09-28-A       wake run 18 cancelled   seq 1365728
   

   Concurrent provider completions showed the same pattern: 10-11-B finished at 01:16:56.958 UTC and 10-12-B 20 ms later. The first wake started. The second was queued, then acknowledged by the first wake's run at 01:17:06.167 UTC and cancelled at the same timestamp.

   The incident's server traces had rotated out, and retained server-child logs did not cover it. Provider logs and persisted database events supplied the lifecycle evidence. Later successful delivery spans cover only the retained trace window, not the entire current-build lifetime.

   Source comparison: terminal-result acknowledgement contract, queued-notice cancellation after acknowledgement.

   Related issues

   This issue, #18859, has a similar queued-notification symptom, but its reported startup queueHeld:true mechanism was not established here. PR #20274 addresses delivery retries and acknowledgement before publication. It was open at triage time; this capture does not demonstrate either failure. PR #14732 addresses rejected-steer recovery. No rejected-steer evidence was found here.

   Fix applied or workaround

   No fix applied. Investigation used read-only SQLite connections and log reads. No T3 source changes, database writes, restarts, or updates were performed. The original parent used terminal result reads to continue its work.

   Filed by

   Draft prepared by Codex, GPT-6, via t3 triage. Not posted.

   👀React with 👀1Reacted by nitedani
7. nitedani commented on Oct 20, 2026

   

   nitedani

   on Oct 20, 2026

   Author

   More actions

   Claude Opus 5.5 (high effort) wrote this; Claude Opus 5.5 (high effort) reviewed it.

   Thanks for digging into this! I have one case from today that acknowledgement doesn't explain, on 0.1.1-nightly.20261020.3411. My parent thread's queue still holds this notice, and I never called task_status on that child:

   t3_queue_list → run …:ordinal:4363  "Delegated task …review-tracker9-vr234-released reached a terminal state. …"
   

   Run 4363 was queued at 10:52:42Z. Two later notices of the same kind (runs 4364 and 4365) started ahead of it, then run 4366, all before the machine restarted at 18:16Z, so a hold after a restart doesn't explain it. That looks like the case @juliusmarminge called a separate bug: a queued run skipped on a normal queue. I can't tell whether it's held now, because t3_queue_read doesn't show queueHeld.
8. t3dotgg commented on Oct 21, 2026

   

   t3dotgg

   on Oct 21, 2026

   Member

   More actions

   Note

   🤖 Claude Opus 5.5 responding on behalf of Theo

   Thanks for the report. One part of this is fixed on main in #21473. Please test on the nightly built from main on 2026-10-21, or on the next stable release. One part is still open: held notices still wait for user action or Resume; normal-queue skipped notices remain unconfirmed. If that part still matters to you, please open a new issue for that part only.

   👀React with 👀1Reacted by nitedani
9. t3dotgg

   closed this as completedon Oct 21, 2026
10. nitedani

    added a commit that references this issue on Oct 28, 2026

    gh-watch.py: the /agent search backup reads every page of its results

    ...

    76d964d