Skip to content

Desktop worker queue: a task queued behind a pre-cancelled task never runs #9463

Open

Description

@Bruno-rose

Where did this happen?

Desktop app (macOS)

Version

Source build, main at 19db630

What happened?

In the desktop WorkerTaskRunner, drainQueue takes a pre-aborted task, rejects it, and moves on to the next slot. The task queued behind it stays queued while the worker is idle, with no timeout running, until another task arrives. The host-service copy of this class already handles this.

No current caller passes a signal, so this cannot happen today.

Expected: the queue keeps draining into the same slot.

Steps to reproduce

  1. With concurrency 1, run a task.
  2. Queue a task with an already aborted signal, then a normal task.
  3. Let the first task finish; the third never starts.

Logs, screenshots, or recordings

Found by model checking every order of events, then reproduced as a failing test against the real code. A fix with that regression test is ready.

Implementation notes

Files

  • apps/desktop/src/lib/trpc/workers/WorkerTaskRunner.ts drainQueue.

Related code

  • packages/host-service/src/workers/WorkerTaskRunner.ts: has the inner while (queue.length && !slot.activeTaskId) loop.

Approach

Copy the host-service loop.

Activity

  1. github-actions commented on Nov 7, 2026

    @github-actions
    Contributor

    The report is accurate. In the desktop WorkerTaskRunner, when drainQueue takes an already-aborted task off the queue, it rejects it and moves on to the next worker slot. It should keep filling the same slot. With concurrency 1 there is no next slot, so the loop ends while that slot sits idle and the next task stays queued. Nothing re-runs the drain until some other runTask call or worker event happens. The queued task has no timeout running, so it can wait forever. The host-service copy of this class already fixed this with an inner loop. As the report says, no desktop caller passes a signal today, so this is a latent bug rather than one users can hit.

    Affected code

    • apps/desktop/src/lib/trpc/workers/WorkerTaskRunner.ts:253-282: in drainQueue, the outer for loop over workerSlots takes one task per slot visit:
      if (task.abortSignal?.aborted) {
          this.rejectTask(task.taskId, new WorkerTaskAbortedError("cancelled"));
          continue;   // goes to the next slot, not the next task for this slot
      }
    • packages/host-service/src/workers/WorkerTaskRunner.ts:312-352: the fixed version, while (this.queue.length > 0 && !slot.activeTaskId) { ... }.
    • Callers: runGitTask in apps/desktop/src/lib/trpc/routers/changes/workers/git-task-runner.ts:63-71 passes options straight through. Searching apps/desktop/src/lib/trpc/routers/changes found no call that sets signal, which matches the report.

    Root cause

    The abort listener is added in runTask (WorkerTaskRunner.ts:154-158). If the signal is already aborted at that point, the listener never fires. So a pre-aborted task is only found when drainQueue takes it off the queue (:264). That branch uses up the slot's turn without giving the slot a task. Afterwards:

    • The slot has activeTaskId === null, so no worker result will arrive to call drainQueue again (:344).
    • The tasks still in the queue have no timeoutHandle, because the timeout is only set when a task is dispatched (:271), so no timer will free them either.

    With concurrency > 1 the same thing happens to the last free slot in the loop. It also happens whenever the number of aborted tasks plus waiting tasks is larger than the number of free slots left in the loop.

    Reproduction (bun:test sketch)

    WorkerTaskRunner.test.ts already has a fake worker passed in through createWorker. Using that:

    1. Create a runner with concurrency: 1.

    2. Call const first = runner.runTask("status", { id: "first" }). The worker takes it and does not reply yet.

    3. Create const ac = new AbortController(); ac.abort();.

    4. Call const aborted = runner.runTask("status", {}, { signal: ac.signal }).

    5. Call const third = runner.runTask("status", { id: "third" }).

    6. Make the fake worker reply { kind: "result", taskId: <first>, ok: true }.

    7. Expected:

      • aborted rejects with WorkerTaskAbortedError of kind "cancelled".
      • The fake worker receives a task message for third.

      Actual: no task message for third is posted, and third never settles.

    Suggested fix

    Copy the host-service inner loop into the desktop drainQueue. Keep the desktop-only checks on the outer loop, slot.cancelledTaskId and slot.terminating. The inner loop only needs !slot.activeTaskId, because a cancelledTaskId cannot be set while the loop runs.

    for (const slot of this.workerSlots.values()) {
        if (slot.activeTaskId || slot.cancelledTaskId || slot.terminating) continue;
        while (this.queue.length > 0 && !slot.activeTaskId) {
            // existing shift / aborted-reject / dispatch body
        }
    }

    A related gap you may want in the same change: the desktop version does not wrap slot.worker.postMessage(request) (:281) in a try/catch. The host-service version does (:339-351). On desktop, a payload that cannot be cloned would throw out of drainQueue and leave the slot occupied until the task times out. The inner loop handles the rejected task correctly once that catch is copied over too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions