Skip to content

fix(orchestration): wake an agent for new mail even while it still holds an unacknowledged batch - #33639

Merged
Jinwoo-H merged 3 commits into
mainfrom
jinwoo/sta-11980-unacked-batch-wake
Oct 25, 2026
Merged

Jinwoo-H merged 3 commits into
mainfrom
jinwoo/sta-11980-unacked-batch-wake

Conversation

@Jinwoo-H

@Jinwoo-H Jinwoo-H commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 9 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​284 $\color{#cf222e}{\Huge{\mathbf{−}}}$​32 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​252
Prod 10 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​127 $\color{#cf222e}{\Huge{\mathbf{−}}}$​48 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​79

ELI5

An agent reads its orchestration mail with check, which hands it a batch it must confirm with check --ack <deliveryId>. Until it confirms, Orca treated the agent as "still busy reading" and stopped telling it about any new mail, forever. So an agent that ran check without --ack (which is exactly what the old nudge text told it to run) and then ended its turn was never woken again. In the ticket, a coordinator's mail sat unread for about 3 hours until a person typed into the chat.

Now the unconfirmed batch only delays the nudge. When the agent's turn ends (chat) or it goes idle (terminal) and mail it has not seen is waiting, Orca nudges it once, names the batch it still holds, and gives the exact command to confirm it and get the new mail.

Root cause

Both pointer lanes returned early whenever the mailbox had an outstanding Delivery:

  • chat (structured) lane: structured-mailbox-pointer-delivery.ts (if (!db || db.hasOutstandingMailboxDelivery?.(mailboxHandle)) return)
  • terminal lane: mailbox-pointer-delivery.ts (same check)

They parked no retry. The turn-end and idle edges do re-run delivery, but they hit the same early return. Nothing expires an outstanding Delivery and nothing re-runs delivery on ack, so only user input could break the wedge. The behaviour has been the same since chat coordinators first got Run mail.

What changed

One rule, in the shared selection both lanes already use (mailbox-pointer-eligibility.ts):

  • selectOrchestrationPointerBatch now returns the mail the held Delivery does not contain, plus the held Delivery's id. Mail inside the held batch is never re-pointed.
  • The nudge for that mail reads: You have N new orchestration messages after delivery <id>, which you have not acknowledged. If you no longer have it, run `<cli> orchestration check --run <run>` to see it again; once it is handled, run `<cli> orchestration check --run <run> --ack <id>`. The replay command comes first so an agent that lost the batch to /clear or compaction never acks mail it has not seen. (Same formatter and send path as every other pointer.)
  • At most one nudge per arrival of newer mail. The dedupe is the existing durable delivered_at stamp on those messages, so no new state or flags.
  • shouldReleaseOrchestrationPointer (terminal lane, between write and Enter) now releases a staged pointer only when a check took that pointer's own mail into a batch, not whenever any batch is open.
  • Why wait for the turn end: a mid-turn steer stamps delivered_at, so a steer an unacked turn ignores would spend that mail's only nudge.
  • Chat lane: if a batch is held and the agent is mid-turn, the mailbox parks on its existing journal / turn-end edge. The send's own admission (beforeRun) re-checks this, because a turn can start (or a queued user send can run first) between the check and the send. A conversation the idle sweep closed counts as not working, so it is woken.
  • Terminal lane: if the agent starts working inside the 500 ms Enter delay, a held-batch reminder's Enter waits on the existing deferred-flight path for the next idle edge (a mid-turn Enter is taken into the running turn, the same hazard). Ordinary pointers are unchanged.
  • No auto-ack, no new timer or poller, no schema change. One new read (getOutstandingMailboxDelivery, checked narrowing, no assertion) on the existing outstanding_deliveries view.

Tests:

  • mailbox-pointer-eligibility.test.ts (real DB): held rows stay read: 0, delivered_at null and are not selected; rows outside are; ack then reselect names the next held batch.
  • structured-chat-coordinator-mail.test.ts: a turn that starts while the nudge reads the chat gets no steer; the nudge arrives at its end. Fails without the admission re-check.
  • structured-chat-coordinator-mail.test.ts: real runtime, RPC and chat host with a fake provider. Coordinator runs check without --ack, mail arrives mid-turn: no steer, one wake at turn end naming the batch; a further turn end with nothing new stays quiet; newer mail while idle wakes it once more; --ack then returns the new mail. Fails on main (no wake).
  • orchestration-mailbox-notification-consistency.test.ts: the terminal lane equivalent (previously asserted the bug: zero pointers), including a turn that starts inside the Enter delay. Fails on main.
  • orchestration-dispatch-mailbox-delivery.spec.ts: worker (Dispatch) mailbox, terminal lane, across a restart: the concurrent follow-up now gets exactly one pointer naming the held batch.
  • tests/e2e/orchestration-idle-mail-delivery.spec.ts (Electron, real PTY, fake agent): a terminal coordinator holding an unacked Run delivery gets exactly one pointer naming it; a turn that starts inside the Enter delay holds the Enter until the next idle; a later idle edge stays quiet; --ack returns the newer mail. Whole spec: 13/13.
  • structured-mailbox-pointer-delivery.test.ts: worker Dispatch mailbox, chat lane scheduling gate: parked while working, sent on the next journal edge. This is the STA-11631 case (chat worker holding an unacked batch).

Real app before / after

Isolated headless Orca built from each tree (ORCA_BACKGROUND_LAUNCH=1, temp ORCA_DEV_USER_DATA_PATH and HOME), a real native-chat Claude coordinator on --model haiku that created and bound the Run. Branch column is from head 4e820ef; main from 843e502. Native chat script: mail 1 arrives, the coordinator runs check --run without --ack, then stays mid-turn (waiting on a file); mail 2 arrives mid-turn; the turn ends; observe for 4 minutes.

origin/main 843e502 this branch
Mail 2 arrives mid-turn not pointed not pointed (held until turn end)
Turn ends nothing one turn: "You have 1 new orchestration message after delivery delivery_949400d6fdf8, which you have not acknowledged. If you no longer have it, run orca-dev orchestration check --run run_61cf291ab98e to see it again; once it is handled, run orca-dev orchestration check --run run_61cf291ab98e --ack delivery_949400d6fdf8."
Agent follows it n/a replayed the batch, ran check ... --ack delivery_949400d6fdf8, got mail 2, acked it, replied "received"
Mail 2 delivered_at after 4 min NULL stamped at the wake
Further wakes in the next 4 min (agent now holds the new batch, nothing newer) none none

Real Claude Code terminal coordinator (claude --model haiku in an Orca terminal, temp HOME/CLAUDE_CONFIG_DIR), same script:

Terminal coordinator origin/main 843e502 this branch (4e820ef)
Mail 2 arrives mid-turn not pointed not pointed (terminal lane only types at idle)
Agent goes idle nothing one pointer: "You have 1 new orchestration message after delivery delivery_a774bdad1b0f, which you have not acknowledged. If you no longer have it, run orca-dev orchestration check --run run_a61671e3af58 to see it again; once it is handled, run ... --ack delivery_a774bdad1b0f."
Agent follows it n/a replayed the batch, acked it, got mail 2, acked that, replied "received"
Mail 2 delivered_at after 4 min NULL stamped at the pointer
Further pointers in the next 4 min none none

An earlier run where mail 2 arrived while the coordinator was already idle and holding an unacked batch showed the same split: main never woke it (delivered_at NULL after 8 min), the branch woke it once and stayed quiet for the next 6 minutes while the agent still did not ack.

Platforms, remote, mobile

  • Platform-neutral main-process logic; nothing OS-specific.
  • Remote / mixed versions: the change is entirely inside the runtime that owns the mailbox. CLI and wire contracts are unchanged; an older CLI's check --ack works as before. No migration.
  • Mobile: no UI change. A phone mirroring a chat sees the nudge as the usual mail-notice turn.
  • Folder workspaces: not involved (mailboxes are per Run / Dispatch).

Risk

  • An agent that holds a batch now gets one extra nudge per batch of newer mail it has not seen. That is the intended change; it never repeats for the same mail.
  • The terminal staged-pointer release is now narrower (only when a check took the pointer's own mail). A pointer staged for mail that a later check did not include still submits, which is correct: that mail is still unseen.

Known gaps

  • If an agent holds an unacked batch and no newer mail arrives, nothing reminds it (the held batch itself is never re-pointed). That recovery wake is out of scope here.
  • Terminal lane: if Orca restarts while a held-batch reminder's Enter is deferred (the agent started working inside the Enter delay), that one wake is lost. The mail is not: it stays unread and check still returns it. Fixing it would need a durable "Enter deferred" pointer phase.

Supersedes #29006.

Fixes STA-11980
Fixes STA-11631

…lds an unacknowledged batch

An unacknowledged check batch used to silence every later nudge for that
mailbox, so a coordinator or worker that ran check without --ack was never
woken again. Both pointer lanes now defer instead: mail the held batch does
not contain gets one nudge at the idle or turn-end edge, naming the batch
and the exact ack command.
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 56e61ba5-168f-4d40-a95e-68a7bef58ae4
Commits

Reviewing files that changed from the base of the PR and between 4e820ef and 318abe7.

Files selected for processing: 1
  • tests/e2e/orchestration-idle-mail-delivery.spec.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

Walkthrough

Mailbox pointer selection now excludes messages in an outstanding delivery and returns that delivery’s ID with selected messages. Pointer notices can include instructions to acknowledge the held delivery. Structured mailbox delivery accounts for whether the session is working. Tests cover delivery timing, pointer content, and acknowledgment behavior.

Priority: Normal

Merge Risk: Minimal · up to 318ab

No actionable issue is established for this change; it is mergeable after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 19 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title names the main change: an agent is woken for new orchestration mail while it still holds an unacknowledged batch. The changeset matches this. The title uses the conventional fix(scope) form …
Description check Passed The description follows the repository template closely. It has the ELI5, root cause, what changed, testing, platform and risk notes, and a before/after table with real-app evidence. It links two Line…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: an agent is woken for new orchestration mail while it still holds an unacknowledged batch. The changeset matches this. The title uses the conventional fix(scope) form and is under 100 characters, though it is slightly long.

Full details: Description check

Explanation

The description follows the repository template closely. It has the ELI5, root cause, what changed, testing, platform and risk notes, and a before/after table with real-app evidence. It links two Linear tickets (STA-11980, STA-11631) under the Linked Issue area, but the template asks for a GitHub issue reference (Fixes #), which is missing. The Visual Proof section gives tables instead of attached screenshots or videos, the Testing checkboxes are not ticked, the Review and Agent skill upstream boundary sections are blank or unchecked, and the AI Disclosure section is empty.

Resolution

Optional improvements: replace the Linear references with a GitHub issue link (Fixes #), or confirm the linked issue meets the outside-contributor rule. Tick the Testing and Checklist items that apply. State the Agent skill upstream boundary status. Complete the AI Disclosure section if an AI tool was used. Attach a video or screenshots for the terminal and chat behavior change, or explain why the tables suffice.

Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…let the agent replay the batch first

- The nudge now says how to see the held delivery again before acking it,
  so an agent that lost it to /clear or compaction never acks unseen mail.
- Chat lane: the send's own admission holds the nudge while a turn runs,
  closing the window where a turn starts during the journal read.
- Terminal lane: a held-batch reminder's Enter waits for the next idle edge
  when the agent starts working inside the Enter delay.
- Checked narrowing replaces the row assertion; tests move the exclusion
  check onto a real database and drop the real-time sleep.
…rminal pane

A terminal coordinator holding an unacked Run delivery gets exactly one
pointer naming it for newer mail; its Enter waits for the next idle edge
when the agent starts working inside the Enter delay; a later idle edge
stays quiet; acking returns the newer mail.
@Jinwoo-H
Jinwoo-H merged commit 47ed6e1 into main Oct 25, 2026
46 checks passed
DoozyX added a commit to DoozyX/orca that referenced this pull request Oct 26, 2026
Merge stablyai/orca main b759a95 into fork main 83fb74d.
66 conflicts, mostly upstream's repo-wide comment audit (stablyai#33249): fork code kept with upstream comments.
Code-level combinations: mailbox pointer delivery (stablyai#33639) with the fork's queued/injected outcome;
held-batch lookup spans every outstanding typed delivery (fork schema v44); launched-agent write guard
keeps the fork settle loop plus upstream's write-if-shell-provable policy; worker-start keeps the fork's
refusal to guess the coordinator (upstream test adapted); monitored-work rows stay out of Claude agent view.
Files over max-lines are split without behavior change (group settings dialog gets its own hook).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant