Repository navigation
fix(orchestration): accept worker reports without the dispatch capability - #23982
Conversation
…ability Worker lifecycle reports and questions no longer depend on the per-dispatch capability token that lives only in the agent's conversation. The host now: - ignores capability_hash/capability_revoked_at for authorization on every row and checks the exact worker process instead (ask gains that check); - refuses a report whose calling terminal is provably another orchestration party (a Run coordinator or another Dispatch's worker), treating env that names no live pane here as absent; - applies one worker-state rule locally and remotely: a stop in flight refuses, while stop_unknown and start_unknown accept and settle. Minting and printing the flag are unchanged, so an older host and older preambles keep working.
|
| if ( | ||
| !callerHandle || | ||
| !callerPaneKey || | ||
| (args.fromPaneKey && isEquivalentPaneKey(callerPaneKey, args.fromPaneKey)) | ||
| ) { | ||
| return |
There was a problem hiding this comment.
Coordinator can impersonate worker reports
A coordinator can run orchestration send --from <worker> --type worker_done with its pane environment unset or stale. This fence then lets the request through because it cannot identify the coordinator’s pane. The later checks look up the pane and process for the declared worker handle, not the calling process, so the coordinator’s report can settle the worker’s task without the worker reporting.
How this was verified: The CLI accepts an explicit --from without pane environment, and the host checks lifecycle authority using that declared handle after the fence returns.
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review. 📝 WalkthroughWalkthroughMessaging routes replace dispatch-capability authorization with caller fencing and worker-state and process-identity checks. Local and remote settlement paths accept applicable unknown worker states. Successful local reports can return unproven workers to ready. Priority: ➖ Normal Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to The updated tests align with process-based worker authority and settlement after unproven stops. No merge-blocking issue is established; normal checks remain appropriate. Security Architecture ReviewSecurity architecture risk: 🟡 Moderate · up to The replacement checks can accept a report in another worker’s name when caller identity is missing or stale. This threatens task-result integrity and can carry forged reports to a connected run home. The demonstrated scope requires access to the runtime and knowledge of a live worker handle; unauthenticated access or broader privilege escalation was not established. Retained concerns
Security review detailsSecurity Blast Radius
Security Findings and Attack Paths
Trust Boundaries and Controls
Resilience and Maintainability Implications
Hardening Proposals
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
Full details: Linked Issues checkExplanation The PR implements first-hand settlement from Resolution Retain local capability authority when a stop becomes unproven and the dispatch remains ✨ Finishing Touches 💡 2📝 Generate docstrings 💡
🛠️ Fix failing CI checks 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — initial review of the orchestration authorization refactor at 4dcc047.
- Per-dispatch token check removed —
verifyDispatchCapabilityandverifyRemoteAttachmentAuthorityare deleted;send-point-to-point,ask-methods,send-remote, andask-remotenow authorize every row (including pre-upgrade rows) from the exact-process check alone.mintDispatchCapability/revokeDispatchCapability/capability_hash/capability_revoked_atare still written for the downgrade path. - Caller-pane fence added (
lifecycle-caller-fence.ts) — refusesworker_done/heartbeat/escalation/decision_gate/askwithconsumer_fencedonly when the caller's env resolves to a live pane that is a different orchestration party (another Run coordinator or another Dispatch's assignee). Absent/scrubbed/foreign/stale env passes; session callers are skipped. - One worker-state rule (
worker-report-admission.ts) —stoppingrefuses;stop_unknown/start_unknownaccept as unverifiable;askgains the process check thatsendalready had. stop_unknownsettlement —settleWorkerReportreconnectsstop_unknownlikestart_unknown, with a newworker stop_unknown → readylifecycle edge andlast_error: nullon the reconnect.- Federation relay settle broadened —
SETTLEABLE_REMOTE_ATTACHMENT_STATES(ready/start_unknown/stop_unknown) with aWHERE dispatch_id = ? AND state = ?optimistic guard.
I read the complete diff end to end, verified the callers of the removed/added symbols, and confirmed the three touched suites (worker-report-authority.test.ts 14, send.test.ts 34, ask.test.ts 18) pass. Two points I specifically probed and cleared: (1) the fence does not false-positive on a stale --from handle after a pane remint, because the runtime still resolves the retained pre-allocated handle to the current pane key; (2) no stop_unknown/start_unknown combination is left permanently unsettled in the current lifecycle. The residual same-user env-scrub bypass and the still-printed --dispatch-capability flag are both called out in the PR body as known/accepted or deferred, and the docs still describing the token belong to that deferred follow-up.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
…nt party - One module owns the unproven/settleable worker states and the refusal rule; local send records it, ask and remote throw it. A stale process is worker_identity_changed on every path. - The caller fence passes the worker's own terminal when its --from handle went stale. - Document the shared-tmux-server limit; drop the dead dispatch_capability_invalid rejection member; tests assert dispatch state, not the capability column.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/main/runtime/rpc/methods/orchestration/messaging/worker-report-authority.test.ts (1)
124-133: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRename the test to match its assertion.
The test title says the worker's own terminal is "not fenced". The test then expects
worker_identity_changedrejection. The caller fence passes, but the process check rejects the report becauseterm_workerno longer resolves. Rename the test to state both results. A future reader could otherwise read the rejection as a regression.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: de3959cc-4450-415b-8815-04711edcc7fc
📒 Files selected for processing (22)
skill-guides/orchestration/references/worker-contract.mdsrc/cli/bundled-skill-guides.tssrc/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.tssrc/main/runtime/orchestration/db-task-dispatch-invariant.test.tssrc/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.tssrc/main/runtime/orchestration/db-task-dispatch-races.test.tssrc/main/runtime/orchestration/db/dispatch-context/worker-report-settlement.tssrc/main/runtime/orchestration/db/federation/federation-relay-enqueue.tssrc/main/runtime/orchestration/db/federation/federation-relay-item.tssrc/main/runtime/orchestration/lifecycle-reconciliation.test.tssrc/main/runtime/orchestration/lifecycle-reconciliation.tssrc/main/runtime/orchestration/worker-report-admission.tssrc/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.tssrc/main/runtime/rpc/methods/orchestration/messaging/ask-methods.tssrc/main/runtime/rpc/methods/orchestration/messaging/ask-remote.tssrc/main/runtime/rpc/methods/orchestration/messaging/lifecycle-caller-fence.tssrc/main/runtime/rpc/methods/orchestration/messaging/send-point-to-point.tssrc/main/runtime/rpc/methods/orchestration/messaging/send-remote.tssrc/main/runtime/rpc/methods/orchestration/messaging/send.test.tssrc/main/runtime/rpc/methods/orchestration/messaging/worker-report-authority.test.tssrc/main/runtime/rpc/orchestration-runtime-update-settlement.test.tssrc/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts
💤 Files with no reviewable changes (6)
- src/main/runtime/orchestration/lifecycle-reconciliation.ts
- src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts
- src/main/runtime/orchestration/db-task-dispatch-races.test.ts
- src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts
- src/main/runtime/orchestration/lifecycle-reconciliation.test.ts
- src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — incremental delta since the prior pullfrog review at 4dcc047: 244d8a2 (fence message wording only) and abfa3ef (the consolidation below).
- One worker-state module —
worker-report-admission.tsmoved up toorchestration/, now exportingUNPROVEN_WORKER_STATES(start_unknown|stop_unknown),SETTLEABLE_WORKER_STATES(ready|start_unknown|stop_unknown),isWorkerStateIn, and a singleworkerReportRefusal/assertWorkerCanReport. The duplicateSETTLEABLE_REMOTE_ATTACHMENT_STATESis gone; federation settle andask/remote send now share the one rule. - Send-path authority collapsed into
lifecycleRefusal—resolveLifecycleAuthoritywas rewritten to returnLifecycleRefusal | nulland delegate the stop/process decision toworkerReportRefusal, dropping the last capability reads. Behavior is equivalent to244d8a2apart from the intendedworker_identity_changedcode/reason rename. - Fence covers the worker's own terminal — the caller fence early-returns when the caller's owned dispatch equals the one
--fromresolves to (newownedDispatchIdhelper), so a stale--fromhandle is settled by the process check (worker_identity_changed) instead of being refused asconsumer_fenced; only a different party is refused. stop_unknownsettles likestart_unknown— newworker stop_unknown → readylifecycle edge,settleWorkerReportreconnectsstop_unknownthroughready(last_error: null), and the relay settle accepts both unproven states with aWHERE ... state = ?optimistic guard.- Docs — the worker-contract reference (and generated bundle) now explains that
send/askreturningconsumer_fencedmeans the command ran from another party's terminal.
I read the full PR diff plus the incremental range-diff, verified the refactor is behavior-preserving against 244d8a2, confirmed no dangling references to the removed symbols, and ran the touched suites (7 files / 124 tests, including worker-report-authority 15, send 34, ask 18; plus 12 federation/settlement files / 62 tests) — all green. I traced the two disposition-changing questions with independent specialists: the fence's same-dispatch early return is closed for any recognized coordinator/other-worker party (the env-scrub bypass is pre-existing from 4dcc047 and already called out in the PR body, so not re-raised), and every reachable start_unknown/stop_unknown report now settles exactly once in the correct direction.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to GitHub limitations.
🟠 Major · Bind null-incarnation reports to an authenticated worker identity. · ask-methods.ts:78-82
src/main/runtime/rpc/methods/orchestration/messaging/ask-methods.ts:78-82
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy liftBind null-incarnation reports to an authenticated worker identity.
A terminal can call
orchestration.sendwith another worker’sfromhandle. When the dispatch has noprocess_incarnation, the current path treats that request as current. A forgedworker_donereport can therefore settle the worker’s task.orchestration.askhas the same bypass.Require an exact process match or another independently authenticated worker identity at both admission points. Do not use the request’s
fromhandle, pane key, or absent compatibility evidence as authority. Preserve unknown-start and unknown-stop recovery only when it supplies that independent identity; otherwise reject the report or question.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: daacb640-db2b-43db-aa6c-febb44102eec
📒 Files selected for processing (2)
src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.tssrc/main/runtime/rpc/methods/orchestration/messaging/send.test.ts
💤 Files with no reviewable changes (1)
- src/main/runtime/rpc/methods/orchestration/messaging/send.test.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

ELI5
When a coordinator hands a task to a worker agent, Orca pastes a secret token into the worker's prompt, and the worker has to repeat it on every "I'm done", heartbeat, escalation, and question. The token only lives in the agent's conversation. It disappears when the conversation is compacted, never shows up in a re-rendered
dispatch-show --preamble, and gets revoked by a stop that didn't actually stop anything. When that happens, the worker's "I'm done" is rejected forever and the task hangs.This PR stops the Orca host from checking that token. It still checks everything it checked before (the report comes from the exact process Orca started in the worker's pane), and adds one new check that keeps the one real protection the token gave: a coordinator or another worker can't report as this worker.
What Changed
Before → after, as a coordinator and its workers see it:
worker_done/askcopied fromdispatch-show --preamble(which never contains the token)--dispatch-capabilitysend --type worker_done --from <worker>from its own terminalconsumer_fenced("This terminal is a Run coordinator and cannot report as …"), nothing recordedconsumer_fencedworker-stopon a tab Orca didn't open (stop ends asstop_unknown), then the worker reportsblockedsucceeded/failedstopping)dispatch_inactive: "is stopping; its worker cannot report until worker-stop settles")worker_identity_changedfromsendandask, local and remote (localsendsaidsender_not_assigneebefore)askfrom a pane whose process changedasknow has the same process check assendMechanism:
send-point-to-point.ts,ask-methods.ts,send-remote.tsandask-remote.tsno longer read the token orcapability_hash/capability_revoked_at; the existing exact-process check (isDispatchProcessCurrentlocally,isRemoteAttachmentProcessCurrenton a connected server) decides for every row, including rows minted before the upgrade.verifyDispatchCapabilityandverifyRemoteAttachmentAuthorityhad no other callers and are deleted.lifecycle-caller-fence.ts). Every CLI request already carries the calling terminal'sORCA_PANE_KEY/ORCA_TERMINAL_HANDLE(since fix(orchestration): preserve active workers across updates #11271). Forworker_done,heartbeat,escalation,decision_gateandask, the host resolves that pane to a live terminal on this host. It refuses only when that terminal is a different pane than--fromand is a different orchestration party: a Run's coordinator or the worker of another active dispatch. The worker's own terminal passes even when its--fromhandle went stale. Anything that names no live pane here (no env, a stale handle after a remint, another Orca's pane, a scrubbed env) counts as absent, and a live pane that is not a party (an agent-teams teammate, the user's shell) passes. Accepted limit: a tmux server keeps the env of the pane that started it, so a worker inside a tmux server that was started from a live coordinator or worker pane is refused. Session callers are skipped because the session binding already fences them.assertCallerHandleMatchesEvidenceis unchanged, so coordinator verbs keep today's behavior.orchestration/worker-report-admission.ts, which also holds the settleable-state lists the settlement paths read).stoppingrefuses (dispatch_inactive); a replaced process refuses (worker_identity_changed);stop_unknownandstart_unknownaccept, because an unproven stop or start is "unverifiable", not "exited" (docs/reference/ssh-execution-boundary.md). Settlement follows:settleWorkerReporttreatsstop_unknownlikestart_unknown(the settlement half of fix(orchestration): keep a wrongly-stopped worker's dispatch capability #21713, with a newstop_unknown → readylifecycle edge), and a connected server's relay settle acceptsstart_unknownandstop_unknownas well asready.Unchanged on purpose: Orca still mints the token and prints the flag in new preambles (that is a follow-up PR),
worker-stopstill refuses to close tabs Orca doesn't own, andcapability_revoked_atis still written.Why
The token's only property is "the caller has read this conversation", which is exactly what breaks on compaction, truncation, re-rendering and secret scanners. The process check already covers a stale process. What the token additionally stopped in practice was a different agent, usually the coordinator, running the worker's command; the caller-pane fence keeps that without anything an agent has to remember.
Alternatives considered:
dispatch --inject,worker-start --terminal, retries) are created after the agent process exists, and a process's environment is fixed at spawn.ORCA_AGENT_LAUNCH_TOKEN) with hook attestation. Never rejects more than the plain pane check above, and returns nothing for hookless agents, agents before their first hook, reconnected SSH panes and structured workers. It adds a dependency, not protection.--fromalone. Lets the coordinator report as the worker, recorded as the worker's own report.What we accept (fail-open, by design):
ORCA_PANE_KEY/ORCA_TERMINAL_HANDLEis unset, stale, or names no live pane here is not fenced, and its report is then judged only by the process check on--from.Security notes: a same-user process that fakes its env can still report as a worker, but it could already read the token from the worker's terminal (terminal reads are not redacted) or type into that terminal. The SSH relay CLI channel is reachable only by the same remote user (credential file
0600, socket under the user's own relay directory), and connected servers don't callorchestration.sendon the Run home (the home pulls relay items), so the token was never a boundary there either.Mixed versions
consumer_fencedinstead of a recordeddispatch_capability_invalidrejection, and the error codedispatch_capability_invalidremains defined for old hosts.Linked Issue
Part of STA-8833 (item 1: the per-dispatch capability). Items 2 (Codex tab title) and 3 (unacknowledged-delivery warnings) are out of scope.
Fixes #21688, #21534, #20565, #16604, #15222
Refs #23656 (item 1 only), #17632 (the token stops being printed in a follow-up PR), #12769, #15124 (recovery through
dispatch-show --preamblenow works, but a truncated preamble still lacks the dispatch id), #16903, #13364 (a rejectedworker_donestill leaves the worker atready; see #15265).Visual Proof
N/A — orchestration authorization and state-machine change with no UI. The end-to-end CLI transcript below is the behavioral proof.
Testing
End to end, before and after (macOS, headless isolated runtime built from
mainata0abcb6818band from this branch; the worker is a fake Codex started in a tab, and every worker command runs inside the worker's own terminal with its real env; the coordinator's commands are typed into the coordinator's terminal):mainsend --type worker_done --from <worker>dispatch_capability_invalid(missing)consumer_fenced, task staysdispatchedaskcopied fromdispatch-show --preambledispatch_capability_invalid(missing)worker_donewithout the flagdispatch_capability_invalid, taskdispatchedcompletedworker_donecopied fromdispatch-show --preambledispatch_capability_invalidworker-stopon a user-opened tab, then the injectedworker_donestop_unknown; report → "capability is revoked", taskblockedstop_unknown; report →completed, workersucceededAutomated: new
worker-report-authority.test.ts(15 cases, including the worker's own terminal with a stale--fromhandle, which fails without the same-dispatch check): coordinator and other-worker panes are fenced for bothsendandaskwith nothing recorded; no env, own pane with a stale handle, a foreign pane, a stale handle, and a live non-party pane are all accepted; a stop in flight refuses reports and questions;succeededandfailedreports afterstop_unknownsettle; a stale process afterstop_unknownis refused; remotestop_unknownandstart_unknownreports settle, and a remote stop in flight refuses. The fence cases fail with the fence disabled and thestop_unknownsuccess case fails without the new lifecycle edge.ask.test.tsgains the process check; existing tests that used the token as an oracle now assert dispatch state.Commands run:
./node_modules/.bin/vitest run --config config/vitest.config.ts src/main/runtime src/main/ssh src/main/ipc src/cli src/shared— 28,609 passed; the 2 failures are fish-shell tests that depend on the local fish install and import nothing changed herepnpm run typecheck:tsc:node,pnpm run typecheck:tsc:cli,pnpm tc:node— cleanoxlinton changed files,pnpm run audit:anti-slop,pnpm run check:code-quality:changed— 0 findingsWindows ("windows 2", Git Bash, worktree at
3a5443552e, which includes this change):vitest runoversrc/main/runtime/orchestration,.../rpc/methods/orchestration/messaging,orchestration-task-list-failure-reasonandorchestration-mutation-recovery— 144 files pass.orchestration-mutation-recovery.test.tsfails 3 cases with Windows argument quoting ("orca-dev" "orchestration" …);main's copy of that file fails the same 3 cases on the same machine, because the test doesn't pinprocess.platform.Not tested: a live SSH/connected-server worker end to end. The remote path is covered by the DB/RPC tests above; the change has no platform-specific code.
Review
Agent skill upstream boundary
docs/reference/agent-skill-sharing-upstream-boundary.mdand copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.Notes
Security, cross-platform, SSH, mobile, compatibility and performance are covered above. The fence costs two in-memory lookups per lifecycle message, plus a few indexed queries only when the caller's env names a live pane other than
--from.Checklist
N/Awith reasonpnpm lint,pnpm typecheck,pnpm test, andpnpm buildpass (or CI will cover; local preferred)