Skip to content

fix(cli,relay): stop reading an unsignalable pid as a dead one (+ unverifiable-collapse sweep result) - #20098

Merged
nwparker merged 3 commits into
mainfrom
nwparker/unverifiable-collapse-pid-probes
Sep 15, 2026
Merged

nwparker merged 3 commits into
mainfrom
nwparker/unverifiable-collapse-pid-probes

Conversation

@nwparker

@nwparker nwparker commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 2 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​84 $\color{#cf222e}{\Huge{\mathbf{−}}}$​1 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​83
Prod 2 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​8 $\color{#cf222e}{\Huge{\mathbf{−}}}$​6 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​2

ELI5

Asking "is this process still alive?" can get three answers: yes, no, and I'm not allowed to ask. Two places in the codebase filed the third answer under "no" and acted on it. This fixes both.

More importantly: this PR is the write-up of a sweep for that whole defect class, and the sweep came back mostly clean. That negative result is the main deliverable.

The sweep result (read this first)

I swept for the class behind PRs #20085 / #20095 / #20096: a value meaning "we could not determine this" being read as "this is gone / dead / absent." Those PRs fixed eleven sites of one narrow version (entry.status nulled on a non-verified probe). The open question was how much bigger the class is.

Answer: not much. Across seven search dimensions, almost everything I checked was already hardened — frequently with the rule stated verbatim in a comment next to the code. The eleven earlier sites were a cluster in one subsystem, not a systemic failure. Nobody could say that before; now it's on the record so the next person doesn't redo it.

Dimensions searched

Dimension Query shape Outcome
Catch → definite value rg "\.catch\(\(\) => " across src/ ~60 hits, triaged by sink; all conservative
Catch block → definite negative multiline rg -U "catch[^\n]*\{\n\s*return (false|null|\[\])" over src/, src/cli, mobile/src all conservative except the two fixed here
Timeout → definite value every withTimeout / withTimeoutResult / Promise.race call site enumerated with its fallback argument all conservative
Tri-state collapse !== 'running'|'live'|'connected'|'ready'|'present'|'found'|'verified'|'own'; Promise<boolean | null>; | 'unknown' type decls no collapse found
Nullish fallback near destruction ?? false / ?? [] / ?? null filtered against kill/remove/delete/purge/prune/reap/retire/evict noise only
process.kill(pid, 0) liveness probes every call site in src/, mobile/, scripts/ read with surrounding context 2 defects — both fixed below
Destructive sinks traced backwards recursive rm, killpg, branch -d/-D, socket unlink, unsubscribe guards correct

Explicitly ruled out

Each of these reads like the defect and is not:

  • worktree-removal-safety.ts / worktree-orphan-gitdir-proof.ts — every catch returns false (refuse to delete). .catch(() => null) on the stat fails isDirectoryStat and refuses. isMissingPathError requires ENOENT/ENOTDIR; EACCES refuses.
  • Branch deletion after worktree removal — branchHasNoUnmergedChangesOnAnyTarget returns false on every probe failure, and an empty targetRefs preserves rather than force-deletes.
  • Workspace cleanup — a failed git read becomes blockers: ['git-status-error'] with clean: null; a null unpushed count becomes 'unknown-base'. A terminal-liveness-unknown blocker and an unverified-removal-consent suite already exist.
  • registered-worktree-roots-cache.ts — isRootGoneFromDisk times out to false ("not gone"), and the prune's own doc comment reads "Nothing is retired for want of evidence."
  • daemon-endpoint-probe.ts — four states, with endpointIsProvenDead as the only licence to replace an endpoint.
  • classifyWindowsTreeKillTarget — own/absent/foreign/unknown, and the caller does .catch(() => 'unknown').
  • Cloud session — OrcaCloudSessionReadResult carries a distinct unreadable alongside decrypt-failed precisely so a locked file cannot license a replace. readFreshOrcaCloudSession does fold unreadable into reconnect-required for the toast, but does not call clearCloudSessionIfUnchanged on that path — cosmetic, not destructive, left alone.
  • withTimeout call sites — every fallback points the conservative way (ssh-connection.ts:1346 falls back to true = "may restrict"; orca-runtime-terminal-create-deduplication.ts:75 throws runtime_unavailable rather than risk a duplicate shell).
  • Every other kill(pid, 0) probe — pty-shell-utils, pack-refs-lock-ownership, codex-app-server-process-teardown, codex-app-server-posix-supervisor, process-tree-termination, runtime-metadata-ownership-watch, runtime-rpc-socket-metadata, agent-session-process-identity-probe, worktree-create-preparation-stale-cleanup, orcad-instance-lock, ensure-virtual-display, grok-hook-owners, configure-process — all already ESRCH-only. After this PR that dimension is exhaustively clean.

What Changed

Two commits, both one-liners in the guard plus tests.

1. orca status calls a live Orca dead — reachable, reporting-level, not destructive

src/cli/runtime/status.ts — when status.get is unreachable, the fallback pid probe used a bare catch { return false }.

kill(pid, 0) refusing with EPERM means the pid belongs to another uid. That is not evidence the process is gone; it is "I'm not allowed to ask." The bare catch converted that into "it's dead", so a live app was reported:

app.running: false,  app.pid: null,
runtime.state: 'stale_bootstrap',  graph.state: 'not_running'

stale_bootstrap is not a loose synonym here — single-instance-lock.ts:29 and runtime-metadata-ownership-watch.ts:8 both define it as "metadata points at a dead pid." This published unverifiable as exited.

Reachable: point the CLI at an Orca running under another account (ORCA_USER_DATA_PATH, or one started with sudo) during a slow status.get. orca status reports that Orca died. An agent reading that reasonably concludes the runtime is gone.

Severity — stated plainly so a reviewer doesn't assume worse: this is reporting-level, not destructive. runtime.reachable is already false on both branches, and every downstream consumer keys off reachable, not app.running — handlers/core.ts:124, agent-hooks.ts:149, terminal-send.ts:33. Nothing is torn down. It under-reports; it does not kill anything.

2. pty.revive retires a pane on an unresolvable refusal — latent

src/relay/pty-handler.ts gated each serialized pane on a hand-rolled process.kill(entry.pid, 0) in a bare try/catch, so any refusal continued and retired the pane.

This is the same shape the three previous PRs fixed, and it's the argument for the shared helper they introduced: the file already imports isProcessAlive (line 14) and already calls it correctly at lines 1790 and 2464 — where reapPtyProvenExited spends nine lines documenting the ESRCH-only rule and cites docs/reference/ssh-execution-boundary.md. The right answer was nine lines away, with the rule written down, and this one path didn't call it. That is precisely the cost of one concept having more than one implementation.

The fix reuses the existing helper. No new predicate.

Latent — I could not construct a caller and am not claiming one. provider.revive(...) has no caller in shipped code: ssh-pty-provider.ts:70 and daemon-pty-router.ts:200 only forward it, and nothing invokes either. It is contract surface exercised by tests. Justified on reuse grounds, not on impact.

Why

docs/reference/ssh-execution-boundary.md fixes the vocabulary at live / unverifiable / exited, with no synonyms, and states that loss of contact is never evidence of process death. exited requires positive evidence of absence from the host that owns the process. Only ESRCH carries that; EPERM carries the opposite — the process exists.

Linked Issue

N/A — swept as a defect class following #20085 / #20095 / #20096 rather than from a filed issue. No issue was opened for this.

Visual Proof

N/A — no UI surface. The CLI change alters two fields of orca status JSON under a fault condition; the relay change alters an RPC guard. Both are covered by assertions below.

Testing

macOS. Cross-platform reasoning: the ESRCH/EPERM distinction is POSIX; on Windows libuv maps an OpenProcess refusal to UV_EPERM, so the fix is correct there too and strictly widens what counts as alive. The CLI suite's Unix-socket fixture is already describe.skipIf(win32), and my new block follows it.

Mutation table — each assertion was verified to fail when, and only when, the specific regression it claims to catch is reintroduced:

Test Mutation Result
CLI: unsignalable app stays running revert to catch { return false } fails — got {running: false, pid: null}, expected {running: true, pid: 424242}
CLI: ESRCH still reports stale bootstrap catch { return true } fails — 1 failed / 10 passed
relay: EPERM pane survives revert to bare catch { continue } fails — spawn called 0 times, expected 1
relay: ESRCH pane is dropped if (false) { continue } fails — spawn called 2 times, expected 1

Gates, each proven capable of failing before being believed:

  • vitest run --config config/vitest.config.ts — 11/11 src/cli/runtime/status.test.ts, 20/20 src/relay/pty-handler-revive.test.ts, and 172 files / 1695 tests across all of src/relay/.

  • oxlint — exit 0 (it prints nothing on success, so the exit code is the only signal).

  • tsc -p config/tsconfig.tc.cli.json and tsc -p config/tsconfig.node.json — empty output. Confirmed live by injecting a type error into status.ts (7 errors, exit 1).

  • Note for anyone following: config/tsconfig.tc.node.json does not exist — that path yields TS5058, which reads like "1 error". The real node target is config/tsconfig.node.json.

  • I manually tested these changes locally

  • Automated tests added/updated, or explained why not below

AI Disclosure

Internal contributor.

Review

Two things worth a reviewer's attention:

  1. The relay guard keeps today's "malformed pid → skip" behaviour (!Number.isInteger(entry.pid) || entry.pid <= 0). I deliberately did not widen it to the !managed.pty.pid || convention used at lines 1790/2464, which treats a missing pid as alive. Consistency would argue for widening; minimality argues against changing behaviour on untrusted client state in a fix aimed at one thing. Happy to go either way.
  2. The cloud-session unreadable → reconnect-required fold described under "ruled out" is a real but cosmetic instance of this class. I left it. Say the word if it should be in scope.

Agent skill upstream boundary

  • Not applicable.

Notes

  • Security — strictly narrows when a process is declared dead. Nothing gains a licence to kill or delete; two paths lose one.
  • Cross-platform — covered under Testing. No path, shell, or accelerator surface touched.
  • Remote SSH — the relay change is on the SSH host's side of the boundary and makes it more conservative about retiring a remote pane. Directly in service of ssh-execution-boundary.md.
  • Mobile — mobile/src was swept (no findings) and is unmodified.
  • Backwards compatibility — no wire change, no RPC params, no stream opcodes, no persisted shape. pty.revive's request and response are unchanged; only the host-local decision inside it changed.
  • Performance — one Number.isInteger per serialized entry on a path that already made a syscall.

Checklist

  • This PR is small and focused
  • I explained what changed and why (including ELI5)
  • Before/after screenshots or videos attached for UI changes, or N/A with reason
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote, and path/shortcut impact considered
  • Lint, typecheck and tests pass locally (scoped; see Testing)

Rebased onto main @ 231e805b1e1e49c765e96b9409f88e09d117346a (2026-09-15).

  • Old head: 959616039443e160d389887231dc9fab0ea26ec4
  • New head: d6ddf2065c05c54132c5dc11f20b0bf36ea190f7

Content unchanged apart from one added commit, fix(lint): clear the casting gate on the pid-probe changes, which main's newly tightened typescript/consistent-type-assertions requires. Verified with git range-diff; prior CI results predate this rebase.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 9d890538-c053-49f0-906e-d79819a9ad89

📥 Commits

Reviewing files that changed from the base of the PR and between c67a747 and d6ddf20.

📒 Files selected for processing (2)
  • src/cli/runtime/status.ts
  • src/relay/pty-handler-revive.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/relay/pty-handler-revive.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The change updates PID liveness checks in CLI status and PTY revival. ESRCH indicates that a process is absent. Other probe errors, including EPERM, indicate that the process remains live but cannot be probed normally. PTY revival also requires a positive integer PID. New tests cover both error outcomes for CLI status and pane revival.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to d6ddf

The change preserves inaccessible live processes and removes only confirmed-absent ones, with focused coverage for both outcomes.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly describes the main change: correcting PID liveness handling in the CLI and relay paths. It is specific and related to the changeset.
Description check ✅ Passed The description is detailed and covers the required explanation, changes, rationale, testing, platform considerations, review notes, and checklist. It does not provide a linked issue, although the tem…
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • CLI status pid fallback — isProcessRunning now returns (error as NodeJS.ErrnoException)?.code !== 'ESRCH', so a pid that only refuses the probe (EPERM, exists under another uid) reports running: true / state: 'starting' rather than stale_bootstrap. Reporting-level only: consumers key off runtime.reachable, which stays false on both branches.
  • Relay pty.revive gate — the hand-rolled process.kill(pid, 0) try/catch is replaced with the already-imported shared isProcessAlive, reusing the ESRCH-only rule; the malformed-pid skip (!Number.isInteger(entry.pid) || entry.pid <= 0) is preserved.
  • Tests — two CLI cases (EPERM → alive, ESRCH → stale) and one relay case (EPERM pane survives, ESRCH pane drops), each verified to fail under the specific regression it claims to pin.

Both points you flagged for review are sound as-is:

  1. Relay guard minimality — keeping !Number.isInteger(entry.pid) || entry.pid <= 0 (skip) rather than widening to the !managed.pty.pid || convention is correct. The two conventions guard opposite directions: reapPtyProvenExited (pty-handler.ts:2464) is destructive, so a missing pid must default to "alive" to avoid retiring something on no evidence; revive re-spawns a shell, so a serialized entry with no usable pid is untrusted client state and skipping it avoids an orphan/duplicate. Both fail safe in their own direction.
  2. Cloud-session unreadable → reconnect-required fold — leaving it out of scope is right; it's toast-level and gates no destructive action.

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) | 𝕏

`orca status` falls back to a `kill(pid, 0)` probe when `status.get` cannot be
reached, and a bare catch read every refusal as absence. EPERM means the pid
exists under another uid -- an Orca reached via ORCA_USER_DATA_PATH, or one
started with sudo -- so a live app was reported `running: false`, `pid: null`,
`runtime.state: stale_bootstrap`, `graph.state: not_running`.

Only ESRCH proves the pid is gone, which is the rule every other liveness probe
in the repo already applies (`isProcessAlive` in relay/pty-shell-utils.ts,
pack-refs-lock-ownership.ts, runtime-metadata-ownership-watch.ts, and
agent-session-process-identity-probe.ts). See
docs/reference/ssh-execution-boundary.md.
…robe

`revive` gated each serialized pane on a hand-rolled `process.kill(pid, 0)` in a
bare try/catch, so any refusal retired the pane. EPERM means the process exists
under another uid; only ESRCH is evidence of absence.

The file already imports `isProcessAlive`, whose ESRCH-only contract
`reapPtyProvenExited` documents 450 lines earlier -- this call site just did not
use it. Reuse it rather than keeping a second implementation of the same
concept. Malformed pids still skip, as before.

See docs/reference/ssh-execution-boundary.md.
@nwparker
nwparker force-pushed the nwparker/unverifiable-collapse-pid-probes branch from 9596160 to c67a747 Compare September 15, 2026 09:35
main tightened typescript/consistent-type-assertions to assertionStyle:
never, which the rebase brings onto these added lines. The CLI probe
narrows instead of casting; the relay test keeps the file's serialize
idiom behind a SAFETY-annotated suppression.
@nwparker
nwparker merged commit 981a482 into main Sep 15, 2026
39 checks passed
@nwparker
nwparker deleted the nwparker/unverifiable-collapse-pid-probes branch September 15, 2026 23:01
ivanvykopal pushed a commit to ivanvykopal/orca that referenced this pull request Sep 20, 2026
…erifiable-collapse sweep result) (stablyai#20098)

* fix(cli): stop reporting an unsignalable Orca pid as a stale bootstrap

`orca status` falls back to a `kill(pid, 0)` probe when `status.get` cannot be
reached, and a bare catch read every refusal as absence. EPERM means the pid
exists under another uid -- an Orca reached via ORCA_USER_DATA_PATH, or one
started with sudo -- so a live app was reported `running: false`, `pid: null`,
`runtime.state: stale_bootstrap`, `graph.state: not_running`.

Only ESRCH proves the pid is gone, which is the rule every other liveness probe
in the repo already applies (`isProcessAlive` in relay/pty-shell-utils.ts,
pack-refs-lock-ownership.ts, runtime-metadata-ownership-watch.ts, and
agent-session-process-identity-probe.ts). See
docs/reference/ssh-execution-boundary.md.

* fix(relay): keep a revived pane whose pid only refuses the liveness probe

`revive` gated each serialized pane on a hand-rolled `process.kill(pid, 0)` in a
bare try/catch, so any refusal retired the pane. EPERM means the process exists
under another uid; only ESRCH is evidence of absence.

The file already imports `isProcessAlive`, whose ESRCH-only contract
`reapPtyProvenExited` documents 450 lines earlier -- this call site just did not
use it. Reuse it rather than keeping a second implementation of the same
concept. Malformed pids still skip, as before.

See docs/reference/ssh-execution-boundary.md.

* fix(lint): clear the casting gate on the pid-probe changes

main tightened typescript/consistent-type-assertions to assertionStyle:
never, which the rebase brings onto these added lines. The CLI probe
narrows instead of casting; the relay test keeps the file's serialize
idiom behind a SAFETY-annotated suppression.
Jinwoo-H added a commit that referenced this pull request Sep 23, 2026
…22341)

* fix(cli): report a denied runtime connection instead of a dead Orca

Inside Codex's macOS Seatbelt sandbox, connect() on the runtime socket fails
with EPERM. The CLI dropped the errno and reported "Could not connect ...
Restart Orca", appended "Orca is not running. Run 'orca open' first.", and
`orca status` answered ok:true with `starting` (its pid probe also gets EPERM).
An agent following that advice restarts a healthy app, which cannot help.

EPERM/EACCES on the metadata read or the socket/pipe connect now fails with a
CLI-local `runtime_access_denied` error: ok:false, non-zero exit,
operation/systemCode/processState:"unverifiable"/retryable:false and nextSteps
that say to re-run with escalated permissions and not restart. CODEX_SANDBOX
only picks the wording. `orca open` stops before launching.

The status pid probe is unchanged: a refused or missing socket proves the
caller reached the endpoint, so a later EPERM probe is another uid and keeps
#20098's `starting`. Missing, refused, stale-pid and timeout paths are
unchanged.

Adapted from the diagnosis and tests in #20487 (and #19605, #13583).

Co-authored-by: lifeodyssey <zhenjiazhou0127@outlook.com>

* docs(skills): tell agents runtime_access_denied means escalate, not restart

The shared CLI-resolution block told every bundled skill to run `orca open`
when a command says Orca is not running. Add the counterpart for the new
access-denied code so sandboxed agents re-run with escalated permissions
instead of launching or restarting Orca. Regenerated stubs and manifest.

* refactor(cli): classify only a denied runtime connect, with a leaner error

A denied metadata read was never observed under a sandbox, and it turned an
unreadable user-data path (the Linux launch contract's root-owned HOME) into
runtime_access_denied instead of "Orca is not running". Keep metadata reads as
on main and classify only the socket/pipe connect.

One helper now maps a socket errno to the error or null; the error data keeps
only systemCode and nextSteps. Tests drop cases already pinned by status.test.ts.

* fix(cli): give not-running advice when a denied socket belongs to a dead Orca

A crashed Orca leaves its metadata and socket file behind, and a sandbox denies
the connect with EPERM before the CLI can see ECONNREFUSED. The sandbox still
reports ESRCH for a gone pid, so a denied connect now probes the metadata pid
and falls through to the ordinary unavailable path when the pid is proven gone.
isProcessRunning moves to its own module so transport and status share it.

* refactor(cli): inline the runtime_access_denied code like other CLI error codes

---------

Co-authored-by: lifeodyssey <zhenjiazhou0127@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant