Skip to content

fix(cli): report denied runtime access without claiming process exit - #20487

Closed
lifeodyssey wants to merge 2 commits into
stablyai:mainfrom
lifeodyssey:codex/fix-runtime-access-diagnostics
Closed

lifeodyssey wants to merge 2 commits into
stablyai:mainfrom
lifeodyssey:codex/fix-runtime-access-diagnostics

Conversation

@lifeodyssey

Copy link
Copy Markdown
Contributor

ELI5

Orca can be healthy while a sandbox prevents a CLI caller from connecting to it. Report that access failure instead of claiming the app stopped and telling the caller to restart it. orca open also stops before launching anything when its initial status check is denied.

What Changed

  • Preserve EPERM / EACCES from metadata reads, local socket connections, and PID probes as runtime_permission_denied.
  • Report unexpected PID-probe errors as runtime_unverifiable; retain the existing stale-PID result for ESRCH after an ordinary connection failure.
  • Return errors through the existing ok: false envelope with the operation, system code, and processState: unverifiable. A metadata PID is retained as an unverified diagnostic hint.
  • Add regression tests and a short diagnostics reference. No runtime-state enum or remote wire change.

Why

A macOS headless delivery workflow independently reproduced this on Orca 1.4.200: the same live runtime was reachable outside the Codex sandbox, while socket connection and signal-0 probing returned EPERM inside it. Historical worker calls also alternated between failure in the sandbox and successful lifecycle delivery with command-scoped host permission, using the same runtime ID.

This adapts the diagnosis and regression approach offered in #19605, whose author explicitly invited reuse and does not plan to maintain that branch. It uses distinct local error codes, as explored in #13583, so generic runtime_unavailable formatting/recovery does not reintroduce restart advice. Unlike #16697, denied access is not projected back to stale_bootstrap.

Linked Issue

Fixes #19604.

Addresses the diagnostics portion of #12528 and #13539. Their sandbox-connectivity work remains open. Downstream incident: lifeodyssey/animichi#1648.

Visual Proof

N/A — no rendered UI changes. CLI behavior was checked against a running packaged 1.4.200 runtime, without restarting it:

Caller Before After
macOS Codex workspace-write sandbox ok: true, stale_bootstrap, app.running: false exit 1, ok: false, runtime_permission_denied, systemCode: EPERM, processState: unverifiable
Authorized host command ready, reachable ready, reachable

Only sanitized results are included; raw logs, tokens, endpoint paths, and private session identifiers are not published.

Testing

  • I manually tested these changes locally: the rebuilt CLI returns the denied-access error inside the sandbox and reaches the same runtime/PID outside it.
  • Automated tests added/updated.
  • New regressions: 17 passed. Running those same tests with the three production files restored to the base revision produced 11 failures and 6 passes.
  • Complete src/cli suite: 1,218 passed, one existing 250 ms timeout-override test failed during concurrent validation. Rerunning its entire runtime-client.test.ts file passed all 15 tests; the initial failure is disclosed rather than counted as a clean full-suite run.
  • CLI typecheck, pnpm run check:code-quality:changed, and pnpm run build:cli passed.
  • Full repository pnpm lint and all node/web/CLI typecheck projects passed. Full repository tests and desktop/native packaging were not run locally and remain for CI; no visible Electron tests were launched on the user's desktop.

AI Disclosure

OpenAI Codex (GPT-6) assisted with diagnosis, implementation, test execution, and self-review under the contributor's direction. Prior art and adapted regression ideas are credited to #19605 and #13583.

Review

Self-review covered error precedence, the error-then-close socket sequence, no request write after a denied connection, no app launch after denied discovery, JSON/human output, secret redaction, and preservation of ordinary missing/refused/starting behavior. No independent review is claimed.

Agent skill upstream boundary

  • Not applicable: no agent skill sources or generated skill artifacts changed.

Notes

  • Cross-platform: pure TypeScript/Node error classification for Unix sockets and named pipes. Tests inject OS errors portably; actual sandbox denial was tested on macOS only. Windows/Linux enforcement remains for CI/platform validation.
  • Remote/SSH: no local probing of a remote PID, no endpoint fallback, no wire framing or protocol changes. Host-reported access-error envelopes retain their error code and do not acquire local restart advice.
  • Agents, integrations, git providers, folder workspaces, and mobile: no provider-specific policy, Git commands, workspace assumptions, or UI changes.
  • Security: no widened permissions, DACL changes, sandbox bypass, or new transport. Messages omit raw OS error text, endpoints, and auth tokens. PID hints are not process-ownership claims.
  • Performance: bounded classification on failure paths; no new polling, subprocesses, or background services.
  • This does not restore sandboxed heartbeat/worker_done delivery or assert exactly-once settlement. That requires an authorized lifecycle channel and remains separate.

Author

GitHub: @lifeodyssey. X/Twitter: not provided.

Checklist

  • This PR is small and focused.
  • I explained what changed and why (including ELI5).
  • Before/after proof included; no rendered UI change.
  • Self-reviewed for correctness, security, and performance.
  • Cross-platform, SSH/remote, and path/shortcut impact considered.
  • Full repository validation is not claimed; scoped results and the initial timing failure are listed above.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 19bd6c53-6b4a-4c90-b20b-ea1e446c9a7b

📥 Commits

Reviewing files that changed from the base of the PR and between fe4b53b and abdd1f4.

📒 Files selected for processing (1)
  • src/cli/runtime/runtime-access-diagnostics.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/cli/runtime/runtime-access-diagnostics.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

The CLI now reports distinct runtime access errors for permission-denied and unverifiable operations. Metadata reads, socket connections, and PID probes preserve these diagnostics instead of reporting runtime absence. Existing handling remains for missing metadata and refused connections. Tests cover structured errors, CLI output, status behavior, and launch prevention. Documentation describes the new error codes, fields, and access boundaries.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to abdd1

No unresolved merge-blocking behavior is identified in the supplied change context.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 6 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: reporting denied runtime access without falsely claiming that the process exited.
Description check ✅ Passed The description follows the required template and covers the change, rationale, linked issue, visual proof, testing, AI disclosure, review, scope boundaries, compatibility, security, performance, and …
Linked Issues check ✅ Passed Issue #19604 requires separate handling for permission denial, absent processes, and other probe failures. The implementation maps EPERM and EACCES from metadata and socket operations to `runtime_…
Out of Scope Changes check ✅ Passed The changed runtime code directly implements issue #19604. The added tests verify the access-error distinctions. The documentation describes the same diagnostic contract. No unrelated runtime-state en…
  • Fix all pre-merge checks with AI

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 6f89a31d-f875-4406-9247-3d9bcd455f01

📥 Commits

Reviewing files that changed from the base of the PR and between fe4237c and fe4b53b.

📒 Files selected for processing (7)
  • docs/reference/runtime-access-diagnostics.md
  • src/cli/runtime/metadata-access-diagnostics.test.ts
  • src/cli/runtime/metadata.ts
  • src/cli/runtime/runtime-access-diagnostics.test.ts
  • src/cli/runtime/runtime-access-error.ts
  • src/cli/runtime/status.ts
  • src/cli/runtime/transport.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues — one optional documentation nit below.

Reviewed changes — reviewed the full diff of PR #20487 (fe4b53b), which classifies local runtime access denials instead of reporting a successful stale_bootstrap / starting status.

  • Access classification — new RuntimeAccessError maps EPERM/EACCES to runtime_permission_denied (and other non-ESRCH PID-probe failures to runtime_unverifiable), carrying operation, systemCode, processState: unverifiable, and an unverified metadata PID hint.
  • Status path — getCliStatus rethrows access errors rather than running a second, denied PID probe; isProcessRunning returns false only for ESRCH and otherwise throws.
  • Transport — the socket error handler preserves EPERM/EACCES as a connect access error while keeping runtime_unavailable + restart advice for every other failure.
  • orca open — an initial access denial rejects before launchOrcaApp, so a sandboxed caller no longer spawns the app.
  • Tests + docs — 17 new regression tests plus docs/reference/runtime-access-diagnostics.md.

ℹ️ Nitpicks

  • The new docs/reference/runtime-access-diagnostics.md is neither linked from AGENTS.md/README.md nor added to the .gitignore allow-list (docs/** still matches it via .gitignore:96). The .gitignore comment scopes tracked docs/reference/*.md to docs linked from those files, so this one is only in the repo because it was force-added. Optional follow-up: add !docs/reference/runtime-access-diagnostics.md (alphabetically) and a link so a clean git add reproduces the page.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — incremental re-review of abdd1f4 since the prior pullfrog review at fe4b53b; the only delta is a test-only change.

  • Strengthened PID-probe assertions — the it.each in runtime-access-diagnostics.test.ts now pins the exact error code (runtime_permission_denied for EPERM/EACCES, runtime_unverifiable otherwise), reason (permission_denied/probe_failed), and asserts data.systemCode is absent when the thrown error carries no code, closing the gap where the prior assertions would have passed under a misclassification.
  • Production behavior unchanged — runtime-access-error.ts, metadata.ts, status.ts, and transport.ts are identical to the previously reviewed fe4b53b. Re-ran src/cli/runtime/runtime-access-diagnostics.test.ts: 13 passed.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Jinwoo-H added a commit that referenced this pull request Sep 23, 2026
…22341)

* fix(cli): report a denied runtime connection instead of a dead Orca

Inside Codex's macOS Seatbelt sandbox, connect() on the runtime socket fails
with EPERM. The CLI dropped the errno and reported "Could not connect ...
Restart Orca", appended "Orca is not running. Run 'orca open' first.", and
`orca status` answered ok:true with `starting` (its pid probe also gets EPERM).
An agent following that advice restarts a healthy app, which cannot help.

EPERM/EACCES on the metadata read or the socket/pipe connect now fails with a
CLI-local `runtime_access_denied` error: ok:false, non-zero exit,
operation/systemCode/processState:"unverifiable"/retryable:false and nextSteps
that say to re-run with escalated permissions and not restart. CODEX_SANDBOX
only picks the wording. `orca open` stops before launching.

The status pid probe is unchanged: a refused or missing socket proves the
caller reached the endpoint, so a later EPERM probe is another uid and keeps
#20098's `starting`. Missing, refused, stale-pid and timeout paths are
unchanged.

Adapted from the diagnosis and tests in #20487 (and #19605, #13583).

Co-authored-by: lifeodyssey <zhenjiazhou0127@outlook.com>

* docs(skills): tell agents runtime_access_denied means escalate, not restart

The shared CLI-resolution block told every bundled skill to run `orca open`
when a command says Orca is not running. Add the counterpart for the new
access-denied code so sandboxed agents re-run with escalated permissions
instead of launching or restarting Orca. Regenerated stubs and manifest.

* refactor(cli): classify only a denied runtime connect, with a leaner error

A denied metadata read was never observed under a sandbox, and it turned an
unreadable user-data path (the Linux launch contract's root-owned HOME) into
runtime_access_denied instead of "Orca is not running". Keep metadata reads as
on main and classify only the socket/pipe connect.

One helper now maps a socket errno to the error or null; the error data keeps
only systemCode and nextSteps. Tests drop cases already pinned by status.test.ts.

* fix(cli): give not-running advice when a denied socket belongs to a dead Orca

A crashed Orca leaves its metadata and socket file behind, and a sandbox denies
the connect with EPERM before the CLI can see ECONNREFUSED. The sandbox still
reports ESRCH for a gone pid, so a denied connect now probes the metadata pid
and falls through to the ordinary unavailable path when the pid is proven gone.
isProcessRunning moves to its own module so transport and status share it.

* refactor(cli): inline the runtime_access_denied code like other CLI error codes

---------

Co-authored-by: lifeodyssey <zhenjiazhou0127@outlook.com>
@nwparker

nwparker commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Closing as superseded — the linked issue (#19604) has been resolved on main by #22341 (9af6a3d798). Thanks for the contribution!

@nwparker nwparker closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLI reports "Orca is not running" when the caller lacks permission to probe it: EPERM from process.kill and the runtime socket is collapsed into absence

2 participants