Skip to content

ci: stop new main-process code from freezing Orca with synchronous launches - #27756

Merged
brennanb2025 merged 6 commits into
mainfrom
brennanb2025/mainio-sync-spawn-ratchet
Oct 12, 2026
Merged

brennanb2025 merged 6 commits into
mainfrom
brennanb2025/mainio-sync-spawn-ratchet

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 5 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​345 $\color{#cf222e}{\Huge{\mathbf{−}}}$​4 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​341
Prod 7 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​557 $\color{#cf222e}{\Huge{\mathbf{−}}}$​8 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​549

ELI5

When Orca's main process starts another program and waits for it to finish, every Orca window freezes until it does. On Windows, starting a program is slow (antivirus scans each one), so these waits are the main cause of the "Orca froze for a moment" reports. This PR does two things. CI now fails if new main-process code adds another one of these waits. And Orca's existing slow-launch diagnostics can now see the waits that go through runProcessSync.

What Changed

The problem. The Electron main process handles every window's input, every terminal and every timer. A synchronous child-process launch (execFileSync, spawnSync, execSync, runProcessSync) blocks all of that until the child exits. So does the synchronous secure-file writer, which runs icacls.exe several times per write on Windows. #27731 shows one such path freezing Orca for about 90 ms per agent notification on Windows. Production code in src/main and src/shared still has dozens of these calls: 118, in 80 files, counting callers of Orca's own synchronous helpers. Nothing stopped new ones from appearing, and the built-in spawn diagnostics couldn't see them:

  • The diagnostics turned on with ORCA_MAIN_THREAD_DIAGNOSTICS=1 (the [main-thread] churn probe) record async spawns only.
  • runProcessSync never reported to the observer they use.

User-facing change: none. Runtime behavior is unchanged. The only difference shows up when the spawn observer is installed (diagnostics on): synchronous launches are then reported, with their full blocking time.

The mechanism.

  1. Ratchet. The new check:main-sync-spawn-ratchet (config/scripts/check-main-sync-spawn-ratchet.mjs) runs in pnpm lint and in the PR static-analysis job next to check:process-host-imports. It is modeled on that check.
    • What it scans: production files in src/main/** and src/shared/**. It parses them with the same TypeScript-based scanner that check uses, and finds calls to the synchronous launch functions imported from child_process or @orca/process-host, and to the synchronous secure-file writers.
    • Import forms it follows: named, aliased, namespace, require, import x = require(...) and dynamic import(). Look-alike names from other modules, type-only imports, and strings or comments are ignored.
    • Orca's own synchronous helpers count too: gitExecFileSync, getWslHome, listWslDistros, wslUncDirectoryExists, isWslAvailable, isPwshAvailable, grantDirAcl, restrictWindowsPathSync, and hardenSecurePath unless the call is visibly async. Only their callers are counted; src/main/wsl.ts itself is untouched.
    • When it fails: config/main-sync-spawn-baseline.txt stores a call count per file, the same format as the focus-read ratchet. The check fails when any file's count rises, including a file that wasn't listed before. When a count falls, the baseline must be lowered with --prune. So the debt can only shrink.
    • Failure messages give advice for the kind of call: process launches point to runProcess/spawnProcess, and each helper points to its async counterpart. For the secure writers, which have no async version yet, the message says to keep the writer and raise the count, never to swap in a plain write.
  2. Observer. runProcessSync now reports to the existing spawn observer, like runProcess already does. Files that call child_process synchronous functions directly still don't report, for example wsl.ts, wsl-availability.ts and win32-utils.ts (11 files, listed in the review comment). Each becomes visible once it moves to the process-host API. It reports the whole call, because that's how long the thread was blocked, and it reports from a finally so failed and timed-out children are counted too. When no observer is installed, it costs one boolean check.

Why

  • Baseline counts, not an outright ban. The existing 80 files include startup-only, quit-only and non-desktop-process code that is legitimately synchronous. Banning them all would mean dozens of suppressions. A shrinking list is how this repo already handles migration debt, for example process-host-direct-import-baseline.txt and the max-lines ratchet.
  • Counts per file, so a file already on the list can't quietly add more.
  • Known limit: counting follows references, not calls. A launch reached through another file's own wrapper doesn't raise that file's count. Two cache-miss helpers, getDefaultWslDistro and resolveCurrentWindowsIdentity, aren't gated yet; they're listed as follow-ups in the review comment.
  • No behavior change in this PR. Moving the hot writers off the main thread is separate, follow-up work. The first one is perf(notifications): stop every Windows notification from freezing Orca #27731; the device-registry and outbox writers wait on the shared async secure writer discussed in fix(runtime): persist mobile notification dismissals without blocking Windows #26754.

Linked Issue

Refs #20497 (synchronous spawns on the main-process IPC thread). Linear: STA-7408.

Visual Proof

N/A. This adds a CI check and a diagnostics hook, with no runtime or UI change, so there is no before/after to measure. The measured effect of removing one of these waits is in #27731: a Windows notification went from 89 ms to 0.5 ms of main-process block.

Testing

  • check-main-sync-spawn-ratchet.test.mjs covers count rises and falls, --prune, helper gating, and:
    • every import form above;
    • look-alike names, type-only imports, and code inside strings or comments;
    • secure-writer imports from the right and wrong modules;
    • the baseline's add and stale-entry failures.
  • spawn-observer.test.ts covers a sync run that exits non-zero, a missing program (throws), a timeout, and no observer installed.
  • node config/scripts/check-main-sync-spawn-ratchet.mjs passes on the tree (80 files, 118 uses), under Node and bun --bun, in about 4 s. check-process-host-imports still passes.
  • Mutation checks show the prune and alias tests fail when their feature is broken.
  • tsc for node and the process-host test project is clean, and oxlint is clean on the changed files.

Agent skill upstream boundary

  • Not applicable, or this change copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.

Notes

  • Cross-platform: the check runs on file text, with no platform-specific behavior.
  • SSH and WSL: the baseline includes files whose synchronous launch runs on other hosts or in other processes (daemon, orcad, CLI). They're listed rather than special-cased, so moving any of them off the synchronous path just deletes a line.

Checklist

  • This PR is small and focused
  • I explained what changed and why
  • N/A visual proof, with reason
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote impact considered
  • pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (CI will cover)

The Electron main process freezes every window while a synchronous child
process runs, and on Windows each launch is slow because antivirus scans
it. Nothing stopped new synchronous launches from landing in main-process
code, and the spawn observer behind ORCA_MAIN_THREAD_DIAGNOSTICS only saw
async spawns, so the worst blocks were invisible to it.

Ratchet: check:main-sync-spawn-ratchet parses production src/main and
src/shared files (test-only paths excluded) and fails when a file not in
config/main-sync-spawn-baseline.txt reaches execFileSync, spawnSync or
execSync from child_process, runProcessSync from @orca/process-host, or
one of the synchronous secure-file writers (they run icacls on Windows).
Named, aliased, namespace, default, require() and import() forms count.
A baseline entry whose file no longer launches synchronously fails as
stale, so the list can only shrink. It runs in lint and in the PR
static-analysis job next to the process-host import check.

Observer: runProcessSync now reports to the spawn observer, timing the
whole call (that is how long the thread was stuck) and reporting from a
finally so failing, throwing and timed-out children are counted too.

Runtime behavior is unchanged except that the diagnostics observer now
sees synchronous launches; with no observer registered the sync path
pays one extra branch.
The sync-spawn ratchet only tracked which files launched synchronously, so a
file already on the list (wsl.ts, win32-utils.ts, git-exec-file.ts) could add
more launches unnoticed, and new callers of the repo's own synchronous
wrappers passed CI.

The baseline now stores a per-file count of uses, in the same format as the
focus-read ratchet: a rise or a new file fails, a lower or zero count fails
until `pnpm check:main-sync-spawn-ratchet --prune` lowers it, and --prune
never raises an entry. Every value reference to a gated function counts
(call, hand-off, alias and its uses, re-export); imports alone count nothing.

Gated wrappers: gitExecFileSync (and the git/runner re-export),
restrictWindowsPathSync, hardenSecurePath when called with sync: true,
grantDirAcl, getWslHome, listWslDistros, wslUncDirectoryExists,
isWslAvailable (and the wsl.ts re-export), isPwshAvailable. Their 18
current caller files join the baseline.

Failure messages now give advice per family: the async process-host API for
child-process launches, the async twin for wrappers, and for the secure
writers a note that no async secure writer exists yet, so the sync one stays
and must never become a plain write (that drops the Windows ACL hardening).

Also refresh two stale blockMs descriptions and drop a no-op line from the
spawn-observer "stays silent" test.
…tests

hardenSecurePath was counted only when `{ sync: true }` was written inline,
so moving the options into a variable or a spread dropped the count and the
check then asked for a prune. It now counts every use except a call with no
options or a spread-free object literal whose `sync` is absent or false.

Also: resolve backslash paths in the importing file as well as the
specifier; still visit heritage and instantiation expressions
(`extends mixin(fn)`, `fn<T>`); state the reference-only counting limit;
and point the secure-writer advice at raising this file's baseline count.

Tests cover the hardenSecurePath shapes, the alias chains that need the
fixpoint loop, and a prune that must not raise a grown row.
@brennanb2025

brennanb2025 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Waiting synchronously for another program holds Electron’s main event loop and can leave Orca’s windows unresponsive. Existing code already contains these waits, the original file allowlist let listed files add more, and diagnostics omitted runProcessSync. This PR adds a shrinking count allowance per file in CI and reports the full synchronous wait through the existing diagnostics observer, including failed launches and timeouts. Existing waits still need the separate migration work; this PR prevents tracked debt from growing.

The review fixed per-file growth enforcement, coverage of known synchronous wrappers, conditional secure-hardening detection, useful secure-writer failure advice, and the workflow’s pinned background-step test. The final independent review found and fixed one further gap: a spread of call arguments could hide sync: true; three regression cases now cover it, and a fresh reviewer verified the fix.

Known limits remain deferred: the scanner counts references rather than tracing arbitrary wrappers, two cache-miss helpers are outside its explicit list, and unusual module-loading/member syntax can escape this scanner. Unconventional test harnesses remain in the baseline, diagnostics combine existing synchronous/asynchronous command buckets, and baseline-command deduplication remains cleanup work. These limits are disclosed in the PR body and do not change execution behavior.

Final verification: final head d7f8d066377 has 48 passing CI jobs, 13 skipped, and no pending or failed jobs; 152 focused tests passed; the ratchet retained 118 tracked references across 80 files; native typechecking passed. Local static checks passed apart from an independently confirmed environment mismatch: installed Bun 1.3.14 lacks bun check, while CI pins Bun 1.4.3. Fresh correctness and architecture reviews, narrow fix review, and the final readiness/performance scan found no open P0, P1, or P2 source issues.

Remote Electron QA now passed on M4Air at d7f8d066377, using standalone OpenCode with DeepSeek V4.1 Flash. The app rendered its startup state, then registered a plain folder through the existing folder action and selected it by clicking its rendered sidebar row; the file explorer showed both fixture files. I inspected both screenshots below. The disposable home and settings were verified, windows stayed hidden and unfocused, and the owned app, helpers and daemon were stopped with no remaining listeners. The driver also reran 97 focused tests and the 118-reference/80-file ratchet successfully.

Diagnostics coverage remains unit-only: success, nonzero exit, missing programs and timeouts were tested through the process API; a live app diagnostics path was not exercised. The native folder picker, Windows, Linux and SSH flows were not exercised in this QA run. No latency improvement is claimed or measured for this prevention-and-reporting change.

Review status: source review, readiness, final CI and the required remote startup/folder QA passed at d7f8d066377; the review loop is complete with the known scanner limits deferred above. No product source changed during QA. The PR remains open, ready for review, and unmerged.

Startup on M4Air:

Startup renders the empty-project state

Plain-folder workspace selected:

Selected plain-folder workspace and its fixture files

@brennanb2025
brennanb2025 marked this pull request as ready for review October 12, 2026 05:31
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 66d44143-ffc5-44fd-a2e0-0fa726a29430

📥 Commits

Reviewing files that changed from the base of the PR and between f3f37b1 and d7f8d06.


📒 Files selected for processing (3)
  • .github/workflows/pr.yml
  • config/scripts/check-main-sync-spawn-ratchet.mjs
  • config/scripts/check-main-sync-spawn-ratchet.test.mjs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.



📝 Walkthrough

Walkthrough

The change adds a scanner that counts synchronous-launch uses in production source and checks them against a decrease-only baseline. The check runs in lint and in the static-analysis preflight. Synchronous process launches now report elapsed time through process exit, including when execution throws. Related diagnostic comments and benchmark documentation distinguish synchronous duration from asynchronous spawn-initiation time.


Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to d7f8d

The ratchet and synchronous timing change are mergeable after normal checks. No actionable failure was established in the reviewed paths.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 9 files. (1 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly identifies the main change: CI now prevents new synchronous launches in the main process. It is concise and directly related to the pull request.
Description check Passed The description is mostly complete. It includes the ELI5 summary, changes, rationale, linked issue, visual-proof explanation, testing details, platform notes, and checklist. It also documents known li…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 9 files. (1 skipped: 1 unsupported.)


  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 13e0584f-1e89-46ea-993e-d134c44262df
📥 Commits

Reviewing files that changed from the base of the PR and between 78144e9 and f3f37b1.

📒 Files selected for processing (12)
  • .github/workflows/pr.yml
  • config/main-sync-spawn-baseline.txt
  • config/scripts/check-main-sync-spawn-ratchet.mjs
  • config/scripts/check-main-sync-spawn-ratchet.test.mjs
  • config/scripts/pr-preflight-gates.test.mjs
  • config/scripts/static-module-specifiers.mjs
  • package.json
  • src/main/diagnostics/main-thread-churn-probe.ts
  • src/packages/process-host/src/run-process.ts
  • src/packages/process-host/src/spawn-observer.test.ts
  • tests/e2e/main-thread-git-cost.spec.ts
  • tests/tools/benchmarks/main-thread-jank-bench.mjs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

Comment on lines 339 to 357
*/
export function runProcessSync(spec: ProcessSpec): ProcessResult {
const resolved = resolveSpawn(spec, process.platform)
// Diagnostics only: spawnSync holds the calling thread until the child exits,
// so the whole call is the block, reported even when it throws.
const spawnStartedAt = hasSpawnObserver() ? performance.now() : null
try {
return runResolvedProcessSync(spec, resolved)
} finally {
if (spawnStartedAt !== null) {
notifySpawnObserver(resolved.file, resolved.args, performance.now() - spawnStartedAt)
}
}
}

function runResolvedProcessSync(spec: ProcessSpec, resolved: ResolvedSpawn): ProcessResult {
const result = nodeSpawnSync(resolved.file, [...resolved.args], {
...resolved.options,
input: spec.input,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

sed -n '310,390p' src/packages/process-host/src/run-process.ts
rg -n 'notifySubprocessSpawn|spawnObserver|SpawnObserver|recordSubprocessSpawn' src/packages/process-host/src src/main/diagnostics/main-thread-churn-probe.ts

Repository: stablyai/orca

Length of output: 5105


🏁 Script executed:

printf '%s\n' '--- spawn-observer.ts ---'
nl -ba src/packages/process-host/src/spawn-observer.ts
printf '%s\n' '--- run-process imports and async path ---'
nl -ba src/packages/process-host/src/run-process.ts | sed -n '1,85p'
printf '%s\n' '--- observer registrations and consumers ---'
rg -n -F --glob '*.{ts,tsx,mjs,js}' -- 'setSpawnObserver' src packages tests 2>/dev/null || true
rg -n -F --glob '*.{ts,tsx,mjs,js}' -- 'recordSubprocessSpawn' src packages tests 2>/dev/null || true
printf '%s\n' '--- observer tests ---'
nl -ba src/packages/process-host/src/spawn-observer.test.ts
printf '%s\n' '--- package exports ---'
rg -n -F --glob 'package.json' -- 'process-host' src packages 2>/dev/null || true

Repository: stablyai/orca

Length of output: 13331


🏁 Script executed:

printf '%s\n' '--- diagnostics recorder ---'
nl -ba src/main/diagnostics/main-thread-churn-probe.ts | sed -n '1,180p'
printf '%s\n' '--- startup registration ---'
nl -ba src/main/startup/main-process-preflight.ts | sed -n '220,270p'
printf '%s\n' '--- recorder tests ---'
nl -ba src/main/diagnostics/main-thread-churn-probe.test.ts | sed -n '1,150p'
printf '%s\n' '--- observer-related process-host tests and contracts ---'
rg -n -F --glob '*.{ts,tsx,mjs,js}' -- 'runProcessSync' src/packages/process-host src/main | head -80
rg -n -i --glob '*.{ts,tsx,mjs,js}' -- 'observer.*throw|throw.*observer|best.?effort|diagnostic' src/packages/process-host/src src/main/diagnostics src/main/startup | head -120

Repository: stablyai/orca

Length of output: 38183


🏁 Script executed:

printf '%s\n' '--- process-host package metadata ---'
cat src/packages/process-host/package.json
printf '%s\n' '--- process-host entrypoints ---'
rg --files src/packages/process-host | sort | sed -n '1,120p'
rg -n -F --glob '*.{ts,tsx}' -- 'from "./spawn-observer"' src/packages/process-host
rg -n -F --glob '*.{ts,tsx}' -- 'setSpawnObserver' src/packages/process-host
printf '%s\n' '--- complete runProcess catch boundary ---'
nl -ba src/packages/process-host/src/run-process.ts | sed -n '61,105p'

Repository: stablyai/orca

Length of output: 9029


Contain observer exceptions in notifySpawnObserver.

setSpawnObserver accepts an arbitrary callback, and notifySpawnObserver invokes it without containment. When that callback throws, runProcessSync can throw after a successful command, or its finally completion can replace the command's original startup error. The same callback can also make runProcess reject after the child has started, despite its documented contract.

Suggested fix
 export function notifySpawnObserver(
   command: string,
   args: readonly string[],
   blockMs: number
 ): void {
-  observer?.(command, args, blockMs)
+  try {
+    observer?.(command, args, blockMs)
+  } catch {
+    // Diagnostics must not change process execution.
+  }
 }

@brennanb2025
brennanb2025 merged commit fa4ef2b into main Oct 12, 2026
61 checks passed
OrcaWin added a commit that referenced this pull request Oct 12, 2026
…ary (#28054)

#27756's script imported baseline helpers from check-focus-read-ratchet.mjs
after #27788 had moved them into ratchet-baseline.mjs, so it crashed on start
and failed every PR that merges main. Use the library's helpers, and record the
two daemon-process launches from #27769 that the broken check never saw.

Co-authored-by: Orca Worker <orca-worker@localhost>
brennanb2025 pushed a commit that referenced this pull request Oct 12, 2026
…ary (#28054)

#27756's script imported baseline helpers from check-focus-read-ratchet.mjs
after #27788 had moved them into ratchet-baseline.mjs, so it crashed on start
and failed every PR that merges main. Use the library's helpers, and record the
two daemon-process launches from #27769 that the broken check never saw.

Co-authored-by: Orca Worker <orca-worker@localhost>
brennanb2025 added a commit that referenced this pull request Oct 12, 2026
…27875)

* fix(browser): annotation card and outline follow the selected element

The grab overlay keeps re-measuring the picked element after the click
(scroll in any box, resize, element size changes), and the isolated
viewport bridge reports that live box so the pending card follows it,
including page zoom. The card's side pick uses its measured height.

* fix(browser): bind annotation card to selected element

* fix(browser): keep document annotation card on selection

* fix(ci): point the main sync-spawn ratchet at the shared ratchet library (#28054)

#27756's script imported baseline helpers from check-focus-read-ratchet.mjs
after #27788 had moved them into ratchet-baseline.mjs, so it crashed on start
and failed every PR that merges main. Use the library's helpers, and record the
two daemon-process launches from #27769 that the broken check never saw.

Co-authored-by: Orca Worker <orca-worker@localhost>

* fix(browser): follow selected element outer-box size

* refactor(chat): give the start-refusal resumable table its own module (#28085)

#27946 and #26684 together put agent-session-failure-words.ts at 304 lines,
over max-lines, which fails lint and the pre-commit hook on every merge of
main. The refusal-code table is standalone with one reader, so it moves out.

Co-authored-by: Orca Worker <orca-worker@localhost>

---------

Co-authored-by: OrcaWin <alpha-eng@stably.ai>
Co-authored-by: Orca Worker <orca-worker@localhost>
@brennanb2025
brennanb2025 deleted the brennanb2025/mainio-sync-spawn-ratchet branch October 12, 2026 23:55
nwparker added a commit that referenced this pull request Oct 13, 2026
C7a (#27933) predates the main sync-spawn ratchet (#27756); drop once C7a carries this row.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant