Skip to content

perf(git): make local Git metadata observation event-driven (no scheduler) - #16404

Merged
nwparker merged 4 commits into
mainfrom
perf/event-driven-git-metadata-standalone
Aug 25, 2026
Merged

nwparker merged 4 commits into
mainfrom
perf/event-driven-git-metadata-standalone

Conversation

@nwparker

@nwparker nwparker commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 5 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​571 $\color{#cf222e}{\Huge{\mathbf{−}}}$​310 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​261
Prod 13 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​1044 $\color{#cf222e}{\Huge{\mathbf{−}}}$​370 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​674

Problem

Local Git-common metadata observation re-scanned every linked worktree on a fixed interval even while completely idle. With E linked worktrees each poll cost roughly:

6*E linked-worktree stats + 6 primary metadata stats + one worktrees listing

That work competed with renderer updates and Git subprocesses. The cost scaled with repo count x worktree count rather than with change rate, which is the actual defect.

Approach

Healthy local observation is now event-driven, with polling kept strictly as a fallback:

native event -> path classification -> existing targeted notification pipeline -> bounded refresh
  • Narrow @parcel/watcher stream over <common>/worktrees, extended from macOS to Linux and Windows, Windows backend pinned explicitly.
  • New shallow watcher mode over the allowlisted primary metadata leaves (HEAD, packed-refs, index, config, config.worktree, logs/HEAD). It watches the containing directory, not each file, so Git's atomic write-and-rename does not orphan the binding.
  • Selected upstream refs stay on the existing bounded stat poll.

No RPC or wire changes. SSH/WSL observation stays provider-owned (subscribeTarget branches on connectionId before any of this code is reached). Folder workspaces are unaffected.

Verified on real hosts

Not reasoned about — measured, on Windows (awin), Ubuntu 24.04, and two Macs.

Check macOS Linux Windows
narrow watch: worktree add / remove / prune pass pass pass
root delete emits delete/worktrees (re-arm trigger) — pass pass
backend: 'windows' accepted n/a n/a pass
open handle blocks git worktree prune n/a n/a does not block
shallow events survive atomic rename pass pass pass
inotify budget n/a 1 instance per event loop n/a

Two long-standing concerns in the previous code turned out to be unfounded:

  • The old comment claimed a Windows directory handle on worktrees/ could interfere with git worktree prune. It does not: remove and prune both succeed with the stream held, the directory is deleted, and parcel reports delete/worktrees.
  • Linux inotify does not cost one instance per watch. libuv shares a single inotify fd per event loop (50 fs.watch calls measured as 1 instance), so the watch budget is not a constraint.

Failure handling

Each of these was reproduced on real hardware before being fixed.

Replaced inode. fs.watch binds an inode, not a path. When the directory is replaced the watcher goes deaf silently — no events, no error. Verified on Linux. Directory bindings are now re-checked on a bounded cadence (2 stats / 30s) and rebound, so the healthy path still performs no primary metadata stats.

Mute host. A host whose notification path is broken accepts the registration and stays silent forever, which is indistinguishable from an idle repo. This is not hypothetical: on a macOS 26.3.1 machine whose fseventsd had grown to ~15GB resident and saturated a core, both fs.watch on a directory and @parcel/watcher delivered zero events across 4/4 runs, while kqueue file watches and stat polling still worked. Since the fallback only triggered on an error that never arrives, metadata would have stayed stale until restart.

A one-shot delivery probe now measures whether the host actually delivers before a shallow subscription is accepted, mirroring the canary that already guards parcel's own delivery. Confirmed on both machines:

Host delivers? probe outcome
macOS 26.3.1 (degraded fseventsd) no false falls back to polling
macOS 26.5.2 yes true uses the watcher

Worth noting for reviewers: on macOS a directory fs.watch is an FSEvents client, where the previous stat poll registered none. That is a real cost on the resource #7068 addressed, and the probe is what keeps its failure mode safe rather than silent.

Testing

  • pnpm run typecheck (all projects) — pass
  • pnpm run lint (full chain incl. max-lines ratchet, reliability gates, electron ratchet, localization) — pass
  • pnpm exec vitest run src/main/ipc/ — 3172 passed, 11 skipped, 1 failure

The single failure is filesystem-watcher-real.test.ts, a live @parcel/watcher integration test. It fails identically on origin/main on the same machine and is caused by that machine's degraded fseventsd; this change touches neither the test nor its subject.

Note on scope

This branch is deliberately independent of the metadata poll scheduler (#16175) and applies directly to main. Root-replacement inode detection was originally bundled into that scheduler commit but is watcher logic, so it is carried here.

Replaces the recurring per-repo metadata scan with native filesystem events on
macOS, Linux, and Windows. Polling is retained purely as a fallback.

- Narrow @parcel/watcher stream over <common>/worktrees, extended from macOS to
  Linux and Windows, with the Windows backend pinned explicitly.
- New shallow watcher mode over the allowlisted primary metadata leaves. It
  watches the containing directory rather than each file, so Git's atomic
  write-and-rename does not orphan the binding.
- Selected upstream refs stay on the existing bounded stat poll.

Verified on real hosts rather than in principle:

- Windows: `git worktree remove` and `git worktree prune` both succeed while the
  narrow stream holds the directory. The historical concern that an open handle
  would block prune does not reproduce.
- Linux: inotify costs one instance per event loop, not one per watch, so the
  watch budget is not a constraint.
- macOS/Linux/Windows: shallow events survive repeated commit, checkout,
  config, and pack-refs cycles.

Failure handling, each reproduced before being fixed:

- fs.watch binds an inode and reports nothing once that inode is replaced, with
  no error. Directory bindings are re-checked on a bounded cadence and rebound.
- A host whose notification path is dead accepts registrations and stays mute
  forever. Observed on a macOS machine whose fseventsd had grown to ~15GB and
  saturated a core. A one-shot delivery probe now fails the shallow subscribe on
  such a host so it falls back to polling instead of showing stale metadata.

This change stands alone on main and does not depend on the metadata poll
scheduler.
@coderabbitai

coderabbitai Bot commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The watcher process now supports filtered shallow subscriptions after verifying native event delivery. Git common-directory monitoring adds shared metadata definitions, snapshot diffing, configurable polling, and reconciliation. Dedicated narrow and primary watchers manage native subscriptions, root replacement, visibility, cleanup, and polling fallback. macOS, Linux, and Windows use the native watcher paths; unsupported platforms use polling. Tests cover shallow events, delivery failures, platform behavior, fallback transitions, stale subscriptions, and watcher rearming.

Merge Risk: 🔵 Low · up to 01a60

The change moves local Git metadata updates from idle polling to event-driven watching with bounded fallback. Edge cases can still leave repository state stale or keep a watcher alive after disposal, while fallback state may degrade upstream-ref updates. These are bounded correctness and resource risks requiring owner awareness, but the supplied evidence does not indicate a release-blocking failure.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description gives detailed problem, approach, failure handling, scope, and testing information. However, it omits required template sections, including ELI5, What Changed, Why, Linked Issue with a… Add the missing required sections. Include a linked issue in the form Fixes #<number>. Add Visual Proof with the required before/after evidence, or state N/A with a brief reason if visual proof is not applicable. Complete the testing ch…
Docstring Coverage ⚠️ Warning Docstring coverage is 11.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 34 functions across 18 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: event-driven local Git metadata observation with polling removed from the normal path.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description gives detailed problem, approach, failure handling, scope, and testing information. However, it omits required template sections, including ELI5, What Changed, Why, Linked Issue with an issue number, Visual Proof or an explicit N/A statement, AI Disclosure, Review, Agent skill upstream boundary, Notes, and Checklist confirmations.

Resolution

Add the missing required sections. Include a linked issue in the form Fixes #&lt;number&gt;. Add Visual Proof with the required before/after evidence, or state N/A with a brief reason if visual proof is not applicable. Complete the testing checkboxes, AI Disclosure, Review, Agent skill upstream boundary, Notes, and Checklist sections.

✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Linux returns a released reservation to the free list, so the second
replacement could land back on the first replacement's inode and look
unchanged to reconciliation. Verified on ext4: releasing yields inodes
[N, N+43, N+43] while holding yields [N, N+43, N+44]. macOS never
recycles, which is why this only failed on CI.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 37469060-3964-4f9b-a40a-8e5414b4b806

📥 Commits

Reviewing files that changed from the base of the PR and between 6103b78 and c81876a.

📒 Files selected for processing (17)
  • src/main/ipc/parcel-watcher-process-entry.test.ts
  • src/main/ipc/parcel-watcher-process-entry.ts
  • src/main/ipc/parcel-watcher-process-protocol.ts
  • src/main/ipc/parcel-watcher-shallow-subscription.test.ts
  • src/main/ipc/parcel-watcher-shallow-subscription.ts
  • src/main/ipc/shallow-watch-delivery-probe.test.ts
  • src/main/ipc/shallow-watch-delivery-probe.ts
  • src/main/ipc/worktree-base-directory-poller.test.ts
  • src/main/ipc/worktree-git-common-metadata-files.ts
  • src/main/ipc/worktree-git-common-narrow-watch.ts
  • src/main/ipc/worktree-git-common-polling.ts
  • src/main/ipc/worktree-git-common-primary-polling.ts
  • src/main/ipc/worktree-git-common-primary-watch.ts
  • src/main/ipc/worktree-git-common-snapshot-diff.ts
  • src/main/ipc/worktree-git-common-watch-reconciliation.ts
  • src/main/ipc/worktree-git-common-watch.test.ts
  • src/main/ipc/worktree-git-common-watch.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.

…allbacks

Both fallbacks tracked their in-flight promise with the same self-comparison
on settle, duplicated verbatim. Hoisting it removes a subtle invariant that
was being hand-maintained in two places.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e2b9f4bf-6c53-425d-bce6-c5ed6b7e86a7

📥 Commits

Reviewing files that changed from the base of the PR and between 4d929e3 and 041cfbb.

📒 Files selected for processing (3)
  • src/main/ipc/single-flight-promise.ts
  • src/main/ipc/worktree-git-common-narrow-watch.ts
  • src/main/ipc/worktree-git-common-primary-watch.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Comment on lines +68 to +92
const ensurePollingFallback = (): Promise<void> =>
pollingFallback.run(() => {
stopExistencePoll()
usingPollingFallback = true
return reconciliation
.unsubscribe()
.catch(() => {})
.then(() =>
startGitCommonPolling(
target.path,
onEvents,
pollIntervalMs,
visibility,
onFullScan,
false
)
)
.then(async (fallback) => {
if (disposed || subscription) {
await fallback.unsubscribe()
return
}
subscription = fallback
})
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

#!/bin/bash
# Verify usingPollingFallback is never reset, and check test coverage for the discard path.
rg -n 'usingPollingFallback' src/main/ipc/
rg -n -C4 'ensurePollingFallback|polling fallback' src/main/ipc/worktree-git-common-watch.test.ts

Repository: stablyai/orca

Length of output: 567


🏁 Script executed:

#!/bin/bash
printf '%s\n' '--- implementation ---'
sed -n '1,230p' src/main/ipc/worktree-git-common-narrow-watch.ts
printf '%s\n' '--- related tests ---'
if [ -f src/main/ipc/worktree-git-common-watch.test.ts ]; then
  sed -n '1,280p' src/main/ipc/worktree-git-common-watch.test.ts
fi
printf '%s\n' '--- related callers ---'
rg -n -C8 'trySubscribe|ensurePollingFallback|startGitCommonPolling|reconciliation' src/main/ipc/worktree-git-common-narrow-watch.ts src/main/ipc/

Repository: stablyai/orca

Length of output: 50369


🏁 Script executed:

#!/bin/bash
printf '%s\n' '--- narrow-watch completion ---'
sed -n '215,275p' src/main/ipc/worktree-git-common-narrow-watch.ts
printf '%s\n' '--- fallback-related tests ---'
rg -n -C12 'WatcherProcessFailure|supervisor_crash_fuse|process_unavailable|fallback|concurrent|tryUpgrade|trySubscribe' src/main/ipc/worktree-git-common-watch.test.ts
printf '%s\n' '--- single-flight contract ---'
rg -n -C8 'export function createSingleFlight|function createSingleFlight|pending\\(' src/main/ipc/single-flight-promise.ts
printf '%s\n' '--- polling startup contract ---'
sed -n '105,190p' src/main/ipc/worktree-git-common-polling.ts

Repository: stablyai/orca

Length of output: 8464


🏁 Script executed:

#!/bin/bash
printf '%s\n' '--- fallback tests ---'
sed -n '330,430p' src/main/ipc/worktree-git-common-watch.test.ts
printf '%s\n' '--- single-flight implementation ---'
cat -n src/main/ipc/single-flight-promise.ts
printf '%s\n' '--- watch composition ---'
rg -n -C20 'startGitCommonNarrowWatch|startGitCommonWatch' src/main/ipc/worktree-git-common-watch.ts

Repository: stablyai/orca

Length of output: 6844


🏁 Script executed:

#!/bin/bash
printf '%s\n' '--- polling snapshot and startup ---'
sed -n '1,125p' src/main/ipc/worktree-git-common-polling.ts
printf '%s\n' '--- reconciliation lifecycle ---'
sed -n '1,125p' src/main/ipc/worktree-git-common-watch-reconciliation.ts
printf '%s\n' '--- narrow-watch test setup and re-arm flow ---'
sed -n '250,350p' src/main/ipc/worktree-git-common-watch.test.ts

Repository: stablyai/orca

Length of output: 12613


Reset usingPollingFallback when fallback startup fails. If startGitCommonPolling rejects, the existence poll is re-armed, but usingPollingFallback remains true. A later native subscription can install, while reconciliation.ensureStarted() skips reconciliation because canStart checks this flag. Reset the flag in the rejection path. The concurrent trySubscribe discard path is prevented by the subscribing and generation checks.

Two independent reviews converged on the same root cause: nothing bounded
how long primary metadata could stay wrong once the shallow watcher stopped
reporting. Four distinct paths led there.

- A terminal watch error arriving while the status-ref poll was still starting
  left the repo with status-ref coverage only. handleWatcherError ran its
  teardown against nulls, then the in-flight poll installed itself, and the
  fallback guard mistook it for coverage and discarded the fallback. Primary
  metadata was then never observed again. The guard no longer treats status-ref
  polling as primary coverage, and startup re-checks watcher liveness after its
  awaits.

- Nothing re-read the six primary files while the watcher was nominally live.
  A lossy notification path, a dropped batch, or inotify queue overflow raises
  no error, so the error-driven fallback never fired and the inode rebind sweep
  does not detect loss. A 15-tick backstop re-stats them, turning permanent
  staleness into one tick. Measured cost is ~0.2 stats/s/repo against the 3/s
  the old poll cost.

- Reconciliation treated any late-observed entry create as a root replacement,
  so an ordinary  tore down a healthy stream ~30s later and
  opened a deaf window. It now also requires the root itself to be recreated.

- The shallow watcher recorded directory identity from a stat issued after
  binding, so a replacement in that gap pinned the dead inode's watcher to the
  new identity and the sweep would never rebind. Identity is now read first,
  which errs toward a harmless extra rebind.

Test helper: replacing the worktrees root frees several inodes at once, so
reserving one still let the recreated root reuse its own. It now verifies the
inode actually changed. Confirmed on ext4, where three holds were needed.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2795034d-db66-409a-9051-b4e3122b38b8

📥 Commits

Reviewing files that changed from the base of the PR and between 041cfbb and 01a6034.

📒 Files selected for processing (4)
  • src/main/ipc/parcel-watcher-shallow-subscription.ts
  • src/main/ipc/worktree-git-common-primary-watch.ts
  • src/main/ipc/worktree-git-common-watch-reconciliation.ts
  • src/main/ipc/worktree-git-common-watch.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Comment on lines +160 to +167
// Why: read identity BEFORE binding. If the directory is replaced in the gap,
// the recorded identity is stale and the first sweep rebinds — the harmless
// direction. Reading after would pin the dead inode's watcher to the new
// identity, and the sweep would then never rebind it.
const identityBeforeBind = directoryIdentitySync(parent)
watchDirectory(parent, fileNames)
if (watchers.has(parent) && identityBeforeBind !== null) {
boundIdentities.set(parent, identityBeforeBind)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Do not record an identity after a failed rebind.

The initial binding checks watchers.has(parent), but refreshBinding records the identity at Line 143 even when nested watchDirectory fails and leaves no watcher. Later refreshes then treat the directory as bound and never retry, causing silent metadata staleness. Guard the assignment with watchers.has(parent).

Proposed fix
     watchDirectory(parent, fileNames, true)
+    if (!watchers.has(parent)) {
+      return
+    }
     boundIdentities.set(parent, identity)

Add a regression test for one failed nested bind followed by a successful retry.

@nwparker
nwparker merged commit e7c5263 into main Aug 25, 2026
48 checks passed
paidaxingyo666 pushed a commit to paidaxingyo666/Manta that referenced this pull request Aug 25, 2026
…uler) (stablyai#16404)

* perf(git): make local Git metadata observation event-driven

Replaces the recurring per-repo metadata scan with native filesystem events on
macOS, Linux, and Windows. Polling is retained purely as a fallback.

- Narrow @parcel/watcher stream over <common>/worktrees, extended from macOS to
  Linux and Windows, with the Windows backend pinned explicitly.
- New shallow watcher mode over the allowlisted primary metadata leaves. It
  watches the containing directory rather than each file, so Git's atomic
  write-and-rename does not orphan the binding.
- Selected upstream refs stay on the existing bounded stat poll.

Verified on real hosts rather than in principle:

- Windows: `git worktree remove` and `git worktree prune` both succeed while the
  narrow stream holds the directory. The historical concern that an open handle
  would block prune does not reproduce.
- Linux: inotify costs one instance per event loop, not one per watch, so the
  watch budget is not a constraint.
- macOS/Linux/Windows: shallow events survive repeated commit, checkout,
  config, and pack-refs cycles.

Failure handling, each reproduced before being fixed:

- fs.watch binds an inode and reports nothing once that inode is replaced, with
  no error. Directory bindings are re-checked on a bounded cadence and rebound.
- A host whose notification path is dead accepts registrations and stays mute
  forever. Observed on a macOS machine whose fseventsd had grown to ~15GB and
  saturated a core. A one-shot delivery probe now fails the shallow subscribe on
  such a host so it falls back to polling instead of showing stale metadata.

This change stands alone on main and does not depend on the metadata poll
scheduler.

* test(git-watch): hold reserved inodes across root replacements

Linux returns a released reservation to the free list, so the second
replacement could land back on the first replacement's inode and look
unchanged to reconciliation. Verified on ext4: releasing yields inodes
[N, N+43, N+43] while holding yields [N, N+43, N+44]. macOS never
recycles, which is why this only failed on CI.

* refactor(git-watch): share one single-flight helper between watcher fallbacks

Both fallbacks tracked their in-flight promise with the same self-comparison
on settle, duplicated verbatim. Hoisting it removes a subtle invariant that
was being hand-maintained in two places.

* fix(git-watch): close the silent-staleness paths in primary metadata

Two independent reviews converged on the same root cause: nothing bounded
how long primary metadata could stay wrong once the shallow watcher stopped
reporting. Four distinct paths led there.

- A terminal watch error arriving while the status-ref poll was still starting
  left the repo with status-ref coverage only. handleWatcherError ran its
  teardown against nulls, then the in-flight poll installed itself, and the
  fallback guard mistook it for coverage and discarded the fallback. Primary
  metadata was then never observed again. The guard no longer treats status-ref
  polling as primary coverage, and startup re-checks watcher liveness after its
  awaits.

- Nothing re-read the six primary files while the watcher was nominally live.
  A lossy notification path, a dropped batch, or inotify queue overflow raises
  no error, so the error-driven fallback never fired and the inode rebind sweep
  does not detect loss. A 15-tick backstop re-stats them, turning permanent
  staleness into one tick. Measured cost is ~0.2 stats/s/repo against the 3/s
  the old poll cost.

- Reconciliation treated any late-observed entry create as a root replacement,
  so an ordinary  tore down a healthy stream ~30s later and
  opened a deaf window. It now also requires the root itself to be recreated.

- The shallow watcher recorded directory identity from a stat issued after
  binding, so a replacement in that gap pinned the dead inode's watcher to the
  new identity and the sweep would never rebind. Identity is now read first,
  which errs toward a harmless extra rebind.

Test helper: replacing the worktrees root frees several inodes at once, so
reserving one still let the recreated root reuse its own. It now verifies the
inode actually changed. Confirmed on ext4, where three holds were needed.

(cherry picked from commit e7c5263)
dallascrilley pushed a commit to dallascrilley/orca that referenced this pull request Aug 27, 2026
…uler) (stablyai#16404)

* perf(git): make local Git metadata observation event-driven

Replaces the recurring per-repo metadata scan with native filesystem events on
macOS, Linux, and Windows. Polling is retained purely as a fallback.

- Narrow @parcel/watcher stream over <common>/worktrees, extended from macOS to
  Linux and Windows, with the Windows backend pinned explicitly.
- New shallow watcher mode over the allowlisted primary metadata leaves. It
  watches the containing directory rather than each file, so Git's atomic
  write-and-rename does not orphan the binding.
- Selected upstream refs stay on the existing bounded stat poll.

Verified on real hosts rather than in principle:

- Windows: `git worktree remove` and `git worktree prune` both succeed while the
  narrow stream holds the directory. The historical concern that an open handle
  would block prune does not reproduce.
- Linux: inotify costs one instance per event loop, not one per watch, so the
  watch budget is not a constraint.
- macOS/Linux/Windows: shallow events survive repeated commit, checkout,
  config, and pack-refs cycles.

Failure handling, each reproduced before being fixed:

- fs.watch binds an inode and reports nothing once that inode is replaced, with
  no error. Directory bindings are re-checked on a bounded cadence and rebound.
- A host whose notification path is dead accepts registrations and stays mute
  forever. Observed on a macOS machine whose fseventsd had grown to ~15GB and
  saturated a core. A one-shot delivery probe now fails the shallow subscribe on
  such a host so it falls back to polling instead of showing stale metadata.

This change stands alone on main and does not depend on the metadata poll
scheduler.

* test(git-watch): hold reserved inodes across root replacements

Linux returns a released reservation to the free list, so the second
replacement could land back on the first replacement's inode and look
unchanged to reconciliation. Verified on ext4: releasing yields inodes
[N, N+43, N+43] while holding yields [N, N+43, N+44]. macOS never
recycles, which is why this only failed on CI.

* refactor(git-watch): share one single-flight helper between watcher fallbacks

Both fallbacks tracked their in-flight promise with the same self-comparison
on settle, duplicated verbatim. Hoisting it removes a subtle invariant that
was being hand-maintained in two places.

* fix(git-watch): close the silent-staleness paths in primary metadata

Two independent reviews converged on the same root cause: nothing bounded
how long primary metadata could stay wrong once the shallow watcher stopped
reporting. Four distinct paths led there.

- A terminal watch error arriving while the status-ref poll was still starting
  left the repo with status-ref coverage only. handleWatcherError ran its
  teardown against nulls, then the in-flight poll installed itself, and the
  fallback guard mistook it for coverage and discarded the fallback. Primary
  metadata was then never observed again. The guard no longer treats status-ref
  polling as primary coverage, and startup re-checks watcher liveness after its
  awaits.

- Nothing re-read the six primary files while the watcher was nominally live.
  A lossy notification path, a dropped batch, or inotify queue overflow raises
  no error, so the error-driven fallback never fired and the inode rebind sweep
  does not detect loss. A 15-tick backstop re-stats them, turning permanent
  staleness into one tick. Measured cost is ~0.2 stats/s/repo against the 3/s
  the old poll cost.

- Reconciliation treated any late-observed entry create as a root replacement,
  so an ordinary  tore down a healthy stream ~30s later and
  opened a deaf window. It now also requires the root itself to be recreated.

- The shallow watcher recorded directory identity from a stat issued after
  binding, so a replacement in that gap pinned the dead inode's watcher to the
  new identity and the sweep would never rebind. Identity is now read first,
  which errs toward a harmless extra rebind.

Test helper: replacing the worktrees root frees several inodes at once, so
reserving one still let the recreated root reuse its own. It now verifies the
inode actually changed. Confirmed on ext4, where three holds were needed.
enrick-dev pushed a commit to enrick-dev/orca that referenced this pull request Oct 15, 2026
…uler) (stablyai#16404)

* perf(git): make local Git metadata observation event-driven

Replaces the recurring per-repo metadata scan with native filesystem events on
macOS, Linux, and Windows. Polling is retained purely as a fallback.

- Narrow @parcel/watcher stream over <common>/worktrees, extended from macOS to
  Linux and Windows, with the Windows backend pinned explicitly.
- New shallow watcher mode over the allowlisted primary metadata leaves. It
  watches the containing directory rather than each file, so Git's atomic
  write-and-rename does not orphan the binding.
- Selected upstream refs stay on the existing bounded stat poll.

Verified on real hosts rather than in principle:

- Windows: `git worktree remove` and `git worktree prune` both succeed while the
  narrow stream holds the directory. The historical concern that an open handle
  would block prune does not reproduce.
- Linux: inotify costs one instance per event loop, not one per watch, so the
  watch budget is not a constraint.
- macOS/Linux/Windows: shallow events survive repeated commit, checkout,
  config, and pack-refs cycles.

Failure handling, each reproduced before being fixed:

- fs.watch binds an inode and reports nothing once that inode is replaced, with
  no error. Directory bindings are re-checked on a bounded cadence and rebound.
- A host whose notification path is dead accepts registrations and stays mute
  forever. Observed on a macOS machine whose fseventsd had grown to ~15GB and
  saturated a core. A one-shot delivery probe now fails the shallow subscribe on
  such a host so it falls back to polling instead of showing stale metadata.

This change stands alone on main and does not depend on the metadata poll
scheduler.

* test(git-watch): hold reserved inodes across root replacements

Linux returns a released reservation to the free list, so the second
replacement could land back on the first replacement's inode and look
unchanged to reconciliation. Verified on ext4: releasing yields inodes
[N, N+43, N+43] while holding yields [N, N+43, N+44]. macOS never
recycles, which is why this only failed on CI.

* refactor(git-watch): share one single-flight helper between watcher fallbacks

Both fallbacks tracked their in-flight promise with the same self-comparison
on settle, duplicated verbatim. Hoisting it removes a subtle invariant that
was being hand-maintained in two places.

* fix(git-watch): close the silent-staleness paths in primary metadata

Two independent reviews converged on the same root cause: nothing bounded
how long primary metadata could stay wrong once the shallow watcher stopped
reporting. Four distinct paths led there.

- A terminal watch error arriving while the status-ref poll was still starting
  left the repo with status-ref coverage only. handleWatcherError ran its
  teardown against nulls, then the in-flight poll installed itself, and the
  fallback guard mistook it for coverage and discarded the fallback. Primary
  metadata was then never observed again. The guard no longer treats status-ref
  polling as primary coverage, and startup re-checks watcher liveness after its
  awaits.

- Nothing re-read the six primary files while the watcher was nominally live.
  A lossy notification path, a dropped batch, or inotify queue overflow raises
  no error, so the error-driven fallback never fired and the inode rebind sweep
  does not detect loss. A 15-tick backstop re-stats them, turning permanent
  staleness into one tick. Measured cost is ~0.2 stats/s/repo against the 3/s
  the old poll cost.

- Reconciliation treated any late-observed entry create as a root replacement,
  so an ordinary  tore down a healthy stream ~30s later and
  opened a deaf window. It now also requires the root itself to be recreated.

- The shallow watcher recorded directory identity from a stat issued after
  binding, so a replacement in that gap pinned the dead inode's watcher to the
  new identity and the sweep would never rebind. Identity is now read first,
  which errs toward a harmless extra rebind.

Test helper: replacing the worktrees root frees several inodes at once, so
reserving one still let the recreated root reuse its own. It now verifies the
inode actually changed. Confirmed on ext4, where three holds were needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant