Skip to content

Let worktree creation proceed during stale preparation reclamation - #18967

Merged
nwparker merged 6 commits into
mainfrom
nwparker/unblock-create-from-stale-reclamation
Sep 6, 2026
Merged

nwparker merged 6 commits into
mainfrom
nwparker/unblock-create-from-stale-reclamation

Conversation

@nwparker

@nwparker nwparker commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 4 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​430 $\color{#cf222e}{\Huge{\mathbf{−}}}$​158 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​272
Prod 3 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​43 $\color{#cf222e}{\Huge{\mathbf{−}}}$​36 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​7

Stale preparations left by a crashed process could delay a new checkout until old files were removed. Preparation now proceeds after the shared host scan while bounded reclamation continues in the background. Cleanup stays tracked until it settles, so subsequent requests share the scan and repo maintenance remains gated.

Removal also uses one locked-worktree removal command instead of unlocking first. If the removal command cannot start, the ownership lock survives so the preparation remains identifiable; each discard uses one fewer Git subprocess. This does not guarantee ownership or registration survives arbitrary failures inside Git.

Stacked on #18951. Ownership selection, preservation of branch-attached workspaces, the four-worker cleanup limit, and native/WSL routing remain intact. SSH and folder workspaces remain excluded by the existing preparation-pool guard. No wire, mobile-facing schema, dependency, or UI change. Double-force locked removal is documented and implemented in Git 2.25.5; the real-binary CI contract now tests it explicitly.

Validation:

  • 60 focused tests pass across the preparation registry, real Git, WSL root routing, base OIDs, and maintenance gating.
  • The stalled-removal regression fails against the previous scheduling implementation and passes here. It covers shared scans, native/WSL host-key isolation, and settling cleanup during reset.
  • A production-path real-Git test creates and finalizes a clean worktree on the expected branch while stale removal is held before runner invocation. Releasing removal leaves exactly the main and new checkout registered.
  • A real-Git removal-launch-failure test reproduces lost ownership with the old unlock-then-remove sequence, verifies the lock/content remain with direct locked removal, and verifies subsequent successful removal.
  • 18 binary compatibility tests pass on local Git 2.55.0. The baseline/intermediate/current CI matrix passed on Git 2.25.5, 2.38.1, and 2.49.1.
  • Node typecheck, lint, formatting, and diff checks pass. Independent review and elegance passes found no required correction.

Assessed risk: low. New materialization can overlap stale cleanup in disk usage and I/O; nearly full or slower drives and live Windows/WSL concurrent removal remain unverified. No stronger eventual-cleanup guarantee is introduced. A separate paired-order real-Git scheduling probe exercised simultaneous removal/reset on this Mac, but substantial load variability makes it supporting evidence only; this PR does not claim a universal latency improvement.

This follow-up is included in the combined trial below and remains unmerged.

CI follow-up: the previous run reproduced expired fixed-date artifact fixtures (unrelated to worktree changes) after 2026-09-06 UTC. Reused upstream main commit 71f2c5d (#18955), test-only, rather than introducing a separate fix; all 34 affected artifact tests pass locally. One native-chat waitFor timeout from that run did not reproduce locally. Current-head full CI result is recorded below.

Full current-head CI passed at 986ebfe7248, including all eight test shards, Linux/Windows packaging, typecheck, static analysis, and Git compatibility.

Current composer-free trial: v1.4.198-adhoc.20260906024508, source 6d989819c5ef5d85b8253d763450de9df4c88d57, includes current main 681119dc05b (including A), cancellation #18951, background stale cleanup #18967, and fetch/checkout overlap #18998. Parked draft composer #18876 is excluded.

Combined full typecheck, 60 focused tests, and background Electron existing Create flow passed. Downloaded ARM64 DMG passed SHA512/size, extraction identity, exact source/version, code signature and notarization. The actual background packaged app passed immediate Create, prepared Create, cancellation without additional workspaces/terminals, and real shell input/output. No draft editor appeared. Owned workspaces were clean and removed, terminal inventory returned to zero, the owned app stopped, and its remaining owned detached preparation was removed.

On the 23,608-file fixture: immediate Create took 7.317 s to observed selection; a second Create after nine seconds of composer dwell took 683 ms. Backend spans were 6.644 s and 515 ms. These are individual potentially warm-cache samples, with a blank terminal and no setup hook. They demonstrate the prepared path, not a universal cold speedup or agent readiness. Screenshots inspected. Windows release packaging is still running; no live Windows/WSL UI validation.


Post-review measurement (readiness review, 2026-09-05)

The original scheduling probe (20.51 s → 17.02 s, ~17%) was a single sample under variable machine load. Re-measured with noise control: an interleaved ABABAB design, 3 rounds × 5 iterations, n = 15 per cell, on a 10,000-file repo with N dead-owner stale preparations seeded. Baseline = the three touched source files swapped to origin/main content; branch = HEAD. Git 2.55.0, macOS, load average 15–40 throughout (the interleaving cancels it).

Condition Metric Baseline (main) This branch
3 stale checkouts present startPreparation → ready 1653 ms (1562–2094) 1165 ms (1100–1562)
through finalizePreparedWorktree 1888 ms (1680–2190) 1331 ms (1216–1787)
0 stale checkouts (control) ready 841 ms (768–927) 849 ms (756–1090)
finalize 1061 ms (911–1186) 1096 ms (916–1224)
  • With stale checkouts: −488 ms ready and −557 ms finalize, both −30%. The distributions do not overlap at the median (branch max ≈ baseline min).
  • Without stale checkouts: no regression — the +8 ms / +35 ms deltas sit well inside the spread, with fully overlapping ranges.

The win is conditional on stale checkouts existing and scales with their size and count; it should not be quoted as a universal number. The original 17% figure is directionally consistent but under-measured.

Background-deletion vs concurrent-adopt: audited, no data-loss race found

The top risk for this change is the background reclaimer deleting a directory a concurrent Create just adopted. It cannot:

  1. Reclaim requires a lockOwnerPid parsed from the Git lock reason and !isProcessAlive(lockOwnerPid) (worktree-create-preparation-stale-cleanup.ts:48-52). Live preparations embed process.pid, so the current process's own entries always fail the dead-pid check.
  2. Removal additionally requires pathOwnerPid === lockOwnerPid (:57) — the directory must still sit at .orca-preparing/<pid>-<uuid>. An adopted worktree has already been moved out by finalizePreparedWorktree, so pathOwnerPid is null and only the unlock branch can apply. Never a delete.
  3. A concurrent preparation adds its worktree unlocked first and locks last, so between add and lock it carries no preparation lock reason and isWorktreeCreatePreparation filters it out of staleWorktrees entirely. The scan is also taken before the new checkout starts.
  4. The reclaimer only acts on paths present in that scan result; it never enumerates the directory. A second live Orca's preparations have their own live pid and are skipped.

Exercised by the real-Git test that holds a dead-owner removal mid-flight while a new preparation is created and finalized (worktree-create-preparation-real-git.test.ts:91-155).

Verification

oxlint and oxfmt --check clean on all 6 touched files; 42/42 on the two preparation suites, 369/369 across a 39-file sweep; real-binary Git compat suite 18/18 on Git 2.55.0.

Counsel validation

Four-seat multi-model counsel ran in findings-severity mode; three seats reported (one aborted on context length). All three independently rated the three candidate findings false-positives and dropped them, so nothing is release-blocking:

  • C1 (claimed P0, background reclaimer deletes a live/adopted worktree) — dropped. Discard requires a parsed lock pid that is dead and pathOwnerPid === lockOwnerPid; live preparations stamp process.pid on both path and lock; the scan snapshot is taken before mkdir/worktree add, so it cannot contain the current create's path; an adopted worktree sits outside .orca-preparing, making pathOwnerPid null so only unlock can apply. PID reuse skips reclaim (a leak, never a delete).
  • C2 (claimed P1, unhandled rejection / stuck / unbounded map) — dropped. cleanup.catch(() => {}).finally(...) with an identity guard, await scanned.catch, per-discard .catch, bounded WORKTREE_REMOVAL_REGISTRATION_TIMEOUT_MS, and a map single-flighted per host key.
  • C3 (claimed P1, remove --force --force compat) — dropped. Double-force locked removal is the documented form on the Git 2.25 baseline, matching the existing move -f -f contract; windowsLongPathGitArgs only prepends -c core.longpaths=true and does not rewrite subcommand flags. Compat CI covers 2.25.5 / 2.38.1 / 2.49.1.

Additional P2 observations raised during counsel, none blocking:

  • The real-Git reclamation test gates remove before the underlying call, so add/finalize complete first; a genuinely concurrent worktree add/move versus remove --force --force is still untested. A lock clash would fall back to a normal add, and that fallback path is unproven.
  • Joiners skip a fresh scan until background removals settle, so newly stale entries can wait up to 30s x ceil(n/4).
  • retryPendingPreparationDiscards runs only on a new cleanup, not on joiners; after three attempts, failed discards leak until process exit and are then reclaimed by dead-pid cleanup.
  • Pre-existing: listWorktreeGraph softens a failed listing to [] (worktree-listing.ts:41-42), so a failed enumeration is indistinguishable from "no stale checkouts". Rescanned on the next prepare.

@nwparker
nwparker marked this pull request as ready for review September 6, 2026 01:11

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the four commits in this PR, which split stale-preparation reclamation from the shared host scan so a new checkout no longer waits for old file removal, and switch discard to a single locked-removal command.

  • Stale reclamation no longer blocks checkout — startStalePreparationCleanup now returns after the shared listWorktreeGraph scan while the 4-worker discard loop runs in the background; the in-flight entry is cleared only once reclamation settles, so hasPendingStalePreparationCleanup (and idle repo maintenance gating) stays true through reclamation — unchanged duration, just the create no longer awaits it.
  • Single locked-worktree removal — performDiscardPreparedWorktree replaces unlock-then-remove with git worktree remove --force --force, preserving the ownership lock when removal cannot start. Double-force locked removal is documented at the Git 2.25.0 baseline, matching the existing move -f -f assumption.
  • Test coverage — adds a stalled-removal regression (unit + real-Git) proving creation proceeds and the scan is shared per host key, a real-Git removal-launch-failure test proving lock/content survive, and a Git binary-compatibility case for double-force removal; also fixes expired artifact share fixtures (expiresAt → Date.now() + 30d).

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) | 𝕏

@nwparker
nwparker changed the base branch from nwparker/cancel-obsolete-worktree-preparation to main September 6, 2026 05:17
Resolves #18951 squash-merge conflicts by keeping the backgrounded stale
cleanup alongside main's discard-retry review fix. Moves the checkout
cancellation tests into worktree-create-preparation-cancellation.test.ts
to keep the registry suite under the test-file line limit.
@nwparker
nwparker merged commit a63a457 into main Sep 6, 2026
27 checks passed
@coderabbitai

coderabbitai Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The changes update prepared worktree discard to use git worktree remove --force --force. Stale cleanup now separates worktree scanning from background reclamation and provides asynchronous test reset handling. Preparation pool startup uses the new cleanup API. New tests cover removal failures, stalled stale cleanup, cancellation causes, retry suppression, TTL expiry, caller abort propagation, and locked worktree removal.

Merge Risk: 🟡 Moderate · up to 05e79

Repository maintenance may overlap an active worktree removal, risking conflicting Git operations. Track retry completion before merging.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 7 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the primary change: allowing worktree creation to continue while stale preparation reclamation runs.
Description check ✅ Passed The description is detailed and covers the change, rationale, linked issue reference, testing, risks, compatibility, and validation results. It does not follow every template heading exactly and omits…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 1b48f03e-423b-48f3-a34c-05d8225d9229

📥 Commits

Reviewing files that changed from the base of the PR and between e2270fe and 05e79e3.

📒 Files selected for processing (7)
  • src/main/git/worktree-create-preparation-real-git.test.ts
  • src/main/git/worktree-create-preparation.ts
  • src/main/worktree-create-preparation-cancellation.test.ts
  • src/main/worktree-create-preparation-pool.ts
  • src/main/worktree-create-preparation-stale-cleanup.ts
  • src/main/worktree-create-preparation.test.ts
  • src/shared/git-binary-compatibility.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 3 remain after this review.

...options,
includeCreatePreparations: true
})
void retryPendingPreparationDiscards(cleanupKey)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Track pending discard retries in settled.

retryPendingPreparationDiscards(cleanupKey) starts discardPreparedWorktree() through discardPreparationWithRetry(). Its promise is tracked by the test-only inFlightDiscards registry, but not by entry.settled. After scanned resolves, another preparation can start. The cleanup finally block can then remove staleCleanupInFlight while the retry is still running, so hasPendingStalePreparationCleanup() releases maintenance gating early. resetStalePreparationCleanupForTests() also waits only for scan cleanup.

Include the retry promise in settled. Add a gated-retry test that keeps the stale-cleanup reset pending until the retry settles.

Proposed fix
-  void retryPendingPreparationDiscards(cleanupKey)
+  const retries = retryPendingPreparationDiscards(cleanupKey)
   const scan = listWorktreeGraph(repoPath, {
     ...options,
     includeCreatePreparations: true
   })
@@
-  const entry = { scanned, settled: cleanup }
+  const settled = Promise.allSettled([retries, cleanup]).then(() => {})
+  const entry = { scanned, settled }
   staleCleanupInFlight.set(cleanupKey, entry)
-  void cleanup
+  void settled
     .catch(() => {})
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
void retryPendingPreparationDiscards(cleanupKey)
const retries = retryPendingPreparationDiscards(cleanupKey)
const scan = listWorktreeGraph(repoPath, {
...options,
includeCreatePreparations: true
})
const settled = Promise.allSettled([retries, cleanup]).then(() => {})
const entry = { scanned, settled }
staleCleanupInFlight.set(cleanupKey, entry)
void settled
.catch(() => {})

nwparker added a commit that referenced this pull request Sep 6, 2026
Resolve squash-rewrite conflicts from #18951/#18967: keep main's
cancellation test split and fs.watch null-filename fix, plus this
branch's fetch/checkout overlap real-git tests.
enrick-dev pushed a commit to enrick-dev/orca that referenced this pull request Oct 15, 2026
…tablyai#18967)

* Stop obsolete worktree preparations when evicted or expired

* Let worktree preparation proceed during stale reclamation

* Verify creation during stalled stale worktree reclamation

* Preserve preparation ownership until Git removal starts

* test: keep artifact share fixtures unexpired across calendar dates (stablyai#18955)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant