Skip to content

fix(runtime): park a mirrored pane's resume until its PTY handle lands - #19882

Merged
nwparker merged 23 commits into
mainfrom
nwparker/fix-19735-handle-gap-wait
Sep 17, 2026
Merged

nwparker merged 23 commits into
mainfrom
nwparker/fix-19735-handle-gap-wait

Conversation

@nwparker

@nwparker nwparker commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 9 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​1606 0 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​1606
Prod 6 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​535 $\color{#cf222e}{\Huge{\mathbf{−}}}$​34 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​501

Closes #19735

Defect

findUnhydratedHostMirrorForPane consumed hasHostSessionMirrorHydrated as "this pane's liveness is decidable", but it only means "the host's tab rows arrived". The PTY handle lands one relay round trip later. On that frame a published web-terminal-* row with an empty ptyIdsByTabId read as not-live, recordPaneIsOwnedByPreservedPane returned false, and the sweep resumed a session the host was still running — a duplicate resume tab.

Per docs/reference/ssh-execution-boundary.md, an empty handle map for a row the host is still publishing is unverifiable, not exited.

This is distinct from #19736: that fix lives in activeOrQueuedResumeClaimsProviderSession, which only fires when an agent status entry exists for the pane. In this window there is none, so it does not close this case (measured on origin/main and with #19736 applied).

Why not just drop the short-circuit

The hasHostSessionMirrorHydrated short-circuit is load-bearing: without it a pane whose host genuinely holds no PTY parks forever, because the environment park is only replayed by mirror settlement, which has already happened.

Fix

A per-pane handle-arrival wait with a bounded release (host-mirror-handle-gap-wait.ts). Once the mirror is hydrated, a published row whose layout still binds a leaf to a PTY minted by the same environment, and which has no published handle yet, parks on its own waiter. Three exits, each replaying the sweep:

  • the pane's handle lands — ptyIdsByTabId[tabId] becomes non-empty. The replayed sweep then decides on real evidence: a handle for this leaf keeps the pane (no resume); a handle for another leaf of the tab lets the existing resume path fire (the control case is unchanged).
  • the row is retracted — the host has spoken; ordinary recovery owns the pane and the resume fires.
  • the deadline expires — HOST_MIRROR_HANDLE_GAP_DEADLINE_MS (= WEB_SESSION_TAB_RPC_TIMEOUT_MS, 15 s). A handle that has not landed within one RPC budget is not coming on this connection, so the wait records an expired verdict for the pane on the current connection generation and releases it to the resume path. The deadline decides resume, not an indefinite hold. A reconnect bumps the connection generation; a wait armed on the previous one records no verdict and the replay re-parks, so the new connection gets its own full budget (see the correction below).

Re-parking an already-parked pane (a second activation mid-wait) replaces the replay closure but keeps the original deadline, so re-activation cannot extend the wait. The store subscription is armed only while a waiter exists and is torn down on the last release.

Sibling consumers of hasHostSessionMirrorHydrated: the only production consumer is this liveness helper; the settle receipts in mirror-settle.ts write it and were left untouched.

Tests

host-mirror-handle-gap-resume.test.ts keeps the repro's failing case and the control, and adds one case per release path plus the reconnect re-arm and re-park deadline invariants.

Failing repro before the change:

× does not resume a published mirrored pane whose handle has not landed yet
AssertionError: expected 1 to be +0

Mutation testing (one mutation at a time, against this file + the two sibling park tests)

Mutation Result
Remove the handle-lands exit killed: only the two handle-lands cases fail
Remove the retraction exit killed: only the retraction case fails
Deadline never fires killed: only the deadline case fails
Deadline fires but records no verdict killed: deadline + reconnect cases fail
Expired verdict ignores connection generation killed: only the reconnect re-arm case fails
Handle park ignores the environment binding killed: sibling parks the resume for a still-published web-mirror tab fails (pane bound to a foreign PTY must not enter the handle park)
Handle park never taken (pre-fix behaviour) killed: repro + all release-path cases fail
Released waiter not removed before run killed: handle-lands case fails (deadline re-fires)
Re-park extends the deadline survived initially; added keeps the original deadline when a second sweep re-parks the same pane, now killed by exactly that case

No survivors.

Live reproduction

Not reproduced on the real hosts, for the structural reasons already noted on the issue: on direct SSH the pane is an ordinary local row with a live PTY so ownership resolves, and on a paired runtime the client's sleepingAgentSessionsByPaneKey stays empty because the hibernation record is captured by the host. The seam test is the evidence.

Gates

pnpm tc, pnpm test src/renderer/src/lib/ plus the mirror settle/receipt/ordering suites (544 files, 5224 passing), oxlint on the changed files, pnpm run check:code-quality:changed (0 new findings), pnpm format.

Correction after review — adversarial combination testing

The sentence above originally read "A reconnect bumps the connection generation and re-arms a fresh wait." That was only true for a wait that had already expired, which is exactly the case the reconnect test covered. For a wait still in flight it was false, and the gap was a live defect in this PR's own bounded-wait guarantee:

  • A wait armed on generation 0 fires after a reconnect. recordExpiredWait read the current generation, so it stamped the verdict against generation 1. hasHostMirrorHandleWaitExpired then agreed, findUnhydratedHostMirrorForPane returned null, and the pane was resumed after 1 ms on a connection that had had no chance to publish the handle — [Bug]: Opening an unread remote worktree creates a duplicate resume-session tab #19735's fork reached through the guard that exists to prevent it.
  • The mutation table below has an entry "Expired verdict ignores connection generation → killed: only the reconnect re-arm case fails." That mutation and that test both concerned the already-expired case. The test and the doc comment agreed with each other and were both wrong about the live case, which is why review did not catch it.

Fixed in fix(runtime): void a handle-gap verdict the reconnect made stale: the waiter records the generation it was armed on and records no verdict when the generation has moved. Killed by a targeted mutation (drop the generation gate → 1 failure across 99 tests, exactly the new assertion).

Three sibling attacks on the same window are now pinned too, all clean: two panes in one environment where only one handle lands, a handle published by a foreign environment, and an environment tearing its rows down mid-park (no waiter and no scheduled timer left behind).

How to assert on this module, because the obvious shape cannot fail

Recorded as a comment at the top of the test file, and worth repeating here. "Did the waiter release" is not an observable. A waiter released for the wrong reason is immediately re-parked by the replayed sweep, so the store, the record and the parked count all read identically one tick later — a mutation that released every waiter on any tab's handle survived twelve tests written that way. What a spurious release actually costs is the deadline, since the re-park starts a fresh budget. Assertions therefore have to advance the clock: park, advance part of the budget, act, then advance to the original deadline and require the pane to decide on schedule. Rewritten that way, the same mutation dies against exactly the two intended cases.

Also carried here: cross-workspace resume dedup

fix(terminal): a live pane owns its transcript in any workspace.

activeOrQueuedResumeClaimsProviderSession was scoped to the record's own workspace on both terms and additionally required entry.state !== 'done'. A record whose peer pane has finished a turn and still holds a live PTY matched nothing, so the sweep launched a second agent onto a transcript the peer is still writing.

This is reachable, not hypothetical: canonicalizeTerminalSessionWorktreeId re-keys tabsByWorktree, tabGroups, tabGroupLayouts, activeTabIdByWorktree and activeGroupIdByWorktree onto the canonical worktree id and does not re-key sleepingAgentSessionsByPaneKey, whose records carry worktreeId inside them. Adopting an orphaned terminal is a direct producer of the divergence.

Split into two arms rather than widening the existing condition. The unscoped arm demands hard evidence — the entry's exact pane holds a live PTY — on the principle that a provider session id names one transcript, so no workspace boundary makes a live PTY less live. The scoped arm keeps its scope and its state !== 'done' term, because a status row with no live PTY is a claim about the past.

Relationship to #19736 — it supersedes it rather than sitting beside it, and this was measured both ways on this branch, which does not carry #19736. Applying #19736's own parameterised checks exact live ownership before resuming … cases here (one per resumable agent x checkpoint state x live-PTY scenario): all 87 tests in that file pass with this change alone, and 28 fail without it — every ['pty-host-old-1'] scenario, i.e. exactly the cell where the peer pane's own PTY is live, in both the working and done states, for every resumable agent. #19736's one-line export of stablePaneHasLivePty is carried here because the unscoped arm needs it. If #19736 lands first, this becomes a pure widening and its tests should be kept. Both the same-workspace and cross-workspace cells are pinned in resume-sleeping-agent-session-provider-claim.test.ts either way.

Mutations: re-scope the live arm with entry.worktreeId === record.worktreeId → 1 failure, exactly the fork assertion. Drop the stablePaneHasLivePty requirement → kills the counterweight plus 9 pre-existing ownership cases.

Gates for both commits

pnpm tc clean; pnpm test src/renderer/src/lib src/renderer/src/runtime — 717 files, 6698 passing, 0 failures (log grepped, not trusted to exit code); pnpm run check:code-quality:changed — 0 new findings across 6 changed files.


Consolidation pass: eleven more commits folded in (2026-09-11)

A parallel effort had left fixes to this PR's own code sitting on separate branches, so merging this PR would have silently missed them. They are now commits here instead. src/renderer/src/lib/host-mirror-handle-gap-wait.ts does not exist on main — this PR introduces it — so every one of these is a fix to this PR, not a neighbouring concern.

Commit What it closes
isolate one pane's replay from the handle-gap drain one replay throwing stranded every pane queued behind it, and surfaced at the mirror apply's own setState
drain a removed environment's handle-gap verdicts on teardown the only trigger that fires for an environment that will never record again
reconcile three branches' handle-gap verdict rules into one loop three agents had written three different scopes for the same sweep; two of the three shapes were wrong in opposite directions
repair the teardown suite the union broke a break visible only once the verdict rules compose
a handle-gap verdict answers for its pane, not for the tab id a republished tab id inherited its predecessor's verdict and skipped its own wait
the two guards on the park-time binding are not redundant docs
a published handle retires the verdict it answered the next gap on a pane got no wait at all once the first expired
the reused-tab-id class is closed at read time, not still open docs — the docstring had contradicted its own named test for one commit
pin replay containment on the deadline path too the deadline path into releaseWaiter was unpinned
isolate one worktree's replay from the mirror-hydration drain the same containment hole in the sibling drain
pin sleeping-agent resume on a failed SSH target coverage

Double-applies avoided

Two changes on those branches were already on this PR under different SHAs, and re-applying them would have been a regression rather than a duplicate:

  • void a handle-gap verdict the reconnect made stale — patch-id identical to cdf4956d300, already here.
  • a live pane owns its transcript in any workspace — this PR's 16393707da6 is the newer and strictly larger version (it also covers the same-workspace half and carries the stablePaneHasLivePty export). The branch copy is an earlier revision.

Also: contain a parked resume replay that throws (on nwparker/adv3-failure-fixes) is the same fix a second agent found independently — isolate one pane's replay from the handle-gap drain already closes it. Only its extra deadline-path test was taken; its implementation hunk was dropped, so the two spellings of the guard do not ship side by side.

Relay and mobile commits that shared those branches were not folded here — they belong to the relay cluster and are left for it. Two commits touched a relay test as well as this map; the relay halves were dropped and that is recorded in the commit messages.

One commit deliberately left off this PR

an outage is not a handle-gap verdict is a handle-gap fix and by rights belongs here, but it calls runtimeHostConnectionStateForEntry, which #19873 introduces and main does not have. Cherry-picked here alone it turns this suite red with 22 failures from that one missing symbol. Rather than inline a second definition of what a runtime's connection state means, it is #20059, based on this branch and landing after both this and #19873.

A coverage gap the reconcile left, found by mutation

The reconciled verdict loop replaced a retention suite that carried one assertion the split suites did not: the store subscription is held for exactly as long as something needs it. Checked both directions rather than assumed:

  • Retain — drop the verdict term from stopStoreSubscriptionIfIdle, so a verdict with no waiter behind it loses the subscription its drain needs: already caught, 2 failures in host-mirror-handle-gap-landed-handle.test.ts.
  • Release — never release the subscription at all: caught by nothing. That mutation passes all 272 tests across the 33 other handle-gap and session-tabs suites. A leaked subscription rescans every parked pane on every store write for the life of the session and no test notices.

pin the store subscription the reconciled loop can leak closes the release direction. That term is also exactly what the reconcile moved — it now counts verdicts as well as waiters — so it is the part of this map most likely to drift again. Asserted with a spy on useAppStore.subscribe rather than a new test-only export, since the production surface should not grow just to say whether it is subscribed.

Verification of the folded state

  • src/renderer/src/lib/host-mirror*, src/renderer/src/runtime/web-session-tabs-sync*, resume-sleeping-agent*, host-session-mirror-hydration-drain — 334 passed / 42 files, log grepped rather than trusted to the wrapper's exit code.
  • tsc -p config/tsconfig.tc.web.json — 0 errors (and confirmed it is a real config, not the TS5058 "1 error" shape a wrong path produces).
  • Every commit added by fast-forward; nothing on this branch was rewritten or force-pushed.

Rebase record (2026-09-11)

Rebased onto main at 5fa62feda7c264c6eda99ba4788ffebab9aae26c. No conflicts across all 17 commits.

SHA
old head 1200ad221b139f745df27f4cc95a6f75f74df26c
new head cb29f42c1e703f363973d751f4e4c61860410ce7

#20059 stacks on this branch and was replayed onto the new head.


Rebased onto main @ caa465d1da886b2697c85c18bf36f79f2d3b9e68 (2026-09-15).

  • Old head: cb29f42c1e703f363973d751f4e4c61860410ce7
  • New head: e652df229548fda5f37b3d2f961a5698bfcf24b1

Content unchanged apart from one added commit, fix(lint): carry SAFETY rationales for the handle-gap fixtures. The casting gate is changed-lines-only and reads its rule config from the merge base, so this branch's old base skipped it entirely and its previous green did not cover these 26 sites. Verified with git range-diff; prior CI results predate this rebase.


Adversarial re-audit (2026-09-16) — 5 commits added

Re-audited from the head above, in repeated design/implement/review/verify rounds, with 29 mutations run per round and three independent adversarial reviews. Every mutation is killed; the two rounds of review that produced findings are recorded below, including the ones that landed on this audit's own commits.

One real defect in this PR's new module

releaseDueWaiters snapshotted the due keys and then re-looked-up each one. A replay earlier in the loop writes to the store (the sweep reaches createTab), and zustand notifies re-entrantly with no queue, so the nested pass can release and re-park a pane still queued in the outer loop. Two things then go wrong, and neither guard covers the other:

  • the outer releaseWaiter(key) finds the re-park and replays it a second time off one store write — a whole worktree resume sweep run twice on one frame;
  • parkUntilHostMirrorHandleLands re-parks a still-parked pane by mutating the waiter in place, so worktreeId can move (adoption re-keys the rows) while object identity survives. The snapshot's "retracted" verdict is then about a workspace the waiter is no longer filed under.

Both directions were conservative — hold longer, never resume early — which is why no outcome assertion could see either. Fixed by snapshotting the waiter alongside its key, then re-judging it against the live store before releasing.

The re-judge deliberately reads useAppStore.getState() rather than the notified state, and no test can tell the difference: the two agree on every sequence the sweep can produce, because a replay's only store write is createTab, which appends a freshly minted tab id. They are kept apart because state is the staler read and its failure direction is to release a pane whose row has come back — resolving unverifiable to exited. That reasoning is recorded at the call site so it is not "simplified" away.

Assertions that could not fail

  • returns to baseline under churn across all three drains asserted nothing. It ran 300 expiries and then cleared every environment's verdicts by name before counting, so the map was empty by construction — deleting the entire prune loop in recordExpiredWait left it green. It now asserts the bound before the teardown clear (300 expiries must leave one live verdict per environment, measured: exactly 3), and each round binds its pane to the environment recording it, since the old fixture stored the empty match value on two rounds in three. Renamed: there are four drains, not three.
  • host-mirror-handle-gap-replay-containment.test.ts was the same fix imported twice. Its deadline case was a strict subset of the drain suite's; its store-write case differed only by also asserting later store listeners still run. One mutation — rethrowing from the replay catch — killed all five cases across both files. Its unique assertion moved into the drain suite and the file was dropped: 83 lines, no coverage lost. Its fixture also seeded no layout bindings and used tab ids isWebTerminalSurfaceTabId rejects, so those panes could never have reached the park path it claimed to exercise.
  • Two fixtures asserted on a store shape the mirror apply cannot produce — ptyIdsByTabId[tab] holding a PTY id no leaf of that tab binds. terminal-build.ts builds one from the other, so they cannot disagree. Both now seed producible shapes.

A residual this PR does not close, now visible instead of hidden

The decidability gate above this wait is tab-granular while everything below it is leaf-aware. Those two impossible fixtures were standing in for a real gap: on a split mirrored tab's first frame, a sibling surface that reaches ready publishes a handle for the tab while this surface is still pending-handle with no binding to retain, the pane reads decidable, and the sweep resumes it. That is #19735's own shape, narrowed to split tabs at cold start.

It is not closable inside this module — such a leaf holds no binding, and the binding is what names a pane in a handle-gap verdict — and it is not a regression: the gate is untouched by this PR, and single-leaf tabs (every agent tab Orca creates) are exact. It is a pre-existing gap whose root cause is a model gap, not a missing mechanism: the host publishes status: 'pending-handle' per surface and the client consumes it without retaining it per leaf, so liveness has to be inferred from an empty handle map plus a timer. Recorded at the gate and pinned by name; tracked separately.

Also recorded, at the point where a reader would mistake it for a regression: the unscoped live-PTY arm ignores entry.state, so a finished agent whose shell is still up releases its record and will not auto-resume. That is the intended side of the trade — a live PTY is positive evidence the host holds the transcript, nothing is killed, and forking a transcript is not recoverable while declining to auto-resume is.

Gates (measured, logs grepped for FAIL rather than trusting exit codes)

  • vitest run src/renderer/src/lib src/renderer/src/runtime — 747 files, 7035 passed, 1 skipped, 0 failures.
  • Targeted handle-gap / resume / liveness / hydration-drain tree — 17 files, 105 passed.
  • 29 mutations, no survivors, each killed with a tight blast radius; the two new guards are killed by exactly one case each and do not overlap.
  • pnpm tc — 0 errors, and the gate was proven live by planting a type error and watching it fail before the clean run was trusted.
  • oxlint, oxfmt --check, pnpm run check:code-quality:changed — 0 new findings across 15 changed files.
  • Playwright electron-headless, real Electron: agent-session-quit-resume.spec.ts and finished-agent-ghost-resume.spec.ts — 2 passed, re-run on each head.

Every commit added by fast-forward; nothing on this branch was rewritten or force-pushed.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 8d8c46ad-6879-4898-9aa7-f839aca92535

📥 Commits

Reviewing files that changed from the base of the PR and between cb29f42 and e652df2.

📒 Files selected for processing (9)
  • src/renderer/src/lib/host-mirror-handle-gap-drain.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-landed-handle.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-replay-containment.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-resume.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-subscription-lifetime.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-teardown.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-verdict-union.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-provider-claim.test.ts
  • src/renderer/src/lib/ssh-failed-target-sleeping-agent-resume.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The change adds pane-aware parking for panes whose host-mirrored tab row exists before its PTY handle. Resume sweeps now distinguish mirror hydration from handle gaps. Provider-session checks now recognize live PTYs across workspaces. Tests cover handle-gap release paths, cleanup, replay containment, SSH terminal states, and provider-session claims.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: 🟡 Moderate · up to e652d

Split terminals can bypass the intended bounded recovery wait, and suppressed resumes can unexpectedly navigate. Address both recovery-path regressions before merging.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 39.02% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 41 functions across 16 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR satisfies the coding requirements in issue #19735. The handle-gap wait prevents resume while a published pane lacks its PTY handle and releases on handle arrival, row removal, reconnect, or dea…
Out of Scope Changes check ✅ Passed The source changes and tests remain within issue #19735. Waiter cleanup, pane identity checks, replay isolation, hydration handling, SSH terminal-state controls, and subscription lifecycle management …
Title check ✅ Passed The title clearly and concisely describes the primary change: parking a mirrored pane's resume until its PTY handle arrives.
Description check ✅ Passed The description is substantially complete and directly covers the defect, user impact, implementation, rationale, linked issue, testing, and verification. It does not follow every template heading and…
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

A focused runtime fix that closes the mirror handle-gap window (#19735): a paired-runtime mirrored pane's resume no longer fires on the frame where the host's tab rows have arrived but its PTY handle has not.

  • host-mirrored-pane-liveness.ts — findUnhydratedHostMirrorForPane returns a discriminated UnhydratedHostMirror (mirror vs handle). The new handle kind is gated on the layout still binding a leaf to this environment's remote PTY, so foreign/local bindings never enter the park.
  • host-mirror-handle-gap-wait.ts (new) — a per-tab park with a bounded setTimeout deadline tied to WEB_SESSION_TAB_RPC_TIMEOUT_MS, three replayed exits (handle lands / row retracted / deadline), connection-generation-scoped expiry, and a store subscription armed only while a waiter exists.
  • resume-sleeping-agent-session.ts — the park helper generalizes to parkWorktreeResumeSweepUntilHostMirrorAnswers, with the replay extracted to replayParkedWorktreeResumeSweep.
  • host-mirror-handle-gap-resume.test.ts (new) — the failing repro plus the control and one case per release path, plus the re-park deadline and reconnect re-arm invariants.

The three-exit structure and the environment-binding guard are the two load-bearing pieces, and both are pinned by tests that can genuinely fail (the repro asserts launched === 0 where pre-fix it was 1, and the sibling parks the resume for a still-published web-mirror tab case kills an ignored-binding mutation). The 15 s deadline-decides-resume tradeoff is documented and defensible given mirror settlement will not replay a parked sweep. Local pnpm test on the new + sibling park suites and pnpm tc:web both pass.

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) | 𝕏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 6c79a333-ec90-4d6d-88be-426bdfa51679

📥 Commits

Reviewing files that changed from the base of the PR and between f2d5711 and 1b531f8.

📒 Files selected for processing (4)
  • src/renderer/src/lib/host-mirror-handle-gap-resume.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-wait.ts
  • src/renderer/src/lib/host-mirrored-pane-liveness.ts
  • src/renderer/src/lib/resume-sleeping-agent-session.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 5 remain after this review.

Comment on lines +164 to +166
resumeSleepingAgentSessionsForWorktree(worktreeId, {
...(options?.onSessionLaunched ? { onSessionLaunched: options.onSessionLaunched } : {}),
...(isActive ? {} : { suppressNavigation: true })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Preserve an explicit suppressNavigation request.

When the worktree is active, this reconstruction removes options.suppressNavigation. A delayed replay can then navigate even when the caller requested suppressed navigation.

Combine the original flag with the inactive-worktree condition:

-    ...(isActive ? {} : { suppressNavigation: true })
+    ...(options?.suppressNavigation || !isActive ? { suppressNavigation: true } : {})
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
resumeSleepingAgentSessionsForWorktree(worktreeId, {
...(options?.onSessionLaunched ? { onSessionLaunched: options.onSessionLaunched } : {}),
...(isActive ? {} : { suppressNavigation: true })
resumeSleepingAgentSessionsForWorktree(worktreeId, {
...(options?.onSessionLaunched ? { onSessionLaunched: options.onSessionLaunched } : {}),
...(options?.suppressNavigation || !isActive ? { suppressNavigation: true } : {})

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Two commits landed since the prior review, both against the runtime resume path this PR fixes:

  • Generation-scoped handle-gap expiry — HandleGapWaiter now records the connection generation it was armed on, and the deadline callback only records an expired verdict when that generation still equals the current one. A wait armed milliseconds before a reconnect no longer authorizes a resume on the new connection; it releases without a verdict so the replay re-parks with the new connection's full budget.
  • Cross-workspace live-PTY ownership — activeOrQueuedResumeClaimsProviderSession split a workspace-unscoped arm off the existing scoped arm: a pane whose exact PTY is live (stablePaneHasLivePty, now exported) owns its provider session regardless of which workspace it sits in, dropping the state !== 'done' requirement for that arm. The scoped arm keeps its scope and its done term.
  • Tests — one case per new path: reconnect-before-deadline re-arm, dual-pane partial handle release, foreign-environment handle isolation, teardown-mid-park leaving no timer, and three cross-workspace provider-claim cases.

The mid-wait reconnect hole the first commit closes was a genuine defect in the bounded-wait guarantee, and the fix is exactly the read-the-generation-later shape its test pins (killed by a targeted mutation, per the PR body). The second commit's two-arm split is well-motivated: the unscoped arm demands hard evidence (the entry's exact pane holds a live PTY) while the scoped arm keeps its boundary because a status row with no live PTY is a claim about the past. Local pnpm test on the resume/ownership suites (48 passing) and pnpm tc:web both pass.

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) | 𝕏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: fe6383ce-97ce-48e1-90bb-df00f298e9cd

📥 Commits

Reviewing files that changed from the base of the PR and between 1639370 and 6c7df35.

📒 Files selected for processing (10)
  • src/renderer/src/lib/host-mirror-handle-gap-drain.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-landed-handle.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-replay-containment.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-teardown.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-verdict-union.test.ts
  • src/renderer/src/lib/host-mirror-handle-gap-wait.ts
  • src/renderer/src/lib/ssh-failed-target-sleeping-agent-resume.test.ts
  • src/renderer/src/runtime/host-session-mirror-hydration-drain.test.ts
  • src/renderer/src/runtime/host-session-mirror-hydration.ts
  • src/renderer/src/runtime/web-session-tabs-sync/tracking-lifecycle.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.

typeof ptyId === 'string' && parseRemoteRuntimePtyId(ptyId)?.environmentId === environmentId
)
.sort()
.join('')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Use an unambiguous encoding for paneBinding.

Line 118 removes the boundaries between PTY identifiers. For example, bindings ending in ["a", "bc"] and ["ab", "c"] produce the same string.

If a tab is rebound between colliding multi-leaf layouts, hasHostMirrorHandleWaitExpired can accept the previous pane’s verdict. The new pane then skips its bounded wait and can resume a duplicate session.

Preserve each identifier boundary. Add a regression test with two colliding binding sets.

Proposed fix
     )
     .sort()
-    .join('')
+    .map((ptyId) => `${ptyId.length}:${ptyId}`)
+    .join('')
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
.join('')
.map((ptyId) => `${ptyId.length}:${ptyId}`)
.join('')

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Twelve commits landed since the prior review (1639370): the consolidation pass that folds a parallel effort's fixes to this PR's own code back into the branch. The verdict map grew pane identity and four drains were reconciled into one loop.

  • Reconciled the verdict drains into one loop — recordExpiredWait now applies a generation rule (judged per key, across every environment) and a tab-death rule (recording environment only) with deliberately different scopes, closing two opposite wrong-scope shapes that had been written independently.
  • Closed the reused-tab-id class at read time — expiredGenerationByPane carries the pane's park-time PTY binding, and hasHostMirrorHandleWaitExpired compares it against the pane's current binding, so a republished tab id bound to a freshly minted PTY does not inherit its predecessor's verdict.
  • Added a published-handle drain — retireVerdictsWithLandedHandles retires a verdict once its pane's handle lands, so the next gap on that pane gets its own full wait rather than none.
  • Contained replayed throws — both releaseWaiter and drainParkedWaiters wrap run() in try/catch, so one pane's throwing replay neither strands its sibling panes nor escapes the mirror apply's setState.
  • Counted verdicts in the subscription lifetime — stopStoreSubscriptionIfIdle keeps the store subscription alive for a verdict with no waiter parked behind it and releases it once both are gone.
  • Moved the waiter's worktree with run — re-parking an orphaned-terminal-adopted pane updates worktreeId, so retraction evidence about the old workspace cannot release the live wait.
  • Drained removed-environment verdicts on teardown — clearHostMirrorHandleGapVerdictsForEnvironment wires into session-tabs tracking teardown.
  • Pinned the SSH failed-target resume gate — ssh-failed-target-sleeping-agent-resume.test.ts covers resuming once a sync terminates in offline/error without hydrating, while still declining while unverifiable.

The design is defensible and the mutation-testing evidence in the PR body is credible: the paneBinding join is unambiguous in practice (handles are encodeURIComponent-encoded, so the remote:<env>@@ delimiter cannot appear inside them), and every release/retain path is pinned by a test that can genuinely fail.

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) | 𝕏

@nwparker
nwparker force-pushed the nwparker/fix-19735-handle-gap-wait branch from 1200ad2 to cb29f42 Compare September 11, 2026 08:17
Mirror hydration means the host's tab rows arrived, not that a given pane's
liveness is decidable: the PTY handle lands one relay round trip later. On
that frame the pane read as not-live and the sweep resumed a session the
host was still running, producing a duplicate resume tab.

An empty handle map for a published row is unverifiable, never exited. Park
the pane on a per-pane wait with three bounded exits, each replaying the
sweep: its own handle lands, the row is retracted, or a deadline expires.
The deadline decides resume rather than an indefinite hold, and is scoped
to the connection generation so a reconnect re-arms it.

Closes #19735
The per-pane park bounds itself with one deadline per connection, but the
waiter never recorded WHICH connection it was armed on. A wait armed on
generation 0 that fires after a reconnect stamps its expiry against the
current generation, so hasHostMirrorHandleWaitExpired agrees, the mirror
lookup returns null, and the pane is resumed after 1ms on a connection
that has had no chance to publish the handle. That is #19735's fork with
an extra step, reached through the guard that exists to prevent it.

The module's own doc comment claims the opposite -- "a reconnect bumps
the connection generation and arms a fresh wait" -- and that is true only
for a wait which had ALREADY expired, which is precisely the case the
existing test covered. The test and the comment agreed with each other
and both were wrong about the live case.

The waiter now carries the generation it was armed on and records no
verdict when the generation has moved; the replay re-parks through the
existing machinery and the new connection gets its own full budget. Still
bounded per connection generation, which is what was documented all along.

Also pins the three sibling attacks on the same window: two panes in one
environment where only one handle lands, a handle published by a foreign
environment, and an environment tearing its rows down mid-park (which
leaves no waiter and no scheduled timer).

The test file now leads with how to assert on this module at all, because
the obvious shape cannot fail. "Did the waiter release" is not an
observable here -- a waiter released for the wrong reason is re-parked by
the replayed sweep, so the store reads identically one tick later, and a
mutation releasing every waiter on any tab's handle survived twelve
assertions written that way. What a spurious release costs is the
deadline, so the assertions advance the clock and require the pane to
decide on the ORIGINAL schedule.
The resume dedup was scoped to the record's own workspace on both terms
-- the entry's tab had to be in worktreeTabIds AND entry.worktreeId had
to match -- and additionally required entry.state !== 'done'. A record
whose peer pane has finished a turn and still holds a live PTY therefore
matched nothing, and the sweep launched a second agent onto a transcript
the peer is still writing. Cross-workspace, it matched nothing even while
the peer was mid-turn.

The two ids really do drift. canonicalizeTerminalSessionWorktreeId
re-keys tabsByWorktree, tabGroups, tabGroupLayouts, activeTabIdByWorktree
and activeGroupIdByWorktree onto the canonical worktree id, and does NOT
re-key sleepingAgentSessionsByPaneKey, whose records carry worktreeId
inside them. So adopting an orphaned terminal is a direct producer of a
record naming one workspace while its pane and status row name another.

Split into two arms rather than widening the existing condition. The new
arm carries no workspace scope but demands hard evidence: a provider
session id names one transcript, so a pane whose exact PTY is live right
now already owns it wherever that pane sits, and no workspace boundary
makes a live PTY less live. The scoped arm keeps its scope and its
state !== 'done' term, because a status row with no live PTY is a claim
about the past and must not reach across workspaces.

Relationship to #19736: that PR fixes the SAME-workspace half of this in
the same function, by relaxing only the status term. This arm covers that
cell too -- measured both ways on this branch, which does not carry
#19736: its thirty `checks exact live ownership before resuming` cases
all pass with this change alone, and ten of them fail without it. So this
supersedes #19736 rather than sitting beside it, and #19736's one-line
`export` of stablePaneHasLivePty is carried here because this arm needs
it. If #19736 lands first this becomes a pure widening and its tests
should be kept. Both cells are pinned here either way.
One store write releases every due pane, and the drain runs synchronously inside
a zustand subscriber. `waiter.run()` was unguarded, so a single pane's replay
reached two things it has no business touching:

  - the throw escapes out of `useAppStore.setState`, meaning the mirror apply
    that published the PTY handle throws at its own call site;
  - every pane queued behind the thrower is stranded — waiter still parked,
    deadline still armed — and then decides on a connection whose evidence
    landed long ago.

The deadline path fans out the same way, so a throwing replay also escaped the
timer callback.

Reachable: `resumeSleepingAgentSessionsForWorktree` reaches `state.createTab`
with no guard of its own. The panes in a drain are strangers to each other and
to the frame that released them; none of them should be able to see another's
failure.

The new tests live in their own file because
host-mirror-handle-gap-resume.test.ts drives the waiter through the real resume
sweep and so cannot choose what a replay DOES. Note for anyone extending that
file: per its header, "did the waiter release" is not an observable here — a
spurious release is re-parked immediately and reads identically one tick later.
These tests assert on timer count and on the deadline instead.

Also records two findings next to the code, so they are not rediscovered:
`expiredGenerationByPane` is never pruned for a removed environment (bounded and
inert, since removal advances the generation, but it does not drain — and a
DIFFERENT leak in that same map is being fixed concurrently, so reconcile rather
than patch around it); and sustained reconnect churn holding a pane parked
indefinitely is CORRECT, not the latch-that-never-releases defect, because under
churn liveness genuinely is unverifiable and ssh-execution-boundary.md forbids
resolving that to `exited`. It has the shape of the defect and will eventually
be "fixed" by someone who does not know that.

Mutation: dropping the guard kills exactly the three new assertions and leaves
all twelve existing waiter tests passing.
…ardown

`expiredGenerationByPane` is pruned only by rules that run when a verdict is
RECORDED — the stale-generation sweep here, and the tab-death sweep added
separately (8f16641, env-scoped in c0e4423). An environment that is
REMOVED records nothing ever again, so neither rule can reach its rows and they
survive for the life of the session. Two orphan classes on one map; neither
prune subsumes the other, because both are driven by a recording.

Severity is a leak, not a correctness bug, and the commit pins WHY so nobody
re-derives it: removing an environment advances its connection generation, so a
stranded verdict can never match again even if the id returns. That test exists
to stop the generation advance being "optimised" away later, since it is the
only thing making the stranded row inert.

Hung off `clearWebSessionTabsTrackingForEnvironment` because that is the only
caller that fires for an environment that is going away.

Clears VERDICTS ONLY. Parked waiters deliberately survive, matching
`clearHostSessionMirrorHydration`: a re-pair or effect restart replaces the
connection's evidence, it does not cancel the recovery this client still owes
the pane. A waiter left behind is bounded by its own deadline and replays its
sweep exactly as it would have. Clearing them here would silently drop a parked
resume that nothing else replays.

A measurement worth recording, because it argued me out of a change I was about
to make: on the unfixed map the per-expiry rescan is super-linear — 500/1000/
2000/4000 sequential expiries cost 7.2/15.3/51.8/173.1 ms, doubling ratios
converging on ~3.35 against 4.0 for quadratic. That looked like a case for
reshaping the map to `Map<env, {generation, Set<tabId>}>`. It is not: the
quadratic is a property of the LEAK, not of the scan. Once the tab-death prune
holds the map at roughly one entry per environment the scan is over ~1 entry,
and a counting probe on the fixed map (summing `map.size` across N expiries,
which IS the iteration count and needs no clock) gives exactly N-1 — linear, and
2000x fewer iterations than quadratic at N=4000. The flat prefix loop used here
is the established pattern in this subsystem and needs no restructure.

Two methodology traps this cost, recorded for the next person measuring in this
repo: `vi.useFakeTimers()` fakes `process.hrtime` and `performance.now` as well,
so a timing harness reports the advanced deadline rather than work done — fake
only the timer surface under test. And expiring N panes in one burst measures
the fake-timer harness clearing N timers, not product code; 1000 panes "cost"
~1s that way and almost none of it was ours.

Mutations: a clear that drops nothing kills exactly the two assertions that
claim it drains, and correctly leaves the waiter-survival and generation-advance
tests passing. An UNSCOPED clear kills the same two, via their sibling-
environment half.
… one loop

Three agents changed `recordExpiredWait` on three branches and each verified only
their own. This is the union, resolved into the agreed shape and proved on one
tree. The rules are NOT alternatives — they have different safety properties, and
flattening them to one scope is wrong in both directions. Both wrong shapes were
independently written before this was reconciled, so the comments say why.

GENERATION rule, per key across EVERY environment (adv2-skew's class).
`hasHostMirrorHandleWaitExpired` compares a row against its own environment's
CURRENT generation, so a row whose generation has moved can never return true for
anybody; retiring it cannot cost a reader a verdict, whoever owns it. Scoped to
the recording environment, an environment that reconnects and then goes quiet
strands its rows forever.

TAB-DEATH rule, recording environment ONLY (my class). Row absence is transient
where a generation is not: a sibling mid-republish has no rows for a frame and
would lose a verdict its pane still needs — reproduced before it was narrowed.

Teardown drain (adv2-races' class) is unchanged and orthogonal: it is the only
trigger that fires for a REMOVED environment, whose rows no rule above reaches
because such an environment records no further verdict. Right predicate, wrong
trigger.

The union suite proves all four orphan classes simultaneously, plus the two
properties none of the three rules may break: the verdict stays sticky enough to
break the park/expire/replay loop, and no rule evicts a verdict a live pane still
needs. It uses three environments throughout, because with two at one generation
the candidate rules are indistinguishable and the naive fix survives.

THE FOURTH CLASS IS UNOWNED AND ASSERTED AS A HAZARD. A retracted tab id that is
republished inherits the old pane's verdict and skips its own wait. Unlike every
other gap on this map it is NOT conservative: the others drop a verdict and
re-park, holding longer, while this one retains a verdict and resumes on a handle
that has not landed — the #19735 direction. No rule reaches it: the tab-death
predicate stops matching once the id is republished, the teardown drain fires on
environment teardown rather than tab retraction, and no waiter exists to observe
the retraction because a pane holding a verdict never parks. Closing it needs a
fourth trigger, on row retraction. The suite pins the current behaviour so it
cannot be quietly forgotten.

Union finding, recorded rather than merged: adv2-skew's
`docs(relay): the live-broker wait budget does not bound the call` (6b02982)
is SKIPPED here. It documents the unbounded wait, and adv2-concurrency-fixes
(1673716) fixed exactly that by extracting the loop into
relay-live-broker-wait.ts. The doc and its test pin behaviour the union no longer
has. This is the kind of interaction neither branch could see alone.
Cherry-picked from 46ad377 with the relay half dropped: that commit
also repaired relay-concurrency-policy-flip-mid-mint.test.ts, which does
not exist on this PR and belongs with the relay cluster's own branch.

The handle-gap half is what this PR needs. Neither break was visible on
its own branch -- both only appear once the verdict rules compose.
…tab id

Folds adv2-skew's e8cac05 into the reconciled union. Closes the fourth orphan
class, the only one that was not conservative: a retracted tab id republished as a
different pane inherited the old pane's verdict and skipped its own wait — the
#19735 direction rather than a longer hold.

It needs no fourth trigger, which is why it composes with the three drains rather
than competing with them. Every trigger those rules own fires downstream of the
moment this hazard needs. The verdict instead carries the environment-minted PTY
binding its pane held AT PARK TIME, and only answers for a pane that still holds
it: a republished pane binds a newly minted PTY and serves its own wait, while a
genuine reattach to the same PTY inherits, which is correct — the verdict follows
the PTY, not the id. A transient rowless frame touches neither, so the read-time
check is safe where a retraction-triggered prune would not have been.

TWO MEASUREMENTS, both requested rather than assumed.

1. The record-then-release ordering is load-bearing and IS pinned. `recordExpiredWait`
reads the waiter's park-time binding, so it must run before `releaseWaiter` deletes
the entry. Swapping the two statements fails three cases, so the capture is not
correct merely by accident of statement order.

2. The `''` fallback is a MATCH VALUE, not a null: two panes that both hold no
environment-minted PTY compare equal and inherit, which is the same hazard in a
narrower window. Measured unreachable through the production park path rather than
assumed — the only route in is `kind: 'handle'`, which `findUnhydratedHostMirrorForPane`
reports only when `tabHoldsEnvironmentPtyBinding` finds a binding, reading the SAME
map through the SAME predicate as `paneBindingFor`. It now refuses to answer anyway.
That coupling is two functions in two files with nothing enforcing it, refusing costs
only a re-park, and the direction is conservative.

THE REFUSAL IS WHAT FOUND THE REAL BUG. With `''` matching, any fixture that omits
`terminalLayoutsByTabId` records `''`, compares `'' === ''`, and passes while the
pane-identity check is entirely inert. Making it refuse turned that silence into
four failures across host-mirror-handle-gap-drain and -teardown, whose fixtures seed
no layout binding at all. Both now bind per environment — one shared environment id
filters every other environment's pane back to `''` and restores the no-op.

Mutation-tested on the merged tree: ignoring the binding fails case D and the
mid-wait case; re-reading at expiry fails the mid-wait case and nothing else;
letting `''` match fails the empty-binding case; widening the tab-death rule across
environments still fails the live-verdict case, so pane identity does not weaken the
scoping the sweep was reconciled around.

Also fixes a real-clock race this branch introduced: the revoke-window test read
`Date.now()` separately from `enqueue`'s own stamp, and under load the drift ate
into the window. It now anchors the injected clock to the item's `createdAt`.
Recording a reconciliation result that existed only in a review thread, and
correcting it in the process — measuring the claim changed it.

The claim under review was that the `?? ''` fallback in `recordExpiredWait` is
unreachable by two independent guards, either sufficient alone: the caller's
generation gate (a missing waiter fails `undefined === number`) and the
record-before-release ordering. That is not what the code does.

Measured, by removing each in turn:

  - ordering removed, generation gate kept: the gate does NOT carry it. With the
    waiter already deleted, the gate is false on every expiry, so nothing is ever
    recorded — five failures, and the door is shut by breaking the mechanism rather
    than by refusing ''.
  - generation gate removed, ordering kept: 736 files green, one failure, and it is
    `does not let a wait armed on the previous connection decide the new one` in
    host-mirror-handle-gap-resume.test.ts — a different property entirely.

So the ordering alone makes `''` unreachable, and the generation gate is not a second
guard on it at all: it pins reconnect-void. Both are load-bearing, for different
reasons, which is a stronger argument against removing either than redundancy would
have been — redundancy invites deleting one.

Worth writing in the file because the two sit three lines apart and read as belt and
braces on the same thing. The `''` comment next to them already exists because an
unexplained guard on an unreachable value gets deleted as dead code in a year; a guard
that looks redundant is deleted sooner.

No behaviour change. One comment, corrected against measurement rather than against the
thread it came from.
The fourth eviction trigger on `expiredGenerationByPane`, and the reason it is
not redundant with the three already there or with the two other agents' guards
on this same map.

A verdict records that a pane's 15s handle-gap wait ran out. Nothing retires it
when that pane subsequently publishes its handle, so the NEXT gap on that pane
gets no wait at all — #19735 with the bounded wait removed rather than merely
shortened. Measured on the reconciled union tree (1b621b6) plus the outage
guard: the verdict still answered `true` after the handle landed, and the second
gap resumed with zero panes parked.

Why none of the existing rules reach it, each checked rather than assumed:
  - superseded generation: #19647 in this same stack stops recording
    `status: null` for an unreachable host, so the generation no longer moves
    across an outage on one runtime.
  - dead tab row: the row stays published throughout. It is the HANDLE that
    comes and goes — that is the definition of the gap.
  - removed environment: the environment is still here.
  - read-time pane identity (adv2-skew, cdafc90): the pane keeps the same
    layout binding across the gap BY DESIGN, and the union suite pins that a
    genuine reattach to the same PTY must inherit. That check discriminates a
    different pane behind one tab id; this one discriminates a later gap on the
    same pane.
  - contact lost (adv3-journeys, 2da6624): no outage is involved; this is a
    healthy connection where the host was simply slow once.

Composition proven by mutation on the union tree, four disjoint kills: dropping
this drain kills 2 tests and only mine; dropping the re-park worktree kills 1 and
only mine; dropping the contact guard kills 1 and only theirs; forcing the
contact guard always-true kills 16 across every suite. No mutation kills another
agent's test, so these are three guards on three holes, not three on one.

Also carries `worktreeId` across a re-park: adopting an orphaned terminal re-keys
`tabsByWorktree` without re-keying the record, so a live wait kept releasing on
retraction evidence about the workspace it was no longer about. The park-time
`paneBinding` deliberately does not move with it — that is the pane's identity,
this is only where its rows are filed.
…ill open

The `ExpiredHandleGapVerdict` docstring told the next reader that a retracted
tab id republished under the same id still inherits its predecessor's verdict,
and that closing it "needs a fourth trigger, on row retraction". The test it
names as its own pin says the opposite: class D in
host-mirror-handle-gap-verdict-union.test.ts asserts the verdict does not
answer, and explains it is closed at READ time rather than by any prune.

Provenance, since two sources disagreeing is what made this expensive: the
paragraph was last written in 46ad377 and the read-time pane-identity check
landed one commit later in cdafc90 (adv2-skew). The prose predates its own
fix by a single commit and was never updated. Confirmed by mutation rather than
by reading: dropping `verdict.paneBinding === paneBindingFor(...)` fails exactly
"handles all four orphan classes simultaneously", which is the class-D
assertion, so the read-time check is what closes it.

Rewritten to say what the code does, keeping the part that was always true —
why no trigger could have reached that class — and keeping the distinction the
new PUBLISHED HANDLE drain needs: read-time identity separates two panes behind
one tab id, the drain separates two gaps on one pane. The drain does not close
class D and must not be read as closing it.

Also records why this block specifically keeps going stale: several agents
change this map in parallel, the invariants move faster than the prose, and when
the two disagree the test file is the one that ran.
Cherry-picked from aa98edf (nwparker/adv3-failure-fixes) with its
implementation hunk dropped: a second agent found the same throwing-replay
hole independently, and `15f34014153` already closed it on this branch with
an equivalent guard. Applying both would have been a double-apply, and the
two spellings of the log line would have shipped side by side.

The tests are worth keeping regardless. The first duplicates coverage
15f3401 already has; the second does not -- it drives the throw from the
DEADLINE path rather than the store-write path, which is a separate call into
releaseWaiter and was unpinned.

Spies retargeted from console.error to console.warn, the channel the guard
that actually shipped writes to, so the suite silences what the code emits.
… drain

The same fan-out hazard as the handle-gap drain, one module up. Settling an
environment drains every worktree parked on it in a single loop, called from the
frame apply, with `waiter.run()` unguarded. One replay that throws strands every
waiter queued behind it and surfaces in the caller applying the frame.

Found by looking for the sibling of a defect rather than by a separate
interleaving: both modules park a `run` callback and drain N of them from one
event, so both have the same blast radius. Kept as its own commit because the
two modules route independently.

Mutation: dropping the guard kills exactly the one new assertion.
The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (#16750), and this only closes a coverage gap in it.
The retention suite that `reconcile three branches' handle-gap verdict rules
into one loop` replaced carried an assertion the split suites did not: the
store subscription is held for exactly as long as something needs it.

Measured before writing it, because half of it was already covered:

  RETAIN direction -- drop the verdict term from `stopStoreSubscriptionIfIdle`
  so a verdict with no waiter behind it loses the subscription its drain needs:
  already caught, 2 failures in host-mirror-handle-gap-landed-handle.test.ts.

  RELEASE direction -- never release the subscription at all: caught by
  NOTHING. That mutation passes all 272 tests across the 33 other handle-gap
  and session-tabs suites. A leaked subscription rescans every parked pane on
  every store write for the life of the session and nothing notices.

So this is for the release direction. The retain cases ride along because both
halves of one invariant belong in one file, not because they were missing.
That term is also precisely what the reconcile moved -- it now counts verdicts
as well as waiters -- so it is the part of this map most likely to drift again.

Asserted with a spy on useAppStore.subscribe rather than a new test-only
export: whether the module is subscribed is already observable at the store
boundary, and the production surface should not grow just to say so.
main tightened typescript/consistent-type-assertions to assertionStyle:
never after this branch was written. The gate only ran here once the
rebase put the casting config at the merge base, so these sites are new
to it, not new to the branch. The store seeds are genuinely partial --
dropping the casts does not typecheck -- so each carries its rationale.
@nwparker
nwparker force-pushed the nwparker/fix-19735-handle-gap-wait branch from cb29f42 to e652df2 Compare September 15, 2026 10:16
`releaseDueWaiters` snapshotted the due KEYS and then re-looked-up each one. A
replay earlier in the loop writes to the store — the sweep reaches `createTab`
and `clearSleepingAgentSession` — and zustand notifies re-entrantly with no
queue, so the nested pass can release and re-park a pane still queued in the
outer loop. The outer `releaseWaiter(key)` then found the re-park, cleared its
brand-new deadline and replayed it a second time off one store write, handing
that pane another full budget.

That is the extension `parkUntilHostMirrorHandleLands` already refuses to grant
a re-park, arriving through a different door. The direction is conservative
(hold longer, never resume early), which is why no outcome assertion could see
it; only the replay count separates the two implementations. Snapshot the waiter
alongside its key and release only while the map still holds that same waiter.

Also folds host-mirror-handle-gap-replay-containment.test.ts into the drain
suite, since the guard it pins is the one this commit extends. It was the same
fix imported twice: its deadline case is a strict subset of the drain suite's,
its store-write case differs only by also asserting that later store listeners
still run, and one mutation — rethrowing from the replay catch — killed all five
cases across both files. Its fixture also seeded no layout bindings and used tab
ids `isWebTerminalSurfaceTabId` rejects, so those panes could not have reached
the park path it claimed to exercise. The unique assertion moves across; the
file goes.

Killed by `releases a pane once per store write even when an earlier replay
re-enters the drain`: 2 replay calls instead of 1 without the identity guard.
`returns to baseline under churn across all three drains` asserted nothing. It
ran 300 expiries and then cleared every environment's verdicts by name before
counting, so the map was empty by construction — deleting the whole prune loop
in `recordExpiredWait` left the test green. It now asserts the bound BEFORE the
teardown clear: 300 expiries must leave exactly one live verdict per
environment. It also binds each round's pane to the environment recording it;
the old fixture filed every binding under env-a, so two rounds in three stored
the empty match value the read-time check refuses, and that much of the churn
was synthetic. Renamed: there are four drains, not three.

Two comments described outcomes their assertions do not produce. `c1` reads
false on the read-time generation gate alone, whether or not a drain ever swept
it — the count below is the only assertion that distinguishes retired from
stranded. And the discriminator in `never evicts a live pane verdict` is env-b,
whose row goes absent while env-a records; env-c is a control that holds under
every candidate rule.

Two more fixtures modelled states the mirror apply cannot produce, both leaning
on `ptyIdsByTabId[tab]` holding a PTY id no leaf of that tab is bound to. It
builds one from the other (web-session-tabs-sync/terminal-build.ts), so they can
never disagree. The producible shape is a SPLIT tab whose sibling surface went
`ready` first, which is what both cases now seed — and which makes the residual
they were quietly standing in for visible instead: the decidability gate above
this wait is tab-granular while everything below it is leaf-aware, so a sibling
handle ends the wait for a surface still `pending-handle`. Recorded at the gate
in host-mirrored-pane-liveness.ts, pinned by name, and left open here: it needs
the per-surface status the host already publishes and the client drops on apply.

Also states what the unscoped live-PTY arm trades — a finished agent whose shell
is still up releases its record and will not auto-resume — because it reads as a
regression and is not one. And replaces the subscription-lifetime header's
unreproducible "272 tests across 33 suites" with the measured 326 across 37.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Two commits landed since the prior review (e652df2): a fix to this PR's own drain loop and a test-suite pass that retires assertions which could not fail.

  • Released a due pane once per store write — releaseDueWaiters now snapshots the waiter object alongside its key and releases only while the map still holds that same waiter. A replay earlier in the loop writes to the store, zustand notifies re-entrantly, and the nested pass can release-and-re-park a pane the outer loop still has queued; releasing by key would then clear the re-park's fresh deadline and replay it a second time off one store write, granting the budget extension parkUntilHostMirrorHandleLands deliberately refuses.
  • Corrected the churn test that asserted nothing — returns to baseline under churn across all three drains cleared every environment's verdicts before counting, so deleting the prune loop left it green. It now asserts one live verdict per environment before the teardown clear, and binds each round's pane to the environment recording it (the old fixture filed two rounds in three under env-a, storing the empty match value the read-time check refuses).
  • Replaced two unreachable fixtures with the producible shape — a tab holding a ptyIdsByTabId handle that no leaf binds cannot be produced by the mirror apply, which builds the handle map from the layout's ptyIdsByLeafId. Both cases now seed a split tab whose sibling surface went ready first, exposing the real residual: decidability is tab-granular while everything below the wait is leaf-aware, so a sibling handle ends the wait for a surface still pending-handle. Documented at the gate in host-mirrored-pane-liveness.ts and pinned by name as current behaviour.
  • Folded the replay-containment suite into the drain suite — the removed file's unique assertion (a later store listener still runs) moved across; its store-write and deadline cases were subsets of the drain cases, and its fixture could not exercise the park path it claimed.
  • Stated the unscoped live-PTY trade-off and measured the subscription claim — the entry.state-ignoring arm deliberately declines to auto-resume a finished agent with a live shell (a live PTY is positive evidence; forking a transcript is not recoverable), and the subscription-lifetime header's unreproducible test count was replaced with the measured figure.

The drain fix is the only behavioural change: it is bounded (a re-park always arms a timer) and conservative (hold longer, never resume early), and its identity guard is killed by exactly the one new replay-count assertion. No open pullfrog threads on any prior review, so nothing to retire.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

The identity guard added one commit ago catches only half of how the drain's
snapshot goes stale. It proves the map entry was not REPLACED; it cannot prove
the verdict still holds, because `parkUntilHostMirrorHandleLands` re-parks a
still-parked pane by MUTATING the waiter in place. `worktreeId` moves with `run`
— that is what adopting an orphaned terminal does — and object identity survives
it. So a waiter the snapshot judged retracted, because its tab was absent from
the worktree it was filed under, can be re-filed by an earlier replay in the
same loop and then released on evidence about a workspace it is no longer about.
That is the defect the `existing.worktreeId` assignment exists to prevent,
reached through the drain instead.

Neither guard covers the other: re-judging alone still replays a re-park twice
(it was just made due, so it re-judges due), and identity alone misses the
mutation. Both, in that order. A waiter that is no longer due simply stays
parked — bounded by its own deadline and re-judged on the next store write, so
declining costs at most one frame of latency.

Killed by `does not release on retraction evidence a mid-drain re-park has
already made stale`; the identity half is still killed by `replays a pane once
per store write even when an earlier replay re-enters the drain`. 29 mutations
across these modules, no survivors.

Corrects three claims made in the two preceding commits, each wrong in a way a
future reader would have acted on:

- the duplicate release does NOT extend the pane's budget. `releaseWaiter`
  deletes the waiter before calling `run`, so the re-park takes the `!existing`
  branch and arms a full deadline either way; a second release clears and re-arms
  at the same instant. What it costs is running an entire worktree resume sweep
  twice off one frame. The test is renamed to say so.
- `retainPendingTerminalBindings` carries a `pending-handle` surface's prior
  binding forward, so the split-tab residual cannot arise from a bound leaf going
  pending — it needs a leaf that was NEVER bound, which means a cold start or
  re-pair with no layout to retain from. The fixture staged the impossible
  history; it now seeds the producible first frame, where the point sharpens: no
  wait is armed at all, which the test now asserts directly.
- `clearSleepingAgentSession` cannot re-enter the drain; the subscription's slice
  guard drops that write. Only `createTab` can.

Also: the churn assertion pins that the prune loop runs at all, not which rule
prunes — with one live tab per round either rule alone still reads 3. Says so,
and points at the case that does isolate the generation rule.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

One commit landed since the prior pullfrog review (9f52dbb): a bounded fix to the handle-gap drain loop that closes the second way its snapshot can go stale mid-iteration.

  • Re-judge a due waiter against fresh state — releaseDueWaiters now applies waiterIsReleased against useAppStore.getState() after the existing identity guard. parkUntilHostMirrorHandleLands re-parks a still-parked pane by mutating the waiter in place, so worktreeId can move (orphaned-terminal adoption) while object identity survives; a waiter the snapshot judged retracted is now skipped rather than released on evidence about a workspace it is no longer filed under. The guard only ever narrows (fresh state can flip due→not-due, never the reverse), so the direction is conservative.
  • Corrected three claims the two preceding commits had wrong — the duplicate release re-arms the deadline rather than extending it (what it costs is a second full sweep); the split-tab residual needs a leaf that was never bound, because retainPendingTerminalBindings carries a prior binding through a pending-handle frame; and only createTab (not clearSleepingAgentSession) can re-enter the drain through the subscription's slice guard.
  • Reseeded the residual fixture on the producible frame — seedSplitTabWithOnlySiblingReady now seeds the cold-start first frame and asserts directly that no wait is armed for that pane, and the new drain case does not release on retraction evidence a mid-drain re-park has already made stale pins the new guard.

Independently verified: disabling the second guard makes exactly that new case fail (adoptedReplay called once); with it restored, the seven handle-gap/hydration-drain suites pass 36/36. The retainPendingTerminalBindings doc correction matches the implementation at terminal-build.ts:23-52, so the reseeded fixture models a state the park path can actually reach.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

…h it

The case added one commit ago pinned the right guard through a call sequence
production cannot make. A replay is `resumeSleepingAgentSessionsForWorktree`
closed over ONE worktree, so it re-parks only under that worktree — but the
fixture had the first pane's replay re-park the second under a DIFFERENT one.
That is the same fault the previous commit corrected in the resume fixture, made
one file over.

The reachable route: the second tab has already been re-keyed onto the canonical
worktree id while its sleeping record still names the old one, so the first
pane's sweep legitimately owns it and re-parks it there, mutating the live
waiter in place. The waiter's snapshot verdict — "retracted", because the tab is
absent from the id it was filed under — is stale by the time the loop reaches
it. Restaged that way, with both waiters and the re-park inside one worktree.

Also drops the second global store read. Guard TWO now re-judges against the
same `state` the drain was notified with, because the two provably cannot differ
here: a re-park only happens when `findUnhydratedHostMirrorForPane` finds the
row already filed under the sweeping worktree, which is a row this frame carries.
`useAppStore.getState()` was an unpinnable degree of freedom — swapping it for
`state` left every test green — and it contradicted the drain's own claim to
judge one snapshot.

And corrects two more claims: guard ordering is a cost preference, not a
correctness requirement (either order works; identity first is just cheaper),
and "no wait is armed here" in the split-tab fixture is not caused by the cold
start — it is the tab-granular gate reading the sibling's handle, which is the
residual itself.

29 mutations across these modules, no survivors. Each guard is killed by exactly
one case and they do not overlap: `replays a pane once per store write even when
an earlier replay re-enters the drain` for identity, `does not release on
retraction evidence a mid-drain adoption has already made stale` for re-judging.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

️ No critical issues — minor suggestions inline.

Reviewed changes

One commit landed since the prior pullfrog review (8427968), a test-and-doc pass over the handle-gap drain:

  • Restaged the mid-drain adoption fixture — does not release on retraction evidence a mid-drain adoption has already made stale now re-parks the adoptable pane under the sweeping worktree itself and is named to match, and the case comment claims this is the production route.
  • Dropped the drain's second global store read — releaseDueWaiters' re-judge guard now judges the subscriber's state snapshot instead of useAppStore.getState().
  • Corrected two module claims — the split-tab residual's "no wait armed" cause in the resume fixture, and the guard-ordering rationale.

Both the state swap and the restaged fixture were run past independent review lenses; neither found a production-correctness break, but both doc claims are narrower than they read.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Comment on lines +151 to +157
// Staged the only way production can reach it. A replay is
// `resumeSleepingAgentSessionsForWorktree` closed over ONE worktree and re-parks only under that
// worktree, so a waiter's worktree can only move when a DIFFERENT waiter's replay sweeps the
// workspace the row was adopted into. Here the second tab has already been re-keyed onto the
// canonical id — `canonicalizeTerminalSessionWorktreeId` re-keys `tabsByWorktree` and leaves the
// sleeping record naming the old one — so the first pane's sweep legitimately owns it, while the
// live waiter is still filed under the id its record named.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reachability claim here is contradicted by the sweep it names. resumeSleepingAgentSessionsForWorktree keeps only record.worktreeId === worktreeId records and parks under that same worktree, while findUnhydratedHostMirrorForPane looks the row up under record.worktreeId. The comment's premise — the record still names the old id while the row moved to the canonical one — is exactly what stops the canonical sweep from iterating that record, so it cannot re-park the pane there, and the old-id sweep sees no row to park. The fixture drives run directly, so neither filter is exercised.

Technical details
# The restaged fixture's reachability claim

## Affected sites
- src/renderer/src/lib/host-mirror-handle-gap-drain.test.ts:151-157 — "Staged the only way production can reach it" route claim
- src/renderer/src/lib/host-mirror-handle-gap-drain.test.ts:173-183 — the synthetic `run` that re-parks `SECOND_TAB_ID` under `ADOPTING_WORKTREE_ID`
- src/renderer/src/lib/resume-sleeping-agent-session.ts:235-237, :270 — the sweep filters `record.worktreeId === worktreeId`, then parks under that same id
- src/renderer/src/lib/host-mirrored-pane-liveness.ts:55-58 — row lookup under `record.worktreeId`
- src/main/runtime/workspace-session-worktree-id.ts:3-37 — adoption re-keys tabs/groups only, never `sleepingAgentSessionsByPaneKey`

## Required outcome
- Either the comment describes this as a synthetic drain-level regression guard (the drain cannot see sleeping records, so it is pinned against an in-place `worktreeId` mutation), or the fixture stages a re-park the sweep can actually emit.

## Suggested approach (optional)
- Keep the test — it does kill an in-place `worktreeId` mutation — but correct the mechanism note: the sweep's record filter plus adoption's non-re-keying means a single pane's waiter cannot move between worktrees.

## Open questions for the human (optional)
- Is there a producer of the in-place `worktreeId` move this analysis misses (e.g. the tab-granular `paneWaitKey` letting a second leaf's record re-park the same key)? If not, guard TWO may be dead code, and defensive-only is the honest framing.

Reverts the `state` read the previous commit put in guard TWO, and says why the
difference is deliberately unpinnable rather than leaving the next reader to
"simplify" it back.

The previous commit swapped `useAppStore.getState()` for the subscriber's
`state` because a reviewer noted the swap left every test green. That was
optimising for mutation-killability over the property the module exists to
protect. The two reads do agree on every sequence the sweep can produce — a
replay's only store write is `createTab`, which appends a freshly minted tab id,
so it can neither make an absent tab id present nor touch `ptyIdsByTabId` — which
is exactly why no test separates them. But they are not interchangeable: `state`
is the staler of the two, and its failure direction is to RELEASE a pane whose
row has come back. That is resolving unverifiable to exited, which is #19735.

Holding on evidence that might be stale costs one frame; acting on it forks a
transcript. Take the fresher read, and record in the comment that no test can
fail on this and why that is not a reason to change it.

Both guards remain killed by exactly one case each and they do not overlap:
`replays a pane once per store write even when an earlier replay re-enters the
drain` for identity, `does not release on retraction evidence a mid-drain
adoption has already made stale` for re-judging.

Verified while confirming the previous commit's test premises against production,
both of which hold: a sweep that parks every record writes nothing to the store
(resume-sleeping-agent-session.ts takes `continue` on the park branch), and
`workspace-session-worktree-id.ts` moves `tabsByWorktree` onto the canonical id
while leaving `sleepingAgentSessionsByPaneKey` naming the old one — the stale
`worktreeId` the adoption case depends on.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

One commit landed since the prior pullfrog review (3f707bd): it reverts the guard-TWO read a prior review prompted and records why the difference between the two reads is deliberately unpinnable.

  • Restored the fresh store read in guard TWO — releaseDueWaiters judges the re-parsed waiter against useAppStore.getState() again, so retraction evidence a mid-drain re-park has already made stale can never release a pane whose row has come back. The comment now states that the two reads agree on every sequence the sweep can produce — a replay's only write is createTab, which mints a fresh tab id and never touches ptyIdsByTabId — and that the fresh read is kept anyway because state is the staler one whose failure direction is the unsafe one, with an explicit note not to "simplify" it back.

I verified the reachable-write claim against the sweep: resumeSleepingAgentSessionsForWorktree takes continue on the park branch, so a sweep that parks every record writes nothing, and clearSleepingAgentSession touches a slice the subscription's guard drops. The swap is conservative in both directions and the comment now matches the code.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Opening an unread remote worktree creates a duplicate resume-session tab

1 participant