Repository navigation
fix(remote): stop a colliding path key, a dead conflict state, and a live-PTY removal from losing tabs - #17948
Conversation
📝 WalkthroughWalkthroughThe renderer now reports unplaced terminal worktree paths and retries snapshot synchronization after placement. Placement resolution supports exact target IDs and configurable timeouts. Terminal recovery retains and rebinds handles when PTY IDs change. Remote session projection merges colliding worktree rows by tab ID and preserves valid active tab metadata. Tests cover deferred re-pulling, PTY rebinding, and projection merging. Merge Risk: 🟡 Moderate · up to Remote workspace recovery can leave a target in conflict without attempting recovery when a new host snapshot overlaps a stalled placement retry. This should be resolved before merge to avoid tabs remaining unrecovered after placement becomes available. 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
Full details: Description checkExplanation The description provides detailed context, rationale, linked issue references, implementation details, testing results, and documented follow-up work. It does not include the template headings or the required visual-proof statement, but the content is otherwise substantially complete. Full details: Out of Scope Changes checkExplanation Several production and test changes address objectives outside the only directly linked issue,
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Team
Run ID: 5a539505-47df-4c8c-a2bd-acf6207556bd
📒 Files selected for processing (12)
src/renderer/src/hooks/remote-workspace-deferred-placement-retry.tssrc/renderer/src/hooks/remote-workspace-snapshot-apply.tssrc/renderer/src/hooks/remote-workspace-snapshot-placement.tssrc/renderer/src/hooks/remote-workspace-snapshot-unplaced-tab-adoption.test.tssrc/renderer/src/hooks/remote-workspace-target-sync-types.tssrc/renderer/src/hooks/remote-workspace-target-sync.test.tssrc/renderer/src/hooks/remote-workspace-target-sync.tssrc/renderer/src/runtime/web-session-terminal-orphan-recovery-inventory.tssrc/renderer/src/runtime/web-session-terminal-orphan-recovery-regressions.test.tssrc/renderer/src/runtime/web-session-terminal-pending-handle-recovery.test.tssrc/shared/remote-workspace-session-projection.test.tssrc/shared/remote-workspace-session-projection.ts
Included review availability: Your plan provides up to 10 included reviews per hour; 6 remain after this review.
There was a problem hiding this comment.
ℹ️ No critical issues — one minor robustness nit inline.
Reviewed changes
- Session projection union —
exportRemoteWorkspaceSessionnow unions tab rows across local keys that collapse onto one host path (deduped by tab id) instead of last-writer-wins, and a colliding key'snullactive tab can no longer erase a real one (#15484). - Deferred placement retry — a new 10-minute catalog watch re-pulls a fresh host snapshot once previously-unplaceable paths become placeable, giving
conflictan exit that a bare reconnect or unsolicited push used to be required for (#12902). - PTY-mismatch retention — a pending terminal surface whose listed handle is live under a different PTY is now retained/rebound rather than removed, since the fresh liveness listing is host attestation of life, not death (#11495).
- Type/helper extraction — the target-sync API/deps/result types moved to
remote-workspace-target-sync-types.ts, andexactTargetWorktreeIdsbecameresolveExactDirectSshTargetWorktreeIdsinremote-workspace-snapshot-placement.ts.
DeepSeek Pro (free via Pullfrog for OSS) | 𝕏
…s, and PTY rotation Three losses in the "remote session state never reconciled" cluster, one rule: absence from a client-side set, or a stale client-side expectation, is `unverifiable` by construction and can never authorise removal. #12902 / #15484 — a direct-SSH snapshot whose host paths the local worktree catalog cannot place yet leaves the target in `conflict`, which suppresses uploads and holds terminal authority at `unverifiable`. Nothing re-pulled once the catalog landed, so the tabs stayed missing and the host ledger stayed stale until a reconnect. The apply now reports the paths it dropped and target-sync watches the catalog for them, re-pulling a fresh host snapshot when they become placeable. #15484 — exportRemoteWorkspaceSession keys the host projection by worktree path, which drops the repoId, so two local rows for one remote checkout collapsed and the last one won outright. An empty duplicate row published an empty tab list for a workspace with live panes, and the upload is a wholesale replace-session. Union by tab id instead, matching the `Math.max` its sibling recency map already applied to the same collision. #11495 — orphan recovery retired a leaf whenever a `terminal.list` with `requireFreshPtyLiveness: true` named a different PTY behind a handle than the snapshot frame's pending row did. That is the host attesting the handle is live under a replacement PTY, which is what a host relaunch looks like. Rebind instead of remove. Two tests pinned the removing behaviour and are retargeted with the reasoning.
4997a71 to
8e4484e
Compare
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
- Rejection handling — the deferred-retry IIFE now wraps its body in
try/catch, so agetSnapshot/applySnapshotrejection (relay drop) is swallowed and leaves the target onconflict, matching the IPC bridge's convention instead of surfacing an unhandled promise rejection. - Retry-chain bound — a per-target
chainDepthmap plus anapplyingset cap the arm → apply → re-arm chain atMAX_DEFERRED_PLACEMENT_CHAIN = 3. A watch armed from within the retry's own apply increments the chain, while a fresh arrival resets it to zero, so a path that stays unplaceable across re-pulls is left onconflictrather than looping forever. - Tests — four new tests cover both rejection paths, the chain bound (asserting exactly three pulls), and that out-of-retry watches don't consume the chain budget.
DeepSeek Pro (free via Pullfrog for OSS) | 𝕏
|
Rebased onto current
Blocker: missing
|
…retry chain
The deferred placement retry ran its body as `void (async () => { try {…}
finally {…} })()` with no `catch`. `getSnapshot` is an IPC call that rejects
when the relay drops, and `applySnapshot` can reject with it, so a dropped relay
produced an unhandled rejection in the renderer. Swallow it: the module already
documents that a pull which fails is `unverifiable` and the target is left on
`conflict`.
The retry also re-armed itself through `applyUnsolicitedSnapshot` with no cycle
bound, next to a sibling loop capped at MAX_SNAPSHOT_APPLY_ATTEMPTS = 3. When an
apply reports still-unplaced paths that the catalog nonetheless reports
placeable, `waitForSnapshotWorktreePlacement` returns true immediately and the
arm -> pull -> apply -> arm chain never yields. The added test measures 50 pulls
with no yield before this change.
The bound counts only re-arms where the unplaced set stops shrinking. A chain
that keeps placing rows is converging and is already bounded by that set
emptying, so a raw count would strand a legitimately converging target on
`conflict`; a test pins a five-round convergence that a raw count truncates at
three. Re-arms are also only counted inside a retry's own apply, so a fresh host
snapshot arrival does not spend the budget.
8e4484e to
285bd36
Compare
|
Refined the retry bound in 285bd36 (force-pushed). The first cut bounded the chain on a raw count of 3, mirroring The bound now counts only re-arms where the unplaced set stops shrinking — the spinning case, which is the actual pathology. A new test pins a five-round convergence that the raw count truncated at three, and it fails against the previous revision.
|
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Team
Run ID: 9955c57e-fd98-4a3e-803d-8e6cefc3f178
📒 Files selected for processing (13)
src/renderer/src/hooks/remote-workspace-deferred-placement-retry.test.tssrc/renderer/src/hooks/remote-workspace-deferred-placement-retry.tssrc/renderer/src/hooks/remote-workspace-snapshot-apply.tssrc/renderer/src/hooks/remote-workspace-snapshot-placement.tssrc/renderer/src/hooks/remote-workspace-snapshot-unplaced-tab-adoption.test.tssrc/renderer/src/hooks/remote-workspace-target-sync-types.tssrc/renderer/src/hooks/remote-workspace-target-sync.test.tssrc/renderer/src/hooks/remote-workspace-target-sync.tssrc/renderer/src/runtime/web-session-terminal-orphan-recovery-inventory.tssrc/renderer/src/runtime/web-session-terminal-orphan-recovery-regressions.test.tssrc/renderer/src/runtime/web-session-terminal-pending-handle-recovery.test.tssrc/shared/remote-workspace-session-projection.test.tssrc/shared/remote-workspace-session-projection.ts
🚧 Files skipped from review as they are similar to previous changes (11)
- src/renderer/src/runtime/web-session-terminal-orphan-recovery-inventory.ts
- src/shared/remote-workspace-session-projection.ts
- src/renderer/src/hooks/remote-workspace-snapshot-placement.ts
- src/renderer/src/hooks/remote-workspace-snapshot-unplaced-tab-adoption.test.ts
- src/renderer/src/hooks/remote-workspace-target-sync.ts
- src/renderer/src/runtime/web-session-terminal-orphan-recovery-regressions.test.ts
- src/renderer/src/hooks/remote-workspace-snapshot-apply.ts
- src/renderer/src/hooks/remote-workspace-target-sync-types.ts
- src/renderer/src/runtime/web-session-terminal-pending-handle-recovery.test.ts
- src/renderer/src/hooks/remote-workspace-target-sync.test.ts
- src/shared/remote-workspace-session-projection.test.ts
Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.
| watchers.get(authority.targetId)?.abort() | ||
| watchers.delete(authority.targetId) | ||
| // A watch armed from outside a retry is a fresh snapshot arrival, not a continued chain. | ||
| const previous = applying.has(authority.targetId) ? chains.get(authority.targetId) : undefined |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Keep the retry budget separate from fresh snapshot arrivals.
applying.has(authority.targetId) only shows that an apply is pending. It does not show that this watch call came from that apply. If a fresh host snapshot arrives while an earlier applySnapshot is pending for the same target, this branch reuses the old stalledDepth. At depth two with the same path count, line 68 discards the fresh watch. The target then stays in conflict without a recovery attempt.
Split fresh watch calls from internal retry re-arms, or pass explicit retry provenance. Add a test for a fresh arrival that overlaps a stalled apply.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
- Stalled-chain bound replaces the total-depth cap — the previous
MAX_DEFERRED_PLACEMENT_CHAIN = 3counted eager re-arms as a fixed depth, which would strand a target onconflictthe moment a converging apply needed a fourth round. The retry now tracks{ stalledDepth, unplacedCount }per target and only counts non-shrinking re-arms (worktreePaths.length >= previous.unplacedCount); a chain that keeps placing rows resets to zero and is bounded by the set emptying, while one re-arming on the same-or-larger set still tripsMAX_STALLED_DEFERRED_PLACEMENT_CHAIN = 3. - Convergence regression test — a new test drives five placeable paths that shrink one per round and asserts exactly five pulls, which fails under the old total-depth cap and so genuinely guards the new behavior.
Both rejection-swallowing paths, the stall bound (exactly three pulls), and the converging chain (exactly five pulls) pass, alongside 59 sibling tests across target-sync, orphan-recovery, and session-projection.
DeepSeek Pro (free via Pullfrog for OSS) | 𝕏
…live-PTY removal from losing tabs (stablyai#17948) * fix(ssh): retain remote sessions across late catalogs, path collisions, and PTY rotation Three losses in the "remote session state never reconciled" cluster, one rule: absence from a client-side set, or a stale client-side expectation, is `unverifiable` by construction and can never authorise removal. stablyai#12902 / stablyai#15484 — a direct-SSH snapshot whose host paths the local worktree catalog cannot place yet leaves the target in `conflict`, which suppresses uploads and holds terminal authority at `unverifiable`. Nothing re-pulled once the catalog landed, so the tabs stayed missing and the host ledger stayed stale until a reconnect. The apply now reports the paths it dropped and target-sync watches the catalog for them, re-pulling a fresh host snapshot when they become placeable. stablyai#15484 — exportRemoteWorkspaceSession keys the host projection by worktree path, which drops the repoId, so two local rows for one remote checkout collapsed and the last one won outright. An empty duplicate row published an empty tab list for a workspace with live panes, and the upload is a wholesale replace-session. Union by tab id instead, matching the `Math.max` its sibling recency map already applied to the same collision. stablyai#11495 — orphan recovery retired a leaf whenever a `terminal.list` with `requireFreshPtyLiveness: true` named a different PTY behind a handle than the snapshot frame's pending row did. That is the host attesting the handle is live under a replacement PTY, which is what a host relaunch looks like. Rebind instead of remove. Two tests pinned the removing behaviour and are retargeted with the reasoning. * fix(remote): handle a rejected deferred-placement pull and bound its retry chain The deferred placement retry ran its body as `void (async () => { try {…} finally {…} })()` with no `catch`. `getSnapshot` is an IPC call that rejects when the relay drops, and `applySnapshot` can reject with it, so a dropped relay produced an unhandled rejection in the renderer. Swallow it: the module already documents that a pull which fails is `unverifiable` and the target is left on `conflict`. The retry also re-armed itself through `applyUnsolicitedSnapshot` with no cycle bound, next to a sibling loop capped at MAX_SNAPSHOT_APPLY_ATTEMPTS = 3. When an apply reports still-unplaced paths that the catalog nonetheless reports placeable, `waitForSnapshotWorktreePlacement` returns true immediately and the arm -> pull -> apply -> arm chain never yields. The added test measures 50 pulls with no yield before this change. The bound counts only re-arms where the unplaced set stops shrinking. A chain that keeps placing rows is converging and is already bounded by that set emptying, so a raw count would strand a legitimately converging target on `conflict`; a test pins a five-round convergence that a raw count truncates at three. Re-arms are also only counted inside a retry's own apply, so a fresh host snapshot arrival does not spend the budget.

Four issues investigated. Two fixed, one partly, one already fixed on main. Nothing here retires or kills a PTY — every change is in the retaining direction.
Are these instances of #9229? Partly — and the umbrella is smaller than it looks
#9229 asks for durable generation provenance so a survivor can be classified after an OS-level restart. These four are not four instances of "we cannot enumerate survivors". They are four instances of a narrower rule this codebase has already adopted and enforces almost everywhere:
workspace-terminal-host-authority.ts,confirmSurfaceInventoryAbsence,hasExactTerminalRetirementProofand theresumeSleepingAgentSessionsForWorktreegate all implement it correctly. The bugs are the outliers that were missed. That is why these were landable and #9229's generation-provenance half was not touched — it is a genuinely different problem.#15484 — the ledger published an empty list over live panes — FIXED
exportRemoteWorkspaceSessionkeys the host projection by worktree path, andworktreePathFromIddiscards the repoId. Two localtabsByWorktreekeys for one remote checkout — duplicate repo rows, the normal state while a host catalog reconciles — collapse onto one entry, and it was a plain assignment: last writer wins. A freshly created empty twin iterating second published"/path": []for a workspace with live panes. The upload is a wholesale replace-session, so it deleted them host-side.The same function already folds this exact collision correctly forty lines below —
Math.maxforlastVisitedAtByWorktreePath, with a comment naming the cause. Now unioned by tab id, andactiveTabIdByWorktreePathno longer lets a colliding key'snullerase a real answer.On the two-source divergence: real, but not the loss mechanism — worth recording so nobody else chases it. Main's copy is a superset, never a subset: SSH worktrees stay in the
localpartition by design, main'ssetWorkspaceSessionretains host-admitted tabs a stale renderer replay omits, and the upload chain awaitslocalWritefirst. The collapse is the whole story, and it fires from either source.Two more candidates checked and cleared:
folder:<uuid>keys never reachworktreePathFromId(main'stargetForWorktreerejects them first), andrepoId::/path::workspace:<uuid>instance ids are a paired-runtime construct that never reaches the direct-SSH ledger.#12902 —
conflictwas a state with no exit — FIXED (remaining half)Most of this had already landed: unplaceable paths are reported, the apply waits 10s for placement, and an incomplete pull marks the target
conflictrather thansynced, suppressing uploads so the loss never becomes durable.What was still broken: nothing re-pulled after the wait.
syncAfterConnectruns only on connect;applyUnsolicitedSnapshotonly on a host push. The test file said so out loud — "Not recoverable: promoting that to truth. Nothing re-pulls once the lineage lands."Now a 10-minute catalog watch re-pulls a fresh host snapshot when the dropped paths become placeable. A pull returning null or revision 0 is left alone:
unverifiablemust not clearconflict.#11495 — a live PTY removed because the client's expectation was stale — PARTLY FIXED
terminal.ptyIdcomes from aterminal.listwithrequireFreshPtyLiveness: true— the host has just attested the handle is live, typicallyorphaned: trueunder a fresh incarnation. A PTY id rotating across a host relaunch is the normal case. Mismatch now retains, which is the rebind:buildRetainedTerminalSurfaceemits areadyrow bound to the host's handle, and the handle is the identity the host answers on.Why it was unrecoverable rather than merely lossy:
shouldReplaceTerminalTabreturns true for everyweb-terminal-*id, so the mirror population is a pure function of one frame, andapply-terminal-records.tsthen deletesptyIdsByTabIdandterminalLayoutsByTabId. One bad disposition destroys the only surviving record of how to rebind —lastKnownRelayPtyIdByTabIdis never rehydrated, andremoteSessionIdsByTabIdonly covers repos withconnectionId, which a paired runtime does not have.What the retargeted tests were protecting: both guarded against stale leaves accumulating — already covered properly by the two host-attested branches in the same file. Neither fixture supplies either kind of proof, so what they actually asserted was only that the host had relaunched the PTY. Retargeted with that reasoning in-comment; this is a deliberate contract change, not an incidental one.
#12701 — already fixed, better than proposed
resumeSleepingAgentSessionsForWorktreereturns 0 when host authority isunverifiable, and a direct-SSH target that has not concluded its first pull isunverifiable.runWorktreeAgentActivationGateadditionally fails closed on an unreadable census. No change needed. (One upstream hole handed off below.)Verification
max-linesforced two extractions rather than a disable.Handoffs — stated precisely rather than half-landed
#11495 Step C, the 32→2 magnitude. Implemented, then reverted.
buildMissingWebSessionTabsRemovals(tracking.ts:292) synthesises aremoved: truetombstone — emptying a worktree's entire mirror — for every worktree in the client's tracked set absent from one host inventory. The code concedes its own premise: "if a host ever scopes that map, this turns live worktrees into tombstones." A version-skewed host after an update is exactly that.I built the correct-shaped fix (two consecutive authoritative omissions, mirroring
confirmSurfaceInventoryAbsence) and reverted it: it fails 14 existing tests that are substantive tests of the omission/fence machinery, not incidental assertions. Retargeting them needs someone to establish each one's intent. Touch points recorded in the commit.#11495 Step A. Making the per-frame rebuild preserve
ptyIdsByTabId/terminalLayoutsByTabIdwhen a frame is not authoritative would make the whole subsystem fail toward retaining regardless of which disposition is wrong. Deserves its own design.#12701's remaining hole.
wakeSleepingAgentsForWorktreeInBackgroundrunsclearSleepingAgentSessionsByPaneKey— a permanent delete of durable resume records — and dispatches background mounts before any host-authority check, judging canonicality against atabsByWorktreethat is empty by construction pre-merge. A bare bail-out onunverifiableis not safe: the issue warns it would trade duplicate launches for dropped mobile wakes, so it needs the park-and-replay treatment paired runtimes already get.A documented gap I did not close.
workspace-terminal-host-authority.ts:47-56: theoffline/errorfloor resolves an un-hydrated target tonone, authorising seeding and sleeping-agent resume. Since hydration is now revoked on an unplaceable snapshot, a target can reach that floor having demonstrably answered with tabs. My #12902 fix shrinks the window but the floor still cannot tell a revoked target from one that never answered. Surgical version: keep the bounded escape for seeding, deny it for resume — a second writer onto a live transcript.Fixes #15484, #12902
Refs #11495, #12701, #9229