Skip to content

fix(runtime): refuse a terminal.create tab id hint the host already retired - #22149

Closed
BearMett wants to merge 4 commits into
stablyai:mainfrom
BearMett:fix/host-terminal-create-tombstone-gate
Closed

BearMett wants to merge 4 commits into
stablyai:mainfrom
BearMett:fix/host-terminal-create-tombstone-gate

Conversation

@BearMett

@BearMett BearMett commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

ELI5

When a paired desktop client goes to sleep and wakes up, it tries to reconnect every terminal pane it remembers. If the host user had already closed some of those terminals, the host still let the client recreate them under the very same tab ids, so a dozen closed terminals could reappear on every wake. The host now remembers which panes it retired and says "that tab is gone" instead of spawning a new shell under it.

What Changed

Before: Host on Mac A hosts a worktree; Mac B is paired to it and holds 12 terminal tabs mirrored from A. The host user closes the 12 tabs (PTYs are killed). Mac B wakes from sleep and replays connect() for each remembered pane; the host answers terminal_not_found, and the client falls through to the legacy terminal.create call carrying the pane's original tabId/leafId as a hint and no command. The host adopted the hint on id format alone and spawned a fresh shell under each closed tab. Observed 9 times in one day on a live host; the tab ids were identical each time.

After: the host keeps a bounded, in-memory, pane-keyed ledger of terminal surfaces it retired, written at the single funnel every permanent PTY exit reaches (retireMobileSessionSurfacesForPty). terminal.create resolves its pane identity through resolveHintedTerminalPaneIdentity, which refuses a hint that (a) names a pane in that ledger and (b) the host does not currently publish as a live terminal surface. The refusal is tab_not_found, which the client already treats as definitive absence. An updated client settles the pane quietly; an older client shows the token as a pane error and does not retry. Neither spawns a shell.

The ledger records only exits the host resolved as an operator close (stopRequestedPtyIds → OPERATOR_CLOSE_EXIT_CAUSE, which the renderer's pty:kill tab-close route marks). A shell the user ended with exit under a tab they kept open is not recorded, so its Restart in place still works on a headless host where nothing republishes the surface. A pane adopted from its hint forgets its retirement.

Review follow-ups folded in (commits 2 and 3): the client's connect() catch settles the pane on the host's surface-absence tokens (reusing isMissingHostSessionSurfaceError) and publishes the same onExit/onDisconnect pair the stream's end handler does, so a refused recreate shows the exit overlay instead of a blank pane or a raw error; the gate reuses the inherited mobileSessionSnapshotHasSurface predicate; the ledger is an insertion-ordered set bounded at 4096.

Commit 4: a stop the caller means to undo (worktree sleep, pane hibernation, both already passing keepHistory) is marked reversible, and only an irreversible stop records into the ledger, so a slept pane's wake replay is still adopted. The exit cause is unchanged. The ledger entry is now cleared in registerPty, where a PTY actually binds to the pane, instead of at the create's hint gate, so a spawn that fails after the gate leaves the retirement in place.

Why

Both halves of the evidence are required. The ledger alone would also refuse a shell that exited under a tab the user kept open, and that pane must still restart in place once the host republishes it. Absence alone cannot tell a retired pane from a client-minted id the host has never seen, which is the normal shape of a paired client's first create.

Refusing rather than minting a fresh id: a fresh id still spawns the shell, which is the user-visible bug.

Why a new ledger instead of existing state: terminalSurfaceTombstonesByPaneKey is legacy (only ever deleted from, and cleared on every full session write); terminalTopologyRevisionByRepoId is per-repo with no pane identity; the session-tab snapshot's retirement proofs are keyed by a terminal handle, and a pane the host user created in the desktop renderer that no client ever addressed by handle publishes none, which is exactly the reported topology. The ledger changes nothing the host publishes, so there is no wire change; it is the host-only half of the fix and protects a host whose paired client has not updated.

This complements the client-hydration fix being worked on for #21341; it does not replace it. Known gaps, on purpose: a host restart empties the ledger, so a client that re-injects after a host restart is not covered here (persisting it needs a WorkspaceSessionState schema field and is a follow-up). A close whose kill was never confirmed (stop_unverified, SSH-owned PTYs) does not record either; that branch keeps the surface addressable rather than retiring it, per docs/reference/ssh-execution-boundary.md, so there is nothing for the ledger to refuse against. Not capability-gated: a host-only fix exists to protect a host whose paired client has not updated, and gating it to capable clients would re-enable the resurrection for precisely the clients that cause it.

Linked Issue

Addresses the host side of #21341 (see my comment there for the Mac ↔ Mac trace). Not marked as closing it, since the client-side hydration fix is in progress separately.

Visual Proof

N/A. No UI change; the observable change is that closed terminals stop reappearing on a paired client's wake, and a refused recreate no longer shows a raw error toast on an updated client.

Testing

Automated:

  • src/main/runtime/orca-runtime-tests/terminal-create-retired-pane-hint.spec.ts (integration through the real retirement path): retired pane hint → tab_not_found, no spawn; renderer-created pane (the reported topology, via syncWindowGraph) → same; no-hint create unchanged; hint without retirement adopts; split leaf under a live tab adopts.
  • src/main/runtime/retired-terminal-pane-hint-gate.test.ts: gate composition (republished surface is not refused), pane-key scoping, malformed hints, ledger bound.
  • remote-runtime-pty-transport-host-absence-evidence.test.ts: terminal.create rejecting with tab_not_found settles the pane with exactly one create, no retry, no error surfaced.
  • Red check: disabling the gate makes the first two integration cases fail with a second PTY spawned under the closed tab.

Runs: pnpm tc:node, pnpm tc:web, pnpm test src/main/runtime, pnpm test src/renderer/src/components/terminal-pane, pnpm test src/renderer/src/runtime, pnpm run check:code-quality:changed, tests/e2e/cross-version-wire — all green. One pre-existing failure in agent-session-process-identity-probe.test.ts reproduces on a clean main checkout and is unrelated.

Platforms: developed and unit/integration tested on macOS. The originating repro was a macOS ↔ macOS pairing over Tailscale; I have not yet re-run the live sleep/wake repro against this branch, so the live confirmation is still pending.

  • I manually tested these changes locally (test suites; live pairing repro not yet re-run)
  • Automated tests added/updated, or explained why not below

AI Disclosure

Investigated and implemented with Claude Code (Claude Fable 5.1 orchestrating, Claude Opus 5 for the implementation pass). Log analysis, root-cause trace and the review of every commit were done by me with the model; all tests were run locally.

Review

Independent review points already addressed in the second commit: client settle on tab_not_found (scoped to the create path's catch, not the subscribe classifier), and reuse of the existing published-surface predicate.

Agent skill upstream boundary

  • Not applicable, or this change follows docs/reference/agent-skill-sharing-upstream-boundary.md and copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.

Notes

  • SSH: the ledger is written only on permanent PTY exit; the preservesAbnormalSshSurface branch that keeps a HUB-owned pane addressable through reconnect grace does not write it, so an SSH relay drop is not treated as retirement.
  • Wire compatibility: no published content changes. The only new host behaviour is an error on a terminal.create whose hint names a retired pane; the current client classifies that as non-recoverable and does not retry (verified in remote-runtime-client-error-classification.ts and createWithUnknownOutcomeRecovery).
  • Bound: 4096 ledger records per host process (oldest evicted).

Checklist

  • This PR is small and focused
  • I explained what changed and why (ELI5, the user-facing before/after, the mechanism, and why over the alternatives)
  • Before/after screenshots or videos attached for UI changes, or N/A with reason
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote, and path/shortcut impact considered (or N/A)
  • pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (or CI will cover; local preferred)

…etired

A paired client that wakes replays connect() for every mirrored pane whose host
PTY is gone: terminal.resolvePane answers terminal_not_found, and the fall-through
legacy terminal.create carries the pane's original tabId/leafId as a hint with no
command. canAdoptPaneIdentity only checked the hint's *format*, so the host adopted
the id and spawned a fresh shell under a tab its own user had closed — 12 tabs came
back with identical ids on every wake, 9x in a day (stablyai#21341).

terminal.create now resolves its pane identity through
resolveHintedTerminalPaneIdentity, which refuses a hint naming a pane this host
retired. Refusal rather than a fresh id: a fresh id still spawns the shell, which is
the user-visible bug.

The host had no pane-keyed retirement record to consult. `terminalSurfaceTombstonesByPaneKey`
is legacy — `recordTerminalSurfaceRetirement` only ever deletes from it and
`sanitizeWorkspaceSessionTerminalRetirements` clears the map on every full session write —
`terminalTopologyRevisionByRepoId` is per-repo, and `closedTerminalTabTombstonesByTabId` is a
client breadcrumb written only for direct-SSH worktrees. The session-tab snapshot's retirement
proofs are pane-keyed but carry a terminal handle, so a pane the host user created in the
desktop renderer and no client addressed by handle publishes none — which is exactly the
reported topology. So `retireMobileSessionSurfacesForPty`, the single funnel a permanent PTY
exit reaches, now also records the pane in a bounded in-memory RetiredTerminalPaneLedger. No
published content changes.

Both halves of the evidence are required to refuse. The ledger alone also covers a shell that
exited under a tab the user kept open, and that pane must still restart in place once the host
republishes it; absence alone cannot tell a retired pane from a client-minted id the host has
never seen, which is the normal shape of a paired client's first create.

The error is `tab_not_found`, which the client already treats as definitive surface absence:
mapRuntimeError sends it as {code: runtime_error, message: tab_not_found},
isRecoverableRemoteRuntimeConnectionError rejects that, so createWithUnknownOutcomeRecovery
rethrows on the first attempt and connect()'s catch surfaces it without arming a retry. No
client change needed.
…showing the raw token

Review follow-up to the retired-pane create gate.

connect()'s catch only settled for isRemoteTerminalGoneMessage, so the host's own
`tab_not_found` — positive evidence that the pane it is being asked to recreate is
retired — fell into the generic else and surfaced the token as a pane error. It now
retires the remote terminal id, the same settle the gone branch reaches through
handleRemoteTerminalError. isRemoteTerminalGoneMessage is deliberately left alone:
handleRemoteTerminalError also classifies the subscribe/resubscribe path, where the
same token must stay re-resolvable rather than fatal.

The gate's doc comment claimed every client settles; corrected to say what holds on
both sides of the skew — no client retries, an updated one settles the pane as
absent, an older one surfaces the token.

Also drops publishesLiveSurface, which duplicated the inherited
mobileSessionSnapshotHasSurface. The gate takes an isSurfacePublished predicate now,
so it no longer restates how a published terminal surface is recognised.
…pane id

Review follow-up. Six findings from the independent review of this branch.

The ledger recorded every permanent PTY exit, so a shell the user ended with
`exit` under a tab they kept open was recorded too. On a desktop host the
renderer republishes that surface and the liveness half lets the pane restart in
place, but on a headless host nothing republishes it — the pane's Restart, which
is a terminal.create carrying the same tabId/leafId, was refused. The ledger now
records only when onPtyExit resolved the exit as an operator close. That cause is
already computed there from stopRequestedPtyIds, and the bug case still records:
`pty:kill`, which renderer-kill.ts documents as "the route the renderer takes when
the user closes a terminal tab", marks the stop before the exit lands.

Also:

- a pane adopted from its hint forgets its retirement, so a later transient
  snapshot absence cannot refuse a pane the host is publishing;
- the ledger is an insertion-ordered set — its timestamp was never read — and its
  bound goes to 4096, which is a host-wide figure now that natural churn no longer
  reaches it and operator closes and worker releases are all it has to outlive;
- the client settles the pane visibly. On a fresh connect there is no pty id for
  retireRemoteTerminalId to notify, so the refusal left the pane blank with no
  overlay; it now publishes the same onExit/onDisconnect pair the stream's own end
  handler does. The test asserts the settle rather than only the absence of a toast;
- the new catch branch reuses isMissingHostSessionSurfaceError, the classifier this
  file already has for exactly this pair of tokens.

Not done, deliberately: capability-gating the refusal. A host-only fix exists to
protect a host whose paired client has not updated, and gating it to capable
clients would re-enable the resurrection for precisely the clients that cause it.
An old client gets the existing non-recoverable error path, on a pane that was
already closed, with no retry.
@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 4b3e2753-0f68-411b-961a-d749704645b0

📥 Commits

Reviewing files that changed from the base of the PR and between d597bec and 2835d0b.

📒 Files selected for processing (16)
  • src/main/ipc/pty-runtime-kill-and-exit.test.ts
  • src/main/ipc/pty-ssh-undelivered-kill.test.ts
  • src/main/ipc/pty/ipc/renderer-kill.ts
  • src/main/ipc/pty/runtime/kill.ts
  • src/main/runtime/orca-runtime-invalidate-all-handles-for-pty.ts
  • src/main/runtime/orca-runtime-mark-pty-liveness-unverifiable.ts
  • src/main/runtime/orca-runtime-on-pty-exit.ts
  • src/main/runtime/orca-runtime-persist-terminal-surface-retirements.ts
  • src/main/runtime/orca-runtime-record-agent-prompt-lifecycle-state.ts
  • src/main/runtime/orca-runtime-register-pty.ts
  • src/main/runtime/orca-runtime-stop-requested-pty-ids.ts
  • src/main/runtime/orca-runtime-tests/terminal-create-retired-pane-hint.spec.ts
  • src/main/runtime/retired-terminal-pane-hint-gate.test.ts
  • src/main/runtime/retired-terminal-pane-hint-gate.ts
  • src/main/runtime/retired-terminal-pane-ledger.ts
  • src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • src/main/runtime/orca-runtime-tests/terminal-create-retired-pane-hint.spec.ts
  • src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.ts
  • src/main/runtime/retired-terminal-pane-ledger.ts
  • src/main/runtime/retired-terminal-pane-hint-gate.test.ts
  • src/main/runtime/retired-terminal-pane-hint-gate.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The runtime now tracks deliberately retired terminal pane identities in a bounded in-memory ledger. Terminal creation validates hinted identities against retirement state and published surfaces. PTY stop handling distinguishes reversible stops from permanent exits. The renderer settles tab_not_found and terminal_not_found creation failures without retrying or reporting an error. Tests cover ledger behavior, replay cases, stop propagation, and renderer settlement.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 2835d

Closed panes are rejected on replay while sleeping or hibernated panes remain recoverable; the current change has no evidenced merge-blocking risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 20 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: refusing terminal.create tab ID hints for panes already retired by the host.
Description check ✅ Passed The description is complete and follows the required template. It explains the user impact, implementation, rationale, linked issue, testing, compatibility considerations, AI disclosure, and checklist…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: 96a77a0f-fe12-4d48-9649-8c2cd2fd7a4e

📥 Commits

Reviewing files that changed from the base of the PR and between b964336 and d597bec.

📒 Files selected for processing (11)
  • src/main/runtime/orca-runtime-create-terminal.ts
  • src/main/runtime/orca-runtime-on-pty-exit.ts
  • src/main/runtime/orca-runtime-persist-terminal-surface-retirements.ts
  • src/main/runtime/orca-runtime-runtime-id.ts
  • src/main/runtime/orca-runtime-tests/terminal-create-retired-pane-hint.spec.ts
  • src/main/runtime/orca-runtime.test.ts
  • src/main/runtime/retired-terminal-pane-hint-gate.test.ts
  • src/main/runtime/retired-terminal-pane-hint-gate.ts
  • src/main/runtime/retired-terminal-pane-ledger.ts
  • src/renderer/src/components/terminal-pane/remote-runtime-pty-transport-host-absence-evidence.test.ts
  • src/renderer/src/components/terminal-pane/remote-runtime-pty-transport.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The retirement ledger keys on every operator_close exit, which includes reversible parks — worktree sleep (terminal.sleep) and the pty:kill+keepHistory hibernation fallback. Their retirement already drops the surface from the mobile session snapshot, so a paired client replaying terminal.create with the pane's own ids after a wake is now refused instead of resumed. Inline comment with the mechanism and an open question.

Reviewed changes

  • Retirement ledger (retired-terminal-pane-ledger.ts): in-memory, insertion-ordered, pane-keyed set bounded at 4096, written at the single permanent-exit funnel.
  • Hint gate (retired-terminal-pane-hint-gate.ts): resolveHintedTerminalPaneIdentity replaces the old format-only adoption in orca-runtime-create-terminal.ts, refusing a retired-and-unpublished hint with tab_not_found and forgetting the retirement on adoption.
  • Ledger write site (orca-runtime-persist-terminal-surface-retirements.ts): records only isDeliberateTerminalExit exits, before the handle-keyed proofs, with onPtyExit threading the cause through.
  • Client settle (remote-runtime-pty-transport.ts): connect()'s catch first-matches isMissingHostSessionSurfaceError and settles the pane (retireRemoteTerminalId + onExit(0)/onDisconnect) instead of surfacing the raw token.
  • Tests: integration spec through the real retirement path, gate unit tests, and a transport settle test.

The server-side gate itself reads cleanly, the red-check tests can fail when the gate is disabled, and the client reconciliation was verified safe: no reachable terminal_not_found occurs inside connect()'s try-block, and the only tab_not_found source is the new gate.

ℹ️ Nitpicks

  • The PR body's Notes section says "Bound: 512 ledger records per host process", but MAX_RETIRED_TERMINAL_PANE_RECORDS is 4096. Worth correcting so the description doesn't mislead.
  • The new first-match on isMissingHostSessionSurfaceError depends on waitForHostSessionHandle swallowing the same tab_not_found from session.tabs.activate; a one-line note in the catch that this ordering is load-bearing would protect it against a refactor.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

… ledger

Worktree sleep and pane hibernation stop the PTY through the same calls an
ordinary close takes, so their exits were recorded as retirements and the
wake that replays the pane's own tabId/leafId was refused — the terminal the
user slept came back as an error instead. A stop is now marked reversible at
the two callers that mean to undo it, and only an irreversible one records.
The exit cause is unchanged; only the ledger write is narrowed.

The create's hint gate also cleared a retirement before the shell was
spawned, so a spawn that then failed left the pane unprotected and the next
replay of the same hint was adopted. Clearing now happens in registerPty,
where a PTY is actually bound to the pane.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed commit 2835d0b (delta against the prior pullfrog review at d597bec), which makes the retirement ledger ignore stops the caller means to undo. No new critical or must-address findings; the prior thread is resolved.

  • Reversible stop intent (orca-runtime-mark-pty-liveness-unverifiable.ts): markPtyStopRequested now takes { reversible }, tracked in reversibleStopRequestedPtyIds; a later irreversible request over the same PTY clears the marker, so the caller giving the pane up for good wins.
  • Park paths mark reversible (kill.ts, renderer-kill.ts): stopAndWaitPtyFromRuntimeController and pty:kill pass reversible: opts.keepHistory === true, covering worktree sleep (stopAndWait + keepHistory) and pane hibernation (pty:kill + keepHistory, terminal.stopExact).
  • Ledger write gate (orca-runtime-persist-terminal-surface-retirements.ts): onPtyExit captures the flag before clearing the sets and threads it through; only an irreversible deliberate close records.
  • Forget moves to registration (orca-runtime-register-pty.ts): the gate no longer clears the ledger on adoption, so a spawn that fails after the gate leaves the retirement in place; only a bound PTY forgets it.
  • Tests: reversible-stop adoption, irreversible-after-reversible refusal, late spawn-failure retention, registerPty forget, and pty:kill keepHistory marking reversible.

The prior review's reversible-park concern is fully addressed, and both earlier nitpicks are handled (PR body now says 4096; the load-bearing-order comment was added to the connect() catch).

ℹ️ SSH synthetic-then-real exit re-marks the stop irreversible

The author flagged this as a deliberately-open door: on an SSH-owned PTY, bind-listeners.ts:96 re-marks markPtyStopRequested(payload.id) without reversible when a duplicate real provider exit follows a synthetic one. Since the ledger gate now keys off that flag, a park can still record a retirement in the narrow window where the first (reversible) retirement was not accepted by persistence and a real exit then arrives. Closing it means carrying reversible through rememberSyntheticKillExit/consumeSyntheticKillExit, which is a reasonable follow-up rather than a change required here — the preconditions (SSH host, missed provider exit, transient store-write failure, in-window duplicate) are narrow and no e2e exercises the park-then-recreate path either way.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@BearMett

Copy link
Copy Markdown
Contributor Author

Closing this PR. No replacement PR is open yet; this closes the proposal, not the underlying issue (#21341). Since this PR was opened, main has gained durable closed-tab records for terminals: they are written by the close transaction, kept per host partition, and stored in SQLite. An alternative that refuses a replayed terminal.create hint when the host has a valid tab-close record for that tab id, so the refusal survives a host restart, is prepared locally for further discussion and would address #21341. This PR's in-memory ledger lost the refusal on restart.

What that local draft changes compared with this PR:

  • Dropped: the in-memory retirement ledger and the exit-cause plumbing that fed it. The existing close records are used instead, so a refusal needs a valid tab-close record.
  • Kept, adapted: the client-side settle. The host answers a closed tab's hint with terminal_tab_not_found, and an updated client drops that tab quietly instead of showing the raw error.

Natural exits and stops without a tab close still adopt the hint; what should happen there stays an open question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant