Skip to content

fix(terminal): don't seed or resume while the execution host hasn't answered - #16750

Merged
nwparker merged 1 commit into
mainfrom
nwparker/ssh-04-host-authority-seeding
Aug 28, 2026
Merged

nwparker merged 1 commit into
mainfrom
nwparker/ssh-04-host-authority-seeding

Conversation

@nwparker

@nwparker nwparker commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 4 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​837 0 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​837
Prod 5 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​205 $\color{#cf222e}{\Huge{\mathbf{−}}}$​14 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​191

Stack: 4 of 6 — based on #16748.

ELI5

When you open an SSH workspace, two things race: Orca asks the remote machine "what terminals do you have?", and separately Orca thinks "this workspace looks empty, I should open a terminal."

If the second wins, you get a terminal you didn't ask for. Worse, a background job that wakes sleeping AI agents can fire in the same gap and start a second claude --resume on a session that is still running — two agents writing one transcript, which corrupts it.

The fix: one rule for both — if the machine that owns this workspace hasn't answered yet, don't create anything and don't resume anything. Waiting is recoverable; a duplicate agent is not.

Two honest notes:

  • The resume half is the valuable part and is proven by tests.
  • The seeding half is correct but currently changes nothing you'd notice, because of a separate bug (STA-3593) where the remote's tabs fail to load at all. It starts working when that lands. The PR explains this rather than claiming a win it doesn't have.

Two client behaviours read local tab rows as the verdict on what the execution host is running. Before the host answers, "I hold no pane for this" is unverifiable, not exited — the collapse docs/reference/ssh-execution-boundary.md forbids. One missing predicate gates both.

Symptom 1 — a client that has never held the workspace creates a tab from nothing

src/renderer/src/lib/worktree-initial-terminal-seeding.ts:47,128 seeds a terminal when renderableTabCount === 0. Its only bail-out (:72-77) covered the paired-web-runtime flavor — comment: "while that session is live the host owns terminal creation" — with no equivalent for direct SSH.

Measured: a fresh client with no local row, against a host owning 3 tabs, creates 1 tab from nothing and adopts NONE of the host's 3. The snapshot then arrives, the merge rightly keeps the tab it was never told about, and the union uploads as the new host truth.

(A restart never reaches the predicate — local state restores the row first — which is why two three-cycle restart repros came back flat. A client that has never held the workspace is the case that fires. An earlier "the predicate never fires" result was confounded by the connect helper creating its own tab; retracted by its own author.)

That guard was also asking the wrong question. It asked "am I a client of a live paired session?", which a host desktop window answers no and a paired client answers yes — so both seeded. That is GH #15556, "Paired client + host window BOTH auto-create the initial terminal", closed by this change.

Symptom 2 — the data-corrupting half: duplicate claude --resume on a live transcript

src/renderer/src/components/Terminal.tsx:1554 calls resumeSleepingAgentSessionsForWorktree(activeWorktreeId) twenty lines after the seeding call at :1529-1534 — same startup path, same pre-hydration window, and not SSH-gated at all. Also reached from worktree-activation.ts:123/:211 and wake-sleeping-agents-in-background.ts:212.

The sweep treats "I have no record of this pane" as "that session is dead and needs resuming". Seeding produces a spare empty tab; the sweep launches claude --resume <id> for a session still running on the remote and still owned by a live pane. Two agent processes writing one transcript — STA-3498 observed five concurrent claude processes on one transcript, one new "Terminal N" tab per incident. STA-3500 files this exact race:

"the worktree tab list lives in the remote snapshot and only reaches the renderer store after connect via remoteWorkspace.get → mergeDirectSshRemoteWorkspaceSession — a network round trip. The sweep fires before that lands, so every restart cold-resumes live remote sessions and destroys their records."

Failure is asymmetric: declining to resume is user-recoverable; a duplicate resume corrupts a transcript irreversibly. The fix biases toward not acting.

The fix, and why this shape

New src/renderer/src/lib/workspace-terminal-host-authority.ts answers the one ownership question both paths ask, in the three-verdict vocabulary the renderer already uses for host terminal inventory — HostLiveTerminalProbeVerdict, aliased rather than restated so the two cannot drift:

  • live — a remote execution host owns terminal creation here; it supplies the surface itself.
  • unverifiable — the workspace has a remote execution host that has not answered yet.
  • none — no remote host holds terminals here: the workspace is local (this client is the execution host), or the host answered and holds nothing.

Seeding requires none. The sweep declines on unverifiable without consuming its one-shot (startupResumeWorktreeIdsRef), so sleeping agents are not stranded for the session — the caller re-runs once the verdict lands.

Shape decisions worth reviewing:

  • An ownership question, not a client-liveness one. That is what fixes [Bug]: Paired remote client + host desktop window both auto-create the initial terminal — workspace activation spawns two empty terminals #15556.
  • Folder workspaces resolve to none. The snapshot replaces exactly DirectSshTargetScope.gitWorktreeIds (remote-workspace-snapshot-apply.ts); a folder workspace's rows are never replaced by the host, so waiting on an answer that will never name them would leave it terminal-less for good.
  • A conflict sync phase is unverifiable — the same pair use-app-session-persistence.ts gates uploads on. A conflicting snapshot means the client's picture is not the host's, so it is no basis for deciding the host holds nothing.
  • An unnameable runtime host is unverifiable, not none.
  • Absence of a catalog row is not evidence of a remote owner — refusing to act on it would strand every workspace whose repo has not landed yet, so that path returns none.
  • Explicit launch work stays ungated. Setup/issue commands are a request to create a terminal now.
  • Terminal.tsx subscribes through a retained selector, not a read inside the effect. The verdict flipping to none is what must re-run the passes; and resolution walks the owner catalogs (and worktreesByRepo uncached, for an active ssh: selection), so recomputing per store write would be the STA-3363 render-path multiplier again. Same retained-selector memo as createConnectionIdForFileSelector.

Evidence

Before (both production hunks reverted to main, tests present):

 FAIL  remote-workspace-snapshot-fresh-client-tab-seeding.test.ts > holds one tab when the host still reports one tab
 FAIL  remote-workspace-snapshot-fresh-client-tab-seeding.test.ts > adopts every tab the host owns instead of replacing them with a seeded one
 FAIL  resume-sleeping-agent-session-direct-ssh-hydration-gap.test.ts > does not resume a remote session before the host has answered
 FAIL  resume-sleeping-agent-session-direct-ssh-hydration-gap.test.ts > wakes the same session once the host answers and holds no pane for it
 FAIL  workspace-terminal-host-authority.test.ts > distinguishes an unanswered SSH host from one that answered with nothing
 FAIL  workspace-terminal-host-authority.test.ts > treats a conflicting host snapshot as unanswered
 FAIL  workspace-terminal-host-authority.test.ts > does not seed locally when the runtime owner cannot be named

 Test Files  3 failed (3)
      Tests  7 failed | 3 passed (10)

After:

 Test Files  3 passed (3)     Tests  10 passed (10)
 Test Files 16 passed (16)    Tests 103 passed (103)   # + seeding, resume, sleeping-agent, remote-workspace suites

$ pnpm tc → clean
$ pnpm test config/scripts/pr-e2e-gate-contract.test.mjs → 36 passed
$ pnpm run check:code-quality:changed → 0 new findings across 24 changed files

tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts is new and added to the Docker-SSH lane runner (required by the gate-contract assertion introduced in #16746). It drives two passing cases over a real relay: no tab added when the host snapshot stalls across a relaunch (the case this fix does close), and — honestly named for what it asserts — one tab still seeded for a client with no local row, which this fix does not close. See "What is NOT covered".

Deliberately NOT done — and a limitation you should weigh before merging

This gate is strictly subtractive. It removes the spurious tab; it does not make adoption work.

Measured at the integration layer — fresh client, seedInitialTab: false, host owning 3 tabs, sampled every 3s for 30s:

[adopt 0] rev:2  snapPaths:[repo:3 tabs]  storeTabs:0  hydrated:true
[adopt 1] rev:2  snapPaths:[repo:3 tabs]  storeTabs:1  hydrated:true
[adopt 2..9]                              storeTabs:1  (flat for 30s)

applyDirectSshRemoteWorkspaceSnapshot does run (rev 2 > 0, hydrated true) and the snapshot does carry all 3 tabs. They never arrive — not late, permanently lost. The drop is between snapshot and store: hydrateTabsSession, or importRemoteWorkspaceSession's path→id resolution (the silent continue when uniqueWorktreeIdByPath returns null). That is STA-3593, and it is out of scope here.

So over a real relay, suppressing the spurious create leaves a fresh client on an empty workspace — 0 tabs, not 1 wrong one. For a user with three live host sessions that is worse. Neither layer could see this alone: the unit layer says the gate is strictly subtractive (4 tabs → 3, adoption unaffected), which is true about the store; the integration layer says the tabs never reach the store.

Consequences, stated rather than hidden:

  • ssh-cold-hydration-gap-tab-seeding.spec.ts:286 pins the adopting behaviour with test.fixme rather than deleting it. It is the honest record of what still does not work.
  • STA-3593 should land with or before anyone relies on adoption here. The two passing e2e cases are the ones that hold regardless.

Also not done: the local claude --resume defect (#14228 / STA-4110) reproduces on purely local macOS with no SSH, relay or runtime, so it does not belong in this layer. Three open PRs already target it.

Risk / blast radius

Renderer-side, no wire change. Two call sites, one new predicate. The dangerous direction is over-refusal — a workspace that never gets a terminal — which is why none is the default for every path where a remote owner is not positively established (local, folder workspaces, unresolved catalog rows), and why three of the ten tests are over-refusal tripwires. The selector is memoised on fourteen named store slices; getting that list short would return a stale verdict, getting it long only costs a recompute.

How to verify

pnpm test src/renderer/src/lib/workspace-terminal-host-authority.test.ts \
  src/renderer/src/lib/resume-sleeping-agent-session-direct-ssh-hydration-gap.test.ts \
  src/renderer/src/hooks/remote-workspace-snapshot-fresh-client-tab-seeding.test.ts

# over a real relay (needs Docker)
pnpm test:e2e:ssh-docker -- tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts

To see it fail: git checkout main -- src/renderer/src/lib/worktree-initial-terminal-seeding.ts src/renderer/src/lib/resume-sleeping-agent-session.ts and re-run.


End-to-end verification

Specs this PR routes to

$ git diff --name-only nwparker/ssh-03-merge-preserves-local..nwparker/ssh-04-host-authority-seeding \
    | node config/scripts/pr-e2e-source-routing.mjs
["tests/e2e/ssh-cold-activation-restore.spec.ts",
 "tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts",
 "tests/e2e/ssh-reconnect-tab-destruction.spec.ts"]

$ … | node config/scripts/pr-e2e-source-routing.mjs --ssh-source
true

Under main's router this level routed to its own new spec only — the two restore specs were unreachable from worktree-initial-terminal-seeding.ts and resume-sleeping-agent-session.ts. #16746 adds them.

ssh-cold-hydration-gap-tab-seeding.spec.ts is new here and is added to the lane runner in the same commit, which the gate-contract assertion introduced in #16746 requires — a Docker-gated spec claimed by no runner now fails the contract by name.

Lane result

Covered by the full Docker-SSH lane at the stack tip (below). The routed subset for this level can also be run directly:

pnpm test:e2e:ssh-docker -- tests/e2e/ssh-cold-activation-restore.spec.ts \
                            tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts \
                            tests/e2e/ssh-reconnect-tab-destruction.spec.ts

The new spec has three tests: two green, and one test.fixme. One of the two green tests asserts toBe(1) — it documents that the seeding predicate still fires for a fresh client, and is named accordingly (:218). It is not evidence the seeding gate works. Confirmed in the lane run — the single skip in 20 tests → 17 passed, 2 failed, 1 skipped is this spec's test.fixme at :286, test.fixme('a client with no local row adopts the tabs the host already owns', …), the only test.fixme in the lane's spec set.

That skip is a deliberate record of a real gap, not coverage. Adoption genuinely fails end-to-end: a fresh client gets 0 of the host's 3 tabs, measured flat over 30s with hydrated:true and the snapshot verifiably carrying all three. That is STA-3593's shape and it is not caused by this PR — this PR's gate is strictly subtractive at the unit layer (4 tabs → 3, adoption untouched). The test.fixme pins the intended behaviour so the gap cannot be quietly forgotten.

A reviewer reading the lane output should read that skip as "known broken, deliberately pinned", never as "covered".

$ pnpm test:e2e:ssh-docker
BUILD_SHA=49bb96e0b4c135ce305a2f4dfe2d4b52e0b19cde   DIRTY=0
14 specs / 20 tests → 17 passed, 2 failed, 1 skipped (10.7m)

The two lane failures, resolved — and one is now exempted:

Spec main tip (isolated) tip (full lane) Outcome
ssh-docker-bulk-open-freeze-repro FAILED — harness TypeError: … reading 'workerIndex' at docker-ssh-relay-target.ts:226 — FAILED Not a flake — a 100% failure, and three further arity bugs behind it. Now exempted from the lane; repair tracked in #16764
ssh-port-forward-lifecycle PASSED PASSED (59.6s) FAILED at ssh-port-forward-lifecycle-evidence.ts:159 waiting on getByText('Ports') Flaky in-lane, not a regression

Port-forward was re-run at the same commit and build and passed in 59.6s. It is green on main, green at the tip in isolation, and red only inside the full lane — it is the set's one @headful spec and runs last. Called flaky in-lane rather than "unrelated": a real lane-ordering problem, pre-existing, and worth tracking.

Note on the recorded run: the 14-spec result above predates the bulk-open exemption. The lane now runs 13 specs; bulk-open is not among them. The other 13 results are unaffected — nothing about that spec's failure changes another's outcome, since the runner passes them all to one Playwright invocation with --workers=1 and each spec owns its own fixture container.

Build provenance

resolveWorkspaceTerminalHostAuthority is a new identifier — 0 occurrences in main's source:

$ cat out/renderer/assets/*.js | grep -c resolveWorkspaceTerminalHostAuthority
4

Measured on the lane build:

BUILD_SHA=49bb96e0b4c135ce305a2f4dfe2d4b52e0b19cde   DIRTY=0
tombstone=13   hasLocalTabsRow=2   hostAuthority=4   mayCreate=3

resolveWorkspaceTerminalHostAuthority=4 is this level's symbol; the other three prove the rest of the stack is in the same build.

Provenance trap, reproduced on this tree. Do not grep a single renderer chunk. grep -c <sym> out/renderer/assets/store-*.js | head -1 matched store-DtDSfs7_.js and reported 0 for a symbol present 13 times across store-uJWnDUOY.js, App-CZkf3Bu7.js and quick-commands-menu-events-SBDd6cXN.js. Always use cat out/renderer/assets/*.js | grep -c <sym>.

Also: renderer console.warn never reaches the app log. Only main-process stdio is forwarded by ORCA_E2E_FORWARD_APP_LOGS=1.

What is NOT covered — read this before approving

The seeding half of this change has no measurable end-to-end effect today, and the branch's own spec says so. applyDirectSshRemoteWorkspaceSnapshot calls markRemoteWorkspaceHydrated unconditionally, after the hydrate calls — including when they wrote nothing. So in the same tick adoption yields zero, the verdict flips unverifiable → none, Terminal.tsx re-runs the effect, and it seeds. The gate cannot outlive the failure it guards against, because the same function that fails to adopt is the one that lifts it.

ssh-cold-hydration-gap-tab-seeding.spec.ts:218 is named for what it asserts — one tab seeded, none of the host's three adopted. :293 pins the intended behaviour with test.fixme and will report as skipped.

Making the seeding half effective needs hydration resolved per worktree — or a refusal to say none when the completed apply's replaceWorkspaceKeys did not name this worktree — rather than a per-target "some apply finished" flag. Deliberately not in this PR.

An earlier draft of this description warned that this change could leave a fresh client on an empty workspace (0 tabs), and said STA-3593 must land first. That warning is withdrawn: the risk is not reachable, for the same reason the seeding half is currently inert — hydration is marked on the failure path too, so the verdict flips and the seed happens. The real outcome is the status quo: one spurious tab, host sessions alive. No "adoption produced nothing" guard is needed and STA-3593 is not a prerequisite.

P1-B is not fixable without STA-3593, and the reason is worth recording. Seeding is already gated on renderableTabCount, not only on the verdict:

const shouldAutoCreate =
  hostAuthority === 'none' &&
  shouldAutoCreateInitialTerminal(renderableTabCount, shouldHonourClosedTerminalTombstone)

Enumerating what a per-worktree hydration gate would change, given the count is already consulted:

Apply outcome for worktree W Tabs Seeds today? Per-worktree gate changes it?
Resolved W, host had tabs > 0 No No — the count already stops it
Resolved W, host had an empty row 0 Yes, correctly No — and it must not
Did not resolve W, host genuinely has nothing 0 Yes, correctly No — and it must not
Did not resolve W, host does hold tabs we failed to map (STA-3593) 0 Yes, wrongly Only here

It bites in one row, and there it trades 1 tab where the host has 3 for 0 tabs with nothing to lift it — the sync succeeded (phase: 'synced'), so the bounded floor does not apply, and remoteWorkspaceHydratedTargetIds is add-only. So per-worktree resolution relocates the defect rather than closing it: the information needed to act correctly — which host tabs belong to this worktree — is precisely what got lost. It was considered and deliberately not built.

Rows 3 and 4 can be told apart, and that detection is probably what STA-3593's own fix will want. Compare the snapshot's tabsByWorktreePath entries that carry tabs against what importRemoteWorkspaceSession actually resolved through uniqueWorktreeIdByPath; a shortfall is positive evidence that the host holds tabs this client could not map. Recorded here because it was worked out here — but note it only lets you choose which bad outcome to serve, not produce a terminal for the user, which is why it is not acted on in this PR.

So why merge this? For the resume half, which is genuinely valuable and unit-proven: it declines while the host is unanswered and wakes the same session once the verdict lands, without consuming its one-shot. Preventing one duplicate claude --resume on a live transcript is worth more than preventing one spare tab — declining to resume is user-recoverable, a duplicate resume corrupts a transcript irreversibly. The seeding half is correct code that buys nothing observable yet, and becomes effective when per-worktree resolution or STA-3593 lands. That is a fine reason to merge; it is just not the reason an earlier draft gave.

Also not covered:

  • unverifiable is bounded, but only per target and only on first hydration. Since nothing clears the flag, a disconnected target that hydrated once reads none. This gate does not cover mid-session reconnect or sleep/resume.
  • Folder workspaces resolve none for the resume sweep too, so they keep full duplicate-resume exposure. Not a regression — that is today's behaviour — but it is a scope limit worth naming.
  • The duplicate-claude --resume harm is proven only at the unit layer. No e2e drives two concurrent agents onto one transcript; that would need a real agent process.
  • The local claude --resume defect ([Bug]: Cleanly exited Claude session is auto-resumed with claude --resume after an app restart (local macOS, hibernation off) #14228 / STA-4110) is out of scope — it reproduces on purely local macOS.

Failing-first test

Yes.

 FAIL  …fresh-client-tab-seeding.test.ts > holds one tab when the host still reports one tab
 FAIL  …fresh-client-tab-seeding.test.ts > adopts every tab the host owns instead of replacing them with a seeded one
 FAIL  …hydration-gap.test.ts > does not resume a remote session before the host has answered
 FAIL  …hydration-gap.test.ts > wakes the same session once the host answers and holds no pane for it
 FAIL  …host-authority.test.ts > distinguishes an unanswered SSH host from one that answered with nothing
 FAIL  …host-authority.test.ts > treats a conflicting host snapshot as unanswered
 FAIL  …host-authority.test.ts > does not seed locally when the runtime owner cannot be named
      Tests  7 failed | 3 passed (10)

with both production hunks reverted to main; 10 passed restored. The 3 that pass both ways are over-refusal tripwires — a workspace that never gets a terminal is the dangerous direction here.

@nwparker
nwparker force-pushed the nwparker/ssh-03-merge-preserves-local branch from 344cc8d to 07710c8 Compare August 27, 2026 09:49
@nwparker
nwparker force-pushed the nwparker/ssh-04-host-authority-seeding branch from 395a238 to fdc407c Compare August 27, 2026 09:49
@nwparker
nwparker force-pushed the nwparker/ssh-03-merge-preserves-local branch from 07710c8 to 01862a3 Compare August 27, 2026 09:53
@nwparker
nwparker force-pushed the nwparker/ssh-04-host-authority-seeding branch 3 times, most recently from e302d3b to 9cc5bac Compare August 27, 2026 10:10
@nwparker
nwparker force-pushed the nwparker/ssh-03-merge-preserves-local branch from 01862a3 to 903257d Compare August 28, 2026 02:43
… answered

Two client behaviours read local tab rows as the verdict on what the execution
host is running. Before the host answers, "I hold no pane for this" is
`unverifiable`, not `exited` -- the collapse
`docs/reference/ssh-execution-boundary.md` forbids.

Symptom 1, seeding. `worktree-initial-terminal-seeding.ts:47,128` seeds a
terminal when `renderableTabCount === 0`. Its only bail-out (`:72-77`) covered
the paired-web-runtime flavor -- "while that session is live the host owns
terminal creation" -- with no equivalent for direct SSH. So a client that has
never held the workspace runs the predicate during the hydration gap and creates
a tab from nothing. The snapshot then arrives, the merge rightly keeps the tab it
was never told about, and the union uploads as the new host truth. Measured on a
fresh client against a host owning 3 tabs: **1 tab created from nothing, 0 of the
host's 3 adopted.** (A restart never reaches the predicate -- local state
restores the row first -- which is why restart-only repros came back flat.)

That guard was also the wrong question. It asked "am I a client of a live paired
session?", which a host desktop window answers "no" and a paired client answers
"yes", so both seeded -- #15556.

Symptom 2, sleeping-agent resume, and the data-corrupting half.
`Terminal.tsx:1554` calls `resumeSleepingAgentSessionsForWorktree` twenty lines
after the seeding call at `:1529-1534` -- same startup path, same pre-hydration
window, and not SSH-gated at all. Seeding produces a spare empty tab; the sweep
launches `claude --resume <id>` for a session still running on the remote and
still owned by a live pane. Two agent processes writing one transcript; STA-3498
observed five. STA-3500 files exactly this race. Failure is asymmetric: declining
to resume is user-recoverable, a duplicate resume corrupts a transcript
irreversibly.

`workspace-terminal-host-authority.ts` answers the one ownership question both
paths ask, in the three-verdict vocabulary the renderer already uses for host
terminal inventory (`HostLiveTerminalProbeVerdict`, aliased rather than restated
so the two cannot drift): `live` (a remote host owns creation here),
`unverifiable` (there is a remote host and it has not answered), `none` (local,
or the host answered and holds nothing). Seeding requires `none`; the sweep
declines on `unverifiable` without consuming its one-shot, so the agents are not
stranded for the session once the verdict lands.

Shape notes:
- An ownership question, not a client-liveness one -- that is what fixes #15556.
- Folder workspaces resolve to `none`: the snapshot replaces exactly
  `DirectSshTargetScope.gitWorktreeIds`, so a folder's rows are never replaced by
  the host and waiting for an answer that will never name them would leave it
  terminal-less for good.
- A `conflict` sync phase is `unverifiable`, matching the pair
  `use-app-session-persistence.ts` already gates uploads on.
- Explicit launch work (setup/issue commands) stays ungated -- that is a request
  to create a terminal now.
- `Terminal.tsx` subscribes through a retained selector rather than reading in
  the effect: the verdict flipping to `none` is what must re-run the passes, and
  resolution walks the owner catalogs, so recomputing per store write would be
  the STA-3363 render-path multiplier again.

The `unverifiable` verdict is BOUNDED, and must be. `remoteWorkspaceHydratedTargetIds` is add-only
in practice -- `markRemoteWorkspaceHydrated` has two production call sites, both on success paths,
and `clearRemoteWorkspaceHydrated` has NONE. Four paths return without marking: local-hydration
timeout (`remote-workspace-target-sync.ts:136-145`), a null `remoteWorkspace.get` (`:160-169`), a
falsy apply token (`:172-185`), and never connecting at all. Without a floor, any of them would
leave every git worktree on that target `unverifiable` for the rest of the app session: no initial
terminal, no sleeping-agent resume, escapable only by creating a tab by hand. That is strictly worse
than the behaviour it replaces -- on main the user got a terminal. So a sync that terminates in
`offline` or `error` without ever hydrating resolves `none`: declining to seed is meant to be a
wait, not a permanent refusal. `pulling` still declines, and a target that HAS hydrated stays `none`
even if a later sync errors.

Scope, stated because the doc comment previously overstated it: this gate is
first-hydration-per-target, not per-connection-generation. Since nothing clears the flag, a
disconnected target that hydrated once reads `none`. It does not cover mid-session reconnect or
sleep/resume.

The memo's input list is checked for COMPLETENESS, not just membership. `satisfies readonly
(keyof State)[]` only proves each listed key exists; a field added to the state and forgotten from
the list would type-check while making the memo return a stale verdict -- silent, and it looks like
"the gate did not fire". A conditional type now names the missing key at compile time. Deliberately
not `const x: Missing[] = []`, which passes regardless because an empty array literal is assignable
to every array type.

Known limitation, stated rather than hidden: the SEEDING half of this change has no measurable
end-to-end effect today, and the branch's own e2e spec says so.
`applyDirectSshRemoteWorkspaceSnapshot` calls `markRemoteWorkspaceHydrated` unconditionally AFTER
the hydrate calls -- including when they wrote nothing. So in the same tick adoption yields zero,
the verdict flips `unverifiable` -> `none`, `Terminal.tsx` re-runs the effect, and it seeds. The
gate cannot outlive the failure it guards against, because the same function that fails to adopt is
the one that lifts it.

`ssh-cold-hydration-gap-tab-seeding.spec.ts:218` is named for what it asserts -- one tab, adopted
none -- rather than for the behaviour we want. The fixme at `:293` pins the intended behaviour.

Making the seeding half effective needs hydration resolved PER WORKTREE (or a refusal to say `none`
when the completed apply's `replaceWorkspaceKeys` did not name this worktree) rather than a
per-target "some apply finished" flag. That is deliberately not in this commit.

The RESUME half is the valuable half and is unit-proven: it declines while the host is unanswered
and wakes the same session once the verdict lands, without consuming its one-shot. Preventing one
duplicate `claude --resume` on a live transcript is worth more than preventing one spare tab --
declining to resume is user-recoverable, a duplicate resume corrupts a transcript irreversibly.

Before: 7 failed | 3 passed. After: 10 passed; 103 across the seeding, resume,
authority and remote-workspace suites.
@nwparker
nwparker force-pushed the nwparker/ssh-04-host-authority-seeding branch from 9cc5bac to a7c18c3 Compare August 28, 2026 02:44
@nwparker
nwparker changed the base branch from nwparker/ssh-03-merge-preserves-local to main August 28, 2026 02:44
@nwparker
nwparker merged commit e06a866 into main Aug 28, 2026
2 of 19 checks passed
@coderabbitai

coderabbitai Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5a2bf962-229f-4238-a455-6bb2a1b43aad

📥 Commits

Reviewing files that changed from the base of the PR and between 8b99c83 and a7c18c3.

📒 Files selected for processing (9)
  • config/scripts/run-ssh-docker-e2e.mjs
  • src/renderer/src/components/Terminal.tsx
  • src/renderer/src/hooks/remote-workspace-snapshot-fresh-client-tab-seeding.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session-direct-ssh-hydration-gap.test.ts
  • src/renderer/src/lib/resume-sleeping-agent-session.ts
  • src/renderer/src/lib/workspace-terminal-host-authority.test.ts
  • src/renderer/src/lib/workspace-terminal-host-authority.ts
  • src/renderer/src/lib/worktree-initial-terminal-seeding.ts
  • tests/e2e/ssh-cold-hydration-gap-tab-seeding.spec.ts

📝 Walkthrough

Walkthrough

The change adds workspace terminal host-authority resolution for local, runtime, and direct-SSH workspaces. Terminal seeding and sleeping-agent session resumption now wait for host hydration when authority is unverifiable. New unit tests cover authority states, seeding, and session resumption. New snapshot tests cover cold-start tab merging. New Docker-backed Playwright tests cover stalled hydration and fresh-client behavior. The SSH E2E runner now includes the new specification.

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

brennanb2025 added a commit that referenced this pull request Aug 28, 2026
… records

#16750 landed this test after the branch was written, so it still asserted the
eager clear the four sibling suites were already updated away from.
paidaxingyo666 pushed a commit to paidaxingyo666/Manta that referenced this pull request Aug 30, 2026
… answered (stablyai#16750)

Two client behaviours read local tab rows as the verdict on what the execution
host is running. Before the host answers, "I hold no pane for this" is
`unverifiable`, not `exited` -- the collapse
`docs/reference/ssh-execution-boundary.md` forbids.

Symptom 1, seeding. `worktree-initial-terminal-seeding.ts:47,128` seeds a
terminal when `renderableTabCount === 0`. Its only bail-out (`:72-77`) covered
the paired-web-runtime flavor -- "while that session is live the host owns
terminal creation" -- with no equivalent for direct SSH. So a client that has
never held the workspace runs the predicate during the hydration gap and creates
a tab from nothing. The snapshot then arrives, the merge rightly keeps the tab it
was never told about, and the union uploads as the new host truth. Measured on a
fresh client against a host owning 3 tabs: **1 tab created from nothing, 0 of the
host's 3 adopted.** (A restart never reaches the predicate -- local state
restores the row first -- which is why restart-only repros came back flat.)

That guard was also the wrong question. It asked "am I a client of a live paired
session?", which a host desktop window answers "no" and a paired client answers
"yes", so both seeded -- stablyai#15556.

Symptom 2, sleeping-agent resume, and the data-corrupting half.
`Terminal.tsx:1554` calls `resumeSleepingAgentSessionsForWorktree` twenty lines
after the seeding call at `:1529-1534` -- same startup path, same pre-hydration
window, and not SSH-gated at all. Seeding produces a spare empty tab; the sweep
launches `claude --resume <id>` for a session still running on the remote and
still owned by a live pane. Two agent processes writing one transcript; STA-3498
observed five. STA-3500 files exactly this race. Failure is asymmetric: declining
to resume is user-recoverable, a duplicate resume corrupts a transcript
irreversibly.

`workspace-terminal-host-authority.ts` answers the one ownership question both
paths ask, in the three-verdict vocabulary the renderer already uses for host
terminal inventory (`HostLiveTerminalProbeVerdict`, aliased rather than restated
so the two cannot drift): `live` (a remote host owns creation here),
`unverifiable` (there is a remote host and it has not answered), `none` (local,
or the host answered and holds nothing). Seeding requires `none`; the sweep
declines on `unverifiable` without consuming its one-shot, so the agents are not
stranded for the session once the verdict lands.

Shape notes:
- An ownership question, not a client-liveness one -- that is what fixes stablyai#15556.
- Folder workspaces resolve to `none`: the snapshot replaces exactly
  `DirectSshTargetScope.gitWorktreeIds`, so a folder's rows are never replaced by
  the host and waiting for an answer that will never name them would leave it
  terminal-less for good.
- A `conflict` sync phase is `unverifiable`, matching the pair
  `use-app-session-persistence.ts` already gates uploads on.
- Explicit launch work (setup/issue commands) stays ungated -- that is a request
  to create a terminal now.
- `Terminal.tsx` subscribes through a retained selector rather than reading in
  the effect: the verdict flipping to `none` is what must re-run the passes, and
  resolution walks the owner catalogs, so recomputing per store write would be
  the STA-3363 render-path multiplier again.

The `unverifiable` verdict is BOUNDED, and must be. `remoteWorkspaceHydratedTargetIds` is add-only
in practice -- `markRemoteWorkspaceHydrated` has two production call sites, both on success paths,
and `clearRemoteWorkspaceHydrated` has NONE. Four paths return without marking: local-hydration
timeout (`remote-workspace-target-sync.ts:136-145`), a null `remoteWorkspace.get` (`:160-169`), a
falsy apply token (`:172-185`), and never connecting at all. Without a floor, any of them would
leave every git worktree on that target `unverifiable` for the rest of the app session: no initial
terminal, no sleeping-agent resume, escapable only by creating a tab by hand. That is strictly worse
than the behaviour it replaces -- on main the user got a terminal. So a sync that terminates in
`offline` or `error` without ever hydrating resolves `none`: declining to seed is meant to be a
wait, not a permanent refusal. `pulling` still declines, and a target that HAS hydrated stays `none`
even if a later sync errors.

Scope, stated because the doc comment previously overstated it: this gate is
first-hydration-per-target, not per-connection-generation. Since nothing clears the flag, a
disconnected target that hydrated once reads `none`. It does not cover mid-session reconnect or
sleep/resume.

The memo's input list is checked for COMPLETENESS, not just membership. `satisfies readonly
(keyof State)[]` only proves each listed key exists; a field added to the state and forgotten from
the list would type-check while making the memo return a stale verdict -- silent, and it looks like
"the gate did not fire". A conditional type now names the missing key at compile time. Deliberately
not `const x: Missing[] = []`, which passes regardless because an empty array literal is assignable
to every array type.

Known limitation, stated rather than hidden: the SEEDING half of this change has no measurable
end-to-end effect today, and the branch's own e2e spec says so.
`applyDirectSshRemoteWorkspaceSnapshot` calls `markRemoteWorkspaceHydrated` unconditionally AFTER
the hydrate calls -- including when they wrote nothing. So in the same tick adoption yields zero,
the verdict flips `unverifiable` -> `none`, `Terminal.tsx` re-runs the effect, and it seeds. The
gate cannot outlive the failure it guards against, because the same function that fails to adopt is
the one that lifts it.

`ssh-cold-hydration-gap-tab-seeding.spec.ts:218` is named for what it asserts -- one tab, adopted
none -- rather than for the behaviour we want. The fixme at `:293` pins the intended behaviour.

Making the seeding half effective needs hydration resolved PER WORKTREE (or a refusal to say `none`
when the completed apply's `replaceWorkspaceKeys` did not name this worktree) rather than a
per-target "some apply finished" flag. That is deliberately not in this commit.

The RESUME half is the valuable half and is unit-proven: it declines while the host is unanswered
and wakes the same session once the verdict lands, without consuming its one-shot. Preventing one
duplicate `claude --resume` on a live transcript is worth more than preventing one spare tab --
declining to resume is user-recoverable, a duplicate resume corrupts a transcript irreversibly.

Before: 7 failed | 3 passed. After: 10 passed; 103 across the seeding, resume,
authority and remote-workspace suites.

(cherry picked from commit e06a866)
nwparker added a commit that referenced this pull request Sep 11, 2026
The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (#16750), and this only closes a coverage gap in it.
nwparker added a commit that referenced this pull request Sep 11, 2026
The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (#16750), and this only closes a coverage gap in it.
nwparker added a commit that referenced this pull request Sep 11, 2026
The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (#16750), and this only closes a coverage gap in it.
nwparker added a commit that referenced this pull request Sep 15, 2026
The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (#16750), and this only closes a coverage gap in it.
nwparker added a commit that referenced this pull request Sep 17, 2026
#19882)

* test(repro): #19735 resumes a published mirrored pane before its handle lands

* fix(runtime): park a mirrored pane's resume until its PTY handle lands

Mirror hydration means the host's tab rows arrived, not that a given pane's
liveness is decidable: the PTY handle lands one relay round trip later. On
that frame the pane read as not-live and the sweep resumed a session the
host was still running, producing a duplicate resume tab.

An empty handle map for a published row is unverifiable, never exited. Park
the pane on a per-pane wait with three bounded exits, each replaying the
sweep: its own handle lands, the row is retracted, or a deadline expires.
The deadline decides resume rather than an indefinite hold, and is scoped
to the connection generation so a reconnect re-arms it.

Closes #19735

* fix(runtime): bound the handle-gap expiry map to the current connection

* fix(runtime): void a handle-gap verdict the reconnect made stale

The per-pane park bounds itself with one deadline per connection, but the
waiter never recorded WHICH connection it was armed on. A wait armed on
generation 0 that fires after a reconnect stamps its expiry against the
current generation, so hasHostMirrorHandleWaitExpired agrees, the mirror
lookup returns null, and the pane is resumed after 1ms on a connection
that has had no chance to publish the handle. That is #19735's fork with
an extra step, reached through the guard that exists to prevent it.

The module's own doc comment claims the opposite -- "a reconnect bumps
the connection generation and arms a fresh wait" -- and that is true only
for a wait which had ALREADY expired, which is precisely the case the
existing test covered. The test and the comment agreed with each other
and both were wrong about the live case.

The waiter now carries the generation it was armed on and records no
verdict when the generation has moved; the replay re-parks through the
existing machinery and the new connection gets its own full budget. Still
bounded per connection generation, which is what was documented all along.

Also pins the three sibling attacks on the same window: two panes in one
environment where only one handle lands, a handle published by a foreign
environment, and an environment tearing its rows down mid-park (which
leaves no waiter and no scheduled timer).

The test file now leads with how to assert on this module at all, because
the obvious shape cannot fail. "Did the waiter release" is not an
observable here -- a waiter released for the wrong reason is re-parked by
the replayed sweep, so the store reads identically one tick later, and a
mutation releasing every waiter on any tab's handle survived twelve
assertions written that way. What a spurious release costs is the
deadline, so the assertions advance the clock and require the pane to
decide on the ORIGINAL schedule.

* fix(terminal): a live pane owns its transcript in any workspace

The resume dedup was scoped to the record's own workspace on both terms
-- the entry's tab had to be in worktreeTabIds AND entry.worktreeId had
to match -- and additionally required entry.state !== 'done'. A record
whose peer pane has finished a turn and still holds a live PTY therefore
matched nothing, and the sweep launched a second agent onto a transcript
the peer is still writing. Cross-workspace, it matched nothing even while
the peer was mid-turn.

The two ids really do drift. canonicalizeTerminalSessionWorktreeId
re-keys tabsByWorktree, tabGroups, tabGroupLayouts, activeTabIdByWorktree
and activeGroupIdByWorktree onto the canonical worktree id, and does NOT
re-key sleepingAgentSessionsByPaneKey, whose records carry worktreeId
inside them. So adopting an orphaned terminal is a direct producer of a
record naming one workspace while its pane and status row name another.

Split into two arms rather than widening the existing condition. The new
arm carries no workspace scope but demands hard evidence: a provider
session id names one transcript, so a pane whose exact PTY is live right
now already owns it wherever that pane sits, and no workspace boundary
makes a live PTY less live. The scoped arm keeps its scope and its
state !== 'done' term, because a status row with no live PTY is a claim
about the past and must not reach across workspaces.

Relationship to #19736: that PR fixes the SAME-workspace half of this in
the same function, by relaxing only the status term. This arm covers that
cell too -- measured both ways on this branch, which does not carry
#19736: its thirty `checks exact live ownership before resuming` cases
all pass with this change alone, and ten of them fail without it. So this
supersedes #19736 rather than sitting beside it, and #19736's one-line
`export` of stablePaneHasLivePty is carried here because this arm needs
it. If #19736 lands first this becomes a pure widening and its tests
should be kept. Both cells are pinned here either way.

* fix(runtime): isolate one pane's replay from the handle-gap drain

One store write releases every due pane, and the drain runs synchronously inside
a zustand subscriber. `waiter.run()` was unguarded, so a single pane's replay
reached two things it has no business touching:

  - the throw escapes out of `useAppStore.setState`, meaning the mirror apply
    that published the PTY handle throws at its own call site;
  - every pane queued behind the thrower is stranded — waiter still parked,
    deadline still armed — and then decides on a connection whose evidence
    landed long ago.

The deadline path fans out the same way, so a throwing replay also escaped the
timer callback.

Reachable: `resumeSleepingAgentSessionsForWorktree` reaches `state.createTab`
with no guard of its own. The panes in a drain are strangers to each other and
to the frame that released them; none of them should be able to see another's
failure.

The new tests live in their own file because
host-mirror-handle-gap-resume.test.ts drives the waiter through the real resume
sweep and so cannot choose what a replay DOES. Note for anyone extending that
file: per its header, "did the waiter release" is not an observable here — a
spurious release is re-parked immediately and reads identically one tick later.
These tests assert on timer count and on the deadline instead.

Also records two findings next to the code, so they are not rediscovered:
`expiredGenerationByPane` is never pruned for a removed environment (bounded and
inert, since removal advances the generation, but it does not drain — and a
DIFFERENT leak in that same map is being fixed concurrently, so reconcile rather
than patch around it); and sustained reconnect churn holding a pane parked
indefinitely is CORRECT, not the latch-that-never-releases defect, because under
churn liveness genuinely is unverifiable and ssh-execution-boundary.md forbids
resolving that to `exited`. It has the shape of the defect and will eventually
be "fixed" by someone who does not know that.

Mutation: dropping the guard kills exactly the three new assertions and leaves
all twelve existing waiter tests passing.

* fix(runtime): drain a removed environment's handle-gap verdicts on teardown

`expiredGenerationByPane` is pruned only by rules that run when a verdict is
RECORDED — the stale-generation sweep here, and the tab-death sweep added
separately (8f16641, env-scoped in c0e4423). An environment that is
REMOVED records nothing ever again, so neither rule can reach its rows and they
survive for the life of the session. Two orphan classes on one map; neither
prune subsumes the other, because both are driven by a recording.

Severity is a leak, not a correctness bug, and the commit pins WHY so nobody
re-derives it: removing an environment advances its connection generation, so a
stranded verdict can never match again even if the id returns. That test exists
to stop the generation advance being "optimised" away later, since it is the
only thing making the stranded row inert.

Hung off `clearWebSessionTabsTrackingForEnvironment` because that is the only
caller that fires for an environment that is going away.

Clears VERDICTS ONLY. Parked waiters deliberately survive, matching
`clearHostSessionMirrorHydration`: a re-pair or effect restart replaces the
connection's evidence, it does not cancel the recovery this client still owes
the pane. A waiter left behind is bounded by its own deadline and replays its
sweep exactly as it would have. Clearing them here would silently drop a parked
resume that nothing else replays.

A measurement worth recording, because it argued me out of a change I was about
to make: on the unfixed map the per-expiry rescan is super-linear — 500/1000/
2000/4000 sequential expiries cost 7.2/15.3/51.8/173.1 ms, doubling ratios
converging on ~3.35 against 4.0 for quadratic. That looked like a case for
reshaping the map to `Map<env, {generation, Set<tabId>}>`. It is not: the
quadratic is a property of the LEAK, not of the scan. Once the tab-death prune
holds the map at roughly one entry per environment the scan is over ~1 entry,
and a counting probe on the fixed map (summing `map.size` across N expiries,
which IS the iteration count and needs no clock) gives exactly N-1 — linear, and
2000x fewer iterations than quadratic at N=4000. The flat prefix loop used here
is the established pattern in this subsystem and needs no restructure.

Two methodology traps this cost, recorded for the next person measuring in this
repo: `vi.useFakeTimers()` fakes `process.hrtime` and `performance.now` as well,
so a timing harness reports the advanced deadline rather than work done — fake
only the timer surface under test. And expiring N panes in one burst measures
the fake-timer harness clearing N timers, not product code; 1000 panes "cost"
~1s that way and almost none of it was ours.

Mutations: a clear that drops nothing kills exactly the two assertions that
claim it drains, and correctly leaves the waiter-survival and generation-advance
tests passing. An UNSCOPED clear kills the same two, via their sibling-
environment half.

* fix(runtime): reconcile three branches' handle-gap verdict rules into one loop

Three agents changed `recordExpiredWait` on three branches and each verified only
their own. This is the union, resolved into the agreed shape and proved on one
tree. The rules are NOT alternatives — they have different safety properties, and
flattening them to one scope is wrong in both directions. Both wrong shapes were
independently written before this was reconciled, so the comments say why.

GENERATION rule, per key across EVERY environment (adv2-skew's class).
`hasHostMirrorHandleWaitExpired` compares a row against its own environment's
CURRENT generation, so a row whose generation has moved can never return true for
anybody; retiring it cannot cost a reader a verdict, whoever owns it. Scoped to
the recording environment, an environment that reconnects and then goes quiet
strands its rows forever.

TAB-DEATH rule, recording environment ONLY (my class). Row absence is transient
where a generation is not: a sibling mid-republish has no rows for a frame and
would lose a verdict its pane still needs — reproduced before it was narrowed.

Teardown drain (adv2-races' class) is unchanged and orthogonal: it is the only
trigger that fires for a REMOVED environment, whose rows no rule above reaches
because such an environment records no further verdict. Right predicate, wrong
trigger.

The union suite proves all four orphan classes simultaneously, plus the two
properties none of the three rules may break: the verdict stays sticky enough to
break the park/expire/replay loop, and no rule evicts a verdict a live pane still
needs. It uses three environments throughout, because with two at one generation
the candidate rules are indistinguishable and the naive fix survives.

THE FOURTH CLASS IS UNOWNED AND ASSERTED AS A HAZARD. A retracted tab id that is
republished inherits the old pane's verdict and skips its own wait. Unlike every
other gap on this map it is NOT conservative: the others drop a verdict and
re-park, holding longer, while this one retains a verdict and resumes on a handle
that has not landed — the #19735 direction. No rule reaches it: the tab-death
predicate stops matching once the id is republished, the teardown drain fires on
environment teardown rather than tab retraction, and no waiter exists to observe
the retraction because a pane holding a verdict never parks. Closing it needs a
fourth trigger, on row retraction. The suite pins the current behaviour so it
cannot be quietly forgotten.

Union finding, recorded rather than merged: adv2-skew's
`docs(relay): the live-broker wait budget does not bound the call` (6b02982)
is SKIPPED here. It documents the unbounded wait, and adv2-concurrency-fixes
(1673716) fixed exactly that by extracting the loop into
relay-live-broker-wait.ts. The doc and its test pin behaviour the union no longer
has. This is the kind of interaction neither branch could see alone.

* fix(test): repair the teardown suite the union broke

Cherry-picked from 46ad377 with the relay half dropped: that commit
also repaired relay-concurrency-policy-flip-mid-mint.test.ts, which does
not exist on this PR and belongs with the relay cluster's own branch.

The handle-gap half is what this PR needs. Neither break was visible on
its own branch -- both only appear once the verdict rules compose.

* fix(runtime): a handle-gap verdict answers for its pane, not for the tab id

Folds adv2-skew's e8cac05 into the reconciled union. Closes the fourth orphan
class, the only one that was not conservative: a retracted tab id republished as a
different pane inherited the old pane's verdict and skipped its own wait — the
#19735 direction rather than a longer hold.

It needs no fourth trigger, which is why it composes with the three drains rather
than competing with them. Every trigger those rules own fires downstream of the
moment this hazard needs. The verdict instead carries the environment-minted PTY
binding its pane held AT PARK TIME, and only answers for a pane that still holds
it: a republished pane binds a newly minted PTY and serves its own wait, while a
genuine reattach to the same PTY inherits, which is correct — the verdict follows
the PTY, not the id. A transient rowless frame touches neither, so the read-time
check is safe where a retraction-triggered prune would not have been.

TWO MEASUREMENTS, both requested rather than assumed.

1. The record-then-release ordering is load-bearing and IS pinned. `recordExpiredWait`
reads the waiter's park-time binding, so it must run before `releaseWaiter` deletes
the entry. Swapping the two statements fails three cases, so the capture is not
correct merely by accident of statement order.

2. The `''` fallback is a MATCH VALUE, not a null: two panes that both hold no
environment-minted PTY compare equal and inherit, which is the same hazard in a
narrower window. Measured unreachable through the production park path rather than
assumed — the only route in is `kind: 'handle'`, which `findUnhydratedHostMirrorForPane`
reports only when `tabHoldsEnvironmentPtyBinding` finds a binding, reading the SAME
map through the SAME predicate as `paneBindingFor`. It now refuses to answer anyway.
That coupling is two functions in two files with nothing enforcing it, refusing costs
only a re-park, and the direction is conservative.

THE REFUSAL IS WHAT FOUND THE REAL BUG. With `''` matching, any fixture that omits
`terminalLayoutsByTabId` records `''`, compares `'' === ''`, and passes while the
pane-identity check is entirely inert. Making it refuse turned that silence into
four failures across host-mirror-handle-gap-drain and -teardown, whose fixtures seed
no layout binding at all. Both now bind per environment — one shared environment id
filters every other environment's pane back to `''` and restores the no-op.

Mutation-tested on the merged tree: ignoring the binding fails case D and the
mid-wait case; re-reading at expiry fails the mid-wait case and nothing else;
letting `''` match fails the empty-binding case; widening the tab-death rule across
environments still fails the live-verdict case, so pane identity does not weaken the
scoping the sweep was reconciled around.

Also fixes a real-clock race this branch introduced: the revoke-window test read
`Date.now()` separately from `enqueue`'s own stamp, and under load the drift ate
into the window. It now anchors the injected clock to the item's `createdAt`.

* docs(runtime): the two guards on the park-time binding are not redundant

Recording a reconciliation result that existed only in a review thread, and
correcting it in the process — measuring the claim changed it.

The claim under review was that the `?? ''` fallback in `recordExpiredWait` is
unreachable by two independent guards, either sufficient alone: the caller's
generation gate (a missing waiter fails `undefined === number`) and the
record-before-release ordering. That is not what the code does.

Measured, by removing each in turn:

  - ordering removed, generation gate kept: the gate does NOT carry it. With the
    waiter already deleted, the gate is false on every expiry, so nothing is ever
    recorded — five failures, and the door is shut by breaking the mechanism rather
    than by refusing ''.
  - generation gate removed, ordering kept: 736 files green, one failure, and it is
    `does not let a wait armed on the previous connection decide the new one` in
    host-mirror-handle-gap-resume.test.ts — a different property entirely.

So the ordering alone makes `''` unreachable, and the generation gate is not a second
guard on it at all: it pins reconnect-void. Both are load-bearing, for different
reasons, which is a stronger argument against removing either than redundancy would
have been — redundancy invites deleting one.

Worth writing in the file because the two sit three lines apart and read as belt and
braces on the same thing. The `''` comment next to them already exists because an
unexplained guard on an unreachable value gets deleted as dead code in a year; a guard
that looks redundant is deleted sooner.

No behaviour change. One comment, corrected against measurement rather than against the
thread it came from.

* fix(runtime): a published handle retires the verdict it answered

The fourth eviction trigger on `expiredGenerationByPane`, and the reason it is
not redundant with the three already there or with the two other agents' guards
on this same map.

A verdict records that a pane's 15s handle-gap wait ran out. Nothing retires it
when that pane subsequently publishes its handle, so the NEXT gap on that pane
gets no wait at all — #19735 with the bounded wait removed rather than merely
shortened. Measured on the reconciled union tree (1b621b6) plus the outage
guard: the verdict still answered `true` after the handle landed, and the second
gap resumed with zero panes parked.

Why none of the existing rules reach it, each checked rather than assumed:
  - superseded generation: #19647 in this same stack stops recording
    `status: null` for an unreachable host, so the generation no longer moves
    across an outage on one runtime.
  - dead tab row: the row stays published throughout. It is the HANDLE that
    comes and goes — that is the definition of the gap.
  - removed environment: the environment is still here.
  - read-time pane identity (adv2-skew, cdafc90): the pane keeps the same
    layout binding across the gap BY DESIGN, and the union suite pins that a
    genuine reattach to the same PTY must inherit. That check discriminates a
    different pane behind one tab id; this one discriminates a later gap on the
    same pane.
  - contact lost (adv3-journeys, 2da6624): no outage is involved; this is a
    healthy connection where the host was simply slow once.

Composition proven by mutation on the union tree, four disjoint kills: dropping
this drain kills 2 tests and only mine; dropping the re-park worktree kills 1 and
only mine; dropping the contact guard kills 1 and only theirs; forcing the
contact guard always-true kills 16 across every suite. No mutation kills another
agent's test, so these are three guards on three holes, not three on one.

Also carries `worktreeId` across a re-park: adopting an orphaned terminal re-keys
`tabsByWorktree` without re-keying the record, so a live wait kept releasing on
retraction evidence about the workspace it was no longer about. The park-time
`paneBinding` deliberately does not move with it — that is the pane's identity,
this is only where its rows are filed.

* docs(runtime): the reused-tab-id class is closed at read time, not still open

The `ExpiredHandleGapVerdict` docstring told the next reader that a retracted
tab id republished under the same id still inherits its predecessor's verdict,
and that closing it "needs a fourth trigger, on row retraction". The test it
names as its own pin says the opposite: class D in
host-mirror-handle-gap-verdict-union.test.ts asserts the verdict does not
answer, and explains it is closed at READ time rather than by any prune.

Provenance, since two sources disagreeing is what made this expensive: the
paragraph was last written in 46ad377 and the read-time pane-identity check
landed one commit later in cdafc90 (adv2-skew). The prose predates its own
fix by a single commit and was never updated. Confirmed by mutation rather than
by reading: dropping `verdict.paneBinding === paneBindingFor(...)` fails exactly
"handles all four orphan classes simultaneously", which is the class-D
assertion, so the read-time check is what closes it.

Rewritten to say what the code does, keeping the part that was always true —
why no trigger could have reached that class — and keeping the distinction the
new PUBLISHED HANDLE drain needs: read-time identity separates two panes behind
one tab id, the drain separates two gaps on one pane. The drain does not close
class D and must not be read as closing it.

Also records why this block specifically keeps going stale: several agents
change this map in parallel, the invariants move faster than the prose, and when
the two disagree the test file is the one that ran.

* test(runtime): pin replay containment on the deadline path too

Cherry-picked from aa98edf (nwparker/adv3-failure-fixes) with its
implementation hunk dropped: a second agent found the same throwing-replay
hole independently, and `15f34014153` already closed it on this branch with
an equivalent guard. Applying both would have been a double-apply, and the
two spellings of the log line would have shipped side by side.

The tests are worth keeping regardless. The first duplicates coverage
15f3401 already has; the second does not -- it drives the throw from the
DEADLINE path rather than the store-write path, which is a separate call into
releaseWaiter and was unpinned.

Spies retargeted from console.error to console.warn, the channel the guard
that actually shipped writes to, so the suite silences what the code emits.

* fix(runtime): isolate one worktree's replay from the mirror-hydration drain

The same fan-out hazard as the handle-gap drain, one module up. Settling an
environment drains every worktree parked on it in a single loop, called from the
frame apply, with `waiter.run()` unguarded. One replay that throws strands every
waiter queued behind it and surfaces in the caller applying the frame.

Found by looking for the sibling of a defect rather than by a separate
interleaving: both modules park a `run` callback and drain N of them from one
event, so both have the same blast radius. Kept as its own commit because the
two modules route independently.

Mutation: dropping the guard kills exactly the one new assertion.

* test(runtime): pin sleeping-agent resume on a failed SSH target

The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (#16750), and this only closes a coverage gap in it.

* test(runtime): pin the store subscription the reconciled loop can leak

The retention suite that `reconcile three branches' handle-gap verdict rules
into one loop` replaced carried an assertion the split suites did not: the
store subscription is held for exactly as long as something needs it.

Measured before writing it, because half of it was already covered:

  RETAIN direction -- drop the verdict term from `stopStoreSubscriptionIfIdle`
  so a verdict with no waiter behind it loses the subscription its drain needs:
  already caught, 2 failures in host-mirror-handle-gap-landed-handle.test.ts.

  RELEASE direction -- never release the subscription at all: caught by
  NOTHING. That mutation passes all 272 tests across the 33 other handle-gap
  and session-tabs suites. A leaked subscription rescans every parked pane on
  every store write for the life of the session and nothing notices.

So this is for the release direction. The retain cases ride along because both
halves of one invariant belong in one file, not because they were missing.
That term is also precisely what the reconcile moved -- it now counts verdicts
as well as waiters -- so it is the part of this map most likely to drift again.

Asserted with a spy on useAppStore.subscribe rather than a new test-only
export: whether the module is subscribed is already observable at the store
boundary, and the production surface should not grow just to say so.

* fix(lint): carry SAFETY rationales for the handle-gap fixtures

main tightened typescript/consistent-type-assertions to assertionStyle:
never after this branch was written. The gate only ran here once the
rebase put the casting config at the merge base, so these sites are new
to it, not new to the branch. The store seeds are genuinely partial --
dropping the casts does not typecheck -- so each carries its rationale.

* fix(runtime): release a handle-gap pane once per store write

`releaseDueWaiters` snapshotted the due KEYS and then re-looked-up each one. A
replay earlier in the loop writes to the store — the sweep reaches `createTab`
and `clearSleepingAgentSession` — and zustand notifies re-entrantly with no
queue, so the nested pass can release and re-park a pane still queued in the
outer loop. The outer `releaseWaiter(key)` then found the re-park, cleared its
brand-new deadline and replayed it a second time off one store write, handing
that pane another full budget.

That is the extension `parkUntilHostMirrorHandleLands` already refuses to grant
a re-park, arriving through a different door. The direction is conservative
(hold longer, never resume early), which is why no outcome assertion could see
it; only the replay count separates the two implementations. Snapshot the waiter
alongside its key and release only while the map still holds that same waiter.

Also folds host-mirror-handle-gap-replay-containment.test.ts into the drain
suite, since the guard it pins is the one this commit extends. It was the same
fix imported twice: its deadline case is a strict subset of the drain suite's,
its store-write case differs only by also asserting that later store listeners
still run, and one mutation — rethrowing from the replay catch — killed all five
cases across both files. Its fixture also seeded no layout bindings and used tab
ids `isWebTerminalSurfaceTabId` rejects, so those panes could not have reached
the park path it claimed to exercise. The unique assertion moves across; the
file goes.

Killed by `releases a pane once per store write even when an earlier replay
re-enters the drain`: 2 replay calls instead of 1 without the identity guard.

* test(runtime): retire the handle-gap assertions that could not fail

`returns to baseline under churn across all three drains` asserted nothing. It
ran 300 expiries and then cleared every environment's verdicts by name before
counting, so the map was empty by construction — deleting the whole prune loop
in `recordExpiredWait` left the test green. It now asserts the bound BEFORE the
teardown clear: 300 expiries must leave exactly one live verdict per
environment. It also binds each round's pane to the environment recording it;
the old fixture filed every binding under env-a, so two rounds in three stored
the empty match value the read-time check refuses, and that much of the churn
was synthetic. Renamed: there are four drains, not three.

Two comments described outcomes their assertions do not produce. `c1` reads
false on the read-time generation gate alone, whether or not a drain ever swept
it — the count below is the only assertion that distinguishes retired from
stranded. And the discriminator in `never evicts a live pane verdict` is env-b,
whose row goes absent while env-a records; env-c is a control that holds under
every candidate rule.

Two more fixtures modelled states the mirror apply cannot produce, both leaning
on `ptyIdsByTabId[tab]` holding a PTY id no leaf of that tab is bound to. It
builds one from the other (web-session-tabs-sync/terminal-build.ts), so they can
never disagree. The producible shape is a SPLIT tab whose sibling surface went
`ready` first, which is what both cases now seed — and which makes the residual
they were quietly standing in for visible instead: the decidability gate above
this wait is tab-granular while everything below it is leaf-aware, so a sibling
handle ends the wait for a surface still `pending-handle`. Recorded at the gate
in host-mirrored-pane-liveness.ts, pinned by name, and left open here: it needs
the per-surface status the host already publishes and the client drops on apply.

Also states what the unscoped live-PTY arm trades — a finished agent whose shell
is still up releases its record and will not auto-resume — because it reads as a
regression and is not one. And replaces the subscription-lifetime header's
unreproducible "272 tests across 33 suites" with the measured 326 across 37.

* fix(runtime): re-judge a handle-gap waiter the drain's own replay moved

The identity guard added one commit ago catches only half of how the drain's
snapshot goes stale. It proves the map entry was not REPLACED; it cannot prove
the verdict still holds, because `parkUntilHostMirrorHandleLands` re-parks a
still-parked pane by MUTATING the waiter in place. `worktreeId` moves with `run`
— that is what adopting an orphaned terminal does — and object identity survives
it. So a waiter the snapshot judged retracted, because its tab was absent from
the worktree it was filed under, can be re-filed by an earlier replay in the
same loop and then released on evidence about a workspace it is no longer about.
That is the defect the `existing.worktreeId` assignment exists to prevent,
reached through the drain instead.

Neither guard covers the other: re-judging alone still replays a re-park twice
(it was just made due, so it re-judges due), and identity alone misses the
mutation. Both, in that order. A waiter that is no longer due simply stays
parked — bounded by its own deadline and re-judged on the next store write, so
declining costs at most one frame of latency.

Killed by `does not release on retraction evidence a mid-drain re-park has
already made stale`; the identity half is still killed by `replays a pane once
per store write even when an earlier replay re-enters the drain`. 29 mutations
across these modules, no survivors.

Corrects three claims made in the two preceding commits, each wrong in a way a
future reader would have acted on:

- the duplicate release does NOT extend the pane's budget. `releaseWaiter`
  deletes the waiter before calling `run`, so the re-park takes the `!existing`
  branch and arms a full deadline either way; a second release clears and re-arms
  at the same instant. What it costs is running an entire worktree resume sweep
  twice off one frame. The test is renamed to say so.
- `retainPendingTerminalBindings` carries a `pending-handle` surface's prior
  binding forward, so the split-tab residual cannot arise from a bound leaf going
  pending — it needs a leaf that was NEVER bound, which means a cold start or
  re-pair with no layout to retain from. The fixture staged the impossible
  history; it now seeds the producible first frame, where the point sharpens: no
  wait is armed at all, which the test now asserts directly.
- `clearSleepingAgentSession` cannot re-enter the drain; the subscription's slice
  guard drops that write. Only `createTab` can.

Also: the churn assertion pins that the prune loop runs at all, not which rule
prunes — with one live tab per round either rule alone still reads 3. Says so,
and points at the case that does isolate the generation rule.

* test(runtime): stage the handle-gap adoption the way a sweep can reach it

The case added one commit ago pinned the right guard through a call sequence
production cannot make. A replay is `resumeSleepingAgentSessionsForWorktree`
closed over ONE worktree, so it re-parks only under that worktree — but the
fixture had the first pane's replay re-park the second under a DIFFERENT one.
That is the same fault the previous commit corrected in the resume fixture, made
one file over.

The reachable route: the second tab has already been re-keyed onto the canonical
worktree id while its sleeping record still names the old one, so the first
pane's sweep legitimately owns it and re-parks it there, mutating the live
waiter in place. The waiter's snapshot verdict — "retracted", because the tab is
absent from the id it was filed under — is stale by the time the loop reaches
it. Restaged that way, with both waiters and the re-park inside one worktree.

Also drops the second global store read. Guard TWO now re-judges against the
same `state` the drain was notified with, because the two provably cannot differ
here: a re-park only happens when `findUnhydratedHostMirrorForPane` finds the
row already filed under the sweeping worktree, which is a row this frame carries.
`useAppStore.getState()` was an unpinnable degree of freedom — swapping it for
`state` left every test green — and it contradicted the drain's own claim to
judge one snapshot.

And corrects two more claims: guard ordering is a cost preference, not a
correctness requirement (either order works; identity first is just cheaper),
and "no wait is armed here" in the split-tab fixture is not caused by the cold
start — it is the tab-granular gate reading the sibling's handle, which is the
residual itself.

29 mutations across these modules, no survivors. Each guard is killed by exactly
one case and they do not overlap: `replays a pane once per store write even when
an earlier replay re-enters the drain` for identity, `does not release on
retraction evidence a mid-drain adoption has already made stale` for re-judging.

* fix(runtime): judge a re-parked handle-gap waiter on the live store

Reverts the `state` read the previous commit put in guard TWO, and says why the
difference is deliberately unpinnable rather than leaving the next reader to
"simplify" it back.

The previous commit swapped `useAppStore.getState()` for the subscriber's
`state` because a reviewer noted the swap left every test green. That was
optimising for mutation-killability over the property the module exists to
protect. The two reads do agree on every sequence the sweep can produce — a
replay's only store write is `createTab`, which appends a freshly minted tab id,
so it can neither make an absent tab id present nor touch `ptyIdsByTabId` — which
is exactly why no test separates them. But they are not interchangeable: `state`
is the staler of the two, and its failure direction is to RELEASE a pane whose
row has come back. That is resolving unverifiable to exited, which is #19735.

Holding on evidence that might be stale costs one frame; acting on it forks a
transcript. Take the fresher read, and record in the comment that no test can
fail on this and why that is not a reason to change it.

Both guards remain killed by exactly one case each and they do not overlap:
`replays a pane once per store write even when an earlier replay re-enters the
drain` for identity, `does not release on retraction evidence a mid-drain
adoption has already made stale` for re-judging.

Verified while confirming the previous commit's test premises against production,
both of which hold: a sweep that parks every record writes nothing to the store
(resume-sleeping-agent-session.ts takes `continue` on the park branch), and
`workspace-session-worktree-id.ts` moves `tabsByWorktree` onto the canonical id
while leaving `sleepingAgentSessionsByPaneKey` naming the old one — the stale
`worktreeId` the adoption case depends on.
ivanvykopal pushed a commit to ivanvykopal/orca that referenced this pull request Sep 20, 2026
stablyai#19882)

* test(repro): stablyai#19735 resumes a published mirrored pane before its handle lands

* fix(runtime): park a mirrored pane's resume until its PTY handle lands

Mirror hydration means the host's tab rows arrived, not that a given pane's
liveness is decidable: the PTY handle lands one relay round trip later. On
that frame the pane read as not-live and the sweep resumed a session the
host was still running, producing a duplicate resume tab.

An empty handle map for a published row is unverifiable, never exited. Park
the pane on a per-pane wait with three bounded exits, each replaying the
sweep: its own handle lands, the row is retracted, or a deadline expires.
The deadline decides resume rather than an indefinite hold, and is scoped
to the connection generation so a reconnect re-arms it.

Closes stablyai#19735

* fix(runtime): bound the handle-gap expiry map to the current connection

* fix(runtime): void a handle-gap verdict the reconnect made stale

The per-pane park bounds itself with one deadline per connection, but the
waiter never recorded WHICH connection it was armed on. A wait armed on
generation 0 that fires after a reconnect stamps its expiry against the
current generation, so hasHostMirrorHandleWaitExpired agrees, the mirror
lookup returns null, and the pane is resumed after 1ms on a connection
that has had no chance to publish the handle. That is stablyai#19735's fork with
an extra step, reached through the guard that exists to prevent it.

The module's own doc comment claims the opposite -- "a reconnect bumps
the connection generation and arms a fresh wait" -- and that is true only
for a wait which had ALREADY expired, which is precisely the case the
existing test covered. The test and the comment agreed with each other
and both were wrong about the live case.

The waiter now carries the generation it was armed on and records no
verdict when the generation has moved; the replay re-parks through the
existing machinery and the new connection gets its own full budget. Still
bounded per connection generation, which is what was documented all along.

Also pins the three sibling attacks on the same window: two panes in one
environment where only one handle lands, a handle published by a foreign
environment, and an environment tearing its rows down mid-park (which
leaves no waiter and no scheduled timer).

The test file now leads with how to assert on this module at all, because
the obvious shape cannot fail. "Did the waiter release" is not an
observable here -- a waiter released for the wrong reason is re-parked by
the replayed sweep, so the store reads identically one tick later, and a
mutation releasing every waiter on any tab's handle survived twelve
assertions written that way. What a spurious release costs is the
deadline, so the assertions advance the clock and require the pane to
decide on the ORIGINAL schedule.

* fix(terminal): a live pane owns its transcript in any workspace

The resume dedup was scoped to the record's own workspace on both terms
-- the entry's tab had to be in worktreeTabIds AND entry.worktreeId had
to match -- and additionally required entry.state !== 'done'. A record
whose peer pane has finished a turn and still holds a live PTY therefore
matched nothing, and the sweep launched a second agent onto a transcript
the peer is still writing. Cross-workspace, it matched nothing even while
the peer was mid-turn.

The two ids really do drift. canonicalizeTerminalSessionWorktreeId
re-keys tabsByWorktree, tabGroups, tabGroupLayouts, activeTabIdByWorktree
and activeGroupIdByWorktree onto the canonical worktree id, and does NOT
re-key sleepingAgentSessionsByPaneKey, whose records carry worktreeId
inside them. So adopting an orphaned terminal is a direct producer of a
record naming one workspace while its pane and status row name another.

Split into two arms rather than widening the existing condition. The new
arm carries no workspace scope but demands hard evidence: a provider
session id names one transcript, so a pane whose exact PTY is live right
now already owns it wherever that pane sits, and no workspace boundary
makes a live PTY less live. The scoped arm keeps its scope and its
state !== 'done' term, because a status row with no live PTY is a claim
about the past and must not reach across workspaces.

Relationship to stablyai#19736: that PR fixes the SAME-workspace half of this in
the same function, by relaxing only the status term. This arm covers that
cell too -- measured both ways on this branch, which does not carry
stablyai#19736: its thirty `checks exact live ownership before resuming` cases
all pass with this change alone, and ten of them fail without it. So this
supersedes stablyai#19736 rather than sitting beside it, and stablyai#19736's one-line
`export` of stablePaneHasLivePty is carried here because this arm needs
it. If stablyai#19736 lands first this becomes a pure widening and its tests
should be kept. Both cells are pinned here either way.

* fix(runtime): isolate one pane's replay from the handle-gap drain

One store write releases every due pane, and the drain runs synchronously inside
a zustand subscriber. `waiter.run()` was unguarded, so a single pane's replay
reached two things it has no business touching:

  - the throw escapes out of `useAppStore.setState`, meaning the mirror apply
    that published the PTY handle throws at its own call site;
  - every pane queued behind the thrower is stranded — waiter still parked,
    deadline still armed — and then decides on a connection whose evidence
    landed long ago.

The deadline path fans out the same way, so a throwing replay also escaped the
timer callback.

Reachable: `resumeSleepingAgentSessionsForWorktree` reaches `state.createTab`
with no guard of its own. The panes in a drain are strangers to each other and
to the frame that released them; none of them should be able to see another's
failure.

The new tests live in their own file because
host-mirror-handle-gap-resume.test.ts drives the waiter through the real resume
sweep and so cannot choose what a replay DOES. Note for anyone extending that
file: per its header, "did the waiter release" is not an observable here — a
spurious release is re-parked immediately and reads identically one tick later.
These tests assert on timer count and on the deadline instead.

Also records two findings next to the code, so they are not rediscovered:
`expiredGenerationByPane` is never pruned for a removed environment (bounded and
inert, since removal advances the generation, but it does not drain — and a
DIFFERENT leak in that same map is being fixed concurrently, so reconcile rather
than patch around it); and sustained reconnect churn holding a pane parked
indefinitely is CORRECT, not the latch-that-never-releases defect, because under
churn liveness genuinely is unverifiable and ssh-execution-boundary.md forbids
resolving that to `exited`. It has the shape of the defect and will eventually
be "fixed" by someone who does not know that.

Mutation: dropping the guard kills exactly the three new assertions and leaves
all twelve existing waiter tests passing.

* fix(runtime): drain a removed environment's handle-gap verdicts on teardown

`expiredGenerationByPane` is pruned only by rules that run when a verdict is
RECORDED — the stale-generation sweep here, and the tab-death sweep added
separately (8f16641, env-scoped in c0e4423). An environment that is
REMOVED records nothing ever again, so neither rule can reach its rows and they
survive for the life of the session. Two orphan classes on one map; neither
prune subsumes the other, because both are driven by a recording.

Severity is a leak, not a correctness bug, and the commit pins WHY so nobody
re-derives it: removing an environment advances its connection generation, so a
stranded verdict can never match again even if the id returns. That test exists
to stop the generation advance being "optimised" away later, since it is the
only thing making the stranded row inert.

Hung off `clearWebSessionTabsTrackingForEnvironment` because that is the only
caller that fires for an environment that is going away.

Clears VERDICTS ONLY. Parked waiters deliberately survive, matching
`clearHostSessionMirrorHydration`: a re-pair or effect restart replaces the
connection's evidence, it does not cancel the recovery this client still owes
the pane. A waiter left behind is bounded by its own deadline and replays its
sweep exactly as it would have. Clearing them here would silently drop a parked
resume that nothing else replays.

A measurement worth recording, because it argued me out of a change I was about
to make: on the unfixed map the per-expiry rescan is super-linear — 500/1000/
2000/4000 sequential expiries cost 7.2/15.3/51.8/173.1 ms, doubling ratios
converging on ~3.35 against 4.0 for quadratic. That looked like a case for
reshaping the map to `Map<env, {generation, Set<tabId>}>`. It is not: the
quadratic is a property of the LEAK, not of the scan. Once the tab-death prune
holds the map at roughly one entry per environment the scan is over ~1 entry,
and a counting probe on the fixed map (summing `map.size` across N expiries,
which IS the iteration count and needs no clock) gives exactly N-1 — linear, and
2000x fewer iterations than quadratic at N=4000. The flat prefix loop used here
is the established pattern in this subsystem and needs no restructure.

Two methodology traps this cost, recorded for the next person measuring in this
repo: `vi.useFakeTimers()` fakes `process.hrtime` and `performance.now` as well,
so a timing harness reports the advanced deadline rather than work done — fake
only the timer surface under test. And expiring N panes in one burst measures
the fake-timer harness clearing N timers, not product code; 1000 panes "cost"
~1s that way and almost none of it was ours.

Mutations: a clear that drops nothing kills exactly the two assertions that
claim it drains, and correctly leaves the waiter-survival and generation-advance
tests passing. An UNSCOPED clear kills the same two, via their sibling-
environment half.

* fix(runtime): reconcile three branches' handle-gap verdict rules into one loop

Three agents changed `recordExpiredWait` on three branches and each verified only
their own. This is the union, resolved into the agreed shape and proved on one
tree. The rules are NOT alternatives — they have different safety properties, and
flattening them to one scope is wrong in both directions. Both wrong shapes were
independently written before this was reconciled, so the comments say why.

GENERATION rule, per key across EVERY environment (adv2-skew's class).
`hasHostMirrorHandleWaitExpired` compares a row against its own environment's
CURRENT generation, so a row whose generation has moved can never return true for
anybody; retiring it cannot cost a reader a verdict, whoever owns it. Scoped to
the recording environment, an environment that reconnects and then goes quiet
strands its rows forever.

TAB-DEATH rule, recording environment ONLY (my class). Row absence is transient
where a generation is not: a sibling mid-republish has no rows for a frame and
would lose a verdict its pane still needs — reproduced before it was narrowed.

Teardown drain (adv2-races' class) is unchanged and orthogonal: it is the only
trigger that fires for a REMOVED environment, whose rows no rule above reaches
because such an environment records no further verdict. Right predicate, wrong
trigger.

The union suite proves all four orphan classes simultaneously, plus the two
properties none of the three rules may break: the verdict stays sticky enough to
break the park/expire/replay loop, and no rule evicts a verdict a live pane still
needs. It uses three environments throughout, because with two at one generation
the candidate rules are indistinguishable and the naive fix survives.

THE FOURTH CLASS IS UNOWNED AND ASSERTED AS A HAZARD. A retracted tab id that is
republished inherits the old pane's verdict and skips its own wait. Unlike every
other gap on this map it is NOT conservative: the others drop a verdict and
re-park, holding longer, while this one retains a verdict and resumes on a handle
that has not landed — the stablyai#19735 direction. No rule reaches it: the tab-death
predicate stops matching once the id is republished, the teardown drain fires on
environment teardown rather than tab retraction, and no waiter exists to observe
the retraction because a pane holding a verdict never parks. Closing it needs a
fourth trigger, on row retraction. The suite pins the current behaviour so it
cannot be quietly forgotten.

Union finding, recorded rather than merged: adv2-skew's
`docs(relay): the live-broker wait budget does not bound the call` (6b02982)
is SKIPPED here. It documents the unbounded wait, and adv2-concurrency-fixes
(1673716) fixed exactly that by extracting the loop into
relay-live-broker-wait.ts. The doc and its test pin behaviour the union no longer
has. This is the kind of interaction neither branch could see alone.

* fix(test): repair the teardown suite the union broke

Cherry-picked from 46ad377 with the relay half dropped: that commit
also repaired relay-concurrency-policy-flip-mid-mint.test.ts, which does
not exist on this PR and belongs with the relay cluster's own branch.

The handle-gap half is what this PR needs. Neither break was visible on
its own branch -- both only appear once the verdict rules compose.

* fix(runtime): a handle-gap verdict answers for its pane, not for the tab id

Folds adv2-skew's e8cac05 into the reconciled union. Closes the fourth orphan
class, the only one that was not conservative: a retracted tab id republished as a
different pane inherited the old pane's verdict and skipped its own wait — the
stablyai#19735 direction rather than a longer hold.

It needs no fourth trigger, which is why it composes with the three drains rather
than competing with them. Every trigger those rules own fires downstream of the
moment this hazard needs. The verdict instead carries the environment-minted PTY
binding its pane held AT PARK TIME, and only answers for a pane that still holds
it: a republished pane binds a newly minted PTY and serves its own wait, while a
genuine reattach to the same PTY inherits, which is correct — the verdict follows
the PTY, not the id. A transient rowless frame touches neither, so the read-time
check is safe where a retraction-triggered prune would not have been.

TWO MEASUREMENTS, both requested rather than assumed.

1. The record-then-release ordering is load-bearing and IS pinned. `recordExpiredWait`
reads the waiter's park-time binding, so it must run before `releaseWaiter` deletes
the entry. Swapping the two statements fails three cases, so the capture is not
correct merely by accident of statement order.

2. The `''` fallback is a MATCH VALUE, not a null: two panes that both hold no
environment-minted PTY compare equal and inherit, which is the same hazard in a
narrower window. Measured unreachable through the production park path rather than
assumed — the only route in is `kind: 'handle'`, which `findUnhydratedHostMirrorForPane`
reports only when `tabHoldsEnvironmentPtyBinding` finds a binding, reading the SAME
map through the SAME predicate as `paneBindingFor`. It now refuses to answer anyway.
That coupling is two functions in two files with nothing enforcing it, refusing costs
only a re-park, and the direction is conservative.

THE REFUSAL IS WHAT FOUND THE REAL BUG. With `''` matching, any fixture that omits
`terminalLayoutsByTabId` records `''`, compares `'' === ''`, and passes while the
pane-identity check is entirely inert. Making it refuse turned that silence into
four failures across host-mirror-handle-gap-drain and -teardown, whose fixtures seed
no layout binding at all. Both now bind per environment — one shared environment id
filters every other environment's pane back to `''` and restores the no-op.

Mutation-tested on the merged tree: ignoring the binding fails case D and the
mid-wait case; re-reading at expiry fails the mid-wait case and nothing else;
letting `''` match fails the empty-binding case; widening the tab-death rule across
environments still fails the live-verdict case, so pane identity does not weaken the
scoping the sweep was reconciled around.

Also fixes a real-clock race this branch introduced: the revoke-window test read
`Date.now()` separately from `enqueue`'s own stamp, and under load the drift ate
into the window. It now anchors the injected clock to the item's `createdAt`.

* docs(runtime): the two guards on the park-time binding are not redundant

Recording a reconciliation result that existed only in a review thread, and
correcting it in the process — measuring the claim changed it.

The claim under review was that the `?? ''` fallback in `recordExpiredWait` is
unreachable by two independent guards, either sufficient alone: the caller's
generation gate (a missing waiter fails `undefined === number`) and the
record-before-release ordering. That is not what the code does.

Measured, by removing each in turn:

  - ordering removed, generation gate kept: the gate does NOT carry it. With the
    waiter already deleted, the gate is false on every expiry, so nothing is ever
    recorded — five failures, and the door is shut by breaking the mechanism rather
    than by refusing ''.
  - generation gate removed, ordering kept: 736 files green, one failure, and it is
    `does not let a wait armed on the previous connection decide the new one` in
    host-mirror-handle-gap-resume.test.ts — a different property entirely.

So the ordering alone makes `''` unreachable, and the generation gate is not a second
guard on it at all: it pins reconnect-void. Both are load-bearing, for different
reasons, which is a stronger argument against removing either than redundancy would
have been — redundancy invites deleting one.

Worth writing in the file because the two sit three lines apart and read as belt and
braces on the same thing. The `''` comment next to them already exists because an
unexplained guard on an unreachable value gets deleted as dead code in a year; a guard
that looks redundant is deleted sooner.

No behaviour change. One comment, corrected against measurement rather than against the
thread it came from.

* fix(runtime): a published handle retires the verdict it answered

The fourth eviction trigger on `expiredGenerationByPane`, and the reason it is
not redundant with the three already there or with the two other agents' guards
on this same map.

A verdict records that a pane's 15s handle-gap wait ran out. Nothing retires it
when that pane subsequently publishes its handle, so the NEXT gap on that pane
gets no wait at all — stablyai#19735 with the bounded wait removed rather than merely
shortened. Measured on the reconciled union tree (1b621b6) plus the outage
guard: the verdict still answered `true` after the handle landed, and the second
gap resumed with zero panes parked.

Why none of the existing rules reach it, each checked rather than assumed:
  - superseded generation: stablyai#19647 in this same stack stops recording
    `status: null` for an unreachable host, so the generation no longer moves
    across an outage on one runtime.
  - dead tab row: the row stays published throughout. It is the HANDLE that
    comes and goes — that is the definition of the gap.
  - removed environment: the environment is still here.
  - read-time pane identity (adv2-skew, cdafc90): the pane keeps the same
    layout binding across the gap BY DESIGN, and the union suite pins that a
    genuine reattach to the same PTY must inherit. That check discriminates a
    different pane behind one tab id; this one discriminates a later gap on the
    same pane.
  - contact lost (adv3-journeys, 2da6624): no outage is involved; this is a
    healthy connection where the host was simply slow once.

Composition proven by mutation on the union tree, four disjoint kills: dropping
this drain kills 2 tests and only mine; dropping the re-park worktree kills 1 and
only mine; dropping the contact guard kills 1 and only theirs; forcing the
contact guard always-true kills 16 across every suite. No mutation kills another
agent's test, so these are three guards on three holes, not three on one.

Also carries `worktreeId` across a re-park: adopting an orphaned terminal re-keys
`tabsByWorktree` without re-keying the record, so a live wait kept releasing on
retraction evidence about the workspace it was no longer about. The park-time
`paneBinding` deliberately does not move with it — that is the pane's identity,
this is only where its rows are filed.

* docs(runtime): the reused-tab-id class is closed at read time, not still open

The `ExpiredHandleGapVerdict` docstring told the next reader that a retracted
tab id republished under the same id still inherits its predecessor's verdict,
and that closing it "needs a fourth trigger, on row retraction". The test it
names as its own pin says the opposite: class D in
host-mirror-handle-gap-verdict-union.test.ts asserts the verdict does not
answer, and explains it is closed at READ time rather than by any prune.

Provenance, since two sources disagreeing is what made this expensive: the
paragraph was last written in 46ad377 and the read-time pane-identity check
landed one commit later in cdafc90 (adv2-skew). The prose predates its own
fix by a single commit and was never updated. Confirmed by mutation rather than
by reading: dropping `verdict.paneBinding === paneBindingFor(...)` fails exactly
"handles all four orphan classes simultaneously", which is the class-D
assertion, so the read-time check is what closes it.

Rewritten to say what the code does, keeping the part that was always true —
why no trigger could have reached that class — and keeping the distinction the
new PUBLISHED HANDLE drain needs: read-time identity separates two panes behind
one tab id, the drain separates two gaps on one pane. The drain does not close
class D and must not be read as closing it.

Also records why this block specifically keeps going stale: several agents
change this map in parallel, the invariants move faster than the prose, and when
the two disagree the test file is the one that ran.

* test(runtime): pin replay containment on the deadline path too

Cherry-picked from aa98edf (nwparker/adv3-failure-fixes) with its
implementation hunk dropped: a second agent found the same throwing-replay
hole independently, and `15f34014153` already closed it on this branch with
an equivalent guard. Applying both would have been a double-apply, and the
two spellings of the log line would have shipped side by side.

The tests are worth keeping regardless. The first duplicates coverage
15f3401 already has; the second does not -- it drives the throw from the
DEADLINE path rather than the store-write path, which is a separate call into
releaseWaiter and was unpinned.

Spies retargeted from console.error to console.warn, the channel the guard
that actually shipped writes to, so the suite silences what the code emits.

* fix(runtime): isolate one worktree's replay from the mirror-hydration drain

The same fan-out hazard as the handle-gap drain, one module up. Settling an
environment drains every worktree parked on it in a single loop, called from the
frame apply, with `waiter.run()` unguarded. One replay that throws strands every
waiter queued behind it and surfaces in the caller applying the frame.

Found by looking for the sibling of a defect rather than by a separate
interleaving: both modules park a `run` callback and drain N of them from one
event, so both have the same blast radius. Kept as its own commit because the
two modules route independently.

Mutation: dropping the guard kills exactly the one new assertion.

* test(runtime): pin sleeping-agent resume on a failed SSH target

The terminal-state floor in workspace-terminal-host-authority.ts has three
consumers: initial-terminal seeding, the startup terminal watcher, and
sleeping-agent resume. Seeding is covered end to end by
worktree-agent-activation-seam.test.ts. Resume was covered only at the
predicate, so nothing failed if the floor stopped reaching it — and the
floor's own comment says the cost of losing it is a failed target left
terminal-less with unresumable agents for the rest of the app session.

Pins the resume half directly: an SSH git worktree on a target whose sync
terminated in offline/error with an empty hydrated set resumes its sleeping
agent. Two controls keep the floor from widening into "resume whenever we
are unsure" — an in-flight 'pulling' sync and no sync status at all both stay
unverifiable and resume nothing.

Verified by mutation: emptying TERMINATED_WITHOUT_ANSWER_PHASES fails exactly
the two floor assertions and leaves both controls passing.

Routes independently of the two fixes on this branch: the floor predates this
stack (stablyai#16750), and this only closes a coverage gap in it.

* test(runtime): pin the store subscription the reconciled loop can leak

The retention suite that `reconcile three branches' handle-gap verdict rules
into one loop` replaced carried an assertion the split suites did not: the
store subscription is held for exactly as long as something needs it.

Measured before writing it, because half of it was already covered:

  RETAIN direction -- drop the verdict term from `stopStoreSubscriptionIfIdle`
  so a verdict with no waiter behind it loses the subscription its drain needs:
  already caught, 2 failures in host-mirror-handle-gap-landed-handle.test.ts.

  RELEASE direction -- never release the subscription at all: caught by
  NOTHING. That mutation passes all 272 tests across the 33 other handle-gap
  and session-tabs suites. A leaked subscription rescans every parked pane on
  every store write for the life of the session and nothing notices.

So this is for the release direction. The retain cases ride along because both
halves of one invariant belong in one file, not because they were missing.
That term is also precisely what the reconcile moved -- it now counts verdicts
as well as waiters -- so it is the part of this map most likely to drift again.

Asserted with a spy on useAppStore.subscribe rather than a new test-only
export: whether the module is subscribed is already observable at the store
boundary, and the production surface should not grow just to say so.

* fix(lint): carry SAFETY rationales for the handle-gap fixtures

main tightened typescript/consistent-type-assertions to assertionStyle:
never after this branch was written. The gate only ran here once the
rebase put the casting config at the merge base, so these sites are new
to it, not new to the branch. The store seeds are genuinely partial --
dropping the casts does not typecheck -- so each carries its rationale.

* fix(runtime): release a handle-gap pane once per store write

`releaseDueWaiters` snapshotted the due KEYS and then re-looked-up each one. A
replay earlier in the loop writes to the store — the sweep reaches `createTab`
and `clearSleepingAgentSession` — and zustand notifies re-entrantly with no
queue, so the nested pass can release and re-park a pane still queued in the
outer loop. The outer `releaseWaiter(key)` then found the re-park, cleared its
brand-new deadline and replayed it a second time off one store write, handing
that pane another full budget.

That is the extension `parkUntilHostMirrorHandleLands` already refuses to grant
a re-park, arriving through a different door. The direction is conservative
(hold longer, never resume early), which is why no outcome assertion could see
it; only the replay count separates the two implementations. Snapshot the waiter
alongside its key and release only while the map still holds that same waiter.

Also folds host-mirror-handle-gap-replay-containment.test.ts into the drain
suite, since the guard it pins is the one this commit extends. It was the same
fix imported twice: its deadline case is a strict subset of the drain suite's,
its store-write case differs only by also asserting that later store listeners
still run, and one mutation — rethrowing from the replay catch — killed all five
cases across both files. Its fixture also seeded no layout bindings and used tab
ids `isWebTerminalSurfaceTabId` rejects, so those panes could not have reached
the park path it claimed to exercise. The unique assertion moves across; the
file goes.

Killed by `releases a pane once per store write even when an earlier replay
re-enters the drain`: 2 replay calls instead of 1 without the identity guard.

* test(runtime): retire the handle-gap assertions that could not fail

`returns to baseline under churn across all three drains` asserted nothing. It
ran 300 expiries and then cleared every environment's verdicts by name before
counting, so the map was empty by construction — deleting the whole prune loop
in `recordExpiredWait` left the test green. It now asserts the bound BEFORE the
teardown clear: 300 expiries must leave exactly one live verdict per
environment. It also binds each round's pane to the environment recording it;
the old fixture filed every binding under env-a, so two rounds in three stored
the empty match value the read-time check refuses, and that much of the churn
was synthetic. Renamed: there are four drains, not three.

Two comments described outcomes their assertions do not produce. `c1` reads
false on the read-time generation gate alone, whether or not a drain ever swept
it — the count below is the only assertion that distinguishes retired from
stranded. And the discriminator in `never evicts a live pane verdict` is env-b,
whose row goes absent while env-a records; env-c is a control that holds under
every candidate rule.

Two more fixtures modelled states the mirror apply cannot produce, both leaning
on `ptyIdsByTabId[tab]` holding a PTY id no leaf of that tab is bound to. It
builds one from the other (web-session-tabs-sync/terminal-build.ts), so they can
never disagree. The producible shape is a SPLIT tab whose sibling surface went
`ready` first, which is what both cases now seed — and which makes the residual
they were quietly standing in for visible instead: the decidability gate above
this wait is tab-granular while everything below it is leaf-aware, so a sibling
handle ends the wait for a surface still `pending-handle`. Recorded at the gate
in host-mirrored-pane-liveness.ts, pinned by name, and left open here: it needs
the per-surface status the host already publishes and the client drops on apply.

Also states what the unscoped live-PTY arm trades — a finished agent whose shell
is still up releases its record and will not auto-resume — because it reads as a
regression and is not one. And replaces the subscription-lifetime header's
unreproducible "272 tests across 33 suites" with the measured 326 across 37.

* fix(runtime): re-judge a handle-gap waiter the drain's own replay moved

The identity guard added one commit ago catches only half of how the drain's
snapshot goes stale. It proves the map entry was not REPLACED; it cannot prove
the verdict still holds, because `parkUntilHostMirrorHandleLands` re-parks a
still-parked pane by MUTATING the waiter in place. `worktreeId` moves with `run`
— that is what adopting an orphaned terminal does — and object identity survives
it. So a waiter the snapshot judged retracted, because its tab was absent from
the worktree it was filed under, can be re-filed by an earlier replay in the
same loop and then released on evidence about a workspace it is no longer about.
That is the defect the `existing.worktreeId` assignment exists to prevent,
reached through the drain instead.

Neither guard covers the other: re-judging alone still replays a re-park twice
(it was just made due, so it re-judges due), and identity alone misses the
mutation. Both, in that order. A waiter that is no longer due simply stays
parked — bounded by its own deadline and re-judged on the next store write, so
declining costs at most one frame of latency.

Killed by `does not release on retraction evidence a mid-drain re-park has
already made stale`; the identity half is still killed by `replays a pane once
per store write even when an earlier replay re-enters the drain`. 29 mutations
across these modules, no survivors.

Corrects three claims made in the two preceding commits, each wrong in a way a
future reader would have acted on:

- the duplicate release does NOT extend the pane's budget. `releaseWaiter`
  deletes the waiter before calling `run`, so the re-park takes the `!existing`
  branch and arms a full deadline either way; a second release clears and re-arms
  at the same instant. What it costs is running an entire worktree resume sweep
  twice off one frame. The test is renamed to say so.
- `retainPendingTerminalBindings` carries a `pending-handle` surface's prior
  binding forward, so the split-tab residual cannot arise from a bound leaf going
  pending — it needs a leaf that was NEVER bound, which means a cold start or
  re-pair with no layout to retain from. The fixture staged the impossible
  history; it now seeds the producible first frame, where the point sharpens: no
  wait is armed at all, which the test now asserts directly.
- `clearSleepingAgentSession` cannot re-enter the drain; the subscription's slice
  guard drops that write. Only `createTab` can.

Also: the churn assertion pins that the prune loop runs at all, not which rule
prunes — with one live tab per round either rule alone still reads 3. Says so,
and points at the case that does isolate the generation rule.

* test(runtime): stage the handle-gap adoption the way a sweep can reach it

The case added one commit ago pinned the right guard through a call sequence
production cannot make. A replay is `resumeSleepingAgentSessionsForWorktree`
closed over ONE worktree, so it re-parks only under that worktree — but the
fixture had the first pane's replay re-park the second under a DIFFERENT one.
That is the same fault the previous commit corrected in the resume fixture, made
one file over.

The reachable route: the second tab has already been re-keyed onto the canonical
worktree id while its sleeping record still names the old one, so the first
pane's sweep legitimately owns it and re-parks it there, mutating the live
waiter in place. The waiter's snapshot verdict — "retracted", because the tab is
absent from the id it was filed under — is stale by the time the loop reaches
it. Restaged that way, with both waiters and the re-park inside one worktree.

Also drops the second global store read. Guard TWO now re-judges against the
same `state` the drain was notified with, because the two provably cannot differ
here: a re-park only happens when `findUnhydratedHostMirrorForPane` finds the
row already filed under the sweeping worktree, which is a row this frame carries.
`useAppStore.getState()` was an unpinnable degree of freedom — swapping it for
`state` left every test green — and it contradicted the drain's own claim to
judge one snapshot.

And corrects two more claims: guard ordering is a cost preference, not a
correctness requirement (either order works; identity first is just cheaper),
and "no wait is armed here" in the split-tab fixture is not caused by the cold
start — it is the tab-granular gate reading the sibling's handle, which is the
residual itself.

29 mutations across these modules, no survivors. Each guard is killed by exactly
one case and they do not overlap: `replays a pane once per store write even when
an earlier replay re-enters the drain` for identity, `does not release on
retraction evidence a mid-drain adoption has already made stale` for re-judging.

* fix(runtime): judge a re-parked handle-gap waiter on the live store

Reverts the `state` read the previous commit put in guard TWO, and says why the
difference is deliberately unpinnable rather than leaving the next reader to
"simplify" it back.

The previous commit swapped `useAppStore.getState()` for the subscriber's
`state` because a reviewer noted the swap left every test green. That was
optimising for mutation-killability over the property the module exists to
protect. The two reads do agree on every sequence the sweep can produce — a
replay's only store write is `createTab`, which appends a freshly minted tab id,
so it can neither make an absent tab id present nor touch `ptyIdsByTabId` — which
is exactly why no test separates them. But they are not interchangeable: `state`
is the staler of the two, and its failure direction is to RELEASE a pane whose
row has come back. That is resolving unverifiable to exited, which is stablyai#19735.

Holding on evidence that might be stale costs one frame; acting on it forks a
transcript. Take the fresher read, and record in the comment that no test can
fail on this and why that is not a reason to change it.

Both guards remain killed by exactly one case each and they do not overlap:
`replays a pane once per store write even when an earlier replay re-enters the
drain` for identity, `does not release on retraction evidence a mid-drain
adoption has already made stale` for re-judging.

Verified while confirming the previous commit's test premises against production,
both of which hold: a sweep that parks every record writes nothing to the store
(resume-sleeping-agent-session.ts takes `continue` on the park branch), and
`workspace-session-worktree-id.ts` moves `tabsByWorktree` onto the canonical id
while leaving `sleepingAgentSessionsByPaneKey` naming the old one — the stale
`worktreeId` the adoption case depends on.
enrick-dev pushed a commit to enrick-dev/orca that referenced this pull request Oct 15, 2026
… answered (stablyai#16750)

Two client behaviours read local tab rows as the verdict on what the execution
host is running. Before the host answers, "I hold no pane for this" is
`unverifiable`, not `exited` -- the collapse
`docs/reference/ssh-execution-boundary.md` forbids.

Symptom 1, seeding. `worktree-initial-terminal-seeding.ts:47,128` seeds a
terminal when `renderableTabCount === 0`. Its only bail-out (`:72-77`) covered
the paired-web-runtime flavor -- "while that session is live the host owns
terminal creation" -- with no equivalent for direct SSH. So a client that has
never held the workspace runs the predicate during the hydration gap and creates
a tab from nothing. The snapshot then arrives, the merge rightly keeps the tab it
was never told about, and the union uploads as the new host truth. Measured on a
fresh client against a host owning 3 tabs: **1 tab created from nothing, 0 of the
host's 3 adopted.** (A restart never reaches the predicate -- local state
restores the row first -- which is why restart-only repros came back flat.)

That guard was also the wrong question. It asked "am I a client of a live paired
session?", which a host desktop window answers "no" and a paired client answers
"yes", so both seeded -- stablyai#15556.

Symptom 2, sleeping-agent resume, and the data-corrupting half.
`Terminal.tsx:1554` calls `resumeSleepingAgentSessionsForWorktree` twenty lines
after the seeding call at `:1529-1534` -- same startup path, same pre-hydration
window, and not SSH-gated at all. Seeding produces a spare empty tab; the sweep
launches `claude --resume <id>` for a session still running on the remote and
still owned by a live pane. Two agent processes writing one transcript; STA-3498
observed five. STA-3500 files exactly this race. Failure is asymmetric: declining
to resume is user-recoverable, a duplicate resume corrupts a transcript
irreversibly.

`workspace-terminal-host-authority.ts` answers the one ownership question both
paths ask, in the three-verdict vocabulary the renderer already uses for host
terminal inventory (`HostLiveTerminalProbeVerdict`, aliased rather than restated
so the two cannot drift): `live` (a remote host owns creation here),
`unverifiable` (there is a remote host and it has not answered), `none` (local,
or the host answered and holds nothing). Seeding requires `none`; the sweep
declines on `unverifiable` without consuming its one-shot, so the agents are not
stranded for the session once the verdict lands.

Shape notes:
- An ownership question, not a client-liveness one -- that is what fixes stablyai#15556.
- Folder workspaces resolve to `none`: the snapshot replaces exactly
  `DirectSshTargetScope.gitWorktreeIds`, so a folder's rows are never replaced by
  the host and waiting for an answer that will never name them would leave it
  terminal-less for good.
- A `conflict` sync phase is `unverifiable`, matching the pair
  `use-app-session-persistence.ts` already gates uploads on.
- Explicit launch work (setup/issue commands) stays ungated -- that is a request
  to create a terminal now.
- `Terminal.tsx` subscribes through a retained selector rather than reading in
  the effect: the verdict flipping to `none` is what must re-run the passes, and
  resolution walks the owner catalogs, so recomputing per store write would be
  the STA-3363 render-path multiplier again.

The `unverifiable` verdict is BOUNDED, and must be. `remoteWorkspaceHydratedTargetIds` is add-only
in practice -- `markRemoteWorkspaceHydrated` has two production call sites, both on success paths,
and `clearRemoteWorkspaceHydrated` has NONE. Four paths return without marking: local-hydration
timeout (`remote-workspace-target-sync.ts:136-145`), a null `remoteWorkspace.get` (`:160-169`), a
falsy apply token (`:172-185`), and never connecting at all. Without a floor, any of them would
leave every git worktree on that target `unverifiable` for the rest of the app session: no initial
terminal, no sleeping-agent resume, escapable only by creating a tab by hand. That is strictly worse
than the behaviour it replaces -- on main the user got a terminal. So a sync that terminates in
`offline` or `error` without ever hydrating resolves `none`: declining to seed is meant to be a
wait, not a permanent refusal. `pulling` still declines, and a target that HAS hydrated stays `none`
even if a later sync errors.

Scope, stated because the doc comment previously overstated it: this gate is
first-hydration-per-target, not per-connection-generation. Since nothing clears the flag, a
disconnected target that hydrated once reads `none`. It does not cover mid-session reconnect or
sleep/resume.

The memo's input list is checked for COMPLETENESS, not just membership. `satisfies readonly
(keyof State)[]` only proves each listed key exists; a field added to the state and forgotten from
the list would type-check while making the memo return a stale verdict -- silent, and it looks like
"the gate did not fire". A conditional type now names the missing key at compile time. Deliberately
not `const x: Missing[] = []`, which passes regardless because an empty array literal is assignable
to every array type.

Known limitation, stated rather than hidden: the SEEDING half of this change has no measurable
end-to-end effect today, and the branch's own e2e spec says so.
`applyDirectSshRemoteWorkspaceSnapshot` calls `markRemoteWorkspaceHydrated` unconditionally AFTER
the hydrate calls -- including when they wrote nothing. So in the same tick adoption yields zero,
the verdict flips `unverifiable` -> `none`, `Terminal.tsx` re-runs the effect, and it seeds. The
gate cannot outlive the failure it guards against, because the same function that fails to adopt is
the one that lifts it.

`ssh-cold-hydration-gap-tab-seeding.spec.ts:218` is named for what it asserts -- one tab, adopted
none -- rather than for the behaviour we want. The fixme at `:293` pins the intended behaviour.

Making the seeding half effective needs hydration resolved PER WORKTREE (or a refusal to say `none`
when the completed apply's `replaceWorkspaceKeys` did not name this worktree) rather than a
per-target "some apply finished" flag. That is deliberately not in this commit.

The RESUME half is the valuable half and is unit-proven: it declines while the host is unanswered
and wakes the same session once the verdict lands, without consuming its one-shot. Preventing one
duplicate `claude --resume` on a live transcript is worth more than preventing one spare tab --
declining to resume is user-recoverable, a duplicate resume corrupts a transcript irreversibly.

Before: 7 failed | 3 passed. After: 10 passed; 103 across the seeding, resume,
authority and remote-workspace suites.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Paired remote client + host desktop window both auto-create the initial terminal — workspace activation spawns two empty terminals

1 participant