Repository navigation
ci(e2e): trigger the Docker-SSH lane from SSH source and forbid unclaimed specs - #16746
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
🚧 Files skipped from review as they are similar to previous changes (12)
Included review availability: Your plan provides up to 10 included reviews per hour; 6 remain after this review. 📝 WalkthroughWalkthroughThe PR adds SSH source-change detection to E2E routing and passes the signal through the pull request workflow into the reusable E2E workflow. The Docker SSH isolation job now runs for SSH source changes, uses a 60-minute timeout, and executes the remaining SSH specifications. A dedicated runner invokes 11 Docker SSH specs. New and updated helpers and tests validate remote tab creation, reconnection identity preservation, and tab stability across repeated application restarts. Merge Risk: 🟡 Moderate · up to The PR expands SSH end-to-end routing and adds a dedicated Docker lane, but deletion-only SSH changes may still bypass that lane, while the restart regression test can self-heal missing worktree state and falsely pass. These bounded CI coverage risks require explicit owner follow-up before merge. 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
Full details: Description checkExplanation The description provides detailed rationale, change scope, risks, testing evidence, and verification steps. However, it omits required template sections and information, including a Linked Issue with Resolution Reformat the description using all required template headings. Add a valid Linked Issue entry, state Full details: Docstring CoverageExplanation Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 9 files. (3 skipped: 3 unsupported.)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/pr.yml (1)
774-782: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winInclude deleted SSH source files.
--diff-filter=AMCRremoves deletions beforehasSshSourceChangeruns. A deletion-only SSH source change therefore does not trigger the Docker-SSH lane. IncludeD, or remove the filter if all changed paths are valid routing inputs.Proposed fix
- CHANGED="$(git diff --name-only --diff-filter=AMCR --merge-base "$BASE" "$HEAD")" + CHANGED="$(git diff --name-only --diff-filter=ADMCR --merge-base "$BASE" "$HEAD")"
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1171a48b-584c-4641-99f4-f7bfa32538c6
📒 Files selected for processing (11)
.github/workflows/e2e.yml.github/workflows/pr.ymlconfig/scripts/pr-e2e-gate-contract.test.mjsconfig/scripts/pr-e2e-source-routing.mjsconfig/scripts/run-ssh-docker-e2e.mjspackage.jsontests/e2e/helpers/docker-ssh-relay-connection.tstests/e2e/helpers/docker-ssh-relay-terminal-tabs.tstests/e2e/ssh-cold-activation-restore.spec.tstests/e2e/ssh-reconnect-tab-destruction.spec.tstests/e2e/ssh-restart-tab-accumulation.spec.ts
Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.
| async function waitForRestoredRemoteWorktree( | ||
| page: Page, | ||
| targetId: string, | ||
| worktreeId: string | ||
| ): Promise<void> { | ||
| await waitForSessionReady(page, 60_000) | ||
| await expect.poll(() => waitForActiveWorktree(page), { timeout: 60_000 }).toBe(worktreeId) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Do not repair the state that this test verifies.
waitForActiveWorktree calls setActiveWorktree when the selection is empty. This can hide a failure to restore activeWorktreeId. Poll the stored value without mutation.
Proposed fix
await waitForSessionReady(page, 60_000)
- await expect.poll(() => waitForActiveWorktree(page), { timeout: 60_000 }).toBe(worktreeId)
+ await expect
+ .poll(
+ () => page.evaluate(() => window.__store?.getState().activeWorktreeId ?? null),
+ { timeout: 60_000 }
+ )
+ .toBe(worktreeId)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async function waitForRestoredRemoteWorktree( | |
| page: Page, | |
| targetId: string, | |
| worktreeId: string | |
| ): Promise<void> { | |
| await waitForSessionReady(page, 60_000) | |
| await expect.poll(() => waitForActiveWorktree(page), { timeout: 60_000 }).toBe(worktreeId) | |
| async function waitForRestoredRemoteWorktree( | |
| page: Page, | |
| targetId: string, | |
| worktreeId: string | |
| ): Promise<void> { | |
| await waitForSessionReady(page, 60_000) | |
| await expect | |
| .poll( | |
| () => page.evaluate(() => window.__store?.getState().activeWorktreeId ?? null), | |
| { timeout: 60_000 } | |
| ) | |
| .toBe(worktreeId) |
2c4463d to
21f9c88
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
…gated spec
The Docker-SSH e2e lane only ran when a PR's changed specs happened to include
`ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`.
Editing SSH source itself did not trigger it, and pruning either spec from a
route's list would have silently retired the whole lane. Meanwhile the sharded
lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself
while the shard still reported green -- the exact silent-skip shape
`docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions
that reached users.
Separately, the modules that actually own direct-SSH workspace and tab restore
carry no "ssh" in their names, so the `ssh-terminal-source` route never reached
them. Measured on the real script before this change:
printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
src/main/ipc/remote-workspace-snapshot-normalization.ts \
src/renderer/src/lib/worktree-initial-terminal-seeding.ts \
src/shared/remote-workspace-session-projection.ts \
| node config/scripts/pr-e2e-source-routing.mjs
=> []
Three changes, all pinned by the executable gate contract:
- `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH
routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source
rather than on a spec name surviving in a list. One list, so the two cannot
drift.
- A sibling `ssh-workspace-session-restore` route names the restore seams
(`remote-workspace-*`, `worktree-initial-terminal-seeding`,
`worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the
two restore specs -- a sibling rather than more paths on `ssh-terminal-source`
so a tab-tombstone edit does not run the whole SSH terminal list.
- A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on
the one VM that sets the flag, and the contract now fails by name when any
Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and
`ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock
budgets; needs a real remote codex binary) and the contract asserts each
exemption still corresponds to a real gated spec, so a stale one cannot
quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added
serial specs.
The lane's first act was to surface four latent bugs in a spec that had been
silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites
out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()`
is called with no argument though the helper dereferences `testInfo.workerIndex`
(a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and
`splitActiveTerminalPane` gained a direction. It was invisible because it ran
nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four
more could not be seen.
The `testInfo` bug is fixed here -- correct on its own, and it removes one real
error from `typecheck:e2e` (240 -> 239). The other three are not, because they
are not argument plumbing: repairing them requires choosing which ptyId to
capture and which split direction to use, and both change what the repro
measures.
The spec is therefore added to the exemption list rather than repaired, for two
independent reasons recorded in the runner: it is a perf oracle, not a
correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured
under a deliberate 5-pane flood on a 420s budget -- the same rule already applied
to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in
#16764. Applying an existing written rule to a sibling that plainly
meets it is consistency; inventing a new exemption to dodge a red would not be.
Three hardening fixes to the contract itself:
- Runner text is comment-stripped before the claimed-by-a-lane scan. A substring
scan over raw text lets a spec merely *discussed* in a runner comment count as
claimed -- the silent skip this assertion exists to catch, re-entering through
the documentation. Not live today only because the existing comments write the
spec names without their `tests/e2e/` prefix.
- An exempt spec must not be invoked by any runner. `unreachableSpecs`
short-circuits the unclaimed check, so a spec could be documented as exempt
while a runner still ran it -- an exemption that reads as coverage removal but
changes nothing, leaving the lane red for a reason the file says it excluded.
This is not hypothetical: adding the bulk-open exemption without removing it
from the runner's spec list produced exactly that state, and this assertion is
what caught it.
- The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/`
rather than one fixed string, so a double-quoted or `!==` spelling can no
longer escape the contract.
`ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence
asserting tab-id set identity, not just the active pane's reclaimed ptyId as
`ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated
by a negative control that injected one tab after cycle 1 and correctly failed.
21f9c88 to
4630033
Compare
`ssh-pi-compatible-agent-title.spec.ts` failed its first-ever CI run with `No remote worktree found for /tmp/orca-docker-relay-perf-repo`, thrown from its own private `connectDockerRemote` before it reached a single title assertion. The spec ran in no CI lane until #16746 claimed it, so there is no green baseline it regressed from. `connectDockerRemote` was a stale fork of `connectDockerSshRelayTarget`, missing the repo-owner wait and the authoritative fetch the shared helper grew. It hydrated with a bare `fetchWorktrees(repoId)` and read `worktreesByRepo[repoId][0]`. Deploying the relay bounces the SSH transport on this fixture -- CI logs `[ssh] Reconnecting ... failed handshakes 0/9` half a second after `Relay started successfully`, and that bounce reproduces on clean main. While the target reads back as reconnecting, `getCurrentDirectSshAuthority` resolves nothing, so `fetchWorktrees` returns false and writes no authoritative rows. Both sides of the shared helper sampled a single instant too: it pinned the authority from `ssh.connect` and threw if the live state had rotated by hydration time. Hydration now waits for the target to report a live connected authority, hydrates against that one, and re-derives if the transport rotated mid-flight, bounded by a 60s deadline. The spec drops its private fork and calls the shared helper. No assertion changed; the three OMP/Pi title assertions are untouched.
…gated spec (stablyai#16746) The Docker-SSH e2e lane only ran when a PR's changed specs happened to include `ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`. Editing SSH source itself did not trigger it, and pruning either spec from a route's list would have silently retired the whole lane. Meanwhile the sharded lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself while the shard still reported green -- the exact silent-skip shape `docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions that reached users. Separately, the modules that actually own direct-SSH workspace and tab restore carry no "ssh" in their names, so the `ssh-terminal-source` route never reached them. Measured on the real script before this change: printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \ src/main/ipc/remote-workspace-snapshot-normalization.ts \ src/renderer/src/lib/worktree-initial-terminal-seeding.ts \ src/shared/remote-workspace-session-projection.ts \ | node config/scripts/pr-e2e-source-routing.mjs => [] Three changes, all pinned by the executable gate contract: - `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source rather than on a spec name surviving in a list. One list, so the two cannot drift. - A sibling `ssh-workspace-session-restore` route names the restore seams (`remote-workspace-*`, `worktree-initial-terminal-seeding`, `worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the two restore specs -- a sibling rather than more paths on `ssh-terminal-source` so a tab-tombstone edit does not run the whole SSH terminal list. - A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on the one VM that sets the flag, and the contract now fails by name when any Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and `ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock budgets; needs a real remote codex binary) and the contract asserts each exemption still corresponds to a real gated spec, so a stale one cannot quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added serial specs. The lane's first act was to surface four latent bugs in a spec that had been silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()` is called with no argument though the helper dereferences `testInfo.workerIndex` (a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and `splitActiveTerminalPane` gained a direction. It was invisible because it ran nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four more could not be seen. The `testInfo` bug is fixed here -- correct on its own, and it removes one real error from `typecheck:e2e` (240 -> 239). The other three are not, because they are not argument plumbing: repairing them requires choosing which ptyId to capture and which split direction to use, and both change what the repro measures. The spec is therefore added to the exemption list rather than repaired, for two independent reasons recorded in the runner: it is a perf oracle, not a correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured under a deliberate 5-pane flood on a 420s budget -- the same rule already applied to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in stablyai#16764. Applying an existing written rule to a sibling that plainly meets it is consistency; inventing a new exemption to dodge a red would not be. Three hardening fixes to the contract itself: - Runner text is comment-stripped before the claimed-by-a-lane scan. A substring scan over raw text lets a spec merely *discussed* in a runner comment count as claimed -- the silent skip this assertion exists to catch, re-entering through the documentation. Not live today only because the existing comments write the spec names without their `tests/e2e/` prefix. - An exempt spec must not be invoked by any runner. `unreachableSpecs` short-circuits the unclaimed check, so a spec could be documented as exempt while a runner still ran it -- an exemption that reads as coverage removal but changes nothing, leaving the lane red for a reason the file says it excluded. This is not hypothetical: adding the bulk-open exemption without removing it from the runner's spec list produced exactly that state, and this assertion is what caught it. - The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/` rather than one fixed string, so a double-quoted or `!==` spelling can no longer escape the contract. `ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence asserting tab-id set identity, not just the active pane's reclaimed ptyId as `ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated by a negative control that injected one tab after cycle 1 and correctly failed. (cherry picked from commit 971d987)
…gated spec (stablyai#16746) The Docker-SSH e2e lane only ran when a PR's changed specs happened to include `ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`. Editing SSH source itself did not trigger it, and pruning either spec from a route's list would have silently retired the whole lane. Meanwhile the sharded lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself while the shard still reported green -- the exact silent-skip shape `docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions that reached users. Separately, the modules that actually own direct-SSH workspace and tab restore carry no "ssh" in their names, so the `ssh-terminal-source` route never reached them. Measured on the real script before this change: printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \ src/main/ipc/remote-workspace-snapshot-normalization.ts \ src/renderer/src/lib/worktree-initial-terminal-seeding.ts \ src/shared/remote-workspace-session-projection.ts \ | node config/scripts/pr-e2e-source-routing.mjs => [] Three changes, all pinned by the executable gate contract: - `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source rather than on a spec name surviving in a list. One list, so the two cannot drift. - A sibling `ssh-workspace-session-restore` route names the restore seams (`remote-workspace-*`, `worktree-initial-terminal-seeding`, `worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the two restore specs -- a sibling rather than more paths on `ssh-terminal-source` so a tab-tombstone edit does not run the whole SSH terminal list. - A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on the one VM that sets the flag, and the contract now fails by name when any Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and `ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock budgets; needs a real remote codex binary) and the contract asserts each exemption still corresponds to a real gated spec, so a stale one cannot quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added serial specs. The lane's first act was to surface four latent bugs in a spec that had been silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()` is called with no argument though the helper dereferences `testInfo.workerIndex` (a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and `splitActiveTerminalPane` gained a direction. It was invisible because it ran nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four more could not be seen. The `testInfo` bug is fixed here -- correct on its own, and it removes one real error from `typecheck:e2e` (240 -> 239). The other three are not, because they are not argument plumbing: repairing them requires choosing which ptyId to capture and which split direction to use, and both change what the repro measures. The spec is therefore added to the exemption list rather than repaired, for two independent reasons recorded in the runner: it is a perf oracle, not a correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured under a deliberate 5-pane flood on a 420s budget -- the same rule already applied to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in stablyai#16764. Applying an existing written rule to a sibling that plainly meets it is consistency; inventing a new exemption to dodge a red would not be. Three hardening fixes to the contract itself: - Runner text is comment-stripped before the claimed-by-a-lane scan. A substring scan over raw text lets a spec merely *discussed* in a runner comment count as claimed -- the silent skip this assertion exists to catch, re-entering through the documentation. Not live today only because the existing comments write the spec names without their `tests/e2e/` prefix. - An exempt spec must not be invoked by any runner. `unreachableSpecs` short-circuits the unclaimed check, so a spec could be documented as exempt while a runner still ran it -- an exemption that reads as coverage removal but changes nothing, leaving the lane red for a reason the file says it excluded. This is not hypothetical: adding the bulk-open exemption without removing it from the runner's spec list produced exactly that state, and this assertion is what caught it. - The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/` rather than one fixed string, so a double-quoted or `!==` spelling can no longer escape the contract. `ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence asserting tab-id set identity, not just the active pane's reclaimed ptyId as `ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated by a negative control that injected one tab after cycle 1 and correctly failed.
Stack: 1 of 6 — based on
main. Bottom of the SSH tab-resurrection stack. This one is pure CI/test infrastructure; it makes every fix above it verifiable instead of hopeful.ELI5
We have a set of end-to-end tests that drive a real SSH connection to a Docker container. They only run if a flag is set — and nothing set it. So they quietly skipped, the job went green, and nobody noticed for months.
Worse, the tests that did run were picked by which test files you edited. Change the actual SSH code and no SSH test ran.
The fix, three parts:
It found real problems immediately. One test had rotted four function calls out of date and could never pass. Another only fails when it runs 14th in a row. Both had been invisible.
The gap this closes, demonstrated on the five PRs stacked on top
The clearest statement of why this PR exists is what its router does to the fixes above it.
BEFOREismain's router run on each level's real changed-file list;AFTERis this PR's router on the same list:main's router)[]ssh-cold-activation-restore,ssh-reconnect-tab-destruction[]ssh-cold-activation-restore,ssh-reconnect-tab-destructionssh-port-forward-lifecycle,ssh-reconnect-tab-destructionssh-cold-activation-restore,ssh-reconnect-tab-destructionTwo of the five fixes in this stack routed to zero e2e specs. One of them, #16748, changes
remote-workspace-session-merge.ts— the single most load-bearing file in the direct-SSH restore path. Another, #16752, fixes a P0 that reached a user repeatedly. Neither could have been caught by CI before this PR.ssh-reconnect-tab-destruction.spec.ts— the spec written to guard this exact regression class — previously ran only if you edited that spec file.Reproduce the BEFORE column yourself:
Symptom
Two SSH regression classes could be reintroduced tomorrow with a green PR.
ssh-startup-exec-readiness.spec.tsorpaired-startup-exec-readiness.spec.ts— a side effect of one route's spec list. Pruning either spec would have silently retired the whole lane.main:ORCA_E2E_SSH_DOCKER, so 16 specs self-skipped while the shard passed. This is the exact failuredocs/reference/ssh-reconnect-source-recovery.mdrecords — "the lane was skipping for 15 specs; four regressions reached a user during that window" — surviving in a new form: the lane runs, but is unreachable from the subsystem it protects.Root cause
config/scripts/pr-e2e-source-routing.mjs:16-29. Thessh-terminal-sourceroute lists exactly the specs that would catch a session/tab restore regression, but its matcher covers onlysrc/main/ssh/,src/main/providers/ssh-,src/main/ipc/(ssh-|pty),src/relay/, and a fewterminal-pane/files. The restore modules carry nosshin their names, so nothing reached them.The lane trigger in
.github/workflows/e2e.yml:291was acontains(inputs.test_files, …)over two spec names — a second, undeclared list that had to agree with the routes and had already drifted.The fix, and why this shape
Three changes, all pinned by the executable gate contract at
config/scripts/pr-e2e-gate-contract.test.mjs:hasSshSourceChangederives a newssh_source_changedsignal from the SSH routes themselves (SSH_SOURCE_ROUTE_IDS), plumbedpr.yml→e2e.yml. One list, so the two cannot drift. The two spec clauses stay for their honest purpose: changed-e2e hands those specs to this lane, so editing one must still run it here.ssh-workspace-session-restoreroute names the restore seams (src/main/ipc/remote-workspace*,src/shared/remote-workspace-*,src/renderer/src/hooks/remote-workspace-*,worktree-initial-terminal-seeding,worktree-default-terminal-tabs,initial-terminal) and routes them to the two restore specs. A sibling rather than more paths onssh-terminal-sourceso a tab-tombstone edit does not run the whole SSH terminal list.test:e2e:ssh-dockerrunner (config/scripts/run-ssh-docker-e2e.mjs) claims the remaining Docker-gated specs on the one VM that sets the flag. The contract now fails by name when any Docker-gated spec is claimed by no runner. Running them on this one VM pays the fixture image build once instead of ten times and keeps an SSH regression legible as an SSH-named failure.Two exemptions are recorded in the runner rather than left looking like coverage:
ssh-docker-relay-perf.spec.ts(wall-clock latency thresholds belong to the perf lane; flaky budgets here would cost the lane its credibility) andssh-codex-display-artifacts-repro.spec.ts(needs a real remote codex binary CI runners lack — observed asspawn codex ENOENT). The contract asserts each exemption still corresponds to a real gated spec, so a stale one cannot quietly excuse a gap.Lane timeout 35 → 60 minutes: several added specs budget 4–10 minutes per test, and a lane that times out is a lane nobody trusts.
tests/e2e/ssh-restart-tab-accumulation.spec.tsis new: three restart cycles asserting tab-id set identity, not just the active pane's reclaimedptyIdasssh-cold-activation-restore.spec.ts:241did (a spurious extra tab per restart passed that assertion). It passes today and is kept as a regression fence — validated by a negative control that injected one tab after cycle 1 and correctly failed, which only a real quit/persist/relaunch produces.Evidence
13 of those assertions are red on
mainand green here. The unclaimed-spec assertion caught a real unclaimed spec on its first run.Deliberately NOT done
pr.yml:796-821). 20 of 20 recente2e.ymlruns failed or were cancelled; making it blocking today would halt all merges. The suite has to be fixed first.pr.yml'sRequire-successful-checkscomment has the exact wiring to flip it, and the contract asserts the current state so the gap stays legible.ssh-docker-watcher-isolationjob. Watcher isolation is now one spec of many, so the name understates the job — but renaming changes the GitHub check name and can break required-check config. Recorded in the runner instead.ORCA_E2E_LOCAL_SSH_BROWSER,ORCA_E2E_SSH_CLIENT_HOSTED_BROWSER,ORCA_E2E_NESTED_RUNTIME_SSH,ORCA_E2E_SSH_LOCALHOST,ORCA_RUN_DOCKER_SSH_BROWSER_E2E). They are not Docker-gated so they sit outside this file's contract; enumerated in the runner because nothing else names them.Risk / blast radius
CI-only. No product code. The failure mode of getting the routing wrong is more CI running, not less — the added route only ever widens the spec set. The one behavioural risk is the 60-minute lane budget on a shared runner; it replaces a 35-minute budget that the added specs would have blown.
How to verify
End-to-end verification
Specs this PR routes to
This PR changes no product source, so it routes on its changed spec files. Run against this branch:
falseis correct and is the point: this PR touches no SSH source, so the new signal does not fire. Every level above it reportstrue.The routing change, measured before and after
See "The gap this closes" at the top of this description — the BEFORE/AFTER table across all five stacked PRs is this PR's primary evidence and is stated there rather than repeated here.
(macOS note for reproducing it:
/tmpis a symlink to/private/tmp, so a copy of the script placed there fails its ownimport.meta.url === pathToFileURL(process.argv[1]).hrefentry guard and silently exits 0 with no output. Copy it somewhere real.)Lane result
The lane this PR creates, run at the stack tip:
This is the lane's first real result, and it is the point of the PR: 14 specs that previously ran nowhere on a PR now run somewhere, in 10.7 minutes against a 60-minute budget.
The two lane failures, resolved — and one is now exempted:
mainssh-docker-bulk-open-freeze-reproTypeError: … reading 'workerIndex'atdocker-ssh-relay-target.ts:226ssh-port-forward-lifecyclessh-port-forward-lifecycle-evidence.ts:159waiting ongetByText('Ports')Port-forward was re-run at the same commit and build and passed in 59.6s. It is green on
main, green at the tip in isolation, and red only inside the full lane — it is the set's one@headfulspec and runs last. Called flaky in-lane rather than "unrelated": a real lane-ordering problem, pre-existing, and worth tracking.Worth being explicit about what those two results mean for this PR. Both specs were claimed by no runner and skipped silently while the shard reported green. Surfacing them is this PR working as intended — and the two are different findings:
main, and worse than the first symptom suggested: putting it in a lane revealed it had rotted four call sites out of date againsttests/e2e/helpers/terminal.ts—startDockerSshRelayTarget()called with no argument though the helper dereferencestestInfo.workerIndex,execInTerminalmissing itsptyId(twice), andsplitActiveTerminalPanemissing its direction. A 100% failure, not a flake. It was invisible because it ran nowhere andtypecheck:e2eis red onmainwith 240 errors, so four more could not be seen. Either gap alone would have been caught by the other.main, green at the tip in isolation, red only as the last spec in the lane — and nobody knew that either.That is the coverage gap, demonstrated rather than asserted. Neither is a regression from this stack; both were invisible before this PR gave them a runner.
Both of these were invisible because neither spec was claimed by any runner, so both skipped silently while the shard reported green. One turned out to be genuinely broken on
main; the other is lane-order flaky and briefly looked like a regression this very stack had introduced, until an isolated run on both refs disproved it in 59.6s. A lane that can surface a scare you can then disprove in one run is the point — the alternative is not "no scare", it is "no signal". The counterfactual to a noisy lane here was not a quiet correct one; it was a lane that reported green over a real breakage for months.What was done about bulk-open. The
testInfobug is fixed here — correct on its own, and it removes one real error fromtypecheck:e2e(240 → 239). The other three are not, because they are not argument plumbing: repairing them requires choosing which ptyId to capture and which split direction to use, and both change what the repro measures. The spec is therefore exempted rather than repaired, for two independent reasons recorded in the runner: it is a perf oracle, not a correctness one (SOFT_FREEZE_LAG_MS=2500/HARD_FREEZE_LAG_MS=5000under a deliberate 5-pane flood on a 420s budget — the same rule already applied tossh-docker-relay-perf.spec.ts), and it is known-rotted. Repair is tracked in #16764.Applying an existing written rule to a sibling that plainly meets it is consistency; inventing a new exemption to dodge a red would not be. The lane now runs 13 specs.
One more contract assertion came out of this, and it caught a real bug in this PR.
unreachableSpecsshort-circuits the unclaimed check, so a spec can be documented as exempt while a runner still invokes it — an exemption that reads as coverage removal but changes nothing, leaving the lane red for a reason the file says it excluded. That is exactly the state the first version of this exemption was in. The contract now asserts every exempt spec is absent from every runner, and re-introducing the line fails it by name:Recommendation: track the port-forward flake. The lane's one
@headfulspec failing after 13 predecessors is a lane-hygiene problem this lane makes visible for the first time. It is pre-existing and out of scope here, but it should get a ticket rather than be absorbed as noise — a lane that is intermittently red for known reasons is how the next silent skip gets rationalised.Build provenance
This PR ships no product code, so there is no bundle symbol to grep — nothing it changes reaches
out/. Stating that rather than inventing a symbol. Its artifact is the router itself, which is directly executable and self-verifying (see the two commands above), plus the 36-assertion contract test.Provenance for the lane run is therefore the build SHA:
Clean tree at the stack tip. This level is contained in that build.
What is NOT covered
pr.yml:796-821). Nothing here blocks a PR. That is deliberate — 20 of 20 recente2e.ymlruns failed or were cancelled — and the contract asserts the current state so the gap stays legible.ssh-docker-relay-perf.spec.tsandssh-codex-display-artifacts-repro.spec.ts. The contract asserts each still names a real gated spec so the exemption cannot rot, but neither runs in CI.main, outside this stack, noted so nobody attributes it to the PR:src/shared/runtime-client-export-parity.test.tsfails at026389a3bc0—runtime-types.tsexportsTERMINAL_PTY_DEGRADATION_CAPABILITY,TERMINAL_UNAVAILABLE_ERROR_CODEandterminalUnavailableMessage, none of which are in the test's expected list. Neither file is touched by any level of this stack.typecheck:e2eis red onmainwith 240 errors. None are in files this PR touches (verified per-file). Pre-existing and untouched.Failing-first test
Yes. 13 assertions in
config/scripts/pr-e2e-gate-contract.test.mjsare red onmainand green here, including the unclaimed-spec assertion — which caught a real unclaimed spec on its first run.