Skip to content

ci(e2e): trigger the Docker-SSH lane from SSH source and forbid unclaimed specs - #16746

Merged
nwparker merged 1 commit into
mainfrom
nwparker/ssh-01-e2e-lane
Aug 28, 2026
Merged

nwparker merged 1 commit into
mainfrom
nwparker/ssh-01-e2e-lane

Conversation

@nwparker

@nwparker nwparker commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 6 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​481 $\color{#cf222e}{\Huge{\mathbf{−}}}$​46 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​435
Prod 5 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​162 $\color{#cf222e}{\Huge{\mathbf{−}}}$​6 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​156

Stack: 1 of 6 — based on main. Bottom of the SSH tab-resurrection stack. This one is pure CI/test infrastructure; it makes every fix above it verifiable instead of hopeful.

ELI5

We have a set of end-to-end tests that drive a real SSH connection to a Docker container. They only run if a flag is set — and nothing set it. So they quietly skipped, the job went green, and nobody noticed for months.

Worse, the tests that did run were picked by which test files you edited. Change the actual SSH code and no SSH test ran.

The fix, three parts:

  1. Pick tests by which source files changed, not which test files.
  2. Give the skipping tests a lane that actually runs them.
  3. Add a check that fails by name if someone adds one of these tests and forgets to put it in a lane.

It found real problems immediately. One test had rotted four function calls out of date and could never pass. Another only fails when it runs 14th in a row. Both had been invisible.


The gap this closes, demonstrated on the five PRs stacked on top

The clearest statement of why this PR exists is what its router does to the fixes above it. BEFORE is main's router run on each level's real changed-file list; AFTER is this PR's router on the same list:

Stacked PR BEFORE (main's router) AFTER (this PR)
#16747 normalizer field-loss [] ssh-cold-activation-restore, ssh-reconnect-tab-destruction
#16748 merge preserves local rows [] ssh-cold-activation-restore, ssh-reconnect-tab-destruction
#16750 host-authority seeding its own new spec only + both restore specs
#16751 reattach fence 5 specs 7 — gains ssh-port-forward-lifecycle, ssh-reconnect-tab-destruction
#16752 close tombstone (the headline fix) 3 specs 5 — gains ssh-cold-activation-restore, ssh-reconnect-tab-destruction

Two of the five fixes in this stack routed to zero e2e specs. One of them, #16748, changes remote-workspace-session-merge.ts — the single most load-bearing file in the direct-SSH restore path. Another, #16752, fixes a P0 that reached a user repeatedly. Neither could have been caught by CI before this PR.

ssh-reconnect-tab-destruction.spec.ts — the spec written to guard this exact regression class — previously ran only if you edited that spec file.

Reproduce the BEFORE column yourself:

git show main:config/scripts/pr-e2e-source-routing.mjs > ./route-main.mjs   # not under /tmp on macOS, see note below
git diff --name-only <parent-branch>..<branch> | node ./route-main.mjs

Symptom

Two SSH regression classes could be reintroduced tomorrow with a green PR.

  1. The Docker-SSH e2e lane did not trigger on SSH source. It ran only when a PR's changed specs happened to include ssh-startup-exec-readiness.spec.ts or paired-startup-exec-readiness.spec.ts — a side effect of one route's spec list. Pruning either spec would have silently retired the whole lane.
  2. The modules that own direct-SSH workspace and tab restore routed to ZERO specs. Measured by running the real script on main:
printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
  src/main/ipc/remote-workspace-snapshot-normalization.ts \
  src/renderer/src/lib/worktree-initial-terminal-seeding.ts \
  src/shared/remote-workspace-session-projection.ts \
  | node config/scripts/pr-e2e-source-routing.mjs
→ []
  1. Docker-gated specs claimed by no runner skip silently and report green. The sharded lanes set no ORCA_E2E_SSH_DOCKER, so 16 specs self-skipped while the shard passed. This is the exact failure docs/reference/ssh-reconnect-source-recovery.md records — "the lane was skipping for 15 specs; four regressions reached a user during that window" — surviving in a new form: the lane runs, but is unreachable from the subsystem it protects.

Root cause

config/scripts/pr-e2e-source-routing.mjs:16-29. The ssh-terminal-source route lists exactly the specs that would catch a session/tab restore regression, but its matcher covers only src/main/ssh/, src/main/providers/ssh-, src/main/ipc/(ssh-|pty), src/relay/, and a few terminal-pane/ files. The restore modules carry no ssh in their names, so nothing reached them.

The lane trigger in .github/workflows/e2e.yml:291 was a contains(inputs.test_files, …) over two spec names — a second, undeclared list that had to agree with the routes and had already drifted.

The fix, and why this shape

Three changes, all pinned by the executable gate contract at config/scripts/pr-e2e-gate-contract.test.mjs:

  • hasSshSourceChange derives a new ssh_source_changed signal from the SSH routes themselves (SSH_SOURCE_ROUTE_IDS), plumbed pr.yml → e2e.yml. One list, so the two cannot drift. The two spec clauses stay for their honest purpose: changed-e2e hands those specs to this lane, so editing one must still run it here.
  • A sibling ssh-workspace-session-restore route names the restore seams (src/main/ipc/remote-workspace*, src/shared/remote-workspace-*, src/renderer/src/hooks/remote-workspace-*, worktree-initial-terminal-seeding, worktree-default-terminal-tabs, initial-terminal) and routes them to the two restore specs. A sibling rather than more paths on ssh-terminal-source so a tab-tombstone edit does not run the whole SSH terminal list.
  • A new test:e2e:ssh-docker runner (config/scripts/run-ssh-docker-e2e.mjs) claims the remaining Docker-gated specs on the one VM that sets the flag. The contract now fails by name when any Docker-gated spec is claimed by no runner. Running them on this one VM pays the fixture image build once instead of ten times and keeps an SSH regression legible as an SSH-named failure.

Two exemptions are recorded in the runner rather than left looking like coverage: ssh-docker-relay-perf.spec.ts (wall-clock latency thresholds belong to the perf lane; flaky budgets here would cost the lane its credibility) and ssh-codex-display-artifacts-repro.spec.ts (needs a real remote codex binary CI runners lack — observed as spawn codex ENOENT). The contract asserts each exemption still corresponds to a real gated spec, so a stale one cannot quietly excuse a gap.

Lane timeout 35 → 60 minutes: several added specs budget 4–10 minutes per test, and a lane that times out is a lane nobody trusts.

tests/e2e/ssh-restart-tab-accumulation.spec.ts is new: three restart cycles asserting tab-id set identity, not just the active pane's reclaimed ptyId as ssh-cold-activation-restore.spec.ts:241 did (a spurious extra tab per restart passed that assertion). It passes today and is kept as a regression fence — validated by a negative control that injected one tab after cycle 1 and correctly failed, which only a real quit/persist/relaunch produces.

Evidence

$ pnpm test config/scripts/pr-e2e-gate-contract.test.mjs
 Test Files  1 passed (1)
      Tests  36 passed (36)

13 of those assertions are red on main and green here. The unclaimed-spec assertion caught a real unclaimed spec on its first run.

$ pnpm tc            → clean
$ pnpm run check:code-quality:changed
code quality: 0 new finding(s) across 8 changed file(s).
type-aware code quality: 0 new finding(s) across 8 changed file(s).
React Doctor: 0 new finding(s) across 8 changed file(s).

Deliberately NOT done

  • Making e2e a required check (pr.yml:796-821). 20 of 20 recent e2e.yml runs failed or were cancelled; making it blocking today would halt all merges. The suite has to be fixed first. pr.yml's Require-successful-checks comment has the exact wiring to flip it, and the contract asserts the current state so the gap stays legible.
  • Renaming the ssh-docker-watcher-isolation job. Watcher isolation is now one spec of many, so the name understates the job — but renaming changes the GitHub check name and can break required-check config. Recorded in the runner instead.
  • Five specs and one unit test gated on env vars no workflow sets (ORCA_E2E_LOCAL_SSH_BROWSER, ORCA_E2E_SSH_CLIENT_HOSTED_BROWSER, ORCA_E2E_NESTED_RUNTIME_SSH, ORCA_E2E_SSH_LOCALHOST, ORCA_RUN_DOCKER_SSH_BROWSER_E2E). They are not Docker-gated so they sit outside this file's contract; enumerated in the runner because nothing else names them.

Risk / blast radius

CI-only. No product code. The failure mode of getting the routing wrong is more CI running, not less — the added route only ever widens the spec set. The one behavioural risk is the 60-minute lane budget on a shared runner; it replaces a 35-minute budget that the added specs would have blown.

How to verify

pnpm test config/scripts/pr-e2e-gate-contract.test.mjs

# routing, before vs after
printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
  | node config/scripts/pr-e2e-source-routing.mjs
printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
  | node config/scripts/pr-e2e-source-routing.mjs --ssh-source

# the lane itself (needs Docker)
pnpm test:e2e:ssh-docker

End-to-end verification

Specs this PR routes to

This PR changes no product source, so it routes on its changed spec files. Run against this branch:

$ git diff --name-only main..nwparker/ssh-01-e2e-lane | node config/scripts/pr-e2e-source-routing.mjs
["tests/e2e/ssh-cold-activation-restore.spec.ts",
 "tests/e2e/ssh-reconnect-tab-destruction.spec.ts",
 "tests/e2e/ssh-restart-tab-accumulation.spec.ts"]

$ git diff --name-only main..nwparker/ssh-01-e2e-lane | node config/scripts/pr-e2e-source-routing.mjs --ssh-source
false

false is correct and is the point: this PR touches no SSH source, so the new signal does not fire. Every level above it reports true.

The routing change, measured before and after

See "The gap this closes" at the top of this description — the BEFORE/AFTER table across all five stacked PRs is this PR's primary evidence and is stated there rather than repeated here.

(macOS note for reproducing it: /tmp is a symlink to /private/tmp, so a copy of the script placed there fails its own import.meta.url === pathToFileURL(process.argv[1]).href entry guard and silently exits 0 with no output. Copy it somewhere real.)

Lane result

The lane this PR creates, run at the stack tip:

$ pnpm test:e2e:ssh-docker
BUILD_SHA=49bb96e0b4c135ce305a2f4dfe2d4b52e0b19cde   DIRTY=0
14 specs / 20 tests → 17 passed, 2 failed, 1 skipped (10.7m)

This is the lane's first real result, and it is the point of the PR: 14 specs that previously ran nowhere on a PR now run somewhere, in 10.7 minutes against a 60-minute budget.

The two lane failures, resolved — and one is now exempted:

Spec main tip (isolated) tip (full lane) Outcome
ssh-docker-bulk-open-freeze-repro FAILED — harness TypeError: … reading 'workerIndex' at docker-ssh-relay-target.ts:226 — FAILED Not a flake — a 100% failure, and three further arity bugs behind it. Now exempted from the lane; repair tracked in #16764
ssh-port-forward-lifecycle PASSED PASSED (59.6s) FAILED at ssh-port-forward-lifecycle-evidence.ts:159 waiting on getByText('Ports') Flaky in-lane, not a regression

Port-forward was re-run at the same commit and build and passed in 59.6s. It is green on main, green at the tip in isolation, and red only inside the full lane — it is the set's one @headful spec and runs last. Called flaky in-lane rather than "unrelated": a real lane-ordering problem, pre-existing, and worth tracking.

Note on the recorded run: the 14-spec result above predates the bulk-open exemption. The lane now runs 13 specs; bulk-open is not among them. The other 13 results are unaffected — nothing about that spec's failure changes another's outcome, since the runner passes them all to one Playwright invocation with --workers=1 and each spec owns its own fixture container.

Worth being explicit about what those two results mean for this PR. Both specs were claimed by no runner and skipped silently while the shard reported green. Surfacing them is this PR working as intended — and the two are different findings:

  • bulk-open is genuinely broken on main, and worse than the first symptom suggested: putting it in a lane revealed it had rotted four call sites out of date against tests/e2e/helpers/terminal.ts — startDockerSshRelayTarget() called with no argument though the helper dereferences testInfo.workerIndex, execInTerminal missing its ptyId (twice), and splitActiveTerminalPane missing its direction. A 100% failure, not a flake. It was invisible because it ran nowhere and typecheck:e2e is red on main with 240 errors, so four more could not be seen. Either gap alone would have been caught by the other.
  • port-forward is lane-order flaky — green on main, green at the tip in isolation, red only as the last spec in the lane — and nobody knew that either.

That is the coverage gap, demonstrated rather than asserted. Neither is a regression from this stack; both were invisible before this PR gave them a runner.

Both of these were invisible because neither spec was claimed by any runner, so both skipped silently while the shard reported green. One turned out to be genuinely broken on main; the other is lane-order flaky and briefly looked like a regression this very stack had introduced, until an isolated run on both refs disproved it in 59.6s. A lane that can surface a scare you can then disprove in one run is the point — the alternative is not "no scare", it is "no signal". The counterfactual to a noisy lane here was not a quiet correct one; it was a lane that reported green over a real breakage for months.

What was done about bulk-open. The testInfo bug is fixed here — correct on its own, and it removes one real error from typecheck:e2e (240 → 239). The other three are not, because they are not argument plumbing: repairing them requires choosing which ptyId to capture and which split direction to use, and both change what the repro measures. The spec is therefore exempted rather than repaired, for two independent reasons recorded in the runner: it is a perf oracle, not a correctness one (SOFT_FREEZE_LAG_MS=2500 / HARD_FREEZE_LAG_MS=5000 under a deliberate 5-pane flood on a 420s budget — the same rule already applied to ssh-docker-relay-perf.spec.ts), and it is known-rotted. Repair is tracked in #16764.

Applying an existing written rule to a sibling that plainly meets it is consistency; inventing a new exemption to dodge a red would not be. The lane now runs 13 specs.

One more contract assertion came out of this, and it caught a real bug in this PR. unreachableSpecs short-circuits the unclaimed check, so a spec can be documented as exempt while a runner still invokes it — an exemption that reads as coverage removal but changes nothing, leaving the lane red for a reason the file says it excluded. That is exactly the state the first version of this exemption was in. The contract now asserts every exempt spec is absent from every runner, and re-introducing the line fails it by name:

AssertionError: tests/e2e/ssh-docker-bulk-open-freeze-repro.spec.ts is exempt but still
invoked by a lane runner: expected true to be false

Recommendation: track the port-forward flake. The lane's one @headful spec failing after 13 predecessors is a lane-hygiene problem this lane makes visible for the first time. It is pre-existing and out of scope here, but it should get a ticket rather than be absorbed as noise — a lane that is intermittently red for known reasons is how the next silent skip gets rationalised.

pnpm test:e2e:ssh-docker

Build provenance

This PR ships no product code, so there is no bundle symbol to grep — nothing it changes reaches out/. Stating that rather than inventing a symbol. Its artifact is the router itself, which is directly executable and self-verifying (see the two commands above), plus the 36-assertion contract test.

Provenance for the lane run is therefore the build SHA:

git rev-parse HEAD && git status --porcelain
BUILD_SHA=49bb96e0b4c135ce305a2f4dfe2d4b52e0b19cde   DIRTY=0

Clean tree at the stack tip. This level is contained in that build.

Provenance trap, reproduced on this tree. Do not grep a single renderer chunk. grep -c <sym> out/renderer/assets/store-*.js | head -1 matched store-DtDSfs7_.js and reported 0 for a symbol that is present 13 times in store-uJWnDUOY.js, App-CZkf3Bu7.js and quick-commands-menu-events-SBDd6cXN.js. Always use cat out/renderer/assets/*.js | grep -c <sym>.

Also: renderer console.warn never reaches the app log. Only main-process stdio is forwarded by ORCA_E2E_FORWARD_APP_LOGS=1.

What is NOT covered

  • This PR cannot prove the lane catches a real regression — only that every Docker-gated spec is claimed by a runner and that the restore modules now route somewhere. Whether those specs are good is a separate question this PR does not answer.
  • e2e still does not gate merges (pr.yml:796-821). Nothing here blocks a PR. That is deliberate — 20 of 20 recent e2e.yml runs failed or were cancelled — and the contract asserts the current state so the gap stays legible.
  • The two recorded exemptions are genuinely uncovered, by design: ssh-docker-relay-perf.spec.ts and ssh-codex-display-artifacts-repro.spec.ts. The contract asserts each still names a real gated spec so the exemption cannot rot, but neither runs in CI.
  • Five specs and one unit test gated on env vars no workflow sets remain unreachable; they are enumerated in the runner because nothing else names them.
  • Observed on main, outside this stack, noted so nobody attributes it to the PR: src/shared/runtime-client-export-parity.test.ts fails at 026389a3bc0 — runtime-types.ts exports TERMINAL_PTY_DEGRADATION_CAPABILITY, TERMINAL_UNAVAILABLE_ERROR_CODE and terminalUnavailableMessage, none of which are in the test's expected list. Neither file is touched by any level of this stack.
  • typecheck:e2e is red on main with 240 errors. None are in files this PR touches (verified per-file). Pre-existing and untouched.

Failing-first test

Yes. 13 assertions in config/scripts/pr-e2e-gate-contract.test.mjs are red on main and green here, including the unclaimed-spec assertion — which caught a real unclaimed spec on its first run.

@coderabbitai

coderabbitai Bot commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5073c03f-ed5a-44b4-a2c3-336e8ad9a8bc

📥 Commits

Reviewing files that changed from the base of the PR and between ebe6b55 and 21f9c88.

📒 Files selected for processing (12)
  • .github/workflows/e2e.yml
  • .github/workflows/pr.yml
  • config/scripts/pr-e2e-gate-contract.test.mjs
  • config/scripts/pr-e2e-source-routing.mjs
  • config/scripts/run-ssh-docker-e2e.mjs
  • package.json
  • tests/e2e/helpers/docker-ssh-relay-connection.ts
  • tests/e2e/helpers/docker-ssh-relay-terminal-tabs.ts
  • tests/e2e/ssh-cold-activation-restore.spec.ts
  • tests/e2e/ssh-docker-bulk-open-freeze-repro.spec.ts
  • tests/e2e/ssh-reconnect-tab-destruction.spec.ts
  • tests/e2e/ssh-restart-tab-accumulation.spec.ts
🚧 Files skipped from review as they are similar to previous changes (12)
  • config/scripts/pr-e2e-source-routing.mjs
  • tests/e2e/ssh-docker-bulk-open-freeze-repro.spec.ts
  • .github/workflows/pr.yml
  • .github/workflows/e2e.yml
  • package.json
  • tests/e2e/helpers/docker-ssh-relay-terminal-tabs.ts
  • tests/e2e/ssh-cold-activation-restore.spec.ts
  • tests/e2e/ssh-reconnect-tab-destruction.spec.ts
  • config/scripts/pr-e2e-gate-contract.test.mjs
  • config/scripts/run-ssh-docker-e2e.mjs
  • tests/e2e/helpers/docker-ssh-relay-connection.ts
  • tests/e2e/ssh-restart-tab-accumulation.spec.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 6 remain after this review.


📝 Walkthrough

Walkthrough

The PR adds SSH source-change detection to E2E routing and passes the signal through the pull request workflow into the reusable E2E workflow. The Docker SSH isolation job now runs for SSH source changes, uses a 60-minute timeout, and executes the remaining SSH specifications. A dedicated runner invokes 11 Docker SSH specs. New and updated helpers and tests validate remote tab creation, reconnection identity preservation, and tab stability across repeated application restarts.

Merge Risk: 🟡 Moderate · up to 21f9c

The PR expands SSH end-to-end routing and adds a dedicated Docker lane, but deletion-only SSH changes may still bypass that lane, while the restart regression test can self-heal missing worktree state and falsely pass. These bounded CI coverage risks require explicit owner follow-up before merge.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description provides detailed rationale, change scope, risks, testing evidence, and verification steps. However, it omits required template sections and information, including a Linked Issue with … Reformat the description using all required template headings. Add a valid Linked Issue entry, state N/A with a reason under Visual Proof because this is CI-only, document the tested platforms and complete the Testing checklist, and compl…
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 9 files. (3 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main changes: triggering the Docker-SSH lane from SSH source changes and preventing unclaimed specs.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description provides detailed rationale, change scope, risks, testing evidence, and verification steps. However, it omits required template sections and information, including a Linked Issue with Fixes #, Visual Proof or an explicit N/A explanation, the Agent skill upstream boundary section, Notes, Review, and the checklist.

Resolution

Reformat the description using all required template headings. Add a valid Linked Issue entry, state N/A with a reason under Visual Proof because this is CI-only, document the tested platforms and complete the Testing checklist, and complete or mark the remaining applicable sections and checklist items.

Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 9 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/pr.yml (1)

774-782: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Include deleted SSH source files.

--diff-filter=AMCR removes deletions before hasSshSourceChange runs. A deletion-only SSH source change therefore does not trigger the Docker-SSH lane. Include D, or remove the filter if all changed paths are valid routing inputs.

Proposed fix
-          CHANGED="$(git diff --name-only --diff-filter=AMCR --merge-base "$BASE" "$HEAD")"
+          CHANGED="$(git diff --name-only --diff-filter=ADMCR --merge-base "$BASE" "$HEAD")"

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1171a48b-584c-4641-99f4-f7bfa32538c6

📥 Commits

Reviewing files that changed from the base of the PR and between 026389a and 607718a.

📒 Files selected for processing (11)
  • .github/workflows/e2e.yml
  • .github/workflows/pr.yml
  • config/scripts/pr-e2e-gate-contract.test.mjs
  • config/scripts/pr-e2e-source-routing.mjs
  • config/scripts/run-ssh-docker-e2e.mjs
  • package.json
  • tests/e2e/helpers/docker-ssh-relay-connection.ts
  • tests/e2e/helpers/docker-ssh-relay-terminal-tabs.ts
  • tests/e2e/ssh-cold-activation-restore.spec.ts
  • tests/e2e/ssh-reconnect-tab-destruction.spec.ts
  • tests/e2e/ssh-restart-tab-accumulation.spec.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.

Comment on lines +100 to +106
async function waitForRestoredRemoteWorktree(
page: Page,
targetId: string,
worktreeId: string
): Promise<void> {
await waitForSessionReady(page, 60_000)
await expect.poll(() => waitForActiveWorktree(page), { timeout: 60_000 }).toBe(worktreeId)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Do not repair the state that this test verifies.

waitForActiveWorktree calls setActiveWorktree when the selection is empty. This can hide a failure to restore activeWorktreeId. Poll the stored value without mutation.

Proposed fix
   await waitForSessionReady(page, 60_000)
-  await expect.poll(() => waitForActiveWorktree(page), { timeout: 60_000 }).toBe(worktreeId)
+  await expect
+    .poll(
+      () => page.evaluate(() => window.__store?.getState().activeWorktreeId ?? null),
+      { timeout: 60_000 }
+    )
+    .toBe(worktreeId)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async function waitForRestoredRemoteWorktree(
page: Page,
targetId: string,
worktreeId: string
): Promise<void> {
await waitForSessionReady(page, 60_000)
await expect.poll(() => waitForActiveWorktree(page), { timeout: 60_000 }).toBe(worktreeId)
async function waitForRestoredRemoteWorktree(
page: Page,
targetId: string,
worktreeId: string
): Promise<void> {
await waitForSessionReady(page, 60_000)
await expect
.poll(
() => page.evaluate(() => window.__store?.getState().activeWorktreeId ?? null),
{ timeout: 60_000 }
)
.toBe(worktreeId)

@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

…gated spec

The Docker-SSH e2e lane only ran when a PR's changed specs happened to include
`ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`.
Editing SSH source itself did not trigger it, and pruning either spec from a
route's list would have silently retired the whole lane. Meanwhile the sharded
lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself
while the shard still reported green -- the exact silent-skip shape
`docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions
that reached users.

Separately, the modules that actually own direct-SSH workspace and tab restore
carry no "ssh" in their names, so the `ssh-terminal-source` route never reached
them. Measured on the real script before this change:

    printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
      src/main/ipc/remote-workspace-snapshot-normalization.ts \
      src/renderer/src/lib/worktree-initial-terminal-seeding.ts \
      src/shared/remote-workspace-session-projection.ts \
      | node config/scripts/pr-e2e-source-routing.mjs
    => []

Three changes, all pinned by the executable gate contract:

- `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH
  routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source
  rather than on a spec name surviving in a list. One list, so the two cannot
  drift.
- A sibling `ssh-workspace-session-restore` route names the restore seams
  (`remote-workspace-*`, `worktree-initial-terminal-seeding`,
  `worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the
  two restore specs -- a sibling rather than more paths on `ssh-terminal-source`
  so a tab-tombstone edit does not run the whole SSH terminal list.
- A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on
  the one VM that sets the flag, and the contract now fails by name when any
  Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and
  `ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock
  budgets; needs a real remote codex binary) and the contract asserts each
  exemption still corresponds to a real gated spec, so a stale one cannot
  quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added
  serial specs.

The lane's first act was to surface four latent bugs in a spec that had been
silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites
out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()`
is called with no argument though the helper dereferences `testInfo.workerIndex`
(a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and
`splitActiveTerminalPane` gained a direction. It was invisible because it ran
nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four
more could not be seen.

The `testInfo` bug is fixed here -- correct on its own, and it removes one real
error from `typecheck:e2e` (240 -> 239). The other three are not, because they
are not argument plumbing: repairing them requires choosing which ptyId to
capture and which split direction to use, and both change what the repro
measures.

The spec is therefore added to the exemption list rather than repaired, for two
independent reasons recorded in the runner: it is a perf oracle, not a
correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured
under a deliberate 5-pane flood on a 420s budget -- the same rule already applied
to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in
#16764. Applying an existing written rule to a sibling that plainly
meets it is consistency; inventing a new exemption to dodge a red would not be.

Three hardening fixes to the contract itself:

- Runner text is comment-stripped before the claimed-by-a-lane scan. A substring
  scan over raw text lets a spec merely *discussed* in a runner comment count as
  claimed -- the silent skip this assertion exists to catch, re-entering through
  the documentation. Not live today only because the existing comments write the
  spec names without their `tests/e2e/` prefix.
- An exempt spec must not be invoked by any runner. `unreachableSpecs`
  short-circuits the unclaimed check, so a spec could be documented as exempt
  while a runner still ran it -- an exemption that reads as coverage removal but
  changes nothing, leaving the lane red for a reason the file says it excluded.
  This is not hypothetical: adding the bulk-open exemption without removing it
  from the runner's spec list produced exactly that state, and this assertion is
  what caught it.

- The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/`
  rather than one fixed string, so a double-quoted or `!==` spelling can no
  longer escape the contract.

`ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence
asserting tab-id set identity, not just the active pane's reclaimed ptyId as
`ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated
by a negative control that injected one tab after cycle 1 and correctly failed.
@nwparker
nwparker force-pushed the nwparker/ssh-01-e2e-lane branch from 21f9c88 to 4630033 Compare August 28, 2026 02:31
@nwparker
nwparker merged commit 971d987 into main Aug 28, 2026
35 of 37 checks passed
brennanb2025 added a commit that referenced this pull request Aug 28, 2026
`ssh-pi-compatible-agent-title.spec.ts` failed its first-ever CI run with
`No remote worktree found for /tmp/orca-docker-relay-perf-repo`, thrown from
its own private `connectDockerRemote` before it reached a single title
assertion. The spec ran in no CI lane until #16746 claimed it, so there is no
green baseline it regressed from.

`connectDockerRemote` was a stale fork of `connectDockerSshRelayTarget`,
missing the repo-owner wait and the authoritative fetch the shared helper
grew. It hydrated with a bare `fetchWorktrees(repoId)` and read
`worktreesByRepo[repoId][0]`.

Deploying the relay bounces the SSH transport on this fixture -- CI logs
`[ssh] Reconnecting ... failed handshakes 0/9` half a second after
`Relay started successfully`, and that bounce reproduces on clean main. While
the target reads back as reconnecting, `getCurrentDirectSshAuthority` resolves
nothing, so `fetchWorktrees` returns false and writes no authoritative rows.

Both sides of the shared helper sampled a single instant too: it pinned the
authority from `ssh.connect` and threw if the live state had rotated by
hydration time. Hydration now waits for the target to report a live connected
authority, hydrates against that one, and re-derives if the transport rotated
mid-flight, bounded by a 60s deadline. The spec drops its private fork and
calls the shared helper.

No assertion changed; the three OMP/Pi title assertions are untouched.
paidaxingyo666 pushed a commit to paidaxingyo666/Manta that referenced this pull request Aug 30, 2026
…gated spec (stablyai#16746)

The Docker-SSH e2e lane only ran when a PR's changed specs happened to include
`ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`.
Editing SSH source itself did not trigger it, and pruning either spec from a
route's list would have silently retired the whole lane. Meanwhile the sharded
lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself
while the shard still reported green -- the exact silent-skip shape
`docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions
that reached users.

Separately, the modules that actually own direct-SSH workspace and tab restore
carry no "ssh" in their names, so the `ssh-terminal-source` route never reached
them. Measured on the real script before this change:

    printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
      src/main/ipc/remote-workspace-snapshot-normalization.ts \
      src/renderer/src/lib/worktree-initial-terminal-seeding.ts \
      src/shared/remote-workspace-session-projection.ts \
      | node config/scripts/pr-e2e-source-routing.mjs
    => []

Three changes, all pinned by the executable gate contract:

- `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH
  routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source
  rather than on a spec name surviving in a list. One list, so the two cannot
  drift.
- A sibling `ssh-workspace-session-restore` route names the restore seams
  (`remote-workspace-*`, `worktree-initial-terminal-seeding`,
  `worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the
  two restore specs -- a sibling rather than more paths on `ssh-terminal-source`
  so a tab-tombstone edit does not run the whole SSH terminal list.
- A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on
  the one VM that sets the flag, and the contract now fails by name when any
  Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and
  `ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock
  budgets; needs a real remote codex binary) and the contract asserts each
  exemption still corresponds to a real gated spec, so a stale one cannot
  quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added
  serial specs.

The lane's first act was to surface four latent bugs in a spec that had been
silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites
out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()`
is called with no argument though the helper dereferences `testInfo.workerIndex`
(a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and
`splitActiveTerminalPane` gained a direction. It was invisible because it ran
nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four
more could not be seen.

The `testInfo` bug is fixed here -- correct on its own, and it removes one real
error from `typecheck:e2e` (240 -> 239). The other three are not, because they
are not argument plumbing: repairing them requires choosing which ptyId to
capture and which split direction to use, and both change what the repro
measures.

The spec is therefore added to the exemption list rather than repaired, for two
independent reasons recorded in the runner: it is a perf oracle, not a
correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured
under a deliberate 5-pane flood on a 420s budget -- the same rule already applied
to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in
stablyai#16764. Applying an existing written rule to a sibling that plainly
meets it is consistency; inventing a new exemption to dodge a red would not be.

Three hardening fixes to the contract itself:

- Runner text is comment-stripped before the claimed-by-a-lane scan. A substring
  scan over raw text lets a spec merely *discussed* in a runner comment count as
  claimed -- the silent skip this assertion exists to catch, re-entering through
  the documentation. Not live today only because the existing comments write the
  spec names without their `tests/e2e/` prefix.
- An exempt spec must not be invoked by any runner. `unreachableSpecs`
  short-circuits the unclaimed check, so a spec could be documented as exempt
  while a runner still ran it -- an exemption that reads as coverage removal but
  changes nothing, leaving the lane red for a reason the file says it excluded.
  This is not hypothetical: adding the bulk-open exemption without removing it
  from the runner's spec list produced exactly that state, and this assertion is
  what caught it.

- The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/`
  rather than one fixed string, so a double-quoted or `!==` spelling can no
  longer escape the contract.

`ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence
asserting tab-id set identity, not just the active pane's reclaimed ptyId as
`ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated
by a negative control that injected one tab after cycle 1 and correctly failed.

(cherry picked from commit 971d987)
enrick-dev pushed a commit to enrick-dev/orca that referenced this pull request Oct 15, 2026
…gated spec (stablyai#16746)

The Docker-SSH e2e lane only ran when a PR's changed specs happened to include
`ssh-startup-exec-readiness.spec.ts` or `paired-startup-exec-readiness.spec.ts`.
Editing SSH source itself did not trigger it, and pruning either spec from a
route's list would have silently retired the whole lane. Meanwhile the sharded
lanes set no `ORCA_E2E_SSH_DOCKER`, so every Docker-gated spec skipped itself
while the shard still reported green -- the exact silent-skip shape
`docs/reference/ssh-reconnect-source-recovery.md` blames for four regressions
that reached users.

Separately, the modules that actually own direct-SSH workspace and tab restore
carry no "ssh" in their names, so the `ssh-terminal-source` route never reached
them. Measured on the real script before this change:

    printf '%s\n' src/renderer/src/hooks/remote-workspace-session-merge.ts \
      src/main/ipc/remote-workspace-snapshot-normalization.ts \
      src/renderer/src/lib/worktree-initial-terminal-seeding.ts \
      src/shared/remote-workspace-session-projection.ts \
      | node config/scripts/pr-e2e-source-routing.mjs
    => []

Three changes, all pinned by the executable gate contract:

- `hasSshSourceChange` derives an `ssh_source_changed` signal from the SSH
  routes themselves, plumbed pr.yml -> e2e.yml, so the lane triggers on source
  rather than on a spec name surviving in a list. One list, so the two cannot
  drift.
- A sibling `ssh-workspace-session-restore` route names the restore seams
  (`remote-workspace-*`, `worktree-initial-terminal-seeding`,
  `worktree-default-terminal-tabs`, `initial-terminal`) and routes them to the
  two restore specs -- a sibling rather than more paths on `ssh-terminal-source`
  so a tab-tombstone edit does not run the whole SSH terminal list.
- A new `test:e2e:ssh-docker` runner claims the remaining Docker-gated specs on
  the one VM that sets the flag, and the contract now fails by name when any
  Docker-gated spec is claimed by no runner. `ssh-docker-relay-perf` and
  `ssh-codex-display-artifacts-repro` are recorded exemptions (wall-clock
  budgets; needs a real remote codex binary) and the contract asserts each
  exemption still corresponds to a real gated spec, so a stale one cannot
  quietly excuse a gap. Lane timeout raised 35 -> 60 minutes for the added
  serial specs.

The lane's first act was to surface four latent bugs in a spec that had been
silently skipping. `ssh-docker-bulk-open-freeze-repro.spec.ts` is four call sites
out of date against `tests/e2e/helpers/terminal.ts`: `startDockerSshRelayTarget()`
is called with no argument though the helper dereferences `testInfo.workerIndex`
(a 100% failure, not a flake), `execInTerminal` gained a `ptyId` parameter, and
`splitActiveTerminalPane` gained a direction. It was invisible because it ran
nowhere and `typecheck:e2e` is red on main with 240 pre-existing errors, so four
more could not be seen.

The `testInfo` bug is fixed here -- correct on its own, and it removes one real
error from `typecheck:e2e` (240 -> 239). The other three are not, because they
are not argument plumbing: repairing them requires choosing which ptyId to
capture and which split direction to use, and both change what the repro
measures.

The spec is therefore added to the exemption list rather than repaired, for two
independent reasons recorded in the runner: it is a perf oracle, not a
correctness one (`SOFT_FREEZE_LAG_MS=2500` / `HARD_FREEZE_LAG_MS=5000` measured
under a deliberate 5-pane flood on a 420s budget -- the same rule already applied
to `ssh-docker-relay-perf.spec.ts`), and it is known-rotted. Repair is tracked in
stablyai#16764. Applying an existing written rule to a sibling that plainly
meets it is consistency; inventing a new exemption to dodge a red would not be.

Three hardening fixes to the contract itself:

- Runner text is comment-stripped before the claimed-by-a-lane scan. A substring
  scan over raw text lets a spec merely *discussed* in a runner comment count as
  claimed -- the silent skip this assertion exists to catch, re-entering through
  the documentation. Not live today only because the existing comments write the
  spec names without their `tests/e2e/` prefix.
- An exempt spec must not be invoked by any runner. `unreachableSpecs`
  short-circuits the unclaimed check, so a spec could be documented as exempt
  while a runner still ran it -- an exemption that reads as coverage removal but
  changes nothing, leaving the lane red for a reason the file says it excluded.
  This is not hypothetical: adding the bulk-open exemption without removing it
  from the runner's spec list produced exactly that state, and this assertion is
  what caught it.

- The Docker-gate detector is now `/ORCA_E2E_SSH_DOCKER\s*[!=]==\s*['"]1['"]/`
  rather than one fixed string, so a double-quoted or `!==` spelling can no
  longer escape the contract.

`ssh-restart-tab-accumulation.spec.ts` is a new three-cycle restart fence
asserting tab-id set identity, not just the active pane's reclaimed ptyId as
`ssh-cold-activation-restore.spec.ts:241` did. It passes today; it was validated
by a negative control that injected one tab after cycle 1 and correctly failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant