Fix E2E harness launch delegation and welcome-dialog interference

[Chiara Mooney (chiaramooney)]

Summary

23 of the E2E specs failed on developer machines. The cause was two-part, both in the shared Playwright launcher, and neither involved product code — every change here is under src/test/e2e/, plus a one-line package.json script fix.

Before / after

Run on a developer machine with a normal VS Code instance already running, which is the exact condition that triggers the bug:

	Passed	Failed	Did not run
Before	0	23	172
After	192	0	0

4 tests are flaky (fail once, pass on retry); details below. npx tsc --noEmit is clean and npm run lint holds at the 12 pre-existing warnings / 0 errors baseline.

Root cause

1. Launch delegation — why an isolated --user-data-dir is required

helpers.ts launched VS Code without --user-data-dir. When the developer already has VS Code running — the common case — the newly spawned Code.exe does not create its own window. It hands the request off to the existing instance and then exits immediately. Playwright is left holding a dead process, so app.firstWindow() never resolves and the launch times out.

This is not obvious from the failure output, which just looks like a slow start-up, so it is worth stating plainly for future readers: without an isolated user-data dir, the launch is silently delegated to another process and the test harness loses its window.

The blast radius was large because the 13 manifest-editor specs share a single VS Code instance via shared-context.ts — one launch failure cascaded into all of them, which is why 172 tests never even ran.

Every launch now gets its own --user-data-dir and --extensions-dir created under a temp directory (the latter also keeps the developer's installed extensions out of the run), removed on teardown.

2. Welcome dialog — why dismissal is required

An isolated user-data dir means a brand-new profile, and a brand-new profile means VS Code shows its Welcome / Trust dialog on first start. That dialog holds keyboard focus and swallows the Ctrl+Shift+P used to open the Command Palette. Fixing only part 1 therefore just moves the failure from the launch into getWebviewFrame.

Launch now dismisses the dialog, tolerantly — it is a no-op when no dialog appears, so it is safe on any profile state.

Fixing both parts is what takes the suite green; either alone is not enough.

One launch path

All launches now go through launchVSCodeApp() in helpers.ts. sign-quickpick.spec.ts and input-folder-validation.spec.ts no longer carry their own copies of the launch logic.

test:e2e now builds dist/

The only change outside src/test/e2e/:

-"test:e2e": "npx playwright test --config=playwright.config.ts"
+"test:e2e": "npm run compile && npx playwright test --config=playwright.config.ts"

The extension's main is ./dist/extension.js, built by esbuild. pretest only runs compile-tsc, which builds out/ — that is test-only and does not make the extension loadable. Running the E2E suite against a stale or missing dist/ fails activation with "Cannot find module", and every test then fails in a way that looks like a product bug rather than a build problem. Since this PR is about making the suite reliable, the trap belongs closed here.

Latent spec races this exposed

With the launch fixed, the specs got far enough to reveal pre-existing races that the launch failure had been masking. Fixed here with web-first assertions rather than longer sleeps:

• Command Palette interaction used fixed sleeps plus a blind Enter. On a cold isolated profile the extension can still be activating, so the command was not yet listed and Enter did nothing. runCommand() now waits for the command row to appear, and does not return until the palette has actually gone — the quick-input widget is reused by whatever the command shows next, so a caller that inspects it too early reads the dismissing palette instead of the command's own UI.
• Sign specs read placeholders with a non-retrying getAttribute, and asserted toHaveCount(0) on rows that VS Code leaves in the DOM after dismissal. They now assert on placeholder attributes and on widget visibility.

Remaining flakes (not introduced here)

These fail at most once and pass on retry; all are pre-existing test-ordering / editor-reload races unrelated to launch:

• applications-tab.spec.ts › Show Name on Tiles checkboxes are visible — the test does not re-select the Applications tab, so a preceding test's edit can leave the card unrendered.
• background-task-fixture.spec.ts / push-notifications-fixture.spec.ts › identity fields are populated correctly, and properties-tab.spec.ts › logo path is populated — the custom editor occasionally does not reload on the fixture swap within the wait window.

I tried a content-based wait for the last group and it produced no measured improvement across two full runs, so I reverted it rather than leave an unproven change in the diff. These look like they may be real editor-reload behavior worth a separate investigation, and I left them alone deliberately rather than papering over them.

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[github-actions]

VSIX Build

Artifact	Baseline	Current	Delta
VS Code Extension (VSIX)	27.88 MB	27.88 MB	📈 +0.0 KB (+0.00%)

---

Updated 2026-09-14 19:41:42 UTC · commit 2d5df0d · workflow run