CI Console job times out at 15 min on main since 2026-10-21

CI's Console job (.github/workflows/ci.yml, timeout-minutes: 15) is cancelled at its 15-minute timeout in the Browser lane step on every recent main run: 4cc44f8, fe1daa9, daa279a and 7b5b4d9 (2026-10-21), each 15 min 16 s. Every PR that touches ui/ or e2e/console/ inherits it, so merge-pr.sh refuses them. PR #844 hit it on three heads.

On a local machine, pnpm test:console --project=chromium at the #844 branch failed e2e/console/actions.spec.ts:166 (drain from Fleet), e2e/console/activity.spec.ts:371 and e2e/console/queue.spec.ts:163. Nobody ran those on main, so they may or may not be the cause.

Expected: the Console job finishes inside its timeout on main.

Written by an agent.

feat(warm-pool): status, doctor, console and docs show learned kinds …

Reproduction

Gap. One 15-minute job runs all three browsers, and the lane now needs about 33 minutes. The test pins a proposal: the lane split into shards.

Corrections: it started 2026-10-19 at 60521f0 (#626). The failing tests are not the cause: the first run that timed out had none, and they cost seconds (#853, #863).

branch bug/852-repro, src/test-lanes.test.ts
› the console lane in CI (.github/workflows/ci.yml) › runs the console lane over a matrix of at least four shards, ...
$ pnpm exec vitest run --project unit src/test-lanes.test.ts
AssertionError: shards in the console lane's matrix: expected 0 to be greater than or equal to 4

Console job logs on main (2 workers):

72fe60ed  last pass     348 tests  Chromium 350 s  Chromium 3m10  job 12m19
60521f0f  #626, cut     375 tests  Chromium 583 s  Chromium 6m20  webkit 35/125 done
1680c5c1  today, cut    624 tests  Chromium 976 s  Chromium 9m04  firefox 108/208, webkit 0
ratios: firefox 1.11x, webkit 1.36x Chromium  ->  ~31 min of tests + ~1 min setup
local main, Chromium only, --workers=2: 5.9 min

Root cause

.github/workflows/ci.yml:192-200: the Console job runs pnpm test:console in every browser (the three projects in playwright.config.ts), using the runner's 2 workers, with timeout-minutes: 15. Chromium test time grew 2.8x in three days. #626 tipped it over: five tests in it wait for the wall-clock minute to end (#865), about 270 s per browser.

Simplest fix

Copy the e2e job's layout. Rename the lane job console-lane (Console (<i>/4)) and give it strategy: {fail-fast: false, matrix: {shard: [1, 2, 3, 4]}}. Add --shard="$SHARD/4" to both pnpm test:console lines. Add a new Console job with needs: [changes, console-lane] and if: always() that fails on any needs.*.result other than success or skipped, like Quality does.

Alternatives rejected

Raise the timeout to 40 minutes: every UI PR waits about 35 minutes.
Fix only the minute waits (Five console tests each wait up to a minute of wall clock for a usage minute to close #865): this leaves about 25 minutes.
workers: 4: same runner, and more contention for the "within a second" assertions that already flake (console e2e: Activity:371 and Queue:163 fail on main since #834 #853).

Risk

Branch protection requires the check named Console. Only the aggregate job may carry that name.
Playwright shards by file per project. Check the balance across shards on the first run. overview.spec.ts alone is about 200 s per browser.
A lane skipped by Changes must still leave Console green.

Side findings

Written by an agent.

docs(toolchain): the console lane runs in four shards (#852)

fix(ci): merge-pr re-runs a cancelled console shard; the lane test pi…

fix(ci): shard the console lane over four jobs (#868)

test(console): stop the Overview's route stubs before the fleet shuts…

Navigation Menu

Repository navigation

CI Console job times out at 15 min on main since 2026-10-21 #852

Reproduction

Root cause

Simplest fix

Alternatives rejected

Risk

Side findings

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

CI Console job times out at 15 min on main since 2026-10-21 #852

Description

Activity

V3RON commented on Oct 22, 2026

Reproduction

Root cause

Simplest fix

Alternatives rejected

Risk

Side findings

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions