Skip to content

ci: give web test shards headroom over their slowest run (10 → 15 minutes) - #637

Merged
yaacovcorcos merged 2 commits into
mainfrom
claude/ci-web-timeout-20261023
Oct 23, 2026
Merged

yaacovcorcos merged 2 commits into
mainfrom
claude/ci-web-timeout-20261023

Conversation

@yaacovcorcos

Copy link
Copy Markdown
Contributor

What this changes

The time limit for the three Test Web jobs in .github/workflows/ci.yml goes from 10 to 15 minutes. It's marked as a Scient change, since upstream uses 10.

Why

The slowest web shard now takes about 10 minutes, setup and cleanup included (9 min 50 s to 10 min 18 s on recent runs). With a 10-minute limit, a shard can pass every step and still be cancelled during cleanup.

On #634's merge-queue run, Test Web 1 passed all its tests but was cancelled at 10 min 3 s. The queue counted the cancellation as a failed check and ejected the PR.

Times from that run:

Shard Setup and cleanup Tests Total
1 about 1.4 min 8.3 min (+18 s editor timing checks) about 10 min
2 about 1.2 min 5.3 min about 6.5 min
3 about 1.2 min 8.2 min about 9.5 min

15 minutes keeps roughly 50% headroom over the slowest observed shard, without padding far beyond it. The server shards already follow the same rule: their limit is 35 minutes, set because "a green shard gets cancelled mid-cleanup" otherwise.

Not done here: splitting the web tests into more shards would make them faster. But the shard names are required checks in branch protection, and the queue-reuse script expects three shards, so that's a separate decision.

🤖 Generated with Claude Code

…utes)

The slowest web shard takes about 10 minutes, setup and cleanup included
(9m50s to 10m18s in October 2026). With a 10-minute cap, a shard whose
every step passed was cancelled during cleanup, and the merge queue treated
that as a failed check and ejected #634. The cap now keeps headroom over the
slowest observed shard, as the server shards already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XS labels Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.6 KiB 5.6 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 4.3 KiB 4.3 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.3 KiB 1.3 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 26.2 KiB 26.2 KiB 0 B (0.0%) 29.3 KiB ✅
Codex Live turn messages 2 2 0 (0.0%) 8 ✅
Claude Total thread wire 5.6 KiB 5.6 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 4.3 KiB 4.3 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.3 KiB 1.3 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 26.6 KiB 26.6 KiB 0 B (0.0%) 29.3 KiB ✅
Claude Live turn messages 2 2 0 (0.0%) 8 ✅

Baseline: d5c9445 · PR result: ba4c80c · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Full compact HTTP snapshot with live cursor.

  • Codex decoded thread snapshot: 133.1 KiB
  • Claude decoded thread snapshot: 133.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@yaacovcorcos
yaacovcorcos added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 87e2e1f Oct 23, 2026
45 checks passed
@yaacovcorcos
yaacovcorcos deleted the claude/ci-web-timeout-20261023 branch October 23, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant