Skip to content

[PR] ci(#4437): balance test shards by measured duration, not line count — windows worst leg ~22min down to ~7min - #4439

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-23_ci-shard-duration-weights
Oct 23, 2026
Merged

[PR] ci(#4437): balance test shards by measured duration, not line count — windows worst leg ~22min down to ~7min#4439
ranxianglei merged 3 commits into
masterfrom
2026-10-23_ci-shard-duration-weights

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 23, 2026

Copy link
Copy Markdown
Owner

What

The test-matrix sharding (scripts/run-ci-tests.mjs) balanced shards by file LINE COUNT. Line count is blind to wall time: process-spawning suites (dsh-real-cli etc., one node child per case) cost orders of magnitude more per line than pure assertion files, so "perfectly balanced" layouts still produced 5min..37min legs from equal byte counts. This weights the partition by MEASURED duration and thickens the windows split to /5.

Changes

  • scripts/ci-shard-partition.mjs (+ .d.mts): extracted partitionAll with weight = last measured ms per file (scripts/ci-shard-weights.json); files missing from the table fall back to lineCount × msPerLineFallback. Same deterministic total LPT otherwise (union of shards == allFiles, nothing skipped).
  • scripts/ci-shard-weights.json: 840-file table measured post-[PR] fix(server): closeAllConnections() cuts through the post-response linger (#4380) #4384 (single-file node --test runs on a VM; local-env hangs capped at the 150s timeout and listed under timedOut, local rc=1 env-fails excluded). Regenerate via scripts/measure-test-durations.mjs when balance degrades.
  • scripts/run-ci-tests.mjs: delegates partitioning to the module (the script has top-level side effects and cannot be imported by tests).
  • tests/ci-shard-weights.test.ts: regression pins — measured-value priority, fallback for new files, LPT spreading spawn-heavy monsters, determinism + total-union invariant, weight-table hygiene.
  • .github/workflows/ci.yml: windows-22 CORE 3→5 legs, windows FULL 22/24 each 3→5; ubuntu stays /3. Stale matrix-size comments in the plan job updated to match.

Measured effect

  • At measurement time: max shard 1984s → 794s serial.
  • Recomputed locally against CURRENT master's file set (861 files, 21 newer than the table → fallback weights): n=3 max shard ≈1360s → n=5 ≈816s, spread across the five legs < 1s. With c=4 in-leg concurrency the windows worst leg targets ≈7 min incl. setup.

Behavior change disclosure (owner action item)

New check-run names appear: test (windows-latest, 22, 4|5) and test (windows-latest, 24, 4|5). Branch protection keys off test (os, node, shard) names — please add these four names to required checks; until then failures in the new shards won't block merges. The existing twelve legacy names are unchanged and keep gating. No user-visible behavior change; no config surface change; no kernel/pi-subagents change.

Provenance

Cherry-pick of commit 0569781af from closed PR #4361 (branch 2026-10-23_ci_windows_under_8min), stranded when that PR was superseded by #4377 and closed per owner decision; picked up per the owner invitation in the #4361 thread ("若这里面还有 #4377 没覆盖的点,欢迎在新 PR 里捡起来"). The sibling commit d06d192ff (LIVE_FILES registration) is already on master via #4416 and is deliberately excluded. Original authorship preserved (ework-agent <ework@relay>).

Validation

  • Targeted: node --import tsx --test tests/ci-shard-weights.test.ts → 4/4 pass; npm run typecheck clean; workflow YAML parses.
  • Full-suite verdict: remote CI on this head.

Closes #4437

@ranxianglei ranxianglei changed the title ci(#4437): balance test shards by measured duration, not line count — windows worst leg ~22min down to ~7min [PR] ci(#4437): balance test shards by measured duration, not line count — windows worst leg ~22min down to ~7min Oct 23, 2026
This was referenced Oct 23, 2026
Closed
Closed
ework-agent added 2 commits October 24, 2026 00:33
ework-agent
ci(#4437): balance shards by measured duration, not line count
9f3d208 
Line-count LPT balanced bytes, not seconds: process-spawning suites
cost orders of magnitude more per line than pure assertion files, and
the 'perfectly balanced' layouts still produced 5min..37min legs from
equal byte counts (windows 22-2/22-3 vs 22-1/22-4 on the same run).
With #4384's linger cut landed the per-file wall times are stable
enough to weight the partition by MEASURED duration:

- scripts/ci-shard-partition.mjs: fileWeight = last measured ms per
  file (scripts/ci-shard-weights.json), new files fall back to
  lineCount x msPerLineFallback (median ms/line); same deterministic
  total LPT otherwise.
- scripts/ci-shard-weights.json: 840-file table measured post-#4384
  (single-file node --test runs, VM, ~66min serial total); local-env
  hangs capped at the 150s timeout and listed under timedOut, local
  rc=1 env-fails excluded. Regenerate via
  scripts/measure-test-durations.mjs when shard balance degrades.
- run-ci-tests.mjs delegates partitionAll to the module (this script
  has top-level side effects and cannot be imported by tests).
- ci.yml: windows-22 CORE 3->5 shards, windows FULL 3->5 (both node
  versions) — measured max shard drops 1984s->794s serial (concurrency
  c=4 per shard => ~7min worst-leg target incl. setup). Four NEW check
  names appear (windows 22/24 shards 4/5); branch protection must add
  them before they gate merges.

Cherry-picked from closed PR #4361 (commit 0569781, stranded when
the PR was superseded by #4377; picked up per owner invitation in the
#4361 thread). The sibling LIVE_FILES commit (d06d192) is already on
master via #4416 and is not included.

Closes #4437
ework-agent
fix(ci): dsh-settings-ui weight 150007ms was the pre-#4436 hang cap —…
34a80e0 
… measured ~1s post-fix (#4437)
@ranxianglei
ranxianglei force-pushed the 2026-10-23_ci-shard-duration-weights branch from d2806db to 34a80e0 Compare October 23, 2026 16:34
@ranxianglei

ranxianglei commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

(ai: zai/glm-5.3)

已 rebase 到当前 master(7017777,含挂死修复 #4436——不带上它 PR 自己的 CI 还是会在 dsh-settings-ui 所在分片挂死),另修一处陈旧权重:

本地验证:tests/ci-shard-weights.test.ts 4/4、typecheck 干净。新头 34a80e0。

@github-actions

github-actions Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_ci-shard-duration-weights (eeedfe3)

Latest build — install this exact version

npm install -g billion-context@0.0.5319-pr.4439.d526424

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
eeedfe3 billion-context@0.0.5319-pr.4439.d526424 build
34a80e0 billion-context@0.0.5318-pr.4439.20b8fba build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4439.tgz
npm install -g package

Install instructions are refreshed on each push.

This was referenced Oct 23, 2026
Closed
Merged
@ranxianglei
ranxianglei merged commit fd973ca into master Oct 23, 2026
35 of 36 checks passed
@github-actions

github-actions Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.777-master.fd973ca (rolling master tag).

npm install -g billion-context@0.0.777-master.fd973ca  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

This was referenced Oct 23, 2026
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Rebalance CI test sharding by measured duration instead of line count: slowest Windows leg ~22 min down to ~7 min

1 participant

@ranxianglei