Repository navigation
CI: eight mutant shards, and a 30-minute timeout on each - #215
Conversation
Six shards ran 14 to 24 minutes against the 25-minute timeout, and one was cut at 25:00 on three attempts of one pull request, once 20 ms after it had killed every mutant. The matrix gains a seventh shard; the timeout and the judging step stay. The example and the standard carry the workflow word for word, and the head comment states the measured times. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The review's one major: a seventh shard leaves the slowest leg near 21 minutes against 25, past the comment's own rule. The Maintainer chose eight shards, about 126 mutants each and 18 to 19 minutes at the slowest, and a 30-minute timeout, so neither run-to-run variance nor the next hundred mutants cuts a leg. The comment's figures are the measured ones, and the design no longer counts the shards. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 41 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
1 participant
Why
The mutation check's six shards run close to their 25-minute job timeout, and one is now cut by it. Measured on the last day:
A mutant that names no test runs the whole suite, with the simulation, before it is read as killed; about half the list does, and that is most of a shard's time. The workflow's own comment said to add a shard before the shards neared 15 minutes.
What changes
The matrix in
.github/workflows/verify.ymlgains two shards, eight in all of about 126 mutants each, and themutantsjob's timeout goes from 25 to 30 minutes.MUTANTS_SHARDandMUTANTS_SHARDSalready come fromstrategy.job-indexandstrategy.job-total, so nothing else in the workflow changes: the judging step andverify'sneedsare untouched, no permission or pin moves.examples/workflows/verify.ymland the first block ofdocs/standards/workflows.mdcarry the workflow word for word, and the head comment now states the measured times. Every mutant is in exactly one shard at any count from 1 to 12 (tests/test_tools.py::test_every_mutant_is_in_exactly_one_shard).Part of #180. No decision record: nothing is decided, two numbers grow. The first head carried a seventh shard alone; the review's one major was that it leaves the slowest leg near 21 minutes against 25, past the comment's own rule, and the Maintainer chose eight shards (about 18:40 at the slowest) and a 30-minute timeout.
Reviews
A fresh-context Claude review with a security pass against
docs/standards/workflows.md, and Muse Wonder Full, read the first head, 75b21ef. Findings, all folded into the one fix commit: seven shards is not enough by the comment's own rule (Claude's major; the Maintainer decided eight and a 30-minute timeout); the comment's figures were off (both: 1,008 mutants, not 960; the cut came 22 ms after the last kill, on different legs of three pull requests); the design still said six shards (both). Muse's minor that nothing holds the shard count itself is by design: decision 0026 keeps the count in the matrix, and every count partitions every mutant once.Verify
make verifypassed on a freshgit archiveexport withPATH=/usr/bin:/bin SIM_SEEDS=40at both heads.🤖 Generated with Claude Code