Repository navigation
ci: make stalled PRs and hung tests visible (shepherd:reapprove, pytest --timeout) - #4865
Merged
Merged
Conversation
…st --timeout) Two quiet failures from 2026-10-10: - #4858 sat green-but-unmerged: a push dropped shepherd:merge (an approval is bound to its head, by design) and, with no request label left, every status label was cleared too, so nothing marked the PR. The bot now sets shepherd:reapprove when it drops the label. It outlives the request labels (is:open label:shepherd:reapprove finds every PR that lost its approval) and clears when shepherd:merge or shepherd:hold comes back, or by hand. - Every macOS matrix leg on main ran into the 45-minute job limit on a hung test (#4862) and read as "cancelled", naming nothing. full-suite-gate.yml and smoke-test.yml now run pytest with --timeout=600 (pytest-timeout 2.4.0; slowest real test ~60 s), so a hang fails as a named test. main_watch.py reports each cancelled leg's run time ("after 45 min") and no longer calls a time-limit cancel "not a verdict on the code". docs/ci.md: the new label, its lifecycle, the timeout; drops a duplicated pair of paragraphs. Part of #4840 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AUWeQX7v4QBdErFbghEEEX
Owner
Author
|
Note for merging: this changes |
Contributor
squid-protocol
marked this pull request as ready for review
October 10, 2026 23:59
This was referenced Oct 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #4840
Two quiet failures from 2026-10-10. Each had a fix in place on the spot, but nothing would have made either one visible the next time. This makes both visible.
1. A PR that lost its approval now carries a label:
shepherd:reapproveWhat happened: #4858 was green but didn't merge. A push dropped
shepherd:merge(by design: an approval is bound to the head it was given on). With no request label left, the bot also cleared every status label, so the only trace was a comment. Nobody re-added the label until a person noticed.Now: when the bot drops
shepherd:mergebecause the head moved, it also setsshepherd:reapprove.is:open label:shepherd:reapprovelists every PR waiting for a maintainer to re-approve.shepherd:mergecomes back (orshepherd:holdis set), or when someone removes it by hand. A manual removal sticks.2. A hung test now fails by name instead of hitting the job time limit
What happened: every macOS matrix leg on main ran into the 45-minute
timeout-minuteson a hung test (#4862: noflockon macOS). GitHub reports that as "cancelled", which names nothing, and it read as noise all day.Now:
full-suite-gate.ymlandsmoke-test.ymlrun pytest with--timeout=600(pytest-timeout==2.4.0). The slowest real test is ~60 s (.test_durations), so this is ~10× headroom and well under the 30 and 45-minute job limits. Checked locally with pytest 9.1.1 + xdist 3.8.0: a hung test fails asFAILED …::test_hangs - Failed: Timeout, and the rest of the run carries on.main_watch.pygives each cancelled leg its run time ("… after 45 min"). It also no longer calls a cancel at the time limit "not a verdict on the code".Also
docs/ci.md: the new label in the status table and its lifecycle, the timeout, and a pair of paragraphs that appeared twice is removed.Checks
test_shepherd_gh.py+test_main_watch.py: 69 pass, including 4 new tests.pr_gates --fastPASS.--ratchetsall PASS exceptports(the local CardDemo corpus is not at its pinned ref; same on main).main_watch.py reconcile --dry-runagainst the live repo: no actions. That's correct, since the latest verdict is already on main is red: Full Suite Gate (All OS x Python) #4840.After merge
shepherd-reconcile.ymlonce withensure_labels: true, so the label exists with its colour and description. Until then, the first add creates it with GitHub's default colour.Not done here (worth a look)
🤖 Generated with Claude Code
https://claude.ai/code/session_01AUWeQX7v4QBdErFbghEEEX