Repository navigation
docs(ci): two ways to attribute a red that reachability cannot settle - #2873
Conversation
Conserving tonight's triage findings as a committed page rather than leaving them in issue comments (AGENTS.md: work products get committed). ReadingCiSignals.md already says to attribute by REACHABILITY before adjacency. That only ever answers NO. Two additions for the cases it cannot settle: 1. When the failing test's project DOES reference everything the diff touched — and a plausible mechanism exists — ask whether the changed path EMITTED anything in that test. Measured on #2868: 0 tags in the failing test's own output against 30 elsewhere in the shard. Zero-against-thirty is positive evidence the branch was not entered, because the thirty prove the grep works. A change cannot hang a test through a branch the test never takes. Corroboration in the same breath: two UNRELATED PRs red in the same window on different tests and shards is an ambient population, not either diff. 2. A host-cap kill (exit=124 / HOST_CRASHED) DESTROYS the transcript of the very test that hung; an xUnit methodTimeout kill writes it in full. So the ambient hang family erases its own defining artefact — an investigation that reads a crashed shard's artifacts reads everything except the thing that hung. Hunt the method-timeout instances, which are routinely discarded as flakes and carry the window intact. Includes the 2026-08-31 window where Dropping StreamEndedEvent and ADVANCE_WITHOUT_HANDOFF land 12 ms apart on the same path, then 25 s of silence — /debug's signature for a dropped reactive emission rather than a lock. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds two documentation heuristics to improve CI-red attribution in cases where dependency reachability can’t eliminate a suspect change, by focusing on whether the changed path actually emitted logs in the failing test and on which termination mode preserves useful artifacts.
Changes:
- Document a log-emission check (“did the changed path emit anything in the failing test?”) as positive evidence for/against causal involvement when reachability can’t decide.
- Document the artifact difference between host-cap kills vs xUnit
methodTimeout(transcript destroyed vs preserved) and how that should steer investigation toward method-timeout instances.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Test Results (shard 0)792 tests 788 ✅ 4m 29s ⏱️ Results for commit a7157d9. |
Test Results (shard 4)1 090 tests 1 090 ✅ 5m 30s ⏱️ Results for commit a7157d9. |
Test Results (shard 1)951 tests 951 ✅ 5m 26s ⏱️ Results for commit a7157d9. |
Test Results (shard 5) 8 files 8 suites 5m 53s ⏱️ Results for commit a7157d9. |
Test Results (shard 2)3 030 tests 2 838 ✅ 6m 17s ⏱️ Results for commit a7157d9. |
Test Results (shard 3)1 186 tests 1 186 ✅ 8m 4s ⏱️ Results for commit a7157d9. |
Test Results 44 files 44 suites 35m 43s ⏱️ Results for commit a7157d9. |
Docs only. Conserving tonight's triage findings as a committed page rather than leaving them in issue comments.
ReadingCiSignals.mdalready tells you to attribute by reachability before adjacency. That only ever answers no. Two additions for the cases it cannot settle:1. When reachability cannot exonerate you, ask whether your path EMITTED anything.
#2868 changed four assemblies that
MeshWeaver.Graph.Testdepends on, with a real mechanism to worry about — the change moved a callback onto a different thread. The graph said nothing. This did:A change cannot hang a test through a branch the test never takes. Zero-against-thirty is positive evidence, not an absence of it — the thirty prove the grep works. Re-running confirmed it afterwards, which is the point: a re-run whose result you can predict is a confirmation; one you cannot is a coin toss.
2. A host-cap kill destroys the hung test's transcript; a
methodTimeoutkill keeps it.exit=124,HOST_CRASHED)methodTimeoutThe ambient hang family therefore erases its own defining artefact — an investigation that waits for a CI occurrence and reads the artifacts reads everything except the thing that hung. A crashed shard held 21
Dropping StreamEndedEventand 3ADVANCE_WITHOUT_HANDOFFwhile the hung test's name appeared nowhere. Hunt the method-timeout instances, which carry the window intact and are routinely discarded as ordinary flakes. One found the same night put the pair 12 ms apart on the same path, then 25 s of silence —/debug's signature for a dropped reactive emission rather than a lock.Verified:
MeshWeaver.DocumentationandMeshWeaver.Documentation.Testbuild-c Release -warnaserrorat0 Warning(s) 0 Error(s); the fullMeshWeaver.Documentation.Testsuite is 132/132. Link form matches the file's existing/Doc/Architecture/…convention and the target exists.🤖 Generated with Claude Code