Repository navigation
docs(ci): a sweep goes blind when its SUBJECT moves repos, not when its detector breaks (#890) - #3105
Conversation
…ts detector breaks (#890) Re-measuring #890 today produced a clean, calibrated zero over the last 201 core CI runs — which reads like a fix and is not one. PR #2847 deleted MeshWeaver.Hosting.Monolith.Test and MeshWeaver.PluginCatalog.Test from core at 2026-09-01T14:06:31Z, 2h35m after the last of 11 occurrences, and every occurrence in that issue's history fired inside one of those two suites. Core CI can no longer express the defect, so every observation after that timestamp measures nothing. ReadingCiSignals already covers "the sink appeared later than the window". This is the second shape of the same error and it is harder to see, because the detector is unchanged and stays calibrated while its subject walks out of the repository. The follow-up sweep in the other repo is also not the same command: core streams a test's output into the job log as live [OUTPUT] lines (5,852 in the calibrated occurrence); the Plugins job running the identical suite carries 0, dumping "Standard Output Messages:" post-hoc per failing test and keeping the phase trace only inside a teardown-stragglers artifact. A detector keyed on the framing rather than on the signature the code emits returns a fast, vacuous null there. Both rules demand a positive control — name the run and the suite verdict line — rather than inferring liveness from the sweep's own silence. Docs only; no code, no public surface. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPe8CH4mFU9E74HafWppqL
There was a problem hiding this comment.
🟢 Approval recommended
Documentation-only changes appear consistent with existing conventions and include valid internal links, with no functional code impact.
Pull request overview
Updates the CI troubleshooting guidance by documenting a second “false null” failure mode: sweeps can go silent because the subject (the test suite producing the signal) moved to another repository, even when the detector itself is unchanged.
Changes:
- Adds a new subsection explaining why a previously observed defect signal can disappear when the relevant suites are migrated out of the repo being swept.
- Adds a follow-up subsection warning that the same suite can surface different log “sink” shapes across repos, so detectors must key on the emitted signature, not harness framing.
File summaries
| File | Description |
|---|---|
| src/MeshWeaver.Documentation/Data/Architecture/ReadingCiSignals.md | Adds two new CI-reading subsections to prevent misinterpreting “clean” sweep tails after cross-repo test-suite moves and differing log sinks. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…line.Test still emits Correcting an overstatement in the previous commit. Core still runs Roslyn Emit after the migration (Compiler.Pipeline.Test, Graph.Test); what left is the compile-heavy integration load that made the defect observable — 9s and 100s of surviving suite against the 13m38s that walked out, while the measured onset is ~2 minutes into a compile-heavy assembly. A post-removal null is uninformative because the exposure is gone, not because the repo became immune. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPe8CH4mFU9E74HafWppqL
Test Results (shard 0)234 tests 234 ✅ 2m 14s ⏱️ Results for commit 551ee20. |
Test Results (shard 1)430 tests 430 ✅ 52s ⏱️ Results for commit 551ee20. |
Test Results (shard 3)616 tests 616 ✅ 1m 2s ⏱️ Results for commit 551ee20. |
Test Results (shard 5) 3 files 3 suites 1m 10s ⏱️ Results for commit 551ee20. |
Test Results (shard 4)1 826 tests 1 826 ✅ 1m 57s ⏱️ Results for commit 551ee20. |
Test Results (shard 2)565 tests 563 ✅ 2m 48s ⏱️ Results for commit 551ee20. |
Test Results 14 files 14 suites 10m 6s ⏱️ Results for commit 551ee20. |
Conserves the work product from today's re-triage of #890 (measurement comment). Docs only — one existing page, two new subsections, no code and no public surface.
Why
Re-sweeping #890 across 958 core
dotnet-test.ymlruns today produced 11 occurrences (2026-08-30T00:05Z → 2026-09-01T11:34Z, allcanary=BELOW-ROSLYN, 1.76 % of non-cancelled runs) and then 0 over the last 201 runs. The clean tail reads like a fix; at that rate a 201-run gap is a ~4 % coincidence.It is neither. #2847 deleted
test/MeshWeaver.Hosting.Monolith.Testandtest/MeshWeaver.PluginCatalog.Testfrom core at 2026-09-01T14:06:31Z — 2 h 35 m after the last occurrence — and every occurrence in that issue's entire history fired inside one of those two suites. Core CI can no longer express the defect at all, so every core observation after that timestamp measures nothing.ReadingCiSignalsalready teaches "a sweep's sensitivity is a property of when the sink started existing". This is the second shape of the same error and is harder to spot, because the detector never changes and stays calibrated while its subject walks out of the repository.The second subsection records the follow-up trap, also measured today: the sweep in the other repo is not the same command. Core streams a test's log output into the job log as live
[OUTPUT]lines — 5 852 of them in the calibrated occurrence — while the MeshWeaver.Plugins job running the identical suite carries 0, dumpingStandard Output Messages:post-hoc per failing test and keeping the phase trace only inside ateardown-stragglers-*artifact. A detector keyed on the framing rather than on the signature the code emits returns a fast, entirely vacuous null there.Both rules end the same way: prove the sink is alive with a positive control — name a run and the suite verdict line it printed — rather than inferring liveness from the sweep's own silence.
Verification
dotnet build test/MeshWeaver.Documentation.Test/MeshWeaver.Documentation.Test.csproj -c Release -warnaserror→ 0 Warning(s), 0 Error(s)dotnet test test/MeshWeaver.Documentation.Test(full project, unfiltered) → 193 passed, 0 failed,DocumentationLinkIntegrityTestincluded — the new/Doc/Architecture/CrossRepoPairGatelink resolves.No What's New entry: docs-only, matching #2960.
🚨 Held as a draft deliberately — the auto-arm lane merges every green non-draft PR, and this is a judgement call about doc placement that deserves a human look, not an automatic merge.
🤖 Generated with Claude Code
https://claude.ai/code/session_01PPe8CH4mFU9E74HafWppqL