fix(#890): the first four `compiler=` readings — and the tier-up instruction they made testable, which was wrong by rbuergi · Pull Request #5211 · Systemorph/MeshWeaver

rbuergi · 2026-09-22T14:05:01Z

Doc/Architecture/NodeTypeCompilation (the #890 section) plus one correction in
src/MeshWeaver.Compiler/PrivateRoslynCopy.cs — an XML doc comment and the tail of a diagnostic
string. No behaviour change, no public surface change.

🚨 The correction, found by the automatic review on this PR: the residual this leg prints — and
the two places the page repeated it — told the next reader to "repeat the private emit until it tiers
up". PrivateRoslynCopy.Emit creates its own collectible AssemblyLoadContext per invocation and
unloads it in finally, so N calls are N cold copies: they measure the same tier-0 code N times
and can never tier anything. The instruction could not have been carried out. What is stated instead
is what tiering requires — ONE copy held loaded while the emit is repeated inside it, the unload
deferred to the end; a second entry point, not a loop at the existing one.
EmitCanaryPrivateCompilerLegTest asserts the verdict PREFIX, so the corrected tail keeps it green.

Leg 5 (PrivateRoslynCopy, shipped 2026-09-13) set its closing condition as "the reading on an
occurrence". Four occurrences inside four days each carry one, and they agree — on four different
platform sets and four unrelated branches. The page stopped predicting that reading and now records it,
with the denominator.

occurrence	run / job (MeshWeaver.Plugins)	platform set (core)	`canary=`	`dissect=`	`flat=`	`compiler=`
2026-09-19 08:22Z	`35430730563` / `105867058225`	`3.0.0-ci.8963` (`31be311dc`)	`BELOW-ROSLYN` ×52	`READS-HEALTHY` ×52	`EMITS` ×52	`PRIVATE-COPY-EMITS` ×52
2026-09-21 13:36Z	`35606133491` / `106355181093`	`3.0.0-ci.9088` (`88cd4b7b4`)	`BELOW-ROSLYN` ×40	`READS-HEALTHY` ×40	`SAME-FRAME@…get_ContainingTypeDefinition` ×40	`PRIVATE-COPY-EMITS` ×40
2026-09-21 22:41Z	`35661949332` / `106544668720`	`3.0.0-ci.9108` (`b1aa8a9e5`)	×58	×58	`EMITS` ×6, then `SAME-FRAME@…` ×52	×58
2026-09-22 11:37Z	`35722501930` / `106742555061`	`3.0.0-ci.9168` (`620a4893a`)	×58	×58	`EMITS` ×58	×58

What it changes. BELOW-ROSLYN's closing sentence is void on all three — a Roslyn loaded from
fresh bytes into its own collectible context emits the source the shared copy cannot, same process,
same CLR, same heap. And flat= turns out to be a phase, not a property of an occurrence: the
middle one read EMITS for its first six failures and SAME-FRAME for the following fifty-two, in
one process. That closes the 2026-09-08 vs 2026-09-10 contradiction the page already recorded (one
defect at two depths, not two defects and not an unstable probe) and retires "confined to
GetConsolidatedTypeParameters' recursion" as a general claim.

And the fault is ACQUIRED, then progressive — measured, not inferred. In the newest occurrence
OverlaySelfHealInstanceRecycleTest.OverlaidInstance_SelfRecycles_WhenTypeCompilesGreen passed at
12:52:07.654Z, and that test's assertion is the missing positive: a NodeType reaching
CompilationStatus.Ok with a usable build, whose instance then renders a marker — 33 s before the
first poisoned emit. Nothing arrived broken. The middle occurrence supplies the other half inside one
process (EMITS ×6, then SAME-FRAME ×52). That is the shape a tier-up produces, not the shape a bad
file produces.

The control the page now leads with. Attempt 2 of run 35722501930 re-ran the same job with head
353b8137b unchanged and platform mount: set 3.0.0-ci.9168 (core 620a4893a) unchanged, and reported
Passed! total: 813, failed: 0 with zero canary blocks — against attempt 1's 58 and its exit 124.
One comparison excludes the pull request's diff, the platform set and the suite's content at once. It
is also why the failure mode is expensive to read: a re-run clears it, so it looks like a flake to
whoever re-runs and like the PR's fault to whoever only saw attempt 1.

Why it points at the native code. Of the three candidates leg 5 leaves open — the image, its
mapping, the native code produced for it — only the third gets worse while a process runs. So the
tier-up repeat leg 5 already names is now the only followable step. It does not yet exclude a
corrupted static reachable only from MetadataWriter's call site, which is why the page calls it a
measurement and not a conclusion.

Four dead ends recorded so the next reader does not pay for them again:

the platform set is not the variable — three different sets, and
git diff 6b3fda2a4..620a4893a -- src/MeshWeaver.Compiler src/MeshWeaver.Compiler.Pipeline is
empty, with Microsoft.CodeAnalysis.CSharp (5.9.0) and global.json unchanged across the pair
that brackets the newest occurrence;
the rate, as a floor with what it could not see — 4 of 201 executions of the unit since
2026-09-19T00:00Z (153 success / 39 cancelled / 1 running / 8 failure, the other 4 failures carrying
no canary= and no exit 124); over the narrower 09-21 window, 3 of 102 and 3 of 3 failures, where
the 22 cancelled were read rather than assumed because a cancelled shard can carry the defect —
18 clean, 4 with no retrievable log ⇒ 3 of 96 determined, 6 undetermined. And all 201 ran on a
pull request: the unit was not selected on main once in four days, so "it passes on main" is a
zero denominator, not a green, and a single re-run cannot supply the control either (at ~2 %/run it
comes back clean either way);
core's own CI over the same days as a measured negative with its weakness stated — 24 build runs, 13
failed jobs, zero hits; thin, and core does not drive in-process NodeType compilation at this
volume, so it localises the observation rather than exonerating core;
exit 124 is arithmetic, not a wedged test — after the first poisoned emit, 9 of 11 failures burn
a full 50–120 s assertion window and 138 of 813 tests consume the 900 s cap; the last test to
start was 24 s into a 120 s window when SIGTERM arrived;
the 0 UNHANDLED, 120 first-chance straggler capture is exactly what the design produces — the
capturer was loaded (its 120 records prove the module initializer ran), its first-chance filter
is IsTeardownDisposedStraggler (Autofac/MemoryCache shapes only), and the emit NRE is caught
in EmitPipeline.EmitCompilationToDirectory, so it can never reach AppDomain.UnhandledException.

And the cost, re-derived. ProbeSharedEmitState runs once per failed emit attempt and
EmitToDiskWithRetry makes DiskEmitAttempts of them per compile — 58 probe runs across the 26 NRE
compile failures of the 2026-09-22 occurrence, each one two ~15 MB assembly loads and an unload. So a
loop at that call site is both the shape that measures nothing and the most expensive one available.
One pair of loads and N emits, armed once per process, is the shape that is neither.

Verification. dotnet build test/MeshWeaver.Documentation.Test/MeshWeaver.Documentation.Test.csproj -c Release -warnaserror
→ 0 Warning(s), 0 Error(s); MeshWeaver.Documentation.dll rebuilt (the doc is an embedded
resource, so a --no-build run here is a false pass); then
./MeshWeaver.Documentation.Test -class '*DocumentationLinkIntegrityTest*' → Total: 1, Failed: 0.
The new text adds no internal Doc/… links.

Pairs-with: none — documentation only, no public surface removed.
Mirror-sync: none — no localization catalog key added or re-worded.

🤖 Generated with Claude Code

Leg 5's closing condition was "the reading on an occurrence". Three occurrences inside thirty hours carried one and they agree, so the page no longer predicts that reading — it records it. What the readings settle: - `BELOW-ROSLYN` is void on all three. A second Roslyn loaded from fresh bytes into its own collectible context emits the source the shared copy cannot, in the same process on the same CLR and heap. The dotnet/runtime venue is wrong. - `flat=` is a PHASE, not a property of an occurrence: one occurrence read `flat=EMITS` for its first six failures and `flat=SAME-FRAME` for the next fifty-two, in ONE process. That closes the 2026-09-08 vs 2026-09-10 contradiction this page recorded — one defect read at two depths — and retires "confined to GetConsolidatedTypeParameters' recursion" as a general claim: once it deepens, a top-level member-less class cannot emit either. - Of the three candidates leg 5 leaves (image, mapping, native code), only the third gets worse while a process runs. The tier-up repeat leg 5 already names is now the only followable step, and it is a measurement, not a conclusion. Also recorded, because each was reached for and is a dead end: the platform set is not the variable (three different sets; no diff in src/MeshWeaver.Compiler*, Roslyn pin and global.json unchanged across the bracketing pair); the rate with its denominator, and why `main` supplies no control (the unit was not selected on main once in the window — a zero denominator, not a green); `exit 124` is arithmetic, not a wedged test (138 of 813 tests consume the 900 s cap on assertion windows, and the last test to START was 24 s into a 120 s one); and the `0 UNHANDLED, 120 first-chance` straggler capture is what the design produces — the capturer WAS loaded, its filter admits only Autofac/MemoryCache teardown shapes, and the emit NRE is caught in EmitCompilationToDirectory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…READ, not assumed A `cancelled` shard can CARRY the defect; filtering a population on its conclusion is the exact mistake this thread's 2026-09-12 correction records. So the 22 were read: 18 logs clean of both `canary=BELOW-ROSLYN` and `PROCESS CANNOT EMIT`, 4 with no retrievable log. 3 of 96 determined, 6 undetermined — a floor, stated with what it could not see. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…t model, measured `ProbeSharedEmitState` runs once per failed emit ATTEMPT and `EmitToDiskWithRetry` makes `DiskEmitAttempts` of them per compile: 58 probe runs across 27 reported compile failures in the 2026-09-22 occurrence. Tier-1 promotion needs tens of invocations plus a background compile, so a loop inside the leg multiplies a ~1 s probe by that factor 58 times over and kills the suite sooner with less printed. The repeat belongs once per PROCESS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Copilot review overview

🟡 Changes recommended

The documented measurement, scope, timing, interpretation, and wording corrections remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

Fresh load context prevents valid private compiler tiering measurement · New

What changed in this PR

Documentation-only update for issue #890, recording three compiler=PRIVATE-COPY-EMITS observations and revising the investigation notes.

Changes:

Adds occurrence data, denominators, and phase behavior.
Updates conclusions on platform inputs, timing, and failure analysis.
Verifies documentation build and link integrity.

File	Summary
`src/MeshWeaver.Documentation/Data/Architecture/NodeTypeCompilation.md`	Adds compiler-leg evidence and revises the issue analysis.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

rbuergi · 2026-09-22T14:12:14Z

+unchanged and is now the only followable step: a `PRIVATE-COPY-EMITS` does not separate fresh mapping
+from fresh native code, and the follow-up is to repeat the private emit until it tiers up.


Correct, and it invalidates the instruction rather than just my wording. Verified in
src/MeshWeaver.Compiler/PrivateRoslynCopy.cs: Emit creates new PrivateRoslynLoadContext() (:85)
and calls context.Unload() in its finally (:113-115). So N calls are N cold copies, each starting
at tier 0 with no profile — the one thing a tier-up measurement must not do. "Repeat the private emit
until it tiers up" therefore cannot be carried out by repeating the helper, and that sentence predates
this PR (the leg-5 residual at what is now :1745); my new paragraph inherited it.

I am rewriting both places to state the experiment in terms of what it actually requires — ONE private
copy held loaded while the emit is repeated inside it, on the same method bodies, with the unload
deferred to the end — and to re-derive the cost from that: one context load plus N emits, not N context
loads. That also sharpens the "not inside the probe" point rather than weakening it: Emit is
load-unload per call by construction, so a loop at the call site is the most expensive possible shape
AND measures nothing.

Thank you — this is the second time on this page that a control turned out not to be a control for what
it shares; here what is shared is the coldness.

…e deliberate test injections Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…rrect the one instruction it prints `PrivateRoslynCopy.Emit` creates its own collectible AssemblyLoadContext per invocation and unloads it in `finally`, so N calls are N COLD copies: they measure the same tier-0 code N times and can never tier anything. The residual this leg prints — and the two places the doc page repeats it — told the next reader to "repeat the private emit until it tiers up", which by construction is not an experiment that can answer the question it is offered for. Corrected in the XML doc, in the message an operator actually reads, and in the page. The remedy stated instead is what tiering requires: ONE private copy held loaded while the emit is repeated inside it, the unload deferred to the end — a second entry point, not a loop at the existing one. The cost note is re-derived from that (one pair of ~15 MB loads and N emits, not N pairs), which sharpens rather than softens the "not inside the probe" point: the probe already runs 58 times over the 26 NRE compile failures of a single occurrence. No behaviour change and no public surface change — an XML doc comment, a diagnostic string's tail, and prose. `EmitCanaryPrivateCompilerLegTest` asserts the verdict PREFIX, so the corrected tail keeps it green. Found by the automatic review on PR #5211. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…either inferred The missing positive was "did this process ever emit?". It did, and a test assertion says so rather than a log line: in the 2026-09-22 occurrence OverlaySelfHealInstanceRecycleTest.OverlaidInstance_SelfRecycles_WhenTypeCompilesGreen PASSED at 12:52:07.654Z, and that test asserts a NodeType reaching CompilationStatus.Ok WITH a usable build whose instance then renders a marker. Thirty-three seconds before the first poisoned emit. So nothing arrived broken. The 2026-09-21 22:41Z occurrence supplies the other half from inside one process: flat=EMITS for six failures, then flat=SAME-FRAME for fifty-two. Acquired, then progressive — the shape a tier-up produces, not the shape a bad file produces, which is what makes the tier-up residual worth a measurement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… second instrument 138 TEST_START events and 136 per-test log files agree, out of the 813 the suite totals when healthy — so the number is a recorded count with a cross-check, not a claim about how many tests the runner dispatched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ed negative in core's CI Widening the sweep to 2026-09-19 found one more occurrence — 08:22Z, run 35430730563, job 105867058225, set 3.0.0-ci.8963 (core 31be311), 52 blocks, `flat=EMITS`, `compiler=PRIVATE-COPY-EMITS`. Four occurrences now, on FOUR different platform sets and four unrelated branches, and the `compiler=` reading is four for four. The denominator is stated over that window: 201 executions of the unit, 153 success / 39 cancelled / 1 running / 8 failure, of which 4 are this defect and 4 carry no `canary=` and no `exit 124` at all. And ALL 201 are `pull_request` runs — the unit was not selected on `main` once in four days, so "it passes on main" is a zero denominator over a longer window than the one that first showed it. Core's own CI is a measured negative over the same days (24 build runs, 13 failed jobs, zero hits), recorded WITH its weakness: 13 is thin, and core's suites do not drive in-process NodeType compilation at the Monolith unit's volume, so that zero is close to what a sampling miss would also produce. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…un passes 813/813 Run 35722501930 attempt 1 killed the unit at exit 124 with 58 canary blocks. Attempt 2 of the SAME run — head 353b8137b unchanged, platform mount set 3.0.0-ci.9168 (core 620a489) unchanged, same runner pool — reported "Passed! total: 813, failed: 0" with ZERO canary=BELOW-ROSLYN, in 14 min against the 900 s cap. One comparison excludes the pull request's diff AND the platform set AND the content of the suite, because none of them changed between the two attempts. It is also why this failure mode is expensive to read: a re-run clears it, so it looks like a flake to re-run and like the PR's fault to anyone who only saw attempt 1. It is neither. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rbuergi · 2026-09-22T14:29:24Z

Held as a DRAFT — do not un-draft or arm this. Core main is frozen from our side while
memex.meshweaver.cloud is roll-thrashing (updatePolicy: Continuous, updatePattern: 3.0.0-ci*,
converged: false), so every additional merge into main mints another ci build and starts another
roll. Settling that is a live-deployment decision for the maintainer, not for this PR.

Auto-merge is disabled. Draft is the structural half of the hold: auto-arm.yml triggers on
opened, reopened, ready_for_review, synchronize, so with the PR in draft a future push cannot
silently re-arm it — which is how it came back armed once already (each of my pushes ran
Arm auto-merge, not a human).

The content is finished and verified; nothing here is waiting on work. To release: gh pr ready 5211,
then arm as usual.

github-actions · 2026-09-22T14:41:23Z