Repository navigation
fix(#890): Tiered PGO off in every compiling host — the .NET 10 JIT drops the isinst in Roslyn's inlined ContainingType - #6254
Conversation
… JIT drops the isinst in Roslyn's inlined ContainingType Memex.Portal.Shared.Test part 1/2 poisoned in 4 of 12 main runs since the class split moved InstallNeverInheritsRepoCompileVerdictTest behind the suite's stand-in emits: every Emit in the host then throws NRE in NamedTypeSymbol.get_ContainingTypeDefinition (#890), failing InstallNeverInherits…, both ModulesUpdateIndependentlyOfThePlatformTest cases, or all of ModuleLinkVersionTest — on unrelated PRs. Reproduced in CI's own shape (run 37647154163, the poisoned PR's build-output-3, part 1 launched as the shard launches it, 14 runners per arm): defaults 4 poisoned / 10 clean, DOTNET_TieredPGO=0 0 / 14. The Tier-1 listing of NamedTypeSymbol.AsNestedTypeDefinitionImpl explains it 14 of 14: the inlined SourceMemberContainerTypeSymbol.ContainingType (`_containingSymbol as NamedTypeSymbol`) keeps its isinst in all 10 clean hosts and has none in all 4 poisoned ones, so a top-level type's namespace container reads as its containing type. TieredPGO=false in the root and test Directory.Build.props reaches every host's runtimeconfig (System.Runtime.TieredPGO). TieredPgoIsOffInEveryCompilingHostTest pins it; negative control: a build with -p:TieredPGO=true fails both assertions. Workaround for a runtime defect, documented as one in NodeTypeCompilation.md ("The defect, in one listing"). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The test guard does not verify the root configuration used by production compiling hosts such as mw-plugin-test.
1 open finding
What changed in this PR
Disables Tiered PGO to avoid a .NET 10 JIT defect that corrupts Roslyn compilation hosts.
Changes:
- Disables Tiered PGO for source and test hosts.
- Adds runtime configuration assertions.
- Documents the diagnosis and workaround.
| File | Description |
|---|---|
Directory.Build.props |
Disables Tiered PGO for repository executables. |
test/Directory.Build.props |
Disables Tiered PGO for test hosts. |
test/Memex.Portal.Shared.Test/TieredPgoIsOffInEveryCompilingHostTest.cs |
Verifies the test-host runtime configuration. |
src/MeshWeaver.Documentation/Data/Architecture/NodeTypeCompilation.md |
Records evidence, mechanism, and remediation. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| /// files. Negative control: building this project with <c>-p:TieredPGO=true</c> makes both assertions | ||
| /// fail.</para> | ||
| /// </summary> | ||
| public class TieredPgoIsOffInEveryCompilingHostTest |
There was a problem hiding this comment.
Agreed, and fixed in 4182810. BakeHostRunsWithTieredPgoOffTest in MeshWeaver.PluginTester.Test parses the runtimeconfigs the SDK copies beside it from the referenced root-inheriting executables, mw-plugin-test.runtimeconfig.json and mw-combo-verify.runtimeconfig.json, and asserts "System.Runtime.TieredPGO": false in each.
Negative control, run locally: I deleted <TieredPGO>false</TieredPGO> from the root Directory.Build.props only and left the test tree's copy in place. Both cases went red (Total: 2, Failed: 2). With the property restored, both pass. The doc section and the root-props comment now name both guards.
…ify runtimeconfigs carry TieredPGO off (review) Negative control: deleting the root property alone turns both cases red while the test tree's copy stays. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Test Results (shard 3) 4 files ± 0 4 suites ±0 7m 13s ⏱️ +18s Results for commit 4182810. ± Comparison against base commit e1820b0. This pull request removes 72 and adds 118 tests. Note that renamed tests count towards both. |
Test Results (shard 4) 4 files ± 0 4 suites ±0 16m 42s ⏱️ -8s Results for commit 4182810. ± Comparison against base commit e1820b0. This pull request removes 148 and adds 88 tests. Note that renamed tests count towards both. |
Test Results 22 files ± 0 22 suites ±0 48m 43s ⏱️ -4s Results for commit 4182810. ± Comparison against base commit e1820b0. This pull request removes 40 and adds 28 tests. Note that renamed tests count towards both. |

What was red
Three
Memex.Portal.Shared.Testtests failed on unrelated PRs (#6231 run 37637952561, #6248 run 37639719471) and onmain(041dde2, e383542, cbb3053):InstallNeverInheritsRepoCompileVerdictTest.InstallingAPackageWhoseNodeFileCarriesAVerdict_LandsTheAuthoredDefinitionOnlyModulesUpdateIndependentlyOfThePlatformTest.ABundleThatCarriesAPlatformAssembly_IsRefusedByName_AndTheSameBundleWithoutItLandsModulesUpdateIndependentlyOfThePlatformTest.ThePlatformStaysFixed_WhileAModuleGoesNToNPlus2Live_AndAFloorAboveItIsDeclinedWhileASiblingUpdatesNone of them was an assertion about module adoption. Every one died inside Roslyn:
PROCESS CANNOT EMIT (#890), NRE atNamedTypeSymbol.Microsoft.Cci.ITypeDefinitionMember.get_ContainingTypeDefinition,canary=BELOW-ROSLYN … compiler=PRIVATE-COPY-EMITS image=INTACT. Once the host is in that state, every laterEmitfails, including the tests' own stand-in compiles.Why it started now: the part-1/part-2 class split is every other class of the sorted list. The classes #6128 added shifted that split, which moved
InstallNeverInherits…(and #6128's ownModulesUpdateIndependently…) into part 1, behind the stand-in emits ofModulePlatformMemberLinkTest/ModuleLinkVersionTest. Since then part 1 poisoned in 4 of 12mainruns. Before #6128 those tests ran in part 2 and passed. #6230 and #6196 are not involved: the failures come before any adoption assertion runs.The defect, measured in CI's shape
Experiment branch
exp/890-tieredpgo-split-arm, run 37647154163. It reuses the poisoned PR's ownbuild-output-3artifact and launches part 1 exactly the way the shard does, on 14 runners per arm:DOTNET_JitDisasmcapture)DOTNET_TieredPGO=0The JIT listing matches the outcome on all 14 default-arm hosts. The Tier-1
NamedTypeSymbol:AsNestedTypeDefinitionImplinlinesSourceMemberContainerTypeSymbol.ContainingType, which is_containingSymbol as NamedTypeSymbol. In all 10 clean hosts the inlined cast keeps itsCORINFO_HELP_ISINSTANCEOFCLASS. In all 4 poisoned hosts it is gone (mov rax, [rbx+0x38]; jmp → test rax,rax). So a top-level type's namespace container reads as its containing type, the guard admits it, and the writer dereferences null. This happened on hosts with and without AVX-512 (VEX and EVEX). This is a .NET 10 (10.0.12 linux-x64) JIT defect, not a bug in this repo.The change
<TieredPGO>false</TieredPGO>in the rootDirectory.Build.props(every executable this repo builds, includingmw-plugin-test) and intest/Directory.Build.props(which does not import the root). It reaches each host asSystem.Runtime.TieredPGO: falsein its runtimeconfig. That file is what adotnet <name>.dlllaunch reads.TieredPgoIsOffInEveryCompilingHostTestpins that the switch reaches the host, both viaAppContextand in the runtimeconfig on disk. Negative control: a build with-p:TieredPGO=truefails both assertions (run locally).NodeTypeCompilation.mdgets a new section, "The defect, in one listing — the JIT drops theisinst, and Tiered PGO off removes it".This is a workaround for a runtime defect, and it is labelled as one. Remove it only with the split arm re-run on a runtime that fixes it.
Not in this PR: the portal hosts (
Memex.Portal.*, MeshWeaver.Plugins) need the same property. Until they have it, production NodeType compiles are exposed to the same per-process miscompile. Nobody has yet checked whether production replicas reach this state. No addresses need a recycle: this is a process-start setting that takes effect on the next host start.Pairs-with: none — no public surface removed
Mirror-sync: none — no i18n catalog change
🤖 Generated with Claude Code