ci(#5212): DOTNET_TieredPGO=0 stopgap in CI + satellite lanes; chart renders it from a record by rbuergi · Pull Request #6268 · Systemorph/MeshWeaver

rbuergi · 2026-10-07T17:31:49Z

What

Temporary stopgap, as the maintainer decided: DOTNET_TieredPGO=0 for #5212. The .NET 10 Tier-1 JIT, running with Dynamic PGO, drops a type check in an inlined guarded devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows / getConsolidatedTypeParameters path. After that, every in-process Emit NREs. Evidence: #5212 (comment). The root cause stays tracked in #5212 and in an upstream dotnet/runtime issue (filed separately). Every place this PR sets the switch carries a comment that names #5212 and says "remove when the runtime fix ships". Policy: tiered-pgo-off-5212 (registered in PolicyNotProse).

Relation to #6254 (already merged): #6254 set <TieredPGO>false</TieredPGO> in both Directory.Build.props, so the switch is in the runtimeconfig. That reaches only hosts built from this tree at or after that commit. This PR adds the environment half, which the runtime reads from the process env and which wins over the runtimeconfig. It covers:

satellite lanes, whatever tester or portal pin they hold;
satellite test hosts, which are not built under core's test props;
the portals, whose hosts are built in MeshWeaver.Plugins (that repo has no TieredPGO property today).

Changes

dotnet-test.yml: job env: DOTNET_TieredPGO: "0" on test (the shards), doc-gate and platform-compat. These are the jobs that run in-process emits (test hosts, mw-plugin-test).
Reusable satellite lanes:
- Job env on node-repo-compile-check (compile-check; its dotnet build compiler server is long-lived), node-repo-gate (gate), node-repo-module-pack (build-workspace, pack, tests) and node-repo-publish-bake (publish-bake).
- -e DOTNET_TieredPGO=0 on each emitting docker run: gate run_tester, module-pack build-project, and publish-bake compile and seed run. The runner env does not cross into a container.
Chart (deploy/helm/templates/memex-portal/config.yaml): memex-portal-config renders DOTNET_TieredPGO, but only when config.memex_portal.DOTNET_TieredPGO is present and non-empty. The ConfigMap reaches the container through envFrom, so the runtime sees it as process env. Before this PR the chart did not pass the key through, and a record setting it reached no container.
Docs: a PolicyNotProse register row, and a NodeTypeCompilation paragraph, "The environment half", under the CI: one Roslyn Emit poisons the whole process — every later compile NREs in Cci.MetadataWriter.GetConsolidatedTypeParameters, shard ends exit=124 #890/MeshWeaver#890 recurred 3x in 30h with the leg-5 reading it was closed waiting for — reopen-or-new-path call needed #5212 defect section.

Paired Memex PR

The Systemorph/Memex PR that adds DOTNET_TieredPGO: "0" to the portal records depends on this PR. Its required config-key-coverage reads the chart from this repo's default branch. Measured locally: against main's chart that check reports 4 new rendered-nowhere (config.memex_portal.DOTNET_TieredPGO on build, memex-cloud, memex and pearl). Against this branch it is 0. Merge this one first.

Checks run locally

check-workflow-timeouts.py: 113 jobs, 0 violations.
check-workflow-yaml-keys.py: 45 workflows + 6 composite actions, 0 violations.
actionlint on all 5 touched workflows. dotnet-test.yml and node-repo-compile-check.yml are clean. gate, module-pack and publish-bake report 20 shellcheck info/style findings, the same 20 as on origin/main (I ran it on both), so this PR adds none. The stopgap comment sits above each # shellcheck disable=SC2086 directive so the directive still binds to its docker run.
deploy/aks/scripts/check-chart-invariants.sh: all 17 combinations self-consistent, 7 refusal controls hold.
deploy/aks/scripts/check-values-are-read.sh: 128 keys read.
helm template:
- with --set-string config.memex_portal.DOTNET_TieredPGO=0, the ConfigMap carries DOTNET_TieredPGO: "0";
- without the key, the line does not render;
- with the Memex memex overlay, it renders DOTNET_TieredPGO: "0".
dotnet build src/MeshWeaver.Documentation -c Release -warnaserror: 0 warnings, 0 errors.

Recycle / roll

Nothing in this PR needs a recycle: CI env takes effect on the next run. The chart key does nothing until a record sets it, and it then takes effect on the next pod start (see the Memex PR for the Reconcile list).

Pairs-with: none — no public surface removed
Mirror-sync: none — no i18n catalog change

🤖 Generated with Claude Code

…t renders it from a record TEMPORARY STOPGAP (maintainer decision, policy tiered-pgo-off-5212) for the .NET 10 JIT miscompile in #5212: Tier-1 + Dynamic PGO drops a type check in an inlined guarded devirtualisation on Roslyn's MetadataWriter path, after which every in-process Emit NREs. Remove when the runtime fix ships. - dotnet-test.yml: job env on test, doc-gate, platform-compat. - node-repo-{compile-check,gate,module-pack,publish-bake}.yml: job env on every job that compiles, bakes or runs satellite tests, plus -e DOTNET_TieredPGO=0 on each emitting `docker run` (the runner env does not cross into the container). Covers satellites whatever tester pin they hold, which #890's runtimeconfig half cannot. - chart: memex-portal-config renders DOTNET_TieredPGO when a record's extraPortalConfig sets it (envFrom -> process env), so the Memex records can carry it. - PolicyNotProse register row + NodeTypeCompilation "environment half" paragraph. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Copilot

🟡 Changes recommended

The core build job still runs Roslyn emits without the Tiered PGO workaround.

1 open finding

Disable Dynamic PGO for the build job's Roslyn compiler · New

What changed in this PR

Adds a temporary Tiered PGO mitigation for Roslyn emit failures across CI, satellite workflows, and portal deployments.

Changes:

Sets DOTNET_TieredPGO=0 in relevant CI jobs and emitting containers.
Passes the setting from deployment records into portal containers.
Documents the mitigation and its removal policy.

File	Description
`.github/workflows/dotnet-test.yml`	Applies the workaround to test and runtime-compilation jobs.
`.github/workflows/node-repo-compile-check.yml`	Protects satellite NodeType compilation.
`.github/workflows/node-repo-gate.yml`	Protects gate processes and containers.
`.github/workflows/node-repo-module-pack.yml`	Protects module builds, packing, and tests.
`.github/workflows/node-repo-publish-bake.yml`	Protects bake compilation and seed containers.
`deploy/helm/templates/memex-portal/config.yaml`	Renders the optional portal environment variable.
`src/MeshWeaver.Documentation/Data/Architecture/NodeTypeCompilation.md`	Documents environment-level coverage.
`src/MeshWeaver.Documentation/Data/Architecture/PolicyNotProse.md`	Registers the temporary policy.

🧠 Review effort: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

rbuergi · 2026-10-07T18:47:53Z

+    env:
+      DOTNET_TieredPGO: "0"


Fixed in e7fcf1e: the stopgap's env DOTNET_TieredPGO=0 is now set on the build job as well, since the solution build's Roslyn compiler server emits in-process and is not configured by a runtimeconfig.

github-actions · 2026-10-07T17:45:30Z

Test Results (shard 2)

5 files ±0 5 suites ±0 3m 48s ⏱️ -44s
1 230 tests ±0 1 230 ✅ ±0 0 💤 ±0 0 ❌ ±0
1 231 runs ±0 1 231 ✅ ±0 0 💤 ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

♻️ This comment has been updated with latest results.

github-actions · 2026-10-07T17:46:01Z

Test Results (shard 1)

1 847 tests ±0 1 845 ✅ ±0 4m 52s ⏱️ -2s
4 suites ±0 2 💤 ±0
4 files ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

♻️ This comment has been updated with latest results.

github-actions · 2026-10-07T17:46:27Z

Test Results (shard 0)

3 files ±0 3 suites ±0 7m 29s ⏱️ ±0s
579 tests ±0 388 ✅ ±0 191 💤 ±0 0 ❌ ±0
583 runs ±0 392 ✅ ±0 191 💤 ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

♻️ This comment has been updated with latest results.

github-actions · 2026-10-07T17:48:20Z

Test Results (shard 3)

2 544 tests ±0 2 544 ✅ ±0 8m 21s ⏱️ -4s
4 suites ±0 0 💤 ±0
4 files ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

♻️ This comment has been updated with latest results.

github-actions · 2026-10-07T17:49:56Z

Test Results (shard 5)

2 files ±0 2 suites ±0 8m 39s ⏱️ +6s
1 126 tests ±0 1 126 ✅ ±0 0 💤 ±0 0 ❌ ±0
1 127 runs ±0 1 127 ✅ ±0 0 💤 ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

♻️ This comment has been updated with latest results.

github-actions · 2026-10-07T17:57:43Z

Test Results (shard 4)

4 files ±0 4 suites ±0 15m 29s ⏱️ -1s
3 737 tests ±0 3 737 ✅ ±0 0 💤 ±0 0 ❌ ±0
3 744 runs ±0 3 744 ✅ ±0 0 💤 ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

This pull request removes 40 and adds 16 tests. Note that renamed tests count towards both.


   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---, expected: True)
   --- End of inner exception stack trace ---, isDenial: True)
 ---> (Inner Exception #1) MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.<---
 ---> (Inner Exception #1) MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432<---
 ---> (Inner Exception #1) System.ArgumentException: Value does not fall within the expected range.<---
 ---> (Inner Exception #1) System.InvalidOperationException: boom<---
 ---> (Inner Exception #1) System.InvalidOperationException: source B is misconfigured<---
 ---> (Inner Exception #1) System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known<---
…

Memex.Portal.Shared.Test.InstanceIdRulesMatchTheRegistryTest ‑ TheSetupHostAgreesWithTheRegistry(candidate: "d658ee52-093a-4294-b9c6-3d4d24ea55da")
Memex.Portal.Shared.Test.SessionDenialIsAnAnswerTest ‑ OnlyAVerdictReadsAsADenial(shape: "the same verdict nested, as a late denial dispatch"···, failure: System.InvalidOperationException: write failed
 ---> System.UnauthorizedAccessException: Access denied
   --- End of inner exception stack trace ---, isDenial: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall as an aggregate's FIRST member", fault: System.AggregateException: One or more errors occurred. (Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.)
 ---> MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.
   --- End of inner exception stack trace ---, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall as an aggregate's SECOND member", fault: System.AggregateException: One or more errors occurred. (The operation has timed out.) (Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.)
 ---> System.TimeoutException: The operation has timed out.
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.<---
, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall nested in an aggregate inside a wrapper", fault: System.InvalidOperationException: outer
 ---> System.AggregateException: One or more errors occurred. (Value does not fall within the expected range.) (Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.)
 ---> System.ArgumentException: Value does not fall within the expected range.
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.<---

   --- End of inner exception stack trace ---, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall wrapped as InnerException", fault: System.InvalidOperationException: outer
 ---> MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.
   --- End of inner exception stack trace ---, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "an aggregate of unrelated faults", fault: System.AggregateException: One or more errors occurred. (The operation has timed out.) (Value does not fall within the expected range.)
 ---> System.TimeoutException: The operation has timed out.
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) System.ArgumentException: Value does not fall within the expected range.<---
, expected: False)
MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest ‑ Classifies(shape: "a MIXED aggregate — one transient branch, one genu"···, exception: System.AggregateException: One or more errors occurred. (Failed to connect to 10.42.18.4:5432) (source B is misconfigured)
 ---> MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) System.InvalidOperationException: source B is misconfigured<---
, expected: False)
MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest ‑ Classifies(shape: "a nested aggregate with one genuine leaf", exception: System.AggregateException: One or more errors occurred. (One or more errors occurred. (Failed to connect to 10.42.18.4:5432) (boom)) (Failed to connect to 10.42.18.4:5432)
 ---> System.AggregateException: One or more errors occurred. (Failed to connect to 10.42.18.4:5432) (boom)
 ---> MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) System.InvalidOperationException: boom<---

   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432<---
, expected: False)
MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest ‑ Classifies(shape: "a nested aggregate, all leaves transient", exception: System.AggregateException: One or more errors occurred. (One or more errors occurred. (Failed to connect to 10.42.18.4:5432)) (Failed to connect to 10.42.18.4:5432)
 ---> System.AggregateException: One or more errors occurred. (Failed to connect to 10.42.18.4:5432)
 ---> MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432<---
, expected: True)
…

♻️ This comment has been updated with latest results.

github-actions · 2026-10-07T17:58:11Z

Test Results

22 files ±0 22 suites ±0 48m 40s ⏱️ -46s
11 063 tests ±0 10 870 ✅ ±0 193 💤 ±0 0 ❌ ±0
11 076 runs ±0 10 883 ✅ ±0 193 💤 ±0 0 ❌ ±0

Results for commit 9b1102c. ± Comparison against base commit 0e6abb9.

This pull request removes 40 and adds 16 tests. Note that renamed tests count towards both.


   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---, expected: True)
   --- End of inner exception stack trace ---, isDenial: True)
 ---> (Inner Exception #1) MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.<---
 ---> (Inner Exception #1) MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432<---
 ---> (Inner Exception #1) System.ArgumentException: Value does not fall within the expected range.<---
 ---> (Inner Exception #1) System.InvalidOperationException: boom<---
 ---> (Inner Exception #1) System.InvalidOperationException: source B is misconfigured<---
 ---> (Inner Exception #1) System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known<---
…

Memex.Portal.Shared.Test.InstanceIdRulesMatchTheRegistryTest ‑ TheSetupHostAgreesWithTheRegistry(candidate: "d658ee52-093a-4294-b9c6-3d4d24ea55da")
Memex.Portal.Shared.Test.SessionDenialIsAnAnswerTest ‑ OnlyAVerdictReadsAsADenial(shape: "the same verdict nested, as a late denial dispatch"···, failure: System.InvalidOperationException: write failed
 ---> System.UnauthorizedAccessException: Access denied
   --- End of inner exception stack trace ---, isDenial: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall as an aggregate's FIRST member", fault: System.AggregateException: One or more errors occurred. (Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.)
 ---> MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.
   --- End of inner exception stack trace ---, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall as an aggregate's SECOND member", fault: System.AggregateException: One or more errors occurred. (The operation has timed out.) (Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.)
 ---> System.TimeoutException: The operation has timed out.
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.<---
, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall nested in an aggregate inside a wrapper", fault: System.InvalidOperationException: outer
 ---> System.AggregateException: One or more errors occurred. (Value does not fall within the expected range.) (Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.)
 ---> System.ArgumentException: Value does not fall within the expected range.
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.<---

   --- End of inner exception stack trace ---, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "a stall wrapped as InnerException", fault: System.InvalidOperationException: outer
 ---> MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [pg] did not emit an Initial within the query fan-in's 16s bound for query 'nodeType:NodeType' (user 'system'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.
   --- End of inner exception stack trace ---, expected: True)
MeshWeaver.Compiler.Pipeline.Test.SourcesWatcherStallBackoffTest ‑ The_production_classifier_recognises_a_stall_through_every_wrapping(because: "an aggregate of unrelated faults", fault: System.AggregateException: One or more errors occurred. (The operation has timed out.) (Value does not fall within the expected range.)
 ---> System.TimeoutException: The operation has timed out.
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) System.ArgumentException: Value does not fall within the expected range.<---
, expected: False)
MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest ‑ Classifies(shape: "a MIXED aggregate — one transient branch, one genu"···, exception: System.AggregateException: One or more errors occurred. (Failed to connect to 10.42.18.4:5432) (source B is misconfigured)
 ---> MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) System.InvalidOperationException: source B is misconfigured<---
, expected: False)
MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest ‑ Classifies(shape: "a nested aggregate with one genuine leaf", exception: System.AggregateException: One or more errors occurred. (One or more errors occurred. (Failed to connect to 10.42.18.4:5432) (boom)) (Failed to connect to 10.42.18.4:5432)
 ---> System.AggregateException: One or more errors occurred. (Failed to connect to 10.42.18.4:5432) (boom)
 ---> MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) System.InvalidOperationException: boom<---

   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432<---
, expected: False)
MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest ‑ Classifies(shape: "a nested aggregate, all leaves transient", exception: System.AggregateException: One or more errors occurred. (One or more errors occurred. (Failed to connect to 10.42.18.4:5432)) (Failed to connect to 10.42.18.4:5432)
 ---> System.AggregateException: One or more errors occurred. (Failed to connect to 10.42.18.4:5432)
 ---> MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) MeshWeaver.Messaging.Hub.Test.InfrastructureFaultTest+ProviderException (0x80004005): Failed to connect to 10.42.18.4:5432<---
, expected: True)
…

♻️ This comment has been updated with latest results.

…erver emits in-process) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

# Conflicts: # src/MeshWeaver.Documentation/Data/Architecture/PolicyNotProse.md

Copilot AI balanced review requested due to automatic review settings October 7, 2026 17:31

rbuergi added the tests-before-review label Oct 7, 2026

Copilot started reviewing on behalf of rbuergi October 7, 2026 17:32 View session

Copilot AI reviewed Oct 7, 2026

View reviewed changes

rbuergi and others added 2 commits October 7, 2026 20:47

ci(#5212): DOTNET_TieredPGO=0 on the build job too (Roslyn compiler s…

e7fcf1e

…erver emits in-process) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into w6268b

9b1102c

# Conflicts: # src/MeshWeaver.Documentation/Data/Architecture/PolicyNotProse.md

rbuergi merged commit 18696ad into main Oct 7, 2026
48 checks passed

rbuergi deleted the fix/5212-tieredpgo-off-env branch October 10, 2026 13:10

Navigation Menu

Repository navigation

ci(#5212): DOTNET_TieredPGO=0 stopgap in CI + satellite lanes; chart renders it from a record - #6268

ci(#5212): DOTNET_TieredPGO=0 stopgap in CI + satellite lanes; chart renders it from a record#6268
rbuergi merged 3 commits into
mainfrom
fix/5212-tieredpgo-off-env

rbuergi commented Oct 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

rbuergi Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rbuergi commented Oct 7, 2026

What

Changes

Paired Memex PR

Checks run locally

Recycle / roll

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

rbuergi Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (shard 2)

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (shard 1)

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (shard 0)

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (shard 3)

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (shard 5)

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (shard 4)

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading