Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/dotnet-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -658,6 +658,12 @@ jobs:
# must hold the build too, and `!cancelled()` alone would read a skipped need as fine.
if: ${{ !cancelled() && needs.precheck.outputs.skip != 'true' && needs.stage-gate.result == 'success' }}
timeout-minutes: 25
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# `dotnet build` of the whole solution launches a long-lived Roslyn compiler server that
# emits in-process, so it is exposed to the same Tier-1 JIT defect as the test shards (see
# the `test` job's env); `<TieredPGO>` in a runtimeconfig does not configure that process.
env:
DOTNET_TieredPGO: "0"
runs-on: ubuntu-latest
outputs:
# THE MAIN THE SUITES TEST AGAINST, resolved HERE — as the suites START. A run held at the stage
Expand Down Expand Up @@ -861,6 +867,14 @@ jobs:
# projects. 20 min is ~1.5x the slowest observed shard (14m36s) with the rebalanced
# weights, so a shard that hits this is stuck, not slow.
timeout-minutes: 20
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
env:
DOTNET_TieredPGO: "0"
Comment on lines +876 to +877

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e7fcf1e: the stopgap's env DOTNET_TieredPGO=0 is now set on the build job as well, since the solution build's Roslyn compiler server emits in-process and is not configured by a runtimeconfig.

runs-on: ubuntu-latest
strategy:
# Don't cancel sibling shards when one fails — we want every shard's
Expand Down Expand Up @@ -1891,6 +1905,14 @@ jobs:
# Same wall-clock contract as plugin-gate: depends on `build` like the shards, so it runs
# CONCURRENTLY with them; while it stays under the slowest shard it costs the PR nothing.
timeout-minutes: 12
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
env:
DOTNET_TieredPGO: "0"
outputs:
failure: ${{ steps.gate.outputs.failure }}
steps:
Expand Down Expand Up @@ -2087,6 +2109,14 @@ jobs:
if: ${{ !cancelled() && needs.build.result == 'success' }}
runs-on: ubuntu-latest
timeout-minutes: 10
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
env:
DOTNET_TieredPGO: "0"
permissions:
contents: read
actions: read
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/node-repo-compile-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,14 @@ jobs:
name: Compile every NodeType (vs core)
runs-on: ${{ vars.MW_RUNNER_DOCKER || 'ubuntu-latest' }}
timeout-minutes: 20
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
env:
DOTNET_TieredPGO: "0"
# 🚨 A job-level `permissions:` in a workflow_call lane is a DEMAND on every caller, not a
# grant (MeshWeaver#3933 took MeshWeaver.Plugins down for five hours with one `id-token:
# write` here). Both scopes below sit inside GitHub's RESTRICTED default (contents / packages /
Expand Down
12 changes: 11 additions & 1 deletion .github/workflows/node-repo-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -944,6 +944,13 @@ jobs:
# needs more is STUCK, not slow.
timeout-minutes: 45
env:
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
DOTNET_TieredPGO: "0"
# The self-hosted runner is non-root: setup-dotnet cannot write /usr/share/dotnet there.
# EMPTY on ubuntu-latest, which setup-dotnet reads as unset — the default path is unchanged.
# No step here runs setup-dotnet today; the line is the rule, uniform with module-pack.
Expand Down Expand Up @@ -2187,7 +2194,10 @@ jobs:
if [ "$MODE" = volume ]; then
"$DOTNET" "$HOST_DIR/mw-plugin-test.dll" "$@"
else
docker run --rm --init --user "$(id -u):$(id -g)" -e HOME=/tmp "${EXT_V[@]}" \
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix
# ships. Same switch as the job env, carried INTO the container (the runner env does not
# cross `docker run`).
docker run --rm --init --user "$(id -u):$(id -g)" -e HOME=/tmp -e DOTNET_TieredPGO=0 "${EXT_V[@]}" \
-v "$HOST_DIR:/host:ro" -v "$REPO_MOUNT:/repo" "${TESTER_MOUNTS[@]}" "${DUMP_ARGS[@]}" \
--entrypoint dotnet "$PLATFORM_REF" /host/mw-plugin-test.dll "$@"
fi
Expand Down
27 changes: 26 additions & 1 deletion .github/workflows/node-repo-module-pack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1730,6 +1730,14 @@ jobs:
# `workspace-runner` (inputs): a Docker-capable label — the compiler is the platform image.
runs-on: ${{ inputs.workspace-runner }}
timeout-minutes: 30
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
env:
DOTNET_TieredPGO: "0"
# Inherit for the same reason as `prepare`: OIDC is a caller-selected capability, while basic
# callers must remain able to grant `actions: read` without also granting an identity token.
steps:
Expand Down Expand Up @@ -1954,8 +1962,11 @@ jobs:
echo "global build: ${#projects[@]} container entr(y|ies) in ONE workspace — ${projects[*]}"
# AS THE RUNNER'S UID (a root-owned /out is unwritable to later steps); HOME=/tmp keeps
# runtime probes off read-only image paths. No --extra-refs, ever, on this lane.
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix
# ships. Same switch as the job env, carried INTO the container (the runner env does not
# cross `docker run`).
docker run --rm --init --platform linux/amd64 \
--user "$(id -u):$(id -g)" -e HOME=/tmp \
--user "$(id -u):$(id -g)" -e HOME=/tmp -e DOTNET_TieredPGO=0 \
-v "$GITHUB_WORKSPACE:/repo:ro" \
-v "$tester_app:/tester:ro" \
-v "$out:/out" \
Expand Down Expand Up @@ -2086,6 +2097,13 @@ jobs:
runs-on: ${{ inputs.runner }}
timeout-minutes: 45
env:
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
DOTNET_TieredPGO: "0"
# The self-hosted runner is non-root: setup-dotnet cannot write /usr/share/dotnet there.
# EMPTY on ubuntu-latest, which setup-dotnet reads as unset — the default path is unchanged.
DOTNET_INSTALL_DIR: ${{ inputs.runner != 'ubuntu-latest' && '/home/runner/.dotnet' || '' }}
Expand Down Expand Up @@ -4383,6 +4401,13 @@ jobs:
runs-on: ${{ inputs.runner }}
timeout-minutes: 45
env:
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
DOTNET_TieredPGO: "0"
# The self-hosted runner is non-root: setup-dotnet cannot write /usr/share/dotnet there.
# EMPTY on ubuntu-latest, which setup-dotnet reads as unset — the default path is unchanged.
DOTNET_INSTALL_DIR: ${{ inputs.runner != 'ubuntu-latest' && '/home/runner/.dotnet' || '' }}
Expand Down
18 changes: 16 additions & 2 deletions .github/workflows/node-repo-publish-bake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -574,6 +574,14 @@ jobs:
# refuses an expression here because a caller-supplied bound cannot be proven by reading the file).
# Measured 2026-09-02 the bake runs ~20 min; past 45 it is stuck, not slow.
timeout-minutes: 45
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix ships.
# The .NET 10 Tier-1 JIT under Dynamic PGO drops a type check in an inlined guarded
# devirtualisation on Roslyn's MetadataWriter.PopulateNestedClassTableRows /
# getConsolidatedTypeParameters path, after which every in-process Emit NREs. The
# runtimeconfig half (<TieredPGO>false</TieredPGO>, #890) covers hosts built from THIS tree;
# this env covers every process the job launches, whatever pin it was built at.
env:
DOTNET_TieredPGO: "0"
outputs:
# Read by dispatch-dependents below: a run that baked NOTHING (already sealed) publishes
# nothing and therefore wakes nobody.
Expand Down Expand Up @@ -1703,8 +1711,11 @@ jobs:
# shape since #2922). `--app /app --shared-frameworks /usr/share/dotnet/shared` makes the
# bake's reference set the portal's /app plus its IMPLEMENTATION frameworks — what the
# portal's own runtime compile sees — and keys every bundle to the portal's identity.
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix
# ships. Same switch as the job env, carried INTO the container (the runner env does not
# cross `docker run`).
# shellcheck disable=SC2086 # EXT_MODULE_ARGS is a flag list built above; word-splitting is the point
docker run --rm --init --user "$(id -u):$(id -g)" -e HOME=/tmp "${EXT_V[@]}" \
docker run --rm --init --user "$(id -u):$(id -g)" -e HOME=/tmp -e DOTNET_TieredPGO=0 "${EXT_V[@]}" \
-v "$HOST_DIR:/host:ro" -v "$REPO_MOUNT:/repo" -v "$BAKE:/bake" \
--entrypoint dotnet "$PLATFORM_REF" /host/mw-plugin-test.dll compile /repo \
--app /app --shared-frameworks /usr/share/dotnet/shared \
Expand Down Expand Up @@ -1743,8 +1754,11 @@ jobs:
# only a genuinely incompatible one — or none at all — becomes `Unservable`, said at
# Critical. A named verdict replaces an unreadable compile failure, and NOTHING that
# passes today starts failing: adoption is the path that already carries every gate.
# 🚨 TEMPORARY STOPGAP (#5212, policy tiered-pgo-off-5212) — remove when the runtime fix
# ships. Same switch as the job env, carried INTO the container (the runner env does not
# cross `docker run`).
# shellcheck disable=SC2086
docker run --rm --init --user "$(id -u):$(id -g)" -e HOME=/tmp \
docker run --rm --init --user "$(id -u):$(id -g)" -e HOME=/tmp -e DOTNET_TieredPGO=0 \
-e Modules__RequirePrebuilt=true "${EXT_V[@]}" \
-v "$HOST_DIR:/host:ro" -v "$REPO_MOUNT:/repo" -v "$BAKE:/bake" \
--entrypoint dotnet "$PLATFORM_REF" /host/mw-plugin-test.dll /repo --app /app \
Expand Down
13 changes: 13 additions & 0 deletions deploy/helm/templates/memex-portal/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -813,6 +813,19 @@ data:
{{- if and (hasKey .Values.config.memex_portal "Hosting__TriageAssignments__Enabled") (not (kindIs "invalid" $triageAssignmentsOn)) (ne ($triageAssignmentsOn | toString | trim) "") }}
Hosting__TriageAssignments__Enabled: "{{ $triageAssignmentsOn | toString | trim }}"
{{- end }}
# ---- Runtime JIT switch: Tiered PGO OFF — TEMPORARY STOPGAP (#5212) --------------------------
# Policy tiered-pgo-off-5212; remove when the .NET runtime fix ships. The .NET 10 Tier-1 JIT
# under Dynamic PGO can drop a type check in an inlined guarded devirtualisation on Roslyn's
# MetadataWriter.PopulateNestedClassTableRows / getConsolidatedTypeParameters path; from then on
# every in-process NodeType Emit in that replica NREs (Doc/Architecture/NodeTypeCompilation,
# "The defect, in one listing"). The runtime reads DOTNET_TieredPGO from the process environment,
# which this ConfigMap feeds through envFrom. Set from a Deployment record's extraPortalConfig
# (`DOTNET_TieredPGO: "0"`); rendered only when the key is PRESENT and non-empty, so an instance
# that does not set it keeps the runtime default rather than a chart-chosen one.
{{- $tieredPgo := .Values.config.memex_portal.DOTNET_TieredPGO }}
{{- if and (hasKey .Values.config.memex_portal "DOTNET_TieredPGO") (not (kindIs "invalid" $tieredPgo)) (ne ($tieredPgo | toString | trim) "") }}
DOTNET_TieredPGO: "{{ $tieredPgo | toString | trim }}"
{{- end }}
# Control instance only: the deployments the ONE org webhook's pull-request and check events are
# forwarded to over the control lane (comma-separated ids; Hosting:PrEvents:ForwardTo).
{{- if .Values.config.memex_portal.Hosting__PrEvents__ForwardTo }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2245,6 +2245,26 @@ one: the defect is in the .NET 10 JIT, not in this repository, and the switch is
measurement on a runtime that fixes it — the arm above, re-run. The portal hosts live in
MeshWeaver.Plugins and need the same property there.

**The environment half — policy [`tiered-pgo-off-5212`](../PolicyNotProse), tracked in
[#5212](https://github.com/Systemorph/MeshWeaver/issues/5212).** The runtimeconfig property reaches only
hosts BUILT from this tree at or after the commit that added it; a satellite lane runs the tester and the
portal image at whatever pin it holds, and the portal hosts are built in MeshWeaver.Plugins. So the same
switch is also set as `DOTNET_TieredPGO=0`, which the runtime reads from the process environment and which
wins over the runtimeconfig:

- **Core CI** — job `env` on the `test`, `doc-gate` and `platform-compat` jobs of `dotnet-test.yml`.
- **Satellite lanes** — job `env` on `node-repo-compile-check` (its `dotnet build` compiler server),
`node-repo-gate` (`gate`), `node-repo-module-pack` (`build-workspace`, `pack`, `tests`) and
`node-repo-publish-bake` (`publish-bake`), plus `-e DOTNET_TieredPGO=0` on every `docker run` that
emits, because the runner's environment does not cross into a container.
- **Portals** — `DOTNET_TieredPGO: "0"` in each portal Deployment record's `extraPortalConfig`, rendered by
the chart into the `memex-portal-config` ConfigMap (fed to the container through `envFrom`) only when the
key is present. A record change reaches a running instance only through a Reconcile, and only on the
next process start.

Every one of these carries a comment naming #5212 and is removed when the runtime fix ships, together with
the runtimeconfig property — after the split arm above is re-run on that runtime.

### Framework-version freezing

A compiled NodeType DLL references the MeshWeaver framework assemblies present
Expand Down
Loading