Skip to content

feat(#1660 WS3): commit-deterministic framework identity — the CI bake seeds at boot - #1687

Merged
rbuergi merged 1 commit into
mainfrom
feat/1660-ws3-deterministic-identity
Aug 16, 2026
Merged

rbuergi merged 1 commit into
mainfrom
feat/1660-ws3-deterministic-identity

Conversation

@rbuergi

@rbuergi rbuergi commented Aug 16, 2026 •

Copy link
Copy Markdown
Contributor

#1660 WS3 — commit-deterministic framework identity: the CI bake actually seeds at boot

Fixes the measured regression: every pod boot logged DynamicTypePreWarmer: 282 of 282 dynamic NodeType(s) need building … baked=0 pending=282 frameworkstale=269 — the CI bake existed but
seeded ZERO, because the framework identity differed between the CI run that baked and the CD run
that built the image.

The history, precisely ("we had this")

The fix — one identity, deterministic per commit

Compile inputs (Directory.Build.props): under CIRun nothing per-build reaches any compiled
attribute: InformationalVersion = $(PlatformVersion) (the SDK appends +<checked-out sha>),
FileVersion pinned to $(_VersionNumeric).0. $(Version) keeps the monotonic .ci.<run#> for
NuGet package versions and image tags only — verified it reaches no compiled attribute.

The identity (FrameworkBuildIdentity, one resolution):

  • CI builds (CIRun and GITHUB_ACTIONS — a CI checkout is clean at its commit; a local
    -p:CIRun=true repro can be dirty and stays on the MVID) are stamped
    AssemblyMetadata("MeshWeaverFrameworkIdentity", "g<sha>"); the identity is the commit. A
    commit names the whole tree — every source file and every package pin — so any
    reference-assembly change flips it: the widening over the whole TPA reference set that the old
    🚨 comment demanded (Graph's bare MVID only covers Graph's own inputs), delivered as a strict
    superset rather than a per-host TPA hash (which could never match between the bake host and the
    portal — different processes have different TPAs).
  • Local builds carry no stamp → Graph MVID, content-exact for dirty trees, exactly as before.

Every consumer moved by construction — all flowed through
NodeTypeCompilationHelpers.FrameworkVersion already: HasUsableBuild /
HasStaleFrameworkBuild (staleness gate), ApplyCompileSuccess (CompiledFrameworkVersion
stamp), FileSystemAssemblyStore.FrameworkTag (store filename tag), NodeTypeBakeStatus
(probe/report), BuildProtocolDriver (build-claim fingerprint), PrebuiltAssemblySeeder
(adoption gate + LiveFrameworkMvid), BakeOutput (bake artifact identity → the
baked-assemblies-<identity> artifact key). The one file-side reader,
Plugin.Build/FrameworkIdentity, gained ReadIdentity (metadata-only PE read of the stamp,
MVID fallback) so producers record what the runtime actually compares; ModulePackCommand and
ResolveFrameworkMvid use it. AssemblyCacheGenerations.TagOf accepts the g+7-hex tag shape so
CI generations stay reclaimable.

Delivery to storage (the restored memex-bake outcome, without the extra image):

  • main-cd gains publish-bake (needs promote): downloads the Build-and-Test run's
    baked-assemblies-* artifact for the same commit and publishes it via
    .github/scripts/publish-bake-bundles.sh to the portals' Azure Files data shares:
    prebuilt-bundles/<identity>/<source>/<bundle>.zip. Config is preflighted red (no
    skip-trapdoor): missing BAKE_PUBLISH_TARGETS fails naming exactly what to provision. A missing
    artifact only warns (a reuse-green run legitimately baked nothing).
  • Portal boot: ShippedPrebuiltBundles.SeedPublishedRoot (new config
    PreWarm:PrebuiltBundleRoot, set to /data/prebuilt-bundles in values.aks.yaml) seeds ONLY
    the pod's own identity's subtree before the NodeType sweep — same seeder, same MVID gate
    underneath. Unconfigured → inert.
  • The feat(ci): the compile stage bakes shipped content — boot loads, never compiles (#1660 WS1 step 1) #1674 in-image prebuilt/ lane is untouched and now redundant for this purpose (it was
    never wired into main-cd); it remains the same-build lane for the tester.

"Subscribe to new releases of any dependency and rebuild": in-repo dependency bumps need no
event — a Dependabot merge is a commit, a commit IS a new identity, the next CI run re-bakes. The
cross-repo edge is closed by notify-dependents: one repository_dispatch
(meshweaver-framework-released, payload commit/identity/version) per repo in
BAKE_SUBSCRIBER_REPOS; node repos subscribe with
on: repository_dispatch: { types: [meshweaver-framework-released] } and re-run their gate+bake.
Reporter-class (like the platform-update webhook): unconfigured is a loud notice, never red.

Runtime version display: the run number stays visible without being compiled in — the
container publish injects $(Version) as MESHWEAVER_PLATFORM_VERSION
(ContainerEnvironmentVariable, image config); ShippedReleaseSeed.InstalledPlatformVersion
(self-updater current-version + Admin/PlatformVersion node) and AppVersionService read it first.
The self-updater's target selection keys on ACR tags (unchanged, still .ci.<run#> monotonic);
its IsNewer(target, current) gate keeps its exact semantics because current keeps the run
number via the env var. Memex.Client (not in the solution/CI — MAUI, built separately) compares
at release granularity when its own build carries no run number, so a current client is not
nagged every connect.

Proof (measured, not claimed)

  • Two clean builds of MeshWeaver.Graph, -p:CIRun=true, GITHUB_RUN_NUMBER=1111 vs 2222,
    70 s apart: output DLLs bit-identical (cmp clean), MVID ec8624fdd279496f87e01c70c7a7d051
    both, identity g5be81c02e… both, InformationalVersion = 3.0.0-rc3+5be81c02e….
  • One-line source change → MVID flips to e33b7842b5244735a69d70b9d1739c80 (identity/commit
    unchanged — which is exactly why the stamp is CI-only; see the props comment).
  • Generated AssemblyInfo.cs carries no run number and no ticks (grepped both builds).
  • MeshWeaver.Graph.Test (FrameworkBuildIdentity/AssemblyCacheRetention/PrebuiltAssemblySeeder/
    NodeTypeCompilationHelpers filters): 59/59 green, including new pins: stamped-wins/MVID-
    fallback resolution, PE-read == loaded-assembly-read, store tag attributable by the sweep.
  • MeshWeaver.PluginCatalog.Test (ShippedPrebuiltBundles/ModuleLandingService/ModuleFunnel):
    21/21 green, including the CI-writes → portal-reads round trip
    (PublishedRoot_SeedsOwnIdentitysDirectory_AndIgnoresOtherIdentities) asserting the published
    layout is found by the sweep's store probe (pending=0 shape) and another identity's bundles
    are invisible.
  • DocumentationLinkIntegrityTest green after the doc updates.
  • Full solution -c Release -warnaserror --no-incremental green.

Provisioned as part of this change (already live)

  • Repo variable BAKE_PUBLISH_TARGETS = the three portals' data shares on storage account
    f45d8e671caa74b96a6c37f (memex pvc-f3e4220c…, memex-cloud pvc-d9f747ad…, client-i
    pvc-9e341c82… — resolved from the PVs behind each memex-data claim).
  • Role Storage File Data Privileged Contributor for the CD OIDC principal
    (github-actions-deploy) on that storage account — verified assigned.

Ops follow-ups (not in this PR)

  • Live portals pick up PreWarm__PrebuiltBundleRoot on the next chart-managed apply (or a
    kubectl set env) — the values.aks.yaml change is the persistent home; prod rollout is its own
    decision.
  • BAKE_SUBSCRIBER_REPOS + DEPENDENT_DISPATCH_TOKEN (cross-repo dispatch) are deliberately
    unprovisioned — needs an org token decision; the notify job says so loudly per run. Receiver
    wiring for the node repos is documented in ContinuousDeliveryContract.md (follow-up: plugins /
    education / reinsurance / social-media workflows).
  • prebuilt-bundles/ retention on the shares: bundles accumulate per commit; small, but worth a
    sweep entry later (the assembly-cache retention pattern applies).

Closes the WS3 item of #1660.

🤖 Generated with Claude Code

…e seeds at boot

Every pod boot re-baked all dynamic NodeTypes (baked=0 pending=282) because the
framework identity differed per CI build: InformationalVersion carried
+build.<UtcNow.Ticks> under CIRun (PR #145), so the Build-and-Test run that bakes
and the main-cd run that builds the image could never agree — the defect that
also killed the memex-bake Job (#1347, ded3275).

The pairing the old props comment demanded, in one change:

* Directory.Build.props: nothing per-build reaches a compiled attribute under
  CIRun — InformationalVersion is $(PlatformVersion) (SDK appends the checked-out
  sha), FileVersion is pinned; $(Version) keeps the monotonic .ci.<run#> for
  package versions and image tags ONLY. On GitHub Actions CIRun builds every
  assembly is stamped AssemblyMetadata("MeshWeaverFrameworkIdentity","g<sha>").
* FrameworkBuildIdentity (Graph): ONE identity resolution — the stamped commit
  identity when present (covers the WHOLE TPA reference set: a commit names the
  entire tree, the widening the ticks comment required), Graph MVID locally
  (content-exact for dirty trees). NodeTypeCompilationHelpers.FrameworkVersion
  resolves through it, so every consumer (HasUsableBuild, CompiledFrameworkVersion
  stamps, store FrameworkTag, bake report, build-claim fingerprint, seeder gate,
  bake artifact key) moves at once. Plugin.Build reads the same stamp from PE
  metadata (ReadIdentity); the retention sweep attributes the g-tag shape.
* Delivery: main-cd publish-bake copies the BT run's baked-assemblies-* artifact
  to the portals' data shares (prebuilt-bundles/<identity>/<source>/, script
  .github/scripts/publish-bake-bundles.sh; config preflighted RED, artifact
  absence warns); boot seeds its own identity's bundles before the sweep
  (ShippedPrebuiltBundles.SeedPublishedRoot, PreWarm:PrebuiltBundleRoot,
  /data/prebuilt-bundles on AKS). notify-dependents dispatches
  meshweaver-framework-released to subscriber repos so dependents rebake on
  every framework release (in-repo dependency bumps re-key by commit already).
* Runtime display: $(Version) rides the image config as
  MESHWEAVER_PLATFORM_VERSION; InstalledPlatformVersion/AppVersionService read it
  first, so the self-updater's current-version comparison keeps the run number.

Proven: two CIRun builds of MeshWeaver.Graph with GITHUB_RUN_NUMBER 1111 vs 2222,
70s apart, are BIT-IDENTICAL (mvid ec8624fd…); a one-line source change flips the
MVID (e33b7842…); generated AssemblyInfo carries no run number and no ticks.
Graph.Test 59/59, PluginCatalog.Test 21/21 (incl. the CI-writes→portal-reads
round trip), DocumentationLinkIntegrityTest green, full solution Release
-warnaserror --no-incremental green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 16, 2026 19:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes the framework identity deterministic per commit (CI) so CI-baked NodeType assemblies can be reliably adopted at portal boot, and wires CD to publish the bake bundles to shared portal storage for consumption during warmup.

Changes:

  • Introduces a unified “framework build identity” resolution (CI: g<sha> stamp; local: Graph MVID) and updates all consumers to use it.
  • Adds a CD publish-bake job plus a publishing script to copy CI bake artifacts onto portal Azure Files in an identity-keyed layout, and seeds that layout during portal warmup.
  • Updates runtime version display to use an injected env var for the run-numbered version while keeping compiled assemblies commit-deterministic.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/MeshWeaver.PluginCatalog.Test/ShippedPrebuiltBundlesTest.cs Adds tests for seeding from the CI-published bundle root and inert behavior when unconfigured.
test/MeshWeaver.PluginCatalog.Test/ModuleLandingServiceTest.cs Switches “live framework identity” source to the centralized identity reader.
test/MeshWeaver.PluginCatalog.Test/ModuleFunnelTest.cs Switches “live framework identity” source to the centralized identity reader.
test/MeshWeaver.Graph.Test/FrameworkBuildIdentityTest.cs Pins identity resolution rules and asserts loaded-assembly vs PE-read agreement.
test/MeshWeaver.Graph.Test/AssemblyCacheRetentionTest.cs Extends retention tag attribution tests to support g-prefixed identity tags.
src/MeshWeaver.Plugin.Build/ModulePackCommand.cs Records framework identity in module pack metadata using PE-read identity.
src/MeshWeaver.Plugin.Build/FrameworkIdentity.cs Adds ReadIdentity (stamp-or-MVID) and PE-level stamp decoding for producers.
src/MeshWeaver.Mesh.Contract/PlatformBuildInfo.cs Introduces env-var-based runtime platform version discovery.
src/MeshWeaver.Hosting/ShippedPrebuiltBundles.cs Adds seeding from CI-published root keyed by the running identity.
src/MeshWeaver.Hosting/DynamicTypePreWarmerHostedService.cs Seeds both in-image bundles and published-root bundles before the sweep.
src/MeshWeaver.Graph/Configuration/NodeTypeCompilationHelpers.cs Updates FrameworkVersion to resolve via the new identity scheme.
src/MeshWeaver.Graph/Configuration/FrameworkBuildIdentity.cs Defines the single identity resolution rule and stamp reader for loaded assemblies.
src/MeshWeaver.Graph/Configuration/AssemblyCacheRetention.cs Extends tag parsing to attribute CI commit-identity generations.
src/MeshWeaver.Documentation/Data/WhatsNew/2026-08-16-ci-bake-seeds-at-boot.md Adds a “What’s New” entry describing faster startup via CI bake adoption.
src/MeshWeaver.Documentation/Data/Architecture/ReleaseToProductionPipeline.md Updates pipeline documentation to reflect CI bake publication/adoption.
src/MeshWeaver.Documentation/Data/Architecture/ReleaseProcess.md Updates versioning docs to keep compiled attributes commit-deterministic and move run number to env var.
src/MeshWeaver.Documentation/Data/Architecture/PluginPackaging.md Updates plugin packaging docs to reflect identity (commit stamp / MVID) semantics.
src/MeshWeaver.Documentation/Data/Architecture/NodeTypeCompilation.md Updates compilation docs to describe the new identity scheme.
src/MeshWeaver.Documentation/Data/Architecture/ContinuousDeliveryContract.md Documents post-promote bake publication and dependent-repo dispatch.
src/MeshWeaver.Blazor.Portal/Infrastructure/AppVersionService.cs Reads injected runtime platform version first, falls back to assembly informational version.
memex/Memex.Portal.Shared/ShippedReleaseSeed.cs Uses injected runtime platform version first for the Admin/PlatformVersion node.
memex/Memex.Client/Services/UpdateNotificationService.cs Adjusts “is newer” logic so clients without .ci.N aren’t nagged on every connect.
memex/aspire/Memex.Portal.Distributed/Program.cs Updates commentary about generation cadence under commit-scoped identity.
memex/aspire/Memex.Portal.Distributed/Memex.Portal.Distributed.csproj Injects MESHWEAVER_PLATFORM_VERSION into container image config under CIRun.
Directory.Build.props Makes CI compiled attributes commit-deterministic and stamps MeshWeaverFrameworkIdentity only on GitHub Actions CI builds.
deploy/CHANGE-PROPAGATION.md Updates change propagation docs to reflect CI bake publication replacing the old bake image.
deploy/aks/values.aks.yaml Adds PreWarm__PrebuiltBundleRoot configuration for CI-published bundle consumption.
deploy/aks/DEPLOY-RUNBOOK.md Updates runbook to explain CI bake publication/adoption instead of a bake Job.
.github/workflows/main-cd.yml Adds publish-bake and notify-dependents jobs; requests actions: read.
.github/workflows/dotnet-test.yml Updates job comments to reflect identity-keyed artifacts and cross-workflow adoption.
.github/scripts/publish-bake-bundles.sh Adds a script to publish baked bundles to Azure Files in the identity-keyed layout.
Suppressed comments (1)

src/MeshWeaver.Plugin.Build/ModulePackCommand.cs:194

  • This warning still says "framework MVID", but the value recorded is now the framework identity (CI commit stamp or local MVID). The message should match the new semantics to avoid misleading operators.
            Console.Error.WriteLine(
                $"warning: {FrameworkIdentity.IdentityAssembly}.dll not found at {graphDll} — the "
                + "bundle records no built-against framework MVID (diagnostic metadata only; the "
                + "landing gate is --min-mesh-version).");

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/main-cd.yml
@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

919 tests   918 ✅  10m 59s ⏱️
 10 suites    1 💤
 10 files      0 ❌

Results for commit 1ac75bb.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

1 606 tests   1 600 ✅  6m 32s ⏱️
   11 suites      6 💤
   11 files        0 ❌

Results for commit 1ac75bb.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

   11 files     11 suites   6m 15s ⏱️
2 197 tests 2 006 ✅ 191 💤 0 ❌
2 566 runs  2 375 ✅ 191 💤 0 ❌

Results for commit 1ac75bb.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

1 371 tests   1 370 ✅  6m 32s ⏱️
   11 suites      1 💤
   11 files        0 ❌

Results for commit 1ac75bb.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

2 164 tests   2 061 ✅  7m 18s ⏱️
   11 suites    103 💤
   11 files        0 ❌

Results for commit 1ac75bb.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

2 447 tests   2 443 ✅  8m 7s ⏱️
   11 suites      4 💤
   11 files        0 ❌

Results for commit 1ac75bb.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results

    65 files      65 suites   45m 46s ⏱️
10 704 tests 10 398 ✅ 306 💤 0 ❌
11 073 runs  10 767 ✅ 306 💤 0 ❌

Results for commit 1ac75bb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants