feat(ci): the compile stage bakes shipped content — boot loads, never compiles (#1660 WS1 step 1) by rbuergi · Pull Request #1674 · Systemorph/MeshWeaver

rbuergi · 2026-08-16T12:25:45Z

Implements #1660 workstream 1, step 1: the CI compile stage that already proves the image's shipped content compiles now also produces the compiled assemblies as MVID-keyed prebuilt bundles, the image can ship them, and boot loads instead of compiling.

What shipped

Producer — the content gates persist what they compile

mw-plugin-test gains --bake-output <dir> / --source-sha <sha>: after the gate's checks, every NodeType that reached CompilationStatus.Ok is persisted as one prebuilt-assembly bundle per package (<package>.zip, new BundleWriter in MeshWeaver.Plugin.Packaging — the write half of the existing BundleReader codec), plus framework-mvid.txt naming the identity the bundles are keyed to. This reuses the batch-bake/compile pipeline the gate already runs (NodeTypeBatchBake under the hood) — nothing is re-implemented; the bake is a persist step over the gate's own assembly store.
Fail-red contract: an Ok type with no bytes in the store faults the run (GateReport.FatalError), and a green gate that wrote no bake identity fails the CI step. No continue-on-error, no input-shaped if:.

CI wiring (dotnet-test.yml)

doc-gate (Doc + samples stages) bakes into $RUNNER_TEMP/bake and uploads baked-assemblies-<mvid>; the step summary lists per-package assembly counts.
plugin-gate bakes the vital plugin modules (Store, Essentials) it already compiles from the build-time sync of Systemorph/MeshWeaver.Plugins and uploads baked-plugins-<mvid>, with the synced plugins-repo commit recorded as each bundle's sourceSha (provenance). Same checkout, same deploy key, same preflight assertion as before — the gate is upgraded from verdict-only to artifact-producing, per the maintainer's direction on NodeType assemblies are built in CI and LOADED at runtime — no runtime Roslyn, no warm-up window (maintainer-directed) #1660.
Both artifacts are stably named and immediately adoptable by every lane of the same run (one solution build ⇒ one Graph MVID). Test lanes are NOT rewired yet.

Image pickup + boot consumer

Memex.Portal.Distributed.csproj accepts -p:PrebuiltBakeDir=<dir> → bundles land in prebuilt/ in the publish output/container image. Inert when unset.
New ShippedPrebuiltBundles.SeedAll (MeshWeaver.Hosting), sequenced inside DynamicTypePreWarmerHostedService after the static repo import settles and before the sweep probes the store: reads each prebuilt/*.zip with BundleReader, gates once on the manifest MVID via PrebuiltAssemblySeeder.DeclineReason, filters to NodeTypes this mesh actually holds (one enumeration — no per-missing-path waits), and adopts via PrebuiltAssemblySeeder.Seed. The sweep then finds those types Baked → pending=0, zero Roslyn for covered shipped content. Best-effort and loud: anything unadoptable degrades to today's compile.

Docs: new Architecture/CiContentBake (+ link from PluginPackaging). DocumentationLinkIntegrityTest green.

Local bake over the repo's shipped content (the required run)

Doc stage (--allow .github/doc-gate.allow):

── Doc: installed (610 written, 0 unchanged); checking 4 NodeType(s)…
bake: Doc → 4 assembly(ies) → …/bake/Doc.zip
bake: framework=daa68696696a4f1abe5d000bad1077d0 source=localrun-doc packages=1 assemblies=4
ALL GREEN.            (wall clock 12:18:35 → 12:18:38 UTC)

Samples stage (--allow .github/samples-gate.allow, staged via .github/scripts/stage-samples-gate.sh):

bake: ACME → 3 · Cornerstone → 3 · FutuRe → 9 · MathDemo → 1 · Northwind → 6 · PythonDemo → 1 · Systemorph → 1
bake: framework=daa68696696a4f1abe5d000bad1077d0 source=localrun-samples packages=7 assemblies=24
GREEN — 8 known-debt failure(s) allowed (shrinking list).   (12:19:26 → 12:19:38 UTC)

28 assemblies, 8 bundles, one framework MVID; the 8 known-debt types are absent from the bundles by design (no usable build to ship — the consumer compiles them as today). Bundle layout verified: meshweaver/manifest.json (nodePath → assembly, frameworkMvid, sourceSha, per-type sourceVersions) + meshweaver/assemblies/<nodePath>.dll/.pdb — byte-compatible with BundleReader/PluginBundleClient.

Seeder-conformance evidence

BakeOutputTest (new, full in-process gate run): the produced bundle round-trips through BundleReader and PrebuiltAssemblySeeder.DeclineReason(manifest.FrameworkMvid) == null on the producing build — the exact gate that holds at consumption. Passed!
ShippedPrebuiltBundlesTest (new, MonolithMeshTestBase): matching-MVID bundle ⇒ adopted, record stamped (CompiledFrameworkVersion = live, LastCompiledVersion set), bytes on the store under the probed key, and NodeTypeBakeStatus.Classify(def, storeHasBytes: true, live) == Baked — the pending=0 pin. Mismatched MVID ⇒ declined whole, record untouched. Absent-node entry ⇒ skipped without hanging. Passed! (2/2)
Existing PrebuiltAssemblySeederGateTest + full MeshWeaver.Graph.Test, MeshWeaver.Hosting.Test, MeshWeaver.PluginCatalog.Test, MeshWeaver.PluginTester.Test suites green (Release, build-then---no-build).

Docker-thinning state (honesty over completeness)

Deferred — and one assumption in the brief was falsified. The CI run number is a compile input (Version → assembly attributes), so main-cd's image legs — which re-dotnet publish from source under their own run number — mint a different Graph MVID than this workflow's artifact. Shipping the Build-and-Test bake into the CD-built image today would produce bundles the seeder declines wholesale: a silently-inert feature that looks shipped. So this PR deliberately does not wire main-cd to consume the artifact. The concrete follow-up plan (bake inside the portal-image leg as the near-term step; thin containerize consuming the compile stage's publish once #1660 WS3 makes the identity deterministic) is posted as a comment on #1660. All plumbing an image producer needs (-p:PrebuiltBakeDir, prebuilt/ seeding, MVID gate) ships here and is inert until used.

Also deferred, tracked in the same comment: widening the plugin bake beyond the vital modules (PR wall-clock budget), and test lanes actually consuming baked-assemblies-<mvid>.

Refs #1660

🤖 Generated with Claude Code

… compiles (#1660 WS1 step 1) The content gates (doc-gate: Doc + samples; plugin-gate: the build-time MeshWeaver.Plugins sync, Store + Essentials) stop being verdict-only: mw-plugin-test --bake-output persists every NodeType the gate compiled to Ok as one prebuilt-assembly bundle per package (BundleWriter — the write half of the BundleReader codec), keyed to the run's framework MVID plus the synced source sha, and CI uploads the sets as baked-assemblies-<mvid> / baked-plugins-<mvid>. A green gate that produced no bake identity fails RED — no skip-trapdoors. The portal publish takes -p:PrebuiltBakeDir to lay bundles into prebuilt/ beside the app, and ShippedPrebuiltBundles seeds them at boot (after the static repo import settles, before the sweep probes the store) through PrebuiltAssemblySeeder — the sweep then reads those types Baked: pending=0 for covered shipped content, zero Roslyn on the pod. Best-effort and loud: anything unadoptable degrades to today's compile path. The MVID gate holds everywhere: a bundle from a DIFFERENT compilation declines wholesale. main-cd's image legs recompile under their own run number, so cross-workflow adoption waits for #1660 WS3 — the docker thinning follow-up is planned on the issue, and this change ships no main-cd wiring rather than a silently-inert artifact hand-off. Pinned by BakeOutputTest (producer output passes the consumer's DeclineReason gate) and ShippedPrebuiltBundlesTest (adopt ⇒ NodeTypeBakeStatus.Classify == Baked; mismatched MVID ⇒ declined whole; absent node ⇒ skipped, never waited on). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot

Pull request overview

Implements #1660 WS1 step 1 by turning the existing “content compiles” CI gates into artifact producers that persist the compiled NodeType assemblies as MVID-keyed bundle zips, and adds a boot-time consumer that seeds those shipped bundles into the assembly store so the dynamic-type sweep can load instead of compile.

Changes:

Extend mw-plugin-test to optionally persist compiled NodeTypes into per-package prebuilt bundle zips (plus framework-mvid.txt) and wire CI to upload these artifacts with stable MVID-keyed names.
Add BundleWriter (producer) and a boot-time shipped-bundle seeder (ShippedPrebuiltBundles.SeedAll) that adopts bundles before the dynamic-type sweep runs.
Add tests that pin producer/consumer conformance against the same MVID gate (PrebuiltAssemblySeeder.DeclineReason) and verify “pending=0 / Baked” classification after adoption.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/MeshWeaver.PluginTester/Program.cs	Adds CLI args for bake output + source SHA provenance.
tools/MeshWeaver.PluginTester/PluginGateRunner.cs	Hooks bake persistence into the gate run so bake failures make the run red.
tools/MeshWeaver.PluginTester/BakeOutput.cs	Implements the bake persistence step that writes per-package bundle zips and `framework-mvid.txt`.
test/MeshWeaver.PluginTester.Test/BakeOutputTest.cs	Pins that baked bundles round-trip via `BundleReader` and pass the live MVID gate.
test/MeshWeaver.PluginCatalog.Test/ShippedPrebuiltBundlesTest.cs	Pins boot consumer adoption behavior (match adopts; mismatch declines; absent types skipped).
test/MeshWeaver.PluginCatalog.Test/ModuleLandingServiceTest.cs	Doc comment fix to reference the updated `DeclineReason(string?)` signature.
src/MeshWeaver.PluginCatalog/ModuleLandingService.cs	Doc comment fix to reference the updated `DeclineReason(string?)` signature.
src/MeshWeaver.Plugin.Packaging/BundleWriter.cs	Adds bundle writer implementing the zip format consumed by `BundleReader`.
src/MeshWeaver.Hosting/ShippedPrebuiltBundles.cs	Adds boot-time seeding of shipped `prebuilt/*.zip` bundles through `PrebuiltAssemblySeeder`.
src/MeshWeaver.Hosting/MeshWeaver.Hosting.csproj	References `MeshWeaver.Plugin.Packaging` so the consumer can read bundles.
src/MeshWeaver.Hosting/DynamicTypePreWarmerHostedService.cs	Sequences shipped-bundle seeding after static import and before the sweep.
src/MeshWeaver.Graph/Configuration/PrebuiltAssemblySeeder.cs	Exposes the MVID identity accessor and adds an overload for explicit live identity in tests.
src/MeshWeaver.Documentation/Data/Architecture/PluginPackaging.md	Links to the new CI content bake documentation page.
src/MeshWeaver.Documentation/Data/Architecture/CiContentBake.md	New architecture doc describing producer/identity/image/consumer flow for WS1 step 1.
memex/aspire/Memex.Portal.Distributed/Memex.Portal.Distributed.csproj	Adds optional publish-time inclusion of prebuilt bundle zips into `prebuilt/`.
.github/workflows/dotnet-test.yml	Wires gates to bake + upload MVID-keyed artifacts and emits step summaries.

Suppressed comments (1)

.github/workflows/dotnet-test.yml:1132

This script section also runs with set -euo pipefail, but mvid_file="$RUNNER_TEMP/bake-plugins/framework-mvid.txt" will fail fast under -u if RUNNER_TEMP is unset, despite other paths in this workflow using ${RUNNER_TEMP:-/tmp}. Using the same fallback here keeps the postcondition check consistent with the gate step.

        set -euo pipefail
        mvid_file="$RUNNER_TEMP/bake-plugins/framework-mvid.txt"
        if [ ! -s "$mvid_file" ]; then
          echo "::error::the plugins gate ran green but wrote no bake identity ($mvid_file) — the bake stage regressed."

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-08-16T12:40:20Z