Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/scripts/compose-gate-host.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# deliberately runs that same adoption code rather than a gate-only reader, so a gate can only judge
# the platform's bake when its process IS the platform host. Until #3022 the gate ran as the TESTER
# image — whose /app is a strict subset of the portal's (measured on 3.0.0-rc9.ci.7534: 88 vs 219
# assemblies; MeshWeaver.Maps, .AI, .ContentCollections.Indexing, the Blazor and hosting halves exist
# assemblies; on that image MeshWeaver.Maps, .AI, .ContentCollections.Indexing — modules since — and the Blazor and hosting halves existed
# only in the portal) — so content binding a portal-shipped assembly could neither compile in the bake
# nor load in the gate, and every such failure read as a CONTENT error on source nobody had changed.
#
Expand All @@ -24,7 +24,7 @@
# tester's identity, which is the exact confusion this host exists to end;
# * the tester's mw-plugin-test.deps.json is NEVER copied — 🚨 load-bearing, not tidiness: with an
# app-local deps.json the dotnet host builds the TPA from THAT file's entries only, and every
# portal-only assembly (Maps…) would be on disk yet unloadable ("could not load file or assembly").
# portal-only assembly (a Blazor half…) would be on disk yet unloadable ("could not load file or assembly").
# Without it the host probes the application directory and every assembly in it is in the TPA
# (documented host behaviour: no deps.json ⇒ all assemblies in the app directory are added).
# The portal's own <Host>.deps.json stays; the runtime reads only <entry>.deps.json.
Expand All @@ -51,9 +51,9 @@ if [ "${1:-}" = "--self-test" ]; then
fail() { echo "SELF-TEST FAILED: $1"; exit 1; }
portal="$tmp/portal"; tester="$tmp/tester"; out="$tmp/host"
mkdir -p "$portal/modules/X" "$tester/razor-generators/linux-x64" "$tester/cs"
printf 'MeshWeaver.Layout=aaaa\nMeshWeaver.Maps=bbbb\n' > "$portal/meshweaver-surface.manifest"
printf 'MeshWeaver.Layout=aaaa\nMeshWeaver.PortalOnly=bbbb\n' > "$portal/meshweaver-surface.manifest"
printf 'PORTAL' > "$portal/MeshWeaver.Layout.dll"
printf 'PORTAL' > "$portal/MeshWeaver.Maps.dll"
printf 'PORTAL' > "$portal/MeshWeaver.PortalOnly.dll"
printf 'PORTAL' > "$portal/Newtonsoft.Json.dll"
printf '{}' > "$portal/Memex.Portal.Distributed.deps.json"
printf '{}' > "$portal/Memex.Portal.Distributed.runtimeconfig.json"
Expand All @@ -70,7 +70,7 @@ if [ "${1:-}" = "--self-test" ]; then

"$self" "$portal" "$tester" "$out" > "$tmp/log" || fail "the happy path exited non-zero: $(cat "$tmp/log")"
[ "$(cat "$out/Newtonsoft.Json.dll")" = "PORTAL" ] || fail "a file both images carry must keep the PORTAL's bytes"
[ "$(cat "$out/MeshWeaver.Maps.dll")" = "PORTAL" ] || fail "portal-only assemblies must be in the host"
[ "$(cat "$out/MeshWeaver.PortalOnly.dll")" = "PORTAL" ] || fail "portal-only assemblies must be in the host"
[ -f "$out/mw-plugin-test.dll" ] || fail "the tester CLI must be in the host"
[ -f "$out/MeshWeaver.Hosting.Monolith.dll" ] || fail "tester-only assemblies must ride"
[ -f "$out/razor-generators/linux-x64/Razor.dll" ] || fail "tester-only subdirectories must ride"
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/node-repo-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ name: Node repo gate (reusable)
# the portal's /app with the tester CLI beside it (compose-gate-host.sh) — started by the portal
# image's own `dotnet`. The tester's /app is a strict subset of the portal's (88 vs 219 assemblies
# on 3.0.0-rc9.ci.7534), so a gate compiling against the tester's /app reports every NodeType
# that binds a portal-only assembly (MeshWeaver.Maps, .AI, .Indexing…) as a CONTENT error.
# that binds a portal-only assembly (the Blazor and hosting halves; on ci.7534 also MeshWeaver.Maps
# and .AI, both modules since) as a CONTENT error.
# secrets:
# acr-username: ${{ secrets.ACR_USERNAME }}
# acr-password: ${{ secrets.ACR_PASSWORD }}
Expand Down
22 changes: 10 additions & 12 deletions memex/Memex.Portal.Shared/Memex.Portal.Shared.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,16 @@
deployment turns each on by listing its DLL, which is what that switch is for. Their
collocated JS and OpenStreetMap's vendored wwwroot/leaflet/ are already addressed as
./_content/<Pack>/... — exactly what the module static-asset provider serves (#1724). -->
<!-- 🚨 DIRECT ON PURPOSE, and it must not be "tidied away" as unused. MeshWeaver.Maps is a
CANONICAL CONTENT-SURFACE assembly (FrameworkBuildIdentity.ContentSurfaceAssemblies), and
until the view packs flipped (#1974) this host reached it only through them — every map
pack references it. Losing it from the COMPILE closure does not fail a build: the surface
manifest is written from @(ReferencePathWithRefAssemblies), so the line simply vanishes,
this host hashes Maps as 'absent' while mw-plugin-test still hashes a real surface id, and
the two hosts of ONE commit resolve DIFFERENT framework identities. Every bake then lands
at an address no portal reads — publication green, pods compiling everything at boot, every
cover serving a compilation-fallback card. That is the 2026-08-17 outage (#1814); it
recurred here the moment the map packs left, and
FrameworkBuildIdentityTest.CanonicalContentSurface_IsRecordedByEverySurfaceManifestHost
is what caught it. No portal code uses these types — the reference IS the point. -->
<!-- 🚨 MeshWeaver.Maps is NOT referenced, and must never be again (maintainer, 2026-09-03:
"all maps should be in plugins and removed from core — move 100% to plugins"). It WAS a
direct reference here while it was a canonical content-surface assembly (#1814 needed
every surface-manifest host to carry it, or two hosts of one commit resolved two
identities). It left the content surface and this repo (#2941) and is a Store MODULE in
MeshWeaver.Plugins — landed from the registry, composed into bakes with the tester's
module flag. A portal host that referenced it again became the SECOND PRODUCER of one
assembly: the bake recorded the module's build (mvid:…) while every portal carried the
app-closure copy (ref:…), and every map gallery was declined at adoption fleet-wide
(#3175). BakeHost refuses that double by name now; this comment is the other half. -->
<!-- MeshWeaver.Blazor.EntityViews is deliberately NOT referenced (the Analysis/#1974 shape):
it ships via the registry as a module, its EntityViewsViewPackModuleAttribute registers
the views when the DLL is listed under Modules:Assemblies, and Modules:Required declares
Expand Down
2 changes: 1 addition & 1 deletion src/MeshWeaver.Cli/BuildPluginCommand.cs
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ await error.WriteLineAsync(
+ "as --image). The tester image only EXECUTES; the portal supplies the reference "
+ "set, the framework identity and the runtime. There is no fallback to the tester's "
+ "/app: it is a strict subset of the portal's (88 vs 219 assemblies on "
+ "3.0.0-rc9.ci.7534), so content binding MeshWeaver.Maps, .AI or .Indexing fails to "
+ "3.0.0-rc9.ci.7534), so content binding a portal-only assembly fails to "
+ "compile against it (MeshWeaver#3022 / #3113).");
return 9;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,26 @@ So the gate moves to the deploy boundary:
This is a *deploy* gate, not a release gate. A broken candidate stops at whichever instances it would
break and rolls everywhere else.

### Availability is a CONSISTENT sealed set, not a present file (#3175)

The first gate a candidate meets — before the combo — is `ReleaseAvailabilityService`: does every
installed package have a sealed bake under the candidate's framework identity? Until 2026-09-03 that
question was answered by PRESENCE, and memex-cloud rolled to ci.7621 on it: every bundle was there,
and the portal then declined SocialMedia at adoption because its NodeTypes had been built against a
`MeshWeaver.Markdown.Collaboration` build the same identity's sealed module set did not carry.

**Available now means CONSISTENT**: every dependency record inside every installed package's sealed
bundle names module builds that ARE the module builds sealed for that identity, and no module is
sealed twice at different builds. Anything else is `SealedSetInconsistent` — a hold naming the
bundle, the NodeType, the module and both MVIDs — and "nothing goes" (maintainer, 2026-09-03: *"we
must have a clear confirmation that all plugins deployed to an instance are available for the
correct platform version; if not ⇒ nothing goes"*). An unreadable module set is `Indeterminate`,
which also holds; a false "available" rolls a fleet, a false "hold" freezes it, and the two are
told apart in the verdict on purpose. The mechanism and its limit — a module an instance installed
from the registry outside any publication is not visible to this check — are in
[ModuleBuildArchitecture](../ModuleBuildArchitecture) → "One producer per (module, framework
identity)".

### The unit of verification is the COMBO, and it includes TESTS

Two things are easy to get subtly wrong here, and both were wrong in earlier drafts of this page.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,67 @@ No component ever publishes from the mesh router — the router names a spokesma
| Module pack tool | 48–79s `dotnet build` × N jobs | ~5s download, built once |
| Orleans test suite | ~90 silo boots + disposal drain | **3 clusters** (the mesh pool; see WritingTests § The Mesh Pool) |

## One producer per (module, framework identity) — and the set is what is gated (#3175)

**A module's bytes have exactly one producer for a given framework identity, and every dependent in
a sealed publication was built against THAT build.** This is not a style preference; it is the
invariant the adoption contract already assumes, and on 2026-09-03 it was broken twice in one
morning, with no diff in any repo:

| where | what the record said | what the consumer held | outcome |
|---|---|---|---|
| every satellite gate (Reinsurance 33727661313, Manufacturing 33727661850) | `'MeshWeaver.Maps' built against mvid:4d04617…` | `live is ref:1D8FDE5B…` | 4 of 240 DECLINED, `GATE FAILED`, nothing sealed, memex-cloud `HOLDING` |
| memex.meshweaver.cloud on ci.7621 | `'MeshWeaver.Markdown.Collaboration' built against mvid:A` | `live is mvid:B` | SocialMedia adopted 0/4, `/Posts` rendered empty (#3174) |

The first row is a **second producer in space**: a portal host had taken a direct project reference
to `MeshWeaver.Maps`, a Store module. The bake composed Maps with `--module` — the id resolver puts
modules first, so every record said `mvid:` — while every portal and every gate host carried the
app-closure copy and resolved the name from its surface manifest as `ref:`. Two schemes, never
equal, every map gallery declined. The maintainer's ruling closes it structurally: *"all maps should
be in plugins and removed from core — move 100% to plugins."* The portal host references no module
project; a module is landed from the registry and composed into bakes, nowhere else.

The second row is a **second producer in time**: core CD's `plugins-modules` rebuilds
`MeshWeaver.Markdown.Collaboration` per platform release to feed the Plugins seal (mvid A), while
the registry's package endpoint serves the Plugins lane's last content-versioned publication (mvid
B) — the bytes a portal actually installs. This row is NOT closed by this change (core holds no
registry credential, so its bake cannot compose the registry's bytes); what changed is that the
availability gate now sees it (below) instead of rolling onto it.

### The two controls

1. **The bake refuses a double by name** (`BakeHost.ShippedByHostProblem`, run by `compile` /
`--bake-output` under both `--app` and in-process). A module composed with `--module` whose
simple name the host also ships in its application directory — or lists in its surface manifest —
fails the bake RED: *"two builds of one assembly name in one bake … remove the assembly from the
platform host's closure or stop composing it."* Caught where both provenances are in one hand,
never sealed and discovered at the fleet. Pinned by
`BakeAgainstPlatformHostTest.AComposedModuleTheHostAlsoShips_IsRefusedByName`.
2. **Availability asserts the SET** (`ReleaseAvailability.IsUpdatable` over the observation
`PublishedBundleCatalogue.ArtifactsForIdentity` makes). For the candidate identity the registry
reads every complete source's sealed module set — each module bundle's entry assembly, PE header
only, for its MVID — and every sealed bundle's per-NodeType dependency record (manifest only,
never assembly bytes). Then: every `mvid:` entry naming a module the set carries must equal the
MVID sealed for it, and no module may be sealed at two MVIDs by two sources. A violation is
`PackageAvailabilityKind.SealedSetInconsistent` — a HOLD that names the bundle, the NodeType, the
module and both builds — and it is consumed unchanged by the self-update poll, CD's post-promote
assertion and `/api/plugins/is-updatable`, because all three call the same predicate. Both
failure directions are pinned (`SealedSetConsistencyTest`): an unreadable or pre-module-sealing
module set is `Indeterminate` (hold, named — never "compatible"), and a module the set does not
carry is not judged, because the gate cannot see the bytes a registry install landed.

### What the gate still cannot see

The registry's package endpoint (`/api/plugins/bundles/<pkg>/<version>`) is content-versioned
(Plugins #931): an instance that installed a module from it holds whatever that lane published last,
and a platform release that rebuilds the module for the seal produces a *consistent* sealed set whose
module MVID differs from the instance's landed one. The set check passes; the instance still
declines. Closing that needs ONE producer in time as well — either the seal composes the registry's
bytes for the root source (`registry-modules` in core CD, which needs a registry credential core
does not hold today), or the instance adopts module bytes for its identity from the sealed
publication it already adopts bundles from. Until one of those lands, `ShippedPrebuiltBundles`'
`dependency record mismatch … live is mvid:` line on a portal is that gap, not a new defect.

## Adoption contract (every repo)

1. Pin the reusable lanes (`node-repo-*.yml`) at a MeshWeaver main SHA — never copy them.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
Name: An update is held when its packages do not fit together
Category: Fix
Description: Before your installation moves to a new platform build, it checks that every installed package has been prepared for that build. It was only checking that the prepared packages existed — not that they had all been prepared against the same pieces — so an update could go through and then leave a package's pages empty.
Icon: ShieldCheckmark
Order: -20260903
---

# An update is held when its packages do not fit together

Every package you install carries pages and views that are compiled ahead of time, for the exact
platform build your installation runs. Before your installation updates itself, it asks whether that
preparation exists for the new build — and refuses the update if a package would otherwise have to
be compiled from scratch at start-up, which is slow and can fail.

That check had a gap. It confirmed that each package **had been prepared**, but not that the
packages had been prepared **against the same pieces**. Some packages build on others — a social
feed on the collaboration tools, a map gallery on the map control — and when the piece a package was
prepared against is not the piece your installation actually receives, the preparation is unusable.
Your installation notices this only after the update, when it quietly discards the prepared package
and the pages that depend on it come up empty or slow. That is what happened to the **Posts** page
on one portal on 2 September.

**The check now looks at the whole set.** For the candidate build it reads what each prepared
package was built against and compares it with what is actually being shipped for that build. If any
package does not fit, the update is **held**, and the **Updates** settings tab names the package, the
piece it depends on, and the two versions that disagree — so the mismatch is fixed in the build
pipeline instead of being discovered on your pages.

The same morning, the map galleries (Apple Maps, Google Maps, OpenStreetMap and the Cornerstone
pricing map) were failing this fit test everywhere, because the map control was being shipped
twice — once inside the platform and once as its own package. The map control now ships only as a
package, and the build refuses to prepare a package against a piece that is shipped both ways.
Loading
Loading