docs: the self-update schema wall, and why a promoted tag is not a deployable tag

[rbuergi]

Why

Two findings from today's fleet work were recorded only in a GitHub issue thread and session notes.
The fleet-wide conserve work products rule says the durable form is a committed doc page — this is
that page, plus the cross-links that route a reader to it.

Documentation only. No product code changes.

Finding 1 — every schema-bumping release is un-takeable by self-update, BY CONSTRUCTION

Tracking issue: #3207 (measured on the memex AKS portal, 2026-09-03).

• Self-update patches the portal image and nothing else. The database migration is a run-once
  Job that only helm upgrade mints. So a build that bumps the schema walks past it,
  DbVersionGate correctly refuses to serve, and the install wedges.
• It is silent. Available=True keeps the old ReplicaSet serving while the new one never
  reaches Ready (Progressing=False ProgressDeadlineExceeded). Three tags wedged between 07:20Z and
  10:27Z, the last on a 5-minute back-off at 14 restarts, and nothing alerted — it surfaced only
  because an unrelated config-drift audit looked at the ReplicaSets.
• It is structural, not a bad build. An install rolls forward until it meets its first
  schema-bumping release and stops there. Clearing one instance at one tag clears today and re-arms
  for the next bump. memex-cloud is behind the same wall at ci.7621 — not configured
  differently, just not arrived yet.
• Why it is consequential: since ci: core publishes ONE event and finishes — withdraw notify-dependents; the release wave is memex's, subscribers are the Deployment records #3185 the release wave belongs to memex, the control instance
  that registers releases and broadcasts meshweaver-framework-released. The wedge does not stop the
  control plane (the old pods keep serving), but it lets the control instance freeze arbitrarily far
  behind the platform it announces — and no change to the control plane's own machinery shipped in a
  schema-bumping release can reach it without an operator.
• The durable remedy is UNDECIDED and is the maintainer's call. The page states the three
  candidate options — (a) self-update triggers the helm run at a schema boundary, (b) schema-bumping
  releases are flagged so an instance HOLDS and asks, (c) status quo, operator-run — with what each
  trades, and deliberately does not pick one. It also records the two standing constraints any
  answer must respect (rolls go through CD; fail-closed stays) and that memex cannot self-update: the portal image is ahead of its schema (db_version 54 < 55) and only helm upgrade can run the migration Job #3207's own first remedy
  proposal was withdrawn by its author for contradicting the first of them.

Finding 2 — a promoted tag is not a deployable tag; the SEAL is the discriminator

Measured on core CD runs 33746020109 (ci.7669) and 33749847612 (ci.7674):

Promote: tag the full set (all-or-nothing)                 success
Verify every image shipped                                 success
Plugins: bake + seal the publication for this identity     FAILURE   <- the one that binds
  |_ Register the publication with memex                   skipped

Both seals FATALed on the one-producer guard (#3175), so both tags are platform-present,
plugin-modules-absent for their framework identity — exactly the half-broken state the standing
availability rule forbids. A peer session nearly walked #3207's helm remedy onto ci.7674 having
checked only that three platform images existed. Both checks that sound like "this build ships"
were green. Image COUNT was never the right question. The skipped register step is the tell: a
failed seal never registers the publication with memex, so the availability predicate has no record
to read either.

The two findings are operationally inseparable — whatever remedy is chosen for finding 1, the target
it picks has to satisfy finding 2 — so the page carries the three-condition checklist (ancestry of
the fix, a memex-migration image at the SAME tag, a green seal) next to the wall itself.

What changed

File	Change
Data/Architecture/SelfUpdateSchemaWall.md	New. The mechanism in four lines · why it is invisible · why it is structural · the control instance stands on it too · what makes a tag a safe target · the open decision · See also
Data/Architecture/DeploymentAKS.md	Two additions inside "Migration under self-update": the fleet-level framing + open decision, and the safe-target trap next to the "a hand roll is not the fix" paragraph. The runbook detail already there is not duplicated
Data/Architecture/ContinuousDeliveryContract.md	New subsection "A promoted tag is not a deployable tag — the seal is the discriminator" under the standing trap (finding 2's rule lives here, where "verify the IMAGE, never the tick" already does), plus a See also entry
Data/Architecture/Deployment.md	One router row
Data/Architecture.md	Topic map entry under Deployment & ops

One correction included: ContinuousDeliveryContract.md said the roll "patches the portal and
migration Deployments". That has been false since #2797 (a guard in MeshWeaver.Plugins fails the
build if a second PATCH returns), it contradicted DeploymentAKS.md, and it is exactly the sentence
that sends an investigation looking for a migration leg that does not exist.

No What's New entry — internal architecture/ops documentation, nothing a user can notice.

Pairs-with: none — documentation only

Verification

• dotnet build src/MeshWeaver.Documentation/MeshWeaver.Documentation.csproj -c Release -warnaserror
  → 0 Warning(s) / 0 Error(s)
• dotnet build test/MeshWeaver.Documentation.Test/MeshWeaver.Documentation.Test.csproj -c Release -warnaserror
  → 0 Warning(s) / 0 Error(s)
• Rebuilt (docs ship as EmbeddedResource) then ran the WHOLE project, not a filter:
  Failed: 0, Passed: 240, Skipped: 0, Total: 240 — .trx written 12:45:50Z against a run started
  12:45:34Z. DocumentationLinkIntegrityTest and MigrationWorkloadModelGuard (which scans this
  doc tree for migration-as-a-Deployment commands and for "the crash loop is normal" prose) are both
  in that set.

🤖 Generated with Claude Code
Claude-Session: https://claude.ai/code/session_01G4xPnGYEbdu8AVvR5jytyj

[Copilot]

🟡 Changes recommended

There is at least one verified documentation defect (a PR reference links to an issue URL) that should be corrected before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds new operational documentation describing the “self-update schema wall” (why schema-bumping releases cannot be taken by pull-based self-update), and documents why a promoted tag is not necessarily deployable unless the plugin “seal” succeeded—then wires this into the existing deployment/CD docs for discoverability.

Changes:

• Add a new architecture page documenting the self-update schema boundary, its failure visibility characteristics, and a 3-condition checklist for selecting a safe helm upgrade target tag.
• Update AKS deployment and CD contract docs to reference the schema wall and correct the previously incorrect statement that self-update patches a migration Deployment.
• Update the deployment router/topic map to include the new page under Deployment & ops.

File summaries

File	Description
src/MeshWeaver.Documentation/Data/Architecture/SelfUpdateSchemaWall.md	New doc page describing the schema wall and deployable-tag checklist.
src/MeshWeaver.Documentation/Data/Architecture/DeploymentAKS.md	Adds fleet-level framing + links to the new schema wall page and safe-target trap.
src/MeshWeaver.Documentation/Data/Architecture/Deployment.md	Adds a router row pointing readers to the new schema wall page.
src/MeshWeaver.Documentation/Data/Architecture/ContinuousDeliveryContract.md	Corrects self-update scope (portal-only) and adds a subsection about seal being the deployability discriminator.
src/MeshWeaver.Documentation/Data/Architecture.md	Adds topic-map entry for the new schema wall page under Deployment & ops.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Test Results (shard 3)

341 tests   339 ✅  40s ⏱️
  3 suites    2 💤
  3 files      0 ❌

Results for commit 5d909d1.

[github-actions]

Test Results (shard 0)

243 tests   243 ✅  2m 5s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 5d909d1.

[github-actions]

Test Results (shard 1)

459 tests   459 ✅  1m 0s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 5d909d1.

[github-actions]

Test Results (shard 5)

    4 files      4 suites   1m 33s ⏱️
1 349 tests 1 349 ✅ 0 💤 0 ❌
1 350 runs  1 350 ✅ 0 💤 0 ❌

Results for commit 5d909d1.

[github-actions]

Test Results (shard 4)

2 201 tests   2 201 ✅  2m 15s ⏱️
    3 suites      0 💤
    3 files        0 ❌

Results for commit 5d909d1.

[github-actions]

Test Results (shard 2)

578 tests   386 ✅  2m 38s ⏱️
  3 suites  192 💤
  3 files      0 ❌

Results for commit 5d909d1.

[github-actions]

Test Results

   15 files     15 suites   10m 14s ⏱️
5 171 tests 4 977 ✅ 194 💤 0 ❌
5 172 runs  4 978 ✅ 194 💤 0 ❌

Results for commit 5d909d1.