You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
No platform pins in module repos — build against the released platform, and be TOLERANT of drift (same major serves) #3842
Stated after the 2026-09-09 incident on memex.systemorph.com (#3583, comments of today): a one-line CSS merge in Plugins (#1555) left every Essentials/Email page dead for hours. Part of the chain was a pin:
12:41:44Z Plugins main, push run for 4c4019b46: FAILURE — "Repo policy gates: 2 staleness problem(s)
in .github/workflows/ci.yml" — the hand-maintained MW_PLATFORM_SET / MW_PLATFORM_REF pin
went stale the moment core released a new framework; every later run on main failed the
same way until Plugins#1560 re-pinned to 3.0.0-ci.8195 at 15:30Z.
So a core release turns every module repo's main red until a human bumps a number, and while it is red the registry still serves HEAD to live portals. Meanwhile the portal itself runs a THIRD version (3.0.0-ci.8057, s414bfb2…), which no pin anywhere names — see also #3768.
What "no pins, tolerant" means concretely
Module repos do not carry a platform pin. The build lane resolves the platform to build against at run time — the latest released (sealed) set, and/or the identities the fleet actually runs (Hosting__Deployment / Hosting__ReportTo know them). No MW_PLATFORM_SET, no MW_PLATFORM_REF, no uses: …@<sha> lane refs that a policy gate has to police for staleness. The staleness gate itself then has nothing to check and goes away.
A red or cancelled main is invisible to portals. The registry serves the last green set, never HEAD.
Readiness is announced. When a module version is actually adopted on a portal, the portal says so (notification + Updates page). Today the only signal between "merged" and "page dead" is the dead page.
Where the pins live today
MeshWeaver.Plugins/.github/workflows/ci.yml: MW_PLATFORM_REF, MW_PLATFORM_SET, MW_IMAGE_DIGEST, and the uses: …@<sha> lane refs; scripts/check-platform-pins.py and the "Repo policy gates / Neither frozen platform input is stale" step that failed today.
Every satellite repo (Crm, Reinsurance, Education, SocialMedia, Manufacturing) fetches the platform's compile-check.py at a pinned ref and carries the same policy inputs, per the "unified build process" doctrine in their AGENTS.md — that doctrine needs the same change.
Refs: #3583 (sources ahead of bundles; fallback requirement), #3768 (identity mismatch between registry and consumer), #2813 (the fingerprint gate).
Retention adaptation is in #3843. The bundle sweep now uses at least 30 days rather than a count quota, preserves the latest sealed publication per source for every represented major plus reported/adopted consumers, and aborts cleanup on unresolved consumer references. 41 focused tests pass; removing age protection makes the two targeted regressions fail. The support policy and retention docs now explicitly supersede the old pin doctrine.
Integration still required with the publication/CI work here: expose the advertised last-green released-set inventory and active build consumption records to protection before publication/consumption, ordered with cleanup. Runtime consumer reports must retain a serving fallback until adoption actually changes; readiness notifications alone cannot retire it. Removing module pins must not release legacy locks or turn an empty pin scan into proof of protection. The local sealed-publication rule is not a substitute for the registry's advertised last-green set.
No live cloud purge settings or credentials were changed. #3843 deliberately does not close #3842 or #3438; the shared inventory and cleanup cutover remain outstanding.
Retention follow-up now has a paired consumer-inventory fix: Systemorph/MeshWeaver.Plugins#1564 preserves optional adopted framework identities and explicit completeness in the existing report receiver; core #3849 reads local NodeType adoption stamps, sends them, and protects the remote identities in retention. Core #3849 stays draft until the receiver is merged and available.
The real-mesh regression proves a portal reports and persists older adopted X alongside running Y; 17 focused tests pass. The receiver has 22 passing tests, including a negative control that fails when identity preservation is removed. Both halves build cleanly and the core documentation suite passes all 368 tests.
#3843 now has green PR CI after correcting the documentation index; #3846 is merged with green post-merge checks. These changes do not arm live deletion. Fleet coverage/freshness and active publisher/consumer coordination are still outstanding under #3438; this does not claim the full cleanup protocol or runtime de-pinning is finished.
Current evidence for the readiness/announcement part of this issue: core CD 8216 (34389781884, source 25a03d2) announced a platform whose Plugins publication was still being baked. In the shipped workflow, notify-platform-update needs promote but not plugins-bake, so an image announcement can precede the complete released artifact set (workflow).
Reinsurance dispatch 34392133001 failed its upstream gate at 19:08:57 UTC: both plugins and crm were unavailable for identity s16349850bbe0df16ecefec772e9c8c30. The actual Plugins seal for that identity was written to both targets at 19:21:15 by core job 102606052000; core CD completed at 19:21:48. Reinsurance's next attempt 34392145204 then recognized Plugins and failed only on missing CRM at 19:23:35.
These are correctly failing availability gates, not runtime-test regressions. The evidence distinguishes an image-published notification from readiness of the full required artifact set. The solution should preserve that distinction and dependency ordering; weakening the gates or manually rerunning against absent upstreams would conceal it. No notification ordering or publication policy was changed in this investigation.
Maintenance coordination: the assessment task is taking the bounded Plugins publish-bake dependency repair (publisher currently starts despite failed portal-hosts aggregate), with regression coverage. No runtime policy, credential, permission or production configuration changes are included. A separate earlier module-pack upload path may expose registry bytes before final validation; changing that sequencing needs its own scope assessment. Please coordinate any overlapping workflow edits here before proceeding. This does not claim the full red-main invisibility rule is implemented.
Confirmed publication-gate violation, now covered by the bounded caller repair in Systemorph/MeshWeaver.Plugins#1583.
Plugins push run 34414173231 had a failed required host aggregate, yet publisher job 102684899327 replaced both targets for framework identity se5d2c163b0d0c1469c5e7af69806b7a8. It verified 45/45 files as its own bytes and sealed at 23:54:20 / 23:54:37 UTC on September 9, from source b078f1bafb1e621845046d539a682a3e0564a76d; publication token Systemorph-MeshWeaver.Plugins-34414173231-1, targets-published=2. This replaced the earlier green core publication. By contrast, publisher 102687384576 in run 34412991750 was an idempotent skip: its success alone did not mean it wrote anything.
PR #1583 adds portal-hosts-gate to the sealed publisher's prerequisites and preserves implicit success(). The existing repository guard has 23 passing mutation regressions; review corrections address valid five-space YAML continuations and remove a duplicate test invocation. The repair changes neither stale-execution policy nor credentials/configuration.
This does not close #3842: individual module-pack jobs can hand bundles to the registry before all repository gates finish, and core main-CD publishes Plugins through its own independent caller. Those paths need their own last-green enforcement. The separate missing-sentinel read race is tracked in #3876.
Follow-up split: #3878 covers the earlier module registry hand-over inside node-repo-module-pack.yml. Packing must stay ahead of compile/runtime validation because those jobs consume its artifacts; only live registry publication moves after the full validation verdict. This is separate from Plugins #1583 and does not claim to solve publication atomicity or the independent core-CD publisher.
Reopened — the 2026-09-10 sweep was not a triage decision
This issue was closed not_planned at 2026-09-10T14:30:29Z by a bulk sweep that PATCHed every open issue in all 13 Systemorph repositories — 311 issues in twelve minutes, no comment on any of them — after a session was asked to "close all issues in all repos". Nothing on this thread was read, and no judgement was made about this issue.
All 82 issues the sweep closed in the MeshWeaver fleet were reviewed afterwards. This one is unresolved: Platform-owner requirement, adoption in flight — MeshWeaver.Crm#81 is still open.
Sixteen of the 82 were left closed — their fix had actually landed, or the author had withdrawn them; those should have read completed, not not planned.
resolves the newest sealed set at run time (resolve-platform.py)
yes
yes
yes
yes
yes
yes
The one remaining reference is FREEZE: ${{ vars.MW_PLATFORM_REF }} in each — the incident-freeze repository VARIABLE the resolver honours, by design (Plugins#1565). PlatformPinUpdater found nothing to move and now sits behind the same switch as the wave (Plugins#1707). The staleness gate that reddened Plugins on 2026-09-09 is gone with the pins. Crm#81 (the last adoption) merged 2026-09-12T06:25Z.
2. Compatibility is a version rule — DELIVERED for the platform surface, and the third-party hole is closed
Same-MAJOR serves: BuildProvenance.StaleAdopted keeps the adopted build serving with an honest status (#3844); the resolver's contract and the process page state the rule (Plugins Hosting/BuildAndReleaseProcess rule 1). The measured hole in the assumption (#4083, the YamlDotNet crash loop) is closed by #4091 (merged 2026-09-12T13:08Z): the link probe compares assembly versions, roll-forward tolerated, roll-back a hard verdict, FileLoadException has a state.
3. A red or cancelled main is invisible to portals — DELIVERED in the shape the maintainer chose
The sealed NodeType publication of Plugins is gated on the whole validation set (publish-bakeneeds: […, test-repos, tests-ratchet, modules-floor, portal-hosts-gate], Plugins#1583). Module bundles are gated on their own suite — per-module deploy, the maintainer's 2026-09-11 directive (Plugins#1676, core #4044/#4062), which reversed the whole-run hand-over of #3878 nine hours after it landed; #3878 is closed on that basis with the production measurement. Core's plugins-bake bakes Plugins main HEAD (main-cd.yml:653-678, git ls-remote refs/heads/main) and seals only if its own bake gate (every NodeType compiled, Tests areas executed in the tester) is green — validation by the publication's own unit, which is the doctrine now.
4. Readiness is announced — PARTIAL, and owned elsewhere
BuildDeliveryHold.EventOf/Notify announces a NodeType's adoption on a hold transition into the Admin/_Notification bell (#3844). The package-level state ("X vN adopted for identity I on this replica") is unbuilt; the scope question is recorded on Doc/Architecture/FrameworkIdentityChurn and tracked as #3583's requirement 3 — not re-opened here.
Two residuals, recorded, not blocking
The resolver is carried as a per-repo copy, and the copies drift: seven copies at six distinct sizes on 2026-09-13 (core 68,150 B; Crm 69,027; Reinsurance 69,035; SocialMedia 69,035; Manufacturing 69,037; Plugins 60,365; Education 46,786), no drift guard. The lanes fetch core's copy for their own resolution, so the copies decide only what a satellite's own jobs resolve. This is the exact concern raised on Plugins#1565's first comment, and Plugins#1565 stays open narrowed to it. Recorded on Doc/Architecture/TheReleaseWave (PR below).
And one caveat on what "delivered" means today: since 2026-09-12T18:11Z every satellite's publish-bake is red at a preflight on the unprovisioned MW_REGISTRY_PUBLISHER_PASSWORD (#4142) — pins have nothing to do with it, but until it is provisioned the no-pin lanes bake nothing.
Read-only: REST contents reads of six ci.ymls, code at cdb2878bb. No secret, registry or mesh touched.
Activity
rbuergi commentedon Sep 9, 2026
Retention adaptation is in #3843. The bundle sweep now uses at least 30 days rather than a count quota, preserves the latest sealed publication per source for every represented major plus reported/adopted consumers, and aborts cleanup on unresolved consumer references. 41 focused tests pass; removing age protection makes the two targeted regressions fail. The support policy and retention docs now explicitly supersede the old pin doctrine.
Integration still required with the publication/CI work here: expose the advertised last-green released-set inventory and active build consumption records to protection before publication/consumption, ordered with cleanup. Runtime consumer reports must retain a serving fallback until adoption actually changes; readiness notifications alone cannot retire it. Removing module pins must not release legacy locks or turn an empty pin scan into proof of protection. The local sealed-publication rule is not a substitute for the registry's advertised last-green set.
No live cloud purge settings or credentials were changed. #3843 deliberately does not close #3842 or #3438; the shared inventory and cleanup cutover remain outstanding.
rbuergi commentedon Sep 9, 2026
Retention follow-up now has a paired consumer-inventory fix: Systemorph/MeshWeaver.Plugins#1564 preserves optional adopted framework identities and explicit completeness in the existing report receiver; core #3849 reads local NodeType adoption stamps, sends them, and protects the remote identities in retention. Core #3849 stays draft until the receiver is merged and available.
The real-mesh regression proves a portal reports and persists older adopted X alongside running Y; 17 focused tests pass. The receiver has 22 passing tests, including a negative control that fails when identity preservation is removed. Both halves build cleanly and the core documentation suite passes all 368 tests.
#3843 now has green PR CI after correcting the documentation index; #3846 is merged with green post-merge checks. These changes do not arm live deletion. Fleet coverage/freshness and active publisher/consumer coordination are still outstanding under #3438; this does not claim the full cleanup protocol or runtime de-pinning is finished.
rbuergi commentedon Sep 9, 2026
Current evidence for the readiness/announcement part of this issue: core CD 8216 (
34389781884, source25a03d2) announced a platform whose Plugins publication was still being baked. In the shipped workflow,notify-platform-updateneedspromotebut notplugins-bake, so an image announcement can precede the complete released artifact set (workflow).Reinsurance dispatch 34392133001 failed its upstream gate at 19:08:57 UTC: both
pluginsandcrmwere unavailable for identitys16349850bbe0df16ecefec772e9c8c30. The actual Plugins seal for that identity was written to both targets at 19:21:15 by core job102606052000; core CD completed at 19:21:48. Reinsurance's next attempt 34392145204 then recognized Plugins and failed only on missing CRM at 19:23:35.These are correctly failing availability gates, not runtime-test regressions. The evidence distinguishes an image-published notification from readiness of the full required artifact set. The solution should preserve that distinction and dependency ordering; weakening the gates or manually rerunning against absent upstreams would conceal it. No notification ordering or publication policy was changed in this investigation.
rbuergi commentedon Sep 9, 2026
Maintenance coordination: the assessment task is taking the bounded Plugins publish-bake dependency repair (publisher currently starts despite failed portal-hosts aggregate), with regression coverage. No runtime policy, credential, permission or production configuration changes are included. A separate earlier module-pack upload path may expose registry bytes before final validation; changing that sequencing needs its own scope assessment. Please coordinate any overlapping workflow edits here before proceeding. This does not claim the full red-main invisibility rule is implemented.
rbuergi commentedon Sep 10, 2026
Confirmed publication-gate violation, now covered by the bounded caller repair in Systemorph/MeshWeaver.Plugins#1583.
Plugins push run 34414173231 had a failed required host aggregate, yet publisher job 102684899327 replaced both targets for framework identity
se5d2c163b0d0c1469c5e7af69806b7a8. It verified 45/45 files as its own bytes and sealed at 23:54:20 / 23:54:37 UTC on September 9, from sourceb078f1bafb1e621845046d539a682a3e0564a76d; publication tokenSystemorph-MeshWeaver.Plugins-34414173231-1,targets-published=2. This replaced the earlier green core publication. By contrast, publisher 102687384576 in run 34412991750 was an idempotent skip: its success alone did not mean it wrote anything.PR #1583 adds
portal-hosts-gateto the sealed publisher's prerequisites and preserves implicitsuccess(). The existing repository guard has 23 passing mutation regressions; review corrections address valid five-space YAML continuations and remove a duplicate test invocation. The repair changes neither stale-execution policy nor credentials/configuration.Durable evidence: BuildReleaseSeparation.
This does not close #3842: individual module-pack jobs can hand bundles to the registry before all repository gates finish, and core main-CD publishes Plugins through its own independent caller. Those paths need their own last-green enforcement. The separate missing-sentinel read race is tracked in #3876.
rbuergi commentedon Sep 10, 2026
Follow-up split: #3878 covers the earlier module registry hand-over inside
node-repo-module-pack.yml. Packing must stay ahead of compile/runtime validation because those jobs consume its artifacts; only live registry publication moves after the full validation verdict. This is separate from Plugins #1583 and does not claim to solve publication atomicity or the independent core-CD publisher.rbuergi commentedon Sep 10, 2026
Reopened — the 2026-09-10 sweep was not a triage decision
This issue was closed
not_plannedat 2026-09-10T14:30:29Z by a bulk sweep that PATCHed every open issue in all 13Systemorphrepositories — 311 issues in twelve minutes, no comment on any of them — after a session was asked to "close all issues in all repos". Nothing on this thread was read, and no judgement was made about this issue.All 82 issues the sweep closed in the MeshWeaver fleet were reviewed afterwards. This one is unresolved: Platform-owner requirement, adoption in flight — MeshWeaver.Crm#81 is still open.
Sixteen of the 82 were left closed — their fix had actually landed, or the author had withdrawn them; those should have read
completed, notnot planned.rbuergi commentedon Sep 13, 2026
Closing — the four requirements, verified on every repository's
mainon 2026-09-13; the two residuals have owners1. No platform pins in module repos — DELIVERED
Audited
.github/workflows/ci.ymlonmainof all six (Plugins, Crm, Reinsurance, Education, SocialMedia, Manufacturing), uncommented lines only:MW_PLATFORM_REF/MW_PLATFORM_SET/MW_IMAGE_DIGEST/MW_PORTAL_IMAGE_DIGESTliteralsuses: Systemorph/MeshWeaver/…@<sha>check-platform-pins.pyinvokedresolve-platform.py)The one remaining reference is
FREEZE: ${{ vars.MW_PLATFORM_REF }}in each — the incident-freeze repository VARIABLE the resolver honours, by design (Plugins#1565).PlatformPinUpdaterfound nothing to move and now sits behind the same switch as the wave (Plugins#1707). The staleness gate that reddened Plugins on 2026-09-09 is gone with the pins. Crm#81 (the last adoption) merged 2026-09-12T06:25Z.2. Compatibility is a version rule — DELIVERED for the platform surface, and the third-party hole is closed
Same-MAJOR serves:
BuildProvenance.StaleAdoptedkeeps the adopted build serving with an honest status (#3844); the resolver's contract and the process page state the rule (PluginsHosting/BuildAndReleaseProcessrule 1). The measured hole in the assumption (#4083, the YamlDotNet crash loop) is closed by #4091 (merged 2026-09-12T13:08Z): the link probe compares assembly versions, roll-forward tolerated, roll-back a hard verdict,FileLoadExceptionhas a state.3. A red or cancelled main is invisible to portals — DELIVERED in the shape the maintainer chose
The sealed NodeType publication of Plugins is gated on the whole validation set (
publish-bakeneeds: […, test-repos, tests-ratchet, modules-floor, portal-hosts-gate], Plugins#1583). Module bundles are gated on their own suite — per-module deploy, the maintainer's 2026-09-11 directive (Plugins#1676, core #4044/#4062), which reversed the whole-run hand-over of #3878 nine hours after it landed; #3878 is closed on that basis with the production measurement. Core'splugins-bakebakes PluginsmainHEAD (main-cd.yml:653-678,git ls-remote refs/heads/main) and seals only if its own bake gate (every NodeType compiled, Tests areas executed in the tester) is green — validation by the publication's own unit, which is the doctrine now.4. Readiness is announced — PARTIAL, and owned elsewhere
BuildDeliveryHold.EventOf/Notifyannounces a NodeType's adoption on a hold transition into theAdmin/_Notificationbell (#3844). The package-level state ("X vN adopted for identity I on this replica") is unbuilt; the scope question is recorded onDoc/Architecture/FrameworkIdentityChurnand tracked as #3583's requirement 3 — not re-opened here.Two residuals, recorded, not blocking
Doc/Architecture/TheReleaseWave(PR below).pluginsregistration wakes every satellite viaupstream-published, ~23 runs per satellite per day) — Roland's decision, recorded on the same page and on Bundles are baked for ONE framework identity per run, so a portal's GitSynced sources stay days behind main (or its types go unadopted) — options 1-4 undecided #3583; not a pin.And one caveat on what "delivered" means today: since 2026-09-12T18:11Z every satellite's publish-bake is red at a preflight on the unprovisioned
MW_REGISTRY_PUBLISHER_PASSWORD(#4142) — pins have nothing to do with it, but until it is provisioned the no-pin lanes bake nothing.Read-only: REST
contentsreads of sixci.ymls, code atcdb2878bb. No secret, registry or mesh touched.PR: #4152