Repository navigation
Module activation entry survives without its landed assemblies — EntityViews views silently absent on a booted pod #2303
Description
What is failing
The module activation reconciliation in MeshWeaver.PluginCatalog (ModuleActivation) found a module the deployment considers ACTIVATED whose landed assemblies are not present in the process: MeshWeaver.Blazor.EntityViews. The activation record and the bytes it points at have diverged, so everything that module contributes — the entity form/edit controls and the Editor / EditForm / Property skin views — is absent on this pod, and no restart will bring it back. The reporter itself is behaving correctly; the defect is in the landing/activation lane it is reporting on.
Probable cause
Moderate confidence on the mechanism, low confidence on which of two paths produced it. The evidence is a single boot-time reconcile line, with no exception and no stack frame, so nothing here proves the write path — but the shape matches the documented "Missing DLL" skip rule in Modules: the entry's modules/<name>/<name>.dll does not exist, so boot skips it loudly and keeps the entry in the sidecar.
What makes this a defect rather than a lost volume: modules/activation.json is a sidecar written beside the module folders, on the same volume, in the same operation, specifically so the two cannot drift. A wiped volume would take the entry with it and this state would be unreachable. That it is reachable points at the entry and the placement not being atomic in practice. Two candidate paths, both in scope:
- Landing wrote the activation entry for a placement that did not complete, or recorded a folder the bytes were not written into — the same failure class as the
ResolveModulePath/ folder-name mismatch fixed on 2026-08-20 ("Installed modules load again after a restart"). - The auto-update pass replaced the entry and lost the bytes.
Plugins/EntityViewscarriesautoUpdate: trueand was re-installed at2026-08-25T13:48Z(version1.1.0,moduleVersion cf663e18a178094b); this fired ~11 h later at a pod boot.RegistryUpdateReconciler's module pass /ModuleUpdateDecisionre-pointing an entry at a landing that never placed would produce exactly this.
Worth checking as part of the same fix: readiness did not catch it. EntityViews is one of the five view packs made optional in Modules:Required on 2026-08-23, so the pod passed its health gate and took traffic with the pack missing. A module that is ACTIVATED-but-absent is a different condition from one that is deliberately not required, and the readiness gate currently cannot tell them apart.
Impact
One pod (memex-portal-deployment-…-z8w2k, memex-cloud), one occurrence — but the count understates it: this is a once-per-process boot diagnostic, so the single line covers that pod's entire lifetime, and the message states the consequence itself — contributed endpoints, views and providers answer as if they were never installed (a 404 or an area-not-found placeholder, no error). Users routed to that pod get blank or placeholder entity edit forms wherever an area returns an EntityViews control; anyone landing on another pod sees the feature work, which makes it read as intermittent. Not an outage; a silent partial feature loss that persists until someone re-installs the package.
Where to look
ModuleLandingService— the entry write vs. byte placement ordering; make the activation entry conditional on a verified placement.MeshBuilder.ResolveModulePathand the boot skip rules — confirm the probed path matches the folder landing actually wrote.RegistryUpdateReconciler's module pass andModuleUpdateDecision— the auto-update path, givenautoUpdate: trueon this package.- The reconcile/health seam in
MeshWeaver.PluginCatalog— consider failing readiness on ACTIVATED-but-absent regardless ofModules:Required. - Context: Doc/Architecture/Modules (activation lanes and skip rules), the package manifest at Plugins/EntityViews.
Prior art
Not a duplicate — no existing incident or issue node covers this component. It is, however, the fourth reported failure in this family (Doc/WhatsNew fixes dated 2026-08-20 ×2 and 2026-08-23), all of them "an installed module silently did not switch on". Each was fixed at a different seam; a maintainer may prefer to treat the invariant — an activation entry must never outlive its bytes — rather than this instance.
Evidence
| Fingerprint | 381b930684c9dfb8 |
| Category | MeshWeaver.PluginCatalog.ModuleActivation |
| Severity | Error |
| Namespace | memex-cloud |
| Pods | memex-portal-deployment-5df4ccb89-z8w2k, memex-portal-deployment-5df4ccb89-txn2l, memex-portal-deployment-5df4ccb89-sprw2 |
| Occurrences | 3 |
| First seen | 2026-08-26 00:34:27Z |
| Last seen | 2026-08-26 00:35:08Z |
Recent log lines
2026-08-26 00:34:27Z memex-portal-deployment-5df4ccb89-z8w2k fail: MeshWeaver.PluginCatalog.ModuleActivation[0]
🚨 1 ACTIVATED module(s) are not loaded in this process and NO RESTART will load them — any HTTP endpoint, view or provider they contribute is silently absent (a 404 with no error) until the package is re-installed: 1 module(s) are ACTIVATED but their landed assemblies are absent — a restart will NOT load them and anything they contribute (endpoints included) stays missing; re-install the package: MeshWeaver.Blazor.EntityViews
2026-08-26 00:34:54Z memex-portal-deployment-5df4ccb89-txn2l fail: MeshWeaver.PluginCatalog.ModuleActivation[0]
🚨 1 ACTIVATED module(s) are not loaded in this process and NO RESTART will load them — any HTTP endpoint, view or provider they contribute is silently absent (a 404 with no error) until the package is re-installed: 1 module(s) are ACTIVATED but their landed assemblies are absent — a restart will NOT load them and anything they contribute (endpoints included) stays missing; re-install the package: MeshWeaver.Blazor.EntityViews
2026-08-26 00:35:08Z memex-portal-deployment-5df4ccb89-sprw2 fail: MeshWeaver.PluginCatalog.ModuleActivation[0]
🚨 1 ACTIVATED module(s) are not loaded in this process and NO RESTART will load them — any HTTP endpoint, view or provider they contribute is silently absent (a 404 with no error) until the package is re-installed: 1 module(s) are ACTIVATED but their landed assemblies are absent — a restart will NOT load them and anything they contribute (endpoints included) stays missing; re-install the package: MeshWeaver.Blazor.EntityViews
Opened automatically from Admin/_LogIncident/381b930684c9dfb8. Recurrences are folded into this issue rather than opening new ones.
Activity
systemorph-com commentedon Sep 17, 2026
Reopened — this fault is happening again.
Still occurring: 12 more since the last update (15 total).
memex-portal-deployment-5df4ccb89-z8w2k,memex-portal-deployment-5df4ccb89-txn2l,memex-portal-deployment-5df4ccb89-sprw2,memex-portal-deployment-5df4ccb89-5w66k,memex-portal-deployment-76c65fb457-z8cw8,memex-portal-deployment-76c65fb457-2w858,memex-portal-deployment-76c65fb457-jr82z,memex-portal-deployment-788cffc666-kh7sl,memex-portal-deployment-788cffc666-9zhzw,memex-portal-deployment-64cc8b9b6d-q264s,memex-portal-deployment-64cc8b9b6d-hr5hh,memex-portal-deployment-6c954fd5bf-fphtz,memex-portal-deployment-6c954fd5bf-b5q52,memex-portal-deployment-6c954fd5bf-czv78,memex-portal-deployment-6c954fd5bf-7b4xqRecent log lines
rbuergi commentedon Sep 17, 2026
Closing — the root cause was found and fixed, and it is named in the code after this issue
This issue asked for the invariant rather than the instance: "an activation entry must never
outlive its bytes." That is exactly what landed, and the fix carries this issue's number.
Root cause — a GC pass racing a concurrent landing, not a lost write. A landing is two writes on
the shared
/datavolume, deliberately ordered bytes-then-entry (LandCore:Directory.Move, thenModuleActivationSidecar.WriteEntry). Nothing serializes them against a GC pass running on anotherreplica: if GC reads the sidecar in the gap, the new generation directory is on disk with no entry
referencing it yet — indistinguishable from an orphan — and it is deleted a moment before the
landing's
WriteEntrycompletes, pointing a real, enabled activation entry at bytes that are gone.Nothing throws: the landing reports success, and the entry only reveals itself as unresolvable when
something next reads it.
On
Systemorph/MeshWeaver@maintoday,src/MeshWeaver.PluginCatalog/ModuleLandingService.csstatesthat in its own words and cites this issue at five sites, including the constant that closes it:
ModuleLandingService.DefaultGarbageMinAge = TimeSpan.FromMinutes(5)— "an UNREFERENCED directoryis not proof of an orphan (Module activation entry survives without its landed assemblies — EntityViews views silently absent on a booted pod #2303)"; GC refuses to reclaim anything younger than the window.
MeshWeaver.Blazor.EntityViews: an ACTIVATED entry whose landed assembly was gone, with noexception or stack frame naming why."
The reporting gap this issue also raised is closed.
ModuleActivationReport.Unresolvable(
src/MeshWeaver.PluginCatalog/PendingModuleActivations.cs) is now a third, distinct state,deliberately not folded into
Pending— "a 'restart required' no restart can clear is the same lieas a green tick over a gate that never ran, and the remedy is different (re-install, not wait)" —
and
InstalledPackageRepairServicere-runs install hooks and re-asserts declared access for everyinstalled package at boot. So ACTIVATED-but-absent and deliberately-not-required are no longer
indistinguishable, which was the readiness question in the filing.
The reopen is three-week-old evidence.
systemorph-com[bot]reopened at 2026-09-17T07:51:44Zwith newest
Last seen: 2026-08-26 16:01:50Z; every pod named is from the 08-26 ReplicaSets. Thatbot's over-eager reopen behaviour is #4577 (root cause fixed in
Systemorph/MeshWeaver.Plugins#2014).
Closing.
Read-only triage 2026-09-17: fresh shallow clone of
Systemorph/MeshWeaver@main, issue timeline via REST. No code change, no mesh write.