Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/MeshWeaver.Documentation/Data/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,7 @@ Each theme starts with its introductory page, followed by related architecture t
- [Sealed Publication Reads](SealedPublicationReads) — a publication is unreadable for ~90s per target per publish and the `plugins` prefix has two writers; the three answers (404/503/412), the generation that pins one instance, and the mix no reader can detect
- [Sealed Publication Generations](SealedPublicationGenerations) — the layout that makes that mix UNREPRESENTABLE (a directory per publication plus a pointer swapped last), the reader contract, the retention rule, and the ordered migration that avoids a new-writer/old-writer half-state
- [Install Completeness](InstallCompleteness) — what an install RECORD declares landed, compared against what is actually in the mesh; the comparison nothing made until #3485, and why only one of its five verdicts is a pass
- [Declared Is Not Landed](DeclaredIsNotLanded) — the three exits an install can take and the one that never looked at the mesh: an UPDATE fetched the manifest diff alone, so a node lost after a previous install survived every subsequent update, each reporting success. The measured case, the batched presence read that fixes it, and the sweep hazard that made the damage read smaller than it was
- [CI Content Bake](CiContentBake)
- [Bundle Delivery Stages](BundleDeliveryStages) — the four independent stages between a merge and a portal serving prebuilt bytes (write · compose · select · deliver), which of #3461 / #3732 / #3768 / #3583 owns each, the instrument that answers for each — and why a reading taken at one stage is not evidence about another
- [Framework Identity Churn](FrameworkIdentityChurn) — the identity moves on every core COMMIT, not on every content change (43 merges, 5 touching the full-MVID set, ≥18 identities in 24h); the commit sha compiled into `AssemblyInformationalVersion` is why, the falsification test that refuted the local fix (0 of 22 control, 22 of 22 and 21 of 21), and the four costed options with every runner-hour labelled as arithmetic
Expand Down
190 changes: 190 additions & 0 deletions src/MeshWeaver.Documentation/Data/Architecture/DeclaredIsNotLanded.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
---
Name: Declared Is Not Landed
Category: Architecture
Description: An install record is a declaration, never an observation. Three exits can write it and until now only one of them looked at the mesh — so a node lost after an install could survive every subsequent update, each reporting success. The measured case on memex.meshweaver.cloud, the exit that was blind, and the sweep hazard that made the damage read smaller than it was.
Icon: <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M4 4h10l6 6v10H4z"/><path d="M14 4v6h6"/><path d="M8 14h5"/><path d="M8 17h3"/></svg>
---

# Declared Is Not Landed

An install record is a statement about the **source**. It names the files a bundle shipped and the
hash of each. Nothing in it observes the mesh, and nothing in the installer's own counters does
either — `InstallResult(Total, Written)` counts what the installer *decided* to write.

That gap has been closed one exit at a time, and this page is about the exit that was still open.

## The three exits an install can take

`CatalogLayoutAreas.InstallOrUpdate` chooses among three, on two questions: does the record's
`ModuleVersion` equal the candidate's, and does the record carry a file map?

| Exit | Taken when | Does it look at the mesh? |
|---|---|---|
| **Skip / heal** | the module hash is EQUAL | **Yes**, since MeshWeaver#3485 — `InstallCompleteness.Observe` reads every declared node path and `SkipOrHeal` runs a full install rather than skipping an incomplete one |
| **Full install** | no record, or a record with no file map | **Yes** — `DecideAndWrite` writes whenever `current is null`, so everything absent is restored |
| **Incremental update** | the module hash MOVED and the record has a file map | **It did not.** Fetched exactly `newManifest.DiffFrom(record.InstalledFiles)` |

The incremental exit's fetch set was a comparison of two *declarations* — the candidate's
`manifest.lock` against the record the installer itself wrote. A node lost after a previous install
has an unchanged file hash, so it never entered `delta.AddedOrChangedFiles`, was never fetched, and
never reached `DecideAndWrite` — the one place that would have restored it. **The
presence-awareness sat one layer below a set the absent file could not enter.**

## 🚨 And it is the exit an update always takes

This is what turns a one-off loss into a permanent one. #3485's heal lives on the module-hash-EQUAL
exit. An *update* never takes that exit by definition, and every new publication moves the hash
again. So a package that loses a node stays short across arbitrarily many updates, each one
reporting success, and the only escape is a boot on which the source's hash happens to equal the
record's **and** the install lane happens to run.

## The measured case

`Plugins/Hosting` on **memex.meshweaver.cloud**, read 2026-09-14T06:0xZ. All reads through `get` /
`search`; nothing was mutated.

**The morning: the files were there and they compiled.** The release node
`Hosting/TriageStatus/Release/20260913081235-bylQeIj_` — created on that portal at
2026-09-13T08:12:35.446Z, `status: Succeeded` — names its inputs:

```
sourceVersions:
Hosting/Deployment/Source/TriageIntake 639248839464110360 (2026-09-13T08:12:26.411Z)
Hosting/TriageStatus/Source/TriageStatusContent 639248839461605390 (…08:12:26.160Z)
Hosting/TriageStatus/Source/TriageStatusLayoutAreas 639248839464108660 (…08:12:26.410Z)
testVersions:
Hosting/TriageStatus/Test/TriageStatusTests …
Hosting/TriageStatus/Test/TriageStatusTestsArea …
```

A release records the source nodes a successful compile actually read. So those five nodes existed
on this portal, nine seconds before the compile that consumed them.

**The evening: the update ran, and wrote only what had changed.** The record was stamped at
2026-09-13T22:03:03.221Z — `installedNodeCount: 224`, `version 1.19.16`. Two nodes carry write
timestamps from that install:

| node | version | written |
|---|---|---|
| `Hosting/Deployment/Source/AksOpsResult` | 1 (new file) | 2026-09-13T22:02:54.601Z |
| `Hosting/Deployment/Source/PlatformBuildInboxWatcher` | 16 (changed file) | 2026-09-13T22:02:57.255Z |

Everything else in `Hosting/Deployment/Source` still carries its older timestamp — 08:09:41,
09-12, 09-09, 09-08, 08-28, 08-19. The update wrote exactly the two files whose content had moved.

**And eleven declared files had no node at all.** Anchored reads, each with a same-anchor positive
control:

| probe | result |
|---|---|
| `namespace:Hosting/Deployment/Source scope:subtree nodeType:Code` | **14**, `truncated: false` — no `TriageIntake` (the other 14 are the control: readable, same query shape) |
| `namespace:Hosting/Deployment/Test scope:subtree nodeType:Code` | **3** — no `TriageIntakeTests` |
| `namespace:Hosting/TriageStatus scope:subtree` | **1** — the Release node alone; no `Source/`, no `Test/` |
| `namespace:Hosting/TriageItem scope:subtree` | **1** — the Release node alone |
| `namespace:Hosting scope:children nodeType:NodeType` | **19**, `truncated: false` — the denominator |
| `namespace:Hosting scope:children nodeType:NodeType content.compilationStatus:Error` | **8 of 19** |

All eleven are named in the record's own `installedFiles` map. The record asserts them; the mesh
does not hold them; the update that stamped the record did not fetch a single one.

## 🚨 The sweep hazard: 8 of 19 understates it, and the 9th reads green

`Hosting/TriageItem` is **not** in the Error set. Its own node says:

```
compilationStatus: Ok
lastCompileSucceededAt: 2026-09-13T08:12:35.5789283Z
lastCompiledVersion: 6
requestedReleaseAt: 2026-09-13T21:56:05.9665591Z
lastReleaseRequestHandledAt: 2026-09-13T21:56:05.9665591Z ← handled without a recompile
lastCompileStartedAt: 2026-09-13T08:12:32.047279Z ← still the morning's
compiledSources: 6 entries — TriageActions, TriageItemContent, TriageItemLayoutAreas,
Hosting/Deployment/Source/TriageIntake, and two Test nodes
currentSourceVersions: {}
currentSourceFingerprint: e3b0c44298fc1c14 ← SHA-256 of the EMPTY STRING
```

Every one of those six `compiledSources` is now absent from the mesh, and the type serves a cached
assembly at `compilationStatus: Ok`.

**So `content.compilationStatus:Error` is a sweep for types that have *attempted* a compile and
failed. A type whose sources all vanished but which has not been re-driven since is in the GREEN
population.** The honest statement of this portal's damage is not "8 of 19 are broken" but "8 of 19
report Error, and at least one more has lost every source it compiled from while reporting Ok".

This joins the list of reasons a `compilationStatus` sweep's number is not a census — alongside the
RLS filtering and the two-executor split already recorded in
[NodeType Compilation](/Doc/Architecture/NodeTypeCompilation). The cross-check that does see it is
the record-vs-mesh comparison on this page: `compiledSources` (or the release node's
`sourceVersions`) against what the mesh holds.

## The fix

`IncrementalUpdate` now asks the mesh before it fetches.

1. Derive the candidate manifest's declared node paths — `newManifest.Files.Keys` through
`PackageInstaller.NodePathForFile`, the installer's own file→node rule and not a second
implementation of part of it.
2. `InstallCompleteness.ObservePresent` — **one** batched `IStorageAdapter.ReadMany` over that
bounded, known set. Never a query (a stale negative would re-fetch a file that is present) and
never N point reads of possibly-absent paths (which is what opens a storm breaker on the owning
hub).
3. `InstallCompleteness.FilesToRestore` — pure, so every arm is pinnable offline — returns the
declared files the delta is not already fetching whose node is absent.
4. The fetch asks for `delta.AddedOrChangedFiles ∪ restore`. `DecideAndWrite` then writes them,
because `current is null`.

It costs one batched read per incremental update — the same read
[Install Completeness](/Doc/Architecture/InstallCompleteness)'s post-install verification already
pays on this exit, moved to where it can still change the outcome instead of only reporting it.

### 🚨 Three answers, never two

`ObservePresent` returns `null` when the mesh was **not read** — no storage adapter, or a batched
read that faulted — and never an empty set. An empty set would say "the mesh holds none of them",
which on a Postgres read that faulted (a satellite table that does not exist answers `42P01`, not
"absent") would make an install re-fetch every file it ships, every time. `FilesToRestore` maps that
`null` to an empty restore set, and the caller **says so at Warning**: an unobserved mesh is not a
clean one, and the line is spelled so it cannot be read as a pass.

### The other silent hole in the same method

`IPackageSource.FetchPackageFiles(package, gitRef, paths)` filters — locally in the interface
default, server-side in `RegistryPackageSource` — and its contract is explicit that *"paths absent
from the package simply don't appear in the result"*. A requested path the source does not serve (a
serving-side casing difference, a stale registry cache, a bundle that disagrees with its own lock)
therefore vanished with no exception and no log line, after which the record was stamped with the
full declared map anyway. The returned set is now compared against the requested one and the
shortfall is named at Error. It reports only; it never fails the update — the install that did run
is a fact, and collapsing "landed short" into "failed" would make a working update a new way to
break.

## The rule this generalises to

> **A decision about what the mesh needs may not be taken from a record the writer wrote.** Any
> comparison whose two sides are both declarations — a lock against a record, a hash against a hash,
> a count against a count — can only tell you what the *source* did. If the decision is "does this
> have to be written", exactly one of the two sides must be a read of the mesh.

The same sentence is why #3485 exists, why the post-install verification exists, and why this page
does. Each of the three closed it on one exit; none of them closed it as a principle, which is how
the third exit stayed open through two fixes aimed at the same defect.

## What this does NOT do

- **It does not explain why the nodes went missing.** The release node proves they were written and
compiled; something removed them between 2026-09-13T08:12:35Z and 21:56:07Z, and no instrument a
session can run names it. This change makes the loss *self-correcting on the next update* instead
of permanent — it is not a diagnosis of the loss.
- **It does not re-arm a PARKED NodeType.** A type that failed a compile serves its cache and does
not re-drive itself when sources arrive; that is MeshWeaver#4208's subject. Restoring the content
and recycling the type are two steps, and on an image without #4208 they stay two steps.

## See also

- [Install Completeness](/Doc/Architecture/InstallCompleteness) — the five verdicts, only one of
which is a pass, and the post-install read-back that reports the outcome on every writing exit
- [NodeType Compilation](/Doc/Architecture/NodeTypeCompilation) — what a compile reads, what a
release records, and the other reasons a `compilationStatus` sweep is not a census
- [CQRS and Content Access](/Doc/Architecture/CqrsAndContentAccess) — why the presence read here is
a batched store read and never a query
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,38 @@ behaves identically. `CodeQueryResolver` is the single expansion/grouping
implementation, so the files shown in the GUI are exactly the files that
compile.

#### 🚨 `@` is ABSOLUTE and never rebases — so a tree carrying one is not relocatable

The grammar has both forms, and the difference is load-bearing:

| entry | `CodeQueryResolver.Expand` yields | addressed against |
|---|---|---|
| `namespace:Source scope:subtree` (no `/` in the value) | `namespace:{selfPath}/Source scope:subtree` | **the declaring NodeType** — rebased by `RebaseRelativeNamespace` |
| `shared=@Other/Type/Source` | `path:Other/Type/Source` **and** `namespace:Other/Type/Source scope:subtree` | **the mesh root** — verbatim, never rebased |

That is deliberate and must stay so. The dominant use of `shared=@…` is a
genuinely cross-partition include — `Store/Plugin` pulls in
`shared=@Store/Coupon/Source`, `@Store/Core/Source`, `@Store/Licensing/Source`
— and rebasing those under the declaring node's prefix would silently redirect
every one of them.

**The consequence, which is a property to know rather than a bug to fix: a tree
containing an absolute cross-reference cannot be imported under a prefix.**
Import `Northwind/Product` — whose `sources` name
`shared=@Northwind/AnalyticsCatalog/Source/Supplier` — as
`MeshWeaver/samples/Graph/Data/Northwind/Product`, and the reference still
resolves against `Northwind/…`, a root that does not exist on that mesh. The
compile then fails with `CS0246` on the shared types while
`failedSourceQueries` names the two unresolved entries exactly. Nothing is
missing; one reference is un-rebased, and nothing in the compiler can rebase it
without breaking the cross-partition case.

The remedy is at the INSTALL boundary, not in the resolver: **install a package
at the root its sources were authored for.** MeshWeaver#4174 is the worked
instance — the mirrored `samples/` subtree was retired and `Northwind` installed
at its own root as a `Store/Plugin` catalog entry, after which the same
`shared=@…` values resolve as authored.

---

## Watching compile progress
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
Name: An update restores content that went missing
Category: Fix
Description: Updating a plugin now brings back any of its files whose node has disappeared from the mesh — not only the files that changed since the last install.
Icon: ArrowSync
Order: -20260914
---

# An update restores content that went missing

Updating an installed plugin used to fetch only the files whose content had changed since the last
install. That is the right thing to do when nothing else moved — but if one of the plugin's nodes
had disappeared from the mesh in the meantime, its file was unchanged, so the update skipped it and
the node stayed gone. Every later update skipped it again, and each one reported success.

An update now checks the mesh before it fetches. Any file the plugin ships whose node is missing is
re-fetched and written back, alongside the files that genuinely changed — so the ordinary "update
the plugin" action repairs the gap instead of stepping over it.

Two smaller things travel with it:

- If the mesh cannot be read at that moment, the update says so and does **not** quietly proceed as
if everything were present.
- If the source returns fewer files than the update asked for, that shortfall is now named instead
of passing unnoticed.

The symptom this removes: a plugin whose views render empty, or whose node type will not compile,
because source files the install record still lists are not actually there.
Loading
Loading