fix(#4126): a module's native payloads are derived by BOTH closures and land where the loader probes by rbuergi · Pull Request #4366 · Systemorph/MeshWeaver

rbuergi · 2026-09-15T05:21:45Z

A module's native payloads are derived by BOTH closures, and they land where the loader probes

Closes #4126.

Since #4239 a registry bundle could carry a RID-specific native payload, and nothing ever
wrote one to disk: carried, declared, ignored. The consumer's landing took the assemblies and
the wwwroot tree and nothing else — so a module that declares an engine published 200, appeared
in the catalog, installed, loaded, rendered, and threw DllNotFoundException at its first
P/Invoke, with nothing in any log connecting the two. The other closure derivation, the
container lane, could not even DECLARE such a package.

Root cause, measured before it was fixed

Not taken from the issue's description — re-derived from a real SQLitePCLRaw.lib.e_sqlite3
publish (2026-09-15), because the two deps.json shapes are the whole container half:

publish	what the deps.json says	what is on disk
portable	29 `runtimeTargets` entries, `assetType: "native"`	the `runtimes/<rid>/native/` tree, preserved
`-r linux-x64` (what every MeshWeaver image is)	a ONE-entry `native` section, key still `runtimes/linux-x64/native/libe_sqlite3.so`	`libe_sqlite3.so`, flattened beside the app

ContainerReferenceSet read neither — only the runtime section — and Resolve matched a
package by the ASSEMBLY it contributes (falling back to <id>.dll). A package contributing only
natives therefore answered Supplied = false against an image that demonstrably ships its engine,
and ProjectBuild reds an unresolved PackageReference. That is why MeshWeaver.AppleMessages
carries a conditional pin and depends on its host.

And the collection ran after the runtime short-circuit in both derivations — the same bug in
the same place the SDK derivation had (#4239's note), skipping exactly the packages it exists for.

What changed

Landing. LandModule / ShelveModule take nativeAssets; the publish endpoint and
PluginBundleClient.LandFromBundle read them off the bundle; GenerationIdOf folds them into the
content address — two bundles differing only in an engine must address two generations, or the
second landing adopts the first's directory and serves the wrong engine under the right version.

🚨 The layout is validated by ONE predicate at every door: NuGetPackageWriter.IsModuleNativeLayout
— the derivation, the packer, the bundle reader, ModulePublish.Validate (a named 400, so a
publisher is told rather than the server blamed) and ModuleLandingService itself, before a byte
touches disk. A payload at any other shape is REFUSED, never landed: it would be bytes that read
as shipped and behave as absent, which is the one outcome the section exists to prevent.

Re-serve. ModuleBundleSource.NativeAssetsOf reads the shelf's tree — an additive call,
not a fourth element on CollectVersion's tuple, because MeshWeaver.Plugins destructures that
into three and widening it would red a consumer's trunk for a capability it does not use.
ServedModule.Natives (an init property — check-record-signatures.py) carries it from the
shelf AND out of a sealed publication, and the download route writes the section back into the
bundle it composes.

The container derivation. ContainerReferenceSet reads both deps.json shapes through one
reader (NativeContributions, shared with the shelf so the two cannot drift), Resolve treats a
native-only package as SUPPLIED, PrivateClosure carries NativeRides and NAMES a declared
payload neither source has, ProjectBuild lays them out under the pack input and writes
module-natives.txt, and the pack lane declares them with --with-native.

🚨 --with-native takes the opposite argument to --with, deliberately: --with refuses a
path component because the flat closure has no place for one; this requires the path, because the
path is what the loader probes. Two flags and two builder manifests, because a lane feeding one to
the other would be wrong whichever way it was written. The pack lane refuses a native with no
builder provenance, exactly as it does a non-MeshWeaver assembly.

🚨 The gate and publish-bake lanes compose into /ext/modules/<Name>/, one level deeper.
ModuleNativeAssets accepts a directory as a module folder only when its PARENT is named
modules, so at the shallower path a module's natives would land on disk and never be probed.

Also fixed at a call site this change already touched: the publish endpoint never passed
accepted.SourceCommit to ShelveModule, so #4158's shelf half recorded nothing — the same
unwired-argument shape as #2221 and as this issue itself.

Evidence

Every positive assertion was driven RED against the pre-fix code, by mutating the real thing:

control	result
`ContainerReferenceSet` reverted to reading only `runtime`	4 of 8 `NativeClosureTest` cases red, incl. `ANativeOnlyPackageIsSUPPLIED…`
the publish endpoint stops passing the natives	2 of 6 `PluginBundlePublishNativesTest` cases red
both lanes' native block removed (`test-module-asset-landing.py` revert arm)	0/2 natives land — red
both lanes' native `cp` removed, the in-lane check kept	the step's own assertion fails closed, naming the files

Plus anti-vacuity controls that pass and must: a package whose payload is DECLARED but absent is
Supplied = false (so the verdict is about the image, not the declaration); a module with no
engine grows no runtimes/ tree; a bundle with no natives declares no section at all; .a/.lib,
runtimes/<rid>/nativeassets/<tfm>/… and assetType: "runtime" are excluded and named.

test-module-asset-landing.py now runs 7 cases per lane and EXECUTES both lanes' real steps,
asserting the bytes, the exact four segments and the depth.

dotnet build MeshWeaver.slnx -c Release -warnaserror → 68 assemblies, 0 Warning(s), 0 Error(s)
NativeClosureTest 8/8 · PluginBundlePublishNativesTest 6/6 · ModulePackCommandTest 32/32 ·
the module suites of Memex.Portal.Shared.Test 76/76 · PrivateClosureTest/ContainerReferenceSetTest/
ModuleLibrariesShelfTest green · doc link integrity green
test-module-asset-landing.py PASS (2 lanes × 7 cases, all four falsification arms red)
check-record-signatures.py ✓ (16 changed C# files, no binary-breaking primary constructors) ·
check-type-forwards.py 0 removed, 1 added (ServedModule.Natives, an init property) ·
check-interface-addition.py 0 obliging members · check-workflow-timeouts.py ✓

test-module-set-consistency.py reports one failure on origin/main too, byte-identical (a
pre-#3732-publisher case) — this diff cannot reach it.

What is deliberately NOT done

Warning → refusal (the issue's stage 5). The pack log's two drop warnings stay warnings:
arming the refusal before a full publish wave has been measured clean takes the fleet's
publishes down instead of the drops — the Arm the registry's own 400: ModulePublish.Validate still accepts a manifest that states no framework identity #3240 shape, which was armed on two measurements
rather than on faith. Filed as A dropped native payload should REFUSE the pack, not warn — arm it on a measured publish wave (#4126 stage 5) #4367, with the measurement it needs stated.
The container lane carries ONE rid's engine, because it resolves against ONE image. A host
on another rid falls back to the runtime's own probing — strictly better than the nothing it
carried before, and stated in the builder's log rather than left to be deduced.
The module-libraries shelf's native half is built (DeclaredNativesOf / NativeFileFor)
and unexercised: no curated package declares one yet.

Conserved on Doc/Architecture/Modules → "LANDED, RE-SERVED, and derived by BOTH lanes",
replacing the stage list the previous pass wrote there, plus a What's New entry.

Pairs-with: none — nothing is removed or renamed; every public change is additive (two optional trailing parameters, one additive static, two init properties), and the sibling repo's existing call sites compile unchanged.

🤖 Generated with Claude Code

…nd land where the loader probes A registry module bundle could CARRY a RID-specific native payload since #4239 and nothing ever wrote one to disk: carried, declared, ignored. The consumer's landing took assemblies and a wwwroot tree and nothing else, so a module that declares an engine installed cleanly, loaded, and threw DllNotFoundException at its first P/Invoke — with nothing in any log connecting the two. The other closure derivation, the container lane, could not even DECLARE such a package. Measured first, on a real SQLitePCLRaw.lib.e_sqlite3 publish rather than from the issue's description (2026-09-15): * a PORTABLE publish declares 29 `runtimeTargets` entries with `assetType: "native"` and keeps a `runtimes/` tree on disk; * `-r linux-x64` — what every MeshWeaver image is — resolves them into a ONE-entry `native` section whose KEY is still runtimes/linux-x64/native/libe_sqlite3.so while the FILE is flattened beside the app. `ContainerReferenceSet` read neither, and `Resolve` matched a package by the ASSEMBLY it contributes, so a native-only package answered Supplied=false against an image that ships the engine. LANDING. LandModule/ShelveModule take `nativeAssets`; the publish endpoint and PluginBundleClient read them off the bundle; GenerationIdOf folds them into the content address (two bundles differing only in an engine must address two generations, or the second adopts the first's); the layout is validated by ONE predicate — NuGetPackageWriter.IsModuleNativeLayout — at the derivation, the packer, the bundle reader, ModulePublish (a named 400) and the landing itself, before a byte touches disk. A payload at any other shape is refused, never landed. RE-SERVE. ModuleBundleSource.NativeAssetsOf reads the shelf's tree (an ADDITIVE call: widening CollectVersion's tuple would rewrite a signature MeshWeaver.Plugins destructures into three), ServedModule.Natives carries it from the shelf and out of a sealed publication, and the download route writes the section back into the bundle it composes. THE CONTAINER DERIVATION. ContainerReferenceSet reads both deps.json shapes through one reader (NativeContributions), Resolve treats a native-only package as SUPPLIED, PrivateClosure carries NativeRides and NAMES a declared payload neither source has, ProjectBuild lays them out and writes module-natives.txt, and the pack lane declares them with --with-native — which takes the opposite argument to --with, deliberately, because the path IS what the loader probes. The gate and publish-bake lanes now compose into /ext/modules/<Name>/: ModuleNativeAssets accepts a module folder only when its PARENT is named `modules`, so the shallower path landed natives that were never probed. Both lanes' real steps are executed by test-module-asset-landing.py, which asserts the bytes, the four segments AND the depth, with a falsification arm per half — all four driven red. Also fixed at the same call site: the publish endpoint never passed `accepted.SourceCommit` to ShelveModule, so #4158's shelf half recorded nothing — the same unwired-argument shape as #2221 and as this issue. Still open, and stated rather than quietly skipped: the pack log's two drop warnings remain warnings (arming the refusal before a clean publish wave has been measured takes the fleet's publishes down instead of the drops — the #3240 shape), the container lane carries ONE rid's engine because it resolves against ONE image, and the module-libraries shelf's native half is built but unexercised. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions · 2026-09-15T05:27:49Z