feat(plugincatalog): pull plugin bundles from the fleet's OCI registry by digest by rbuergi · Pull Request #3726 · Systemorph/MeshWeaver

rbuergi · 2026-09-08T10:54:13Z

Implements the consumer half of Plugin Bundles in the Registry (#3720, refs #3353): a plugin bundle can now be pulled from the fleet's OCI registry by digest, verified, with the instance credential the installation already holds.

What changed

A shared OCI client — MeshWeaver.PluginCatalog.OciRegistryClient

The bearer handshake (/v2/… → 401 challenge → realm with Basic instance:<token> → bearer, cached per pull scope for the client's life) factored OUT of OciTagLister; the lister now resolves its credential and delegates to OciRegistryClient.ListTags. Byte-identical wire behaviour, its four tests unchanged and green.
GetManifest(repo, reference) → OciManifest (bytes, media type, verified digest, Config/Layers/Manifests descriptors). A manifest asked for by digest must hash to it; one asked for by tag must hash to the Docker-Content-Digest the registry states.
GetBlob(repo, digest, openDestination) → OciBlobReceipt, streamed through the Blob I/O pool with an incremental SHA-256; a mismatch faults with OciDigestMismatchException and the caller discards the destination.
IObservable<T> surfaces, one async IO leaf per operation inside IIoPool.Invoke (Http pool for handshake/manifests/tags, Blob pool for blobs). Supporting types: OciDescriptor, OciReference (<registry>/<repo>@sha256:… / :tag parser), OciDigest.

PluginBundleClient learns the artifact path

BundleRef.Artifact (init property, additive). With one, Download fetches the manifest by digest and the application/vnd.meshweaver.bundle.v1.zip layer by digest from the OCI registry, presenting the plugin-registry token it already holds; without one it takes GET /api/plugins/bundles/{plugin}/{version} exactly as before. The landed bytes enter the SAME landing path (LandFromBundle → ModuleLandingService, the link probe, the seeder's MVID rule) — the landing contract is untouched.
A digest mismatch is REFUSED: nothing lands, no HTTP fallback, ledger kind BundleAdoptionKind.ArtifactRefused (new enum member). A transport failure is the ordinary FetchFailed miss.

Registry side — the artifact field, from a seam

IPublicationArtifacts (MeshWeaver.PluginCatalog) with Read() carrying a DEFAULT implementation that answers none; NoPublicationArtifacts registered by AddPluginCatalog via TryAddSingleton; PublicationArtifact(Source, Package, Version, Reference) and PublicationArtifactLookup.ReferenceFor (the one match rule).
/api/plugins/bundles/index.json: each bundle carries artifact — cr.meshweaver.cloud/plugins/<source>/<package>@sha256:<manifest digest> or null.
/api/plugins: PackageManifest.Artifact (additive, nullable) stamped beside source; { packages: [...] } and storageType unchanged, so the setup wizard's SetupRegistryClient keeps working (its SetupIdentityPhaseTest fake is the contract).

Docs: PluginRegistry.md gains "Bundle bytes from the registry"; the design page's consumer table names OciRegistryClient; What's New Category: Feature.

Tests (`test/Memex.Portal.Shared.Test`, where the bundle-client tests already live — there is no `MeshWeaver.PluginCatalog.Test` project)

OciRegistryClientTest — handshake with Basic instance:key and ONE exchange serving manifest + blob; manifest by tag verified against the stated digest; tampered blob REFUSED; tampered manifest REFUSED; refused key faults naming the host; every tag page followed; reference parsing.
PluginBundleArtifactFetchTest — an advertised artifact is fetched from the OCI fake and lands (HTTP bundle route never asked); a tampered artifact lands nothing, no file, no sidecar entry, no HTTP fallback, ledger ArtifactRefused; artifact: null takes the HTTP route as today.
PluginBundleIndexArtifactTest — the index renders artifact for a pushed bundle and null for the rest (fake IPublicationArtifacts on request services); with the platform default every artifact is null; the lookup rule.
OciTagListerTest after the refactor: 5/5 green. Fake registries substitute the NETWORK only; hub, pools, landing service, registration and install path are real.

Verification

dotnet build -c Release -warnaserror, one project per invocation, 0 Error(s): MeshWeaver.PluginCatalog, Memex.Portal.Shared, Memex.Portal.Shared.Test, MeshWeaver.Documentation.Test, MeshWeaver.Compiler.Pipeline.Test, MeshWeaver.ComboAssembler, MeshWeaver.ComboVerifier, MeshWeaver.PluginTester.
dotnet test (Release, fresh .trx): 25/25 across the four classes above; DocumentationLinkIntegrityTest 1/1.

Pairs-with: none — additions only (a new class, a new interface with a defaulted member, an init property on a public record, a nullable property on a public record, a new enum member; no public surface removed).
Implementers: IPublicationArtifacts.Read — a NEW interface whose only member carries a default implementation; NoPublicationArtifacts is the platform's implementer, and a satellite that implements it later overrides Read(). No existing implementer anywhere (the interface did not exist).
Mirror-sync: none — no i18n catalog change.

Refs #3720, #3353.

🤖 Generated with Claude Code

…y by digest A shared OciRegistryClient (bearer handshake factored out of OciTagLister, GetManifest/GetBlob with digest verification, ListTags), PluginBundleClient fetching a bundle's .zip layer by digest when the index names an `artifact` and taking the HTTP route otherwise, the `artifact` field on both registry indexes rendered from the IPublicationArtifacts seam (default: none), tests against fake registries, and the PluginRegistry doc section. Refs #3720, #3353. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot

🟡 Changes recommended

There are repo-rule violations in the new product code (await/async usage under src/) and forbidden static mutable state in a new test, both of which need to be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds the “consumer half” of pulling plugin bundles from the fleet OCI registry by digest, including manifest/blob verification, while preserving the existing HTTP bundle route as a fallback when no artifact reference is advertised. It also introduces a registry-side seam (IPublicationArtifacts) so the bundle index and catalog can advertise an artifact reference when a publication was pushed.

Changes:

Introduces MeshWeaver.PluginCatalog.OciRegistryClient and refactors OciTagLister to reuse it.
Extends PluginBundleClient to fetch bundle bytes from OCI artifacts (by digest, verified) when artifact is present; otherwise uses the existing HTTP endpoint.
Stamps artifact into /api/plugins and /api/plugins/bundles/index.json using IPublicationArtifacts, with tests + docs/What’s New updates.

File summaries

File	Description
`src/MeshWeaver.PluginCatalog/OciRegistryClient.cs`	New OCI Distribution pull client (handshake, manifest/blob fetch, tags) with digest verification.
`memex/Memex.Portal.Shared/SelfUpdate/OciTagLister.cs`	Refactors tag listing to delegate to `OciRegistryClient`.
`src/MeshWeaver.PluginCatalog/PluginBundleClient.cs`	Adds `BundleRef.Artifact` and OCI artifact download path (no HTTP fallback on digest mismatch).
`src/MeshWeaver.PluginCatalog/BundleAdoption.cs`	Adds `BundleAdoptionKind.ArtifactRefused` for digest mismatches.
`memex/Memex.Portal.Shared/Api/PluginBundleEndpoints.cs`	Stamps `artifact` into bundle index entries from `IPublicationArtifacts`.
`memex/Memex.Portal.Shared/Api/PluginRegistryEndpoints.cs`	Stamps `artifact` into `/api/plugins` package entries from `IPublicationArtifacts`.
`src/MeshWeaver.PluginCatalog/PublicationArtifacts.cs`	Adds `IPublicationArtifacts` seam, default implementation, and lookup helper.
`src/MeshWeaver.PluginCatalog/PluginCatalogConfigurationExtensions.cs`	Registers the default `NoPublicationArtifacts` via `TryAddSingleton`.
`src/MeshWeaver.PluginCatalog/Package.cs`	Adds nullable `PackageManifest.Artifact`.
`test/Memex.Portal.Shared.Test/FakeOciRegistry.cs`	Test fake implementing the OCI pull protocol surface.
`test/Memex.Portal.Shared.Test/OciRegistryClientTest.cs`	Tests handshake, digest verification, paging, and parsing.
`test/Memex.Portal.Shared.Test/PluginBundleArtifactFetchTest.cs`	Tests artifact-based bundle adoption vs HTTP fallback and refusal behavior.
`test/Memex.Portal.Shared.Test/PluginBundleIndexArtifactTest.cs`	Tests index `artifact` stamping and lookup matching rules.
`src/MeshWeaver.Documentation/Data/Architecture/PluginRegistry.md`	Documents the dual-route bundle fetch behavior and `artifact` semantics.
`src/MeshWeaver.Documentation/Data/Architecture/PluginBundlesInTheRegistry.md`	Adds the design/contract page for OCI-backed plugin bundles.
`src/MeshWeaver.Documentation/Data/Architecture/ContainerRegistryInMemex.md`	Links to the new “Plugin Bundles in the Registry” page.
`src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-08-plugin-bundles-can-be-pulled-from-the-fleet-registry.md`	Adds a What’s New entry for the feature.

Review details

Suppressed comments (17)

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:206

await/async Task usage in src/ is a known deadlock/identity-loss hazard in this codebase (hub/grain/circuit schedulers can be parked; AccessContext can be lost across continuations). Please rework this I/O leaf to avoid await (e.g., move the work into an IIoPool.InvokeBlocking leaf using synchronous HttpClient.Send / synchronous stream reads) while keeping the public surface reactive.

            next = await ReadTagPageAsync(response, repository, tags, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:219

await/async Task usage in src/ is a known deadlock/identity-loss hazard in this codebase (hub/grain/circuit schedulers can be parked; AccessContext can be lost across continuations). Please rework this I/O leaf to avoid await (e.g., move the work into an IIoPool.InvokeBlocking leaf using synchronous HttpClient.Send / synchronous stream reads) while keeping the public surface reactive.

        using var response = await SendAuthenticatedAsync(uri, repository, key, request =>

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:235

await/async Task usage in src/ is a known deadlock/identity-loss hazard in this codebase. Consider replacing this async read with a blocking I/O leaf inside IIoPool.InvokeBlocking (or another established non-async pattern used in the repo) to keep async continuations off hub/circuit schedulers.

        var bytes = await response.Content.ReadAsByteArrayAsync(ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:259

await/async Task usage in src/ is a known deadlock/identity-loss hazard in this codebase. Please rework this call path to avoid await (prefer IIoPool.InvokeBlocking + sync HTTP APIs) so hub/grain schedulers are never parked and AccessContext is not lost.

        using var response = await SendAuthenticatedAsync(uri, repository, key, null, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:270

await/async Task usage in src/ is a known deadlock/identity-loss hazard. This await using introduces async disposal in product code; prefer a non-async streaming implementation under an IIoPool.InvokeBlocking leaf (sync reads/writes + sync dispose) if possible.

        await using var source = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:278

await/async Task usage in src/ is a known deadlock/identity-loss hazard. This async read loop should be expressed without await in product code (e.g., sync stream read inside IIoPool.InvokeBlocking) to avoid scheduler parking and context loss.

            while ((read = await source.ReadAsync(buffer, ct).ConfigureAwait(false)) > 0)

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:281

await/async Task usage in src/ is a known deadlock/identity-loss hazard. Consider replacing async writes with synchronous writes inside an IIoPool.InvokeBlocking leaf to keep continuations out of hub/circuit schedulers.

                await destination.WriteAsync(buffer.AsMemory(0, read), ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:284

await/async Task usage in src/ is a known deadlock/identity-loss hazard. Prefer a non-async flush/disposal path inside IIoPool.InvokeBlocking (sync IO) rather than awaiting here.

            await destination.FlushAsync(ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:288

await/async Task usage in src/ is a known deadlock/identity-loss hazard. Async disposal here should be avoided in product code per repo guidance; consider switching the transfer leaf to a blocking implementation managed by IIoPool.InvokeBlocking.

            await destination.DisposeAsync().ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:315

await/async Task usage in src/ is a known deadlock/identity-loss hazard in this codebase. Please rework this request path to avoid awaiting in product code (e.g., sync HTTP in an IIoPool.InvokeBlocking leaf) so hub/grain/circuit schedulers are never parked and AccessContext isn't lost across continuations.

        var response = await SendAsync(uri, bearer, configure, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:326

await/async Task usage in src/ is a known deadlock/identity-loss hazard. The bearer exchange call path should avoid await in product code; prefer a blocking leaf on IIoPool.InvokeBlocking (sync HTTP) or an established non-async pattern already used in this repo.

            minted = await ExchangeAsync(response, repository, key, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:329

await/async Task usage in src/ is a known deadlock/identity-loss hazard. This retry send should be implemented without await in product code (move into a blocking pool leaf) to avoid scheduler parking/context loss.

        var retried = await SendAsync(uri, minted, configure, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:347

await/async Task usage in src/ is a known deadlock/identity-loss hazard. Consider replacing this async HttpClient.SendAsync call with synchronous HttpClient.Send inside an IIoPool.InvokeBlocking leaf to avoid async continuations in product code.

        return await http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:389

await/async Task usage in src/ is a known deadlock/identity-loss hazard. The token exchange HTTP call should avoid await in product code; prefer sync HTTP in an IIoPool.InvokeBlocking leaf (token responses are small).

        using var response = await http.SendAsync(request, ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:400

await/async Task usage in src/ is a known deadlock/identity-loss hazard. Please avoid awaiting the token response body in product code; instead read synchronously within an IIoPool.InvokeBlocking leaf or another established non-async pattern.

        using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync(ct).ConfigureAwait(false));

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:424

await/async Task usage in src/ is a known deadlock/identity-loss hazard. This async stream read should be expressed without await in product code (sync read inside an IIoPool.InvokeBlocking leaf), keeping the surface reactive.

        await using var stream = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false);

src/MeshWeaver.PluginCatalog/OciRegistryClient.cs:425

await/async Task usage in src/ is a known deadlock/identity-loss hazard. Please avoid awaiting JSON parsing in product code; parse synchronously within a blocking pool leaf to keep continuations off hub/circuit schedulers.

        using var doc = await JsonDocument.ParseAsync(stream, cancellationToken: ct).ConfigureAwait(false);

Files reviewed: 17/17 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+                throw new InvalidOperationException(
+                    $"{registry} named {next.AbsoluteUri} as the next tags page twice — the listing loops.");
+
+            using var response = await SendAuthenticatedAsync(next, repository, key, null, ct).ConfigureAwait(false);


+    // One packed bundle, shared by both fakes: the OCI registry serves it as the artifact's layer,
+    // the plugin registry serves it on the HTTP route. Immutable bytes, never written after
+    // construction — the same shape as a media-type table.
+    private static readonly byte[] BundleBytes = ModuleBundle();
+
+    private readonly FakeOciRegistry oci = new(OciHost, Token, $"plugins/Plugins/{Plugin}", BundleBytes);
+    private readonly FakePluginRegistry registry = new(BundleBytes);


github-actions · 2026-09-08T11:07:53Z