fix(hub): a hosted hub at ShutDown is retired and replaced, never handed out (#5136)

src/MeshWeaver.Connection.Orleans/OrleansRoutingService.cs

@@ -1709,7 +1709,9 @@ IObservable<bool> ClaimOnce()
                     break;
             }
             inFlight.Remove(attach);
-            podHubClaimSettled.TryRemove(address, out _);
+            // Value-matched: a successor registered under the same address has its own settled
+            // subject in this slot by now, and a predecessor's disposal must not take it out.
+            podHubClaimSettled.TryRemove(new KeyValuePair<Address, AsyncSubject<Unit>>(address, settled));
             attach.Dispose();
             // A round inside its attach window owns the release: it has not returned from Attach()
             // yet, and the CAS above guarantees it will see the DisposeRequested bit on its way out.

src/MeshWeaver.Documentation/Data/Architecture/HubDisposalModel.md

@@ -523,6 +523,40 @@ was armed for — never a successor built under the same address.
 > skips the post pipeline once the hub is past `DisposeHostedHubs`, so that park raced the very
 > shutdown it waited for.
 
+## The third window: a hub at ShutDown, still registered
+
+The two windows above are about a hub accepting work, and a hub disposed before it was built.
+This one is a hub that is **past serving and still findable** (#5136). The removal `Add` arms
+runs inside the ShutDown phase's registrant walk — after `RunLevel` flipped, after
+`CancelCallbacks`, after the reactive dispose actions — and from the flip on the hub refuses
+every delivery at intake and faults a direct `Observe(...)` synchronously. A teardown that wedges
+anywhere in that phase (a registered cleanup blocking on a lock — #4883's shape) never reaches
+the removal, and `GetHubWithOutcome` answered the corpse as `Available` for as long as the wedge
+lasted: one `portal/nodeops-…` hub, four callers, 55 minutes, on a pod serving everything else.
+
+**The rule: an `Always` lookup never answers a hub at `ShutDown`.** It retires the corpse
+(value-matched, so a successor a concurrent lookup registered is untouched) and mints a successor
+through the ordinary single-flight creation. The corpse stays in the collection's disposal join
+until its own `DisposalCompleted` — its scope is a child of the owner's, and the owner must not
+close it under a registrant walk still running. The bound is `ShutDown` and not `IsDisposing` on
+purpose: below `ShutDown` the hub is still draining accepted work and the intake's transient
+`ShuttingDown` NACK is the designed answer (the first window above); a successor minted there
+would overlap accepted writes. `Never` probes — the router's — still find the corpse and are
+refused with that NACK, which is right for a message and wrong for a caller holding the
+reference. The retirement is one `Warning` line, `[HOSTED-RETIRE]`, and it is the first line
+that names a ShutDown-phase wedge by address. The three registries a teardown removes itself
+from are all value-matched now — the hosted-hub registry (#4741), the local route (#5159), and
+the pod-hub claim, whose `Detach` leaves a claim alone when the silo still holds a live route
+for the address, because that route can only be a successor's. Full account and the measured
+controls: [Disposed Scopes and Dying Hubs](../DisposedScopeAndDyingHubs), R2.
+
+> Deterministic repro and control: `AHubAtShutDownIsNotHandedOutTest` (Messaging.Hub.Test)
+> parks the ShutDown turn inside a reactive dispose action — invoked before the registrant walk
+> — asserts the corpse is still registered, and measures both the successor and the owner's
+> join. `PodHubStaleReleaseTest` (Hosting.Orleans.Test) drives the real grain on the two-silo
+> cluster: a stale release leaves a live successor served and pinned; a release with no live
+> route still tombstones.
+
 ---
 
 ## A recycle owes its subscribers a goodbye — and can only say it BEFORE the teardown

src/MeshWeaver.Hosting.Monolith/MonolithRoutingService.cs

@@ -18,12 +18,6 @@ internal class MonolithRoutingService(
     private readonly ConcurrentDictionary<Address, AsyncDelivery> streams = new();
 
 
-    private void UnregisterStream(Address address)
-    {
-        streams.TryRemove(address, out _);
-    }
-
-
     public override IDisposable RegisterStream(Address address, AsyncDelivery callback)
     {
         streams[address] = callback;
@@ -260,7 +254,13 @@ private IMessageDelivery PostNotFound(IMessageDelivery delivery, MeshNode? node,
                     logger.LogDebug("[ROUTE-CREATE] GetHostedHub returned {HubAddr} for {Address}",
                         createdHub?.Address.ToString() ?? "(null)", address);
 
-                    createdHub?.RegisterForDisposal(_ => UnregisterStream(createdHub.Address));
+                    // No second, key-only removal is armed here. The hub's own RegisterStream
+                    // registration removes what it registered (value-matched, above) when the hub
+                    // disposes; a removal by ADDRESS armed on top of it ran in the same registrant
+                    // walk and erased whatever was registered there by then — which, once a
+                    // successor can be minted while its predecessor is still tearing down
+                    // (HostedHubsCollection.RetireCorpse, #5136), is the successor's live route:
+                    // the address goes dark on this host with no exception and nothing to grep.
                     return createdHub;
                 }
                 finally

src/MeshWeaver.Hosting.Orleans/PodHubGrain.cs

@@ -131,6 +131,29 @@ public Task<bool> Attach()
     /// <inheritdoc />
     public Task Detach()
     {
+        // 🚨 A RELEASE REMOVES WHAT ITS REGISTRATION REGISTERED — the same rule the two registries
+        // behind it already keep (#4741 for the hosted-hub registry, #5159 for the local route), and
+        // the third and last place it was missing (#5136). RegisterStream's disposal removes its own
+        // local route, value-matched, BEFORE it releases this claim; so if this silo still has a live
+        // route for the address when the release arrives, it belongs to a SUCCESSOR that registered
+        // under the same address since (RegisterStream is last-writer-wins). This Detach is then the
+        // predecessor's, and it is stale. Honouring it would stamp the terminal Released tombstone on
+        // a LIVE hub — every delivery refused for ten minutes with the one verdict the owner-side
+        // eviction acts on, and the pin cut from indefinite to the tombstone's lifetime — which is
+        // strictly worse than the fault a retire-and-replace exists to cure. The grain is not
+        // reentrant, so this read is ordered against the successor's Attach and every Deliver: there
+        // is no moment at which the route is there and the successor is not.
+        Address address = AddressPath;
+        if (localRoutes?.TryGetLocalRoute(address) is not null)
+        {
+            logger.LogDebug(
+                "[POD-HUB] {Address}: a release arrived while this silo holds a live local route for it — "
+                + "a successor has registered under the address, so the release is the predecessor's and "
+                + "is not honoured. The claim stays as the successor's Attach left it.",
+                AddressPath);
+            return Task.CompletedTask;
+        }
+
         // 🚨 A RELEASE IS A FACT THE CLUSTER MUST KEEP, briefly. Deactivating on idle here (the
         // previous behaviour) threw that fact away: the next delivery to the address re-created the
         // activation on the CALLER's silo (prefer-local), which has no local route either, and the