Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 108 additions & 14 deletions src/MeshWeaver.Documentation/Data/Architecture/RouterTrafficDetection.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
nodeType: Markdown
name: Router Traffic Detection
category: Architecture
description: The ROUTER_TRAFFIC detector has two sites — the receiving hub, which names the two addresses, and the origin, which names the call site. Why the receiver-side line alone could not close an issue in four re-filings and 41,087 lines, what each site can and cannot see, and the seam a violating caller hops onto.
description: The ROUTER_TRAFFIC detector has two sites — the receiving hub, which names the two addresses, and the origin, which names the call site. Why the receiver-side line alone could not close an issue in four re-filings and 41,087 lines, what each site can and cannot see, the seam a violating caller hops onto, and why the ratchet that holds that seam derives its denominator from the framework's own handler registrations instead of listing message types.
icon: /static/NodeTypeIcons/box.svg
---

Expand Down Expand Up @@ -118,30 +118,111 @@ a **ratchet that may only shrink**, one per tree:
| `src/` | `RouterAsNodeOperationOriginRatchetGuard` | `test/RouterNodeOperationOriginSites.allow` | 1 |
| `test/` | `RouterAsTestRequestOriginRatchetGuard` | `test/RouterRequestOriginSites.allow` | 3 |

The `src/` guard matches an `.Observe(…)`/`.Post(…)` whose first argument is a node-lifecycle
request — `CreateNodeRequest`, `CreateNodesRequest`, `CreateOrUpdateNodeRequest`,
`DeleteNodeRequest`, `MoveNodeRequest`, `CopyNodeRequest` — built inline **or** hoisted into a local
first, and reads the receiver as an expression rather than as preceding text. Both tolerances were
measured: over `src/` the shape occurs at 23 sites, of which a construction-anchored scan misses the
five that are `MeshService`'s own verbs, and a literal `NodeOperationIssuingHub()` receiver test
misclassifies the six that hoist the seam into a local or a cached property.
The `src/` guard matches an `.Observe(…)`/`.Post(…)` whose first argument is a **lifecycle message**,
built inline **or** hoisted into a local first, and reads the receiver as an expression rather than
as preceding text. Both tolerances were measured over the tree, not guessed: of the 33 sites it
matches, a construction-anchored scan would miss **7** (every verb of `MeshService`, the reference
implementation, plus the copy helper's hoisted request), and a receiver test that accepted only the
literal `NodeOperationIssuingHub()` text would misclassify the **11** that hoist the seam into a
local or a cached property.

#### The denominator is DERIVED, and that is the point (#4463)

> 🚨 **A list of message types is the same trap one level along.** The guard's first version carried
> a literal list of six node-CRUD names. `DisposeRequest` — a teardown, the most router-hostile
> lifecycle message there is — was not on it, so when `MeshOperations.RecycleCore` posted one
> straight off the DI-injected hub, **only the runtime ORIGIN detector saw it**
> ([#4463](https://github.com/Systemorph/MeshWeaver/issues/4463), measured on `memex`
> 2026-09-16 01:10:13Z). Adding one more name would have bought exactly one more message.

So the guard now **reads its denominator out of production** instead of restating it. A message is
lifecycle when the framework itself registers a lifecycle handler for it, and there are exactly two
such registrations:

| family | read from | today |
|---|---|---|
| hub lifecycle | every `Register<T>(…)` in **`MessageHub`'s constructor** — what a hub answers *because it is a hub* | `DisposeRequest`, `ShutdownRequest`, `PingRequest`, `InitializeHubRequest` |
| node lifecycle | every `.WithHandler<T>(…)` in **`MeshExtensions.WithNodeOperationHandlers`** — what the off-router node-CRUD execution hub answers | `CreateNodeRequest`, `CreateNodesRequest`, `CreateOrUpdateNodeRequest`, `DeleteNodeRequest`, `ValidateDeleteRequest`, `MoveNodeRequest`, `CopyNodeRequest` |

This is a *rule* rather than a *list* because **you cannot add a lifecycle message without passing
through one of those two registrations** — a handler is what makes a message lifecycle in the first
place. A marker attribute, or a name in a test, can be forgotten; a handler cannot, because without
one the message does nothing. On its first run the derivation picked up `ValidateDeleteRequest`,
which the hand-written list had already missed.

The exclusions are sourced the same way: the derived verbs are filtered through **`RouterTrafficRule`
itself**, the pure predicate both runtime sites evaluate, so `HeartBeatEvent` — which
`WithNodeOperationHandlers` also registers, and which the rule calls routing's own liveness — leaves
the denominator automatically, and a future exclusion added there is inherited rather than copied.
Every derived name must also resolve to a real type, so a rename or a move **reddens the guard
instead of silently shrinking what it measures**. That half is a separate test, and it fails on its
own: when the derivation was deliberately broken in rehearsal the ratchet went *green* over 25 sites
with `DisposeRequest` unguarded, and only the derivation test went red.

**One structural exclusion: a self-directed hub-lifecycle post** — no target, or
`o.WithTarget(thatHub.Address)` — is out of the denominator, and not because the detector is quiet
about it. It is because **the remedy does not apply**: the seam's whole effect is to issue the
delivery from a *different* hub, which for a self-directed teardown would send it to the wrong hub
and change what is torn down. `NodeTypeRebindWatcher`, the stale-build convergence, the overlay
self-heal and `LayoutAreaHost`'s own `InitializeHubRequest` are the four sites this covers. The NODE
family is deliberately *not* excluded this way: a target-less `CreateNodeRequest` on the router is
the literal prod 2026-06-11 wedge, and the seam does fix it by moving execution onto the
node-operation hub's block.

Over `src/` this yields **29 lifecycle post sites in the denominator**, 4 self-directed ones
excluded, and one allowed violation (the `#981` entry below).

**Adopting the seam is never a behaviour change**, which is what lets this be a rule rather than a
judgement call: `NodeOperationIssuingHub` returns the hub unchanged unless its address type is the
mesh type. A site already off the router is byte-for-byte unaffected; a site that is not is
corrected.

> 🚨 **What the ratchets still do not see.** They key on node-CRUD request TYPES, so the read seam
> (`ReadIssuingHub`, which has no request type of its own) is recognised where it is used but its
> absence is not reported — and the other message families in #1140's evidence (`ClickedEvent`,
> `UnsubscribeRequest`, `PatchDataChangeRequest`) have no single seam to hop onto at all. For those
> 🚨 **What the ratchets still do not see, and why #1140 is NOT closed by this.** They key on
> messages the framework registers a LIFECYCLE handler for, so the read seam (`ReadIssuingHub`,
> which has no request type of its own) is recognised where it is used but its absence is not
> reported — and the other message families in
> [#1140](https://github.com/Systemorph/MeshWeaver/issues/1140)'s evidence have no single seam to
> hop onto at all. Measured 2026-09-16, one of them is still live in `src/`:
> `JsonSynchronizationStream` posts `new UnsubscribeRequest(reduced.StreamId)` on its own `hub`,
> and hopping *that* is not a no-op — it would change which hub the unsubscribe originates from,
> which is a correlation question, not a routing one.
>
> #1140 is therefore the same defect **family** as #4463 and not the same **defect**. Its evidence
> is receiver-side, where a routed payload arrives packed, so its type list is mostly the honest but
> useless `RawJson`; its stated hypothesis (portal hubs not assigning their own address) was wrong;
> and the bulk of its 1,386 lines was the HOP mis-keying, fixed by keying the rule on the delivery's
> own ends. What is left of it is this residue plus whatever the origin line names next. For those
> the origin line remains the instrument, and it is what makes a new one a five-minute question
> instead of a month-long one.

The one seeded `src/` entry is not debt: it is the #981 self-targeted inner create inside the
`CreateOrUpdateNodeRequest` **handler**, posted on and handled by the hub whose turn loop already
owns the upsert. A self-post is deliberately reported by neither detector site (above), so hopping
it would move the inner create off the hub that is mid-upsert and buy nothing.
it would move the inner create off the hub that is mid-upsert and buy nothing. It stays an explicit
allowance rather than joining the self-directed exclusion, because that exclusion is for HUB
lifecycle only — a target-less node CRUD post on the router is the wedge, not a benign self-post.

### The two callers this rule was written from

`MeshOperations.RecycleCore` (the Recycle tool / Compile button) and
`HubRecycleExtensions.RecycleNode` (the framework's one recycle surface) both post their
`DisposeRequest` through `hub.NodeOperationIssuingHub()`. The second had no production sighting; it
is hopped because it is a **public extension on `IMessageHub`**, so its documented rule — "the caller
holds a hub that outlives the target" — constrains who *should* call it, not who *can*. The seam is
the identity function for every documented caller, so adopting it costs them nothing and removes the
undocumented one.

> 🚨 **A known residual on `RecycleNode`, stated rather than assumed.** Its wait works by queue
> order: the read is issued after the dispose, so it lands behind it *at the target*. That holds
> because both deliveries leave the **same** hub — for every documented caller both seams are the
> identity function. A **root-hub** caller, the one this class's rule already excludes, hops to two
> different off-router hubs (`portal/nodeops-{meshId}` and `portal/reads-{meshId}`) whose action
> blocks are independent, so the read can in principle be answered by the still-live hub and
> `RecycleNode` emits before a fresh activation. That is **not** a cost of adopting the seam: before
> it, a root-hub caller posted the dispose from `mesh/{id}` and the read from `portal/reads-{meshId}`
> — two hubs then as well, with the router on an end of the teardown besides. Closing it properly
> means an explicit disposal-completion barrier instead of relying on queue order, which is a
> different change with its own design.

## Reading a report

Expand All @@ -159,7 +240,20 @@ another process.
and both genuine end roles must still fire.
- `RouterTrafficOnNodeCreateFromTheRootHubTest` — the create seam (`IMeshService.CreateNode` issued
from the root hub keeps the router off both ends) plus the origin site's call-site attribution,
each with its own positive control in the same fixture.
each with its own positive control in the same fixture. It also carries the two RECYCLE surfaces
driven from the ROOT hub — `MeshOperations.Recycle` and `HubRecycleExtensions.RecycleNode`. Those
exist because **every other test of both verbs drives them through a session or client hub, where
the seam is the identity function**, so a revert to `hub.Post` would leave the whole existing
recycle suite green. Reverted in rehearsal, they reproduce #4463's production line byte for byte,
down to the frame: `at MeshWeaver.AI.MeshOperations+<>c__DisplayClass95_0.<RecycleCore>b__3`.
Each asserts a positive anchor first (`status=Recycled`; the node coming back), because the verb
answers a refusal *without posting anything*, and "no router traffic" is trivially true of an
operation that never ran.
- `RouterAsNodeOperationOriginRatchetGuard` — the `src/` ratchet, in two tests that fail
independently: one asserts no new router-issued lifecycle site, the other asserts that the
denominator was actually DERIVED (both families non-empty, every derived name resolving to a real
type, `HeartBeatEvent` dropped by `RouterTrafficRule` rather than by a literal). The second exists
because a broken derivation makes the first go green over an unguarded tree.

## See also

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
Name: Recycling a node no longer competes with the portal's routing
Category: Fix
Description: The Recycle tool and the Compile button used to tear a node's hub down from the mesh's router itself, so a burst of recycles queued behind — and ahead of — everyone else's page loads. Both recycle surfaces now issue the teardown off the router, and the guard that is supposed to catch this class was widened so it can no longer miss a message type.
Icon: ArrowSync
Order: -20260916
---

# Recycling a node no longer competes with the portal's routing

The mesh's root hub is its **router**: every message between two hubs passes through it, one at a
time. Work executed there does not merely run slowly — it takes turns away from routing, and a burst
of it starves the ordinary `SubscribeRequest` traffic that makes pages appear. That is what wedged a
production portal on 2026-06-11.

`Recycle` — the MCP tool, and the **Compile** button that shares its code path — posted its teardown
straight from the router. So did `RecycleNode`, the framework's one recycle surface. Neither was
slow on its own; both put node-hub lifecycle on the one action block that must never queue. Both now
hand the teardown to the mesh's dedicated off-router hub first. For every caller that already held a
session, portal or per-node hub — which is nearly all of them — nothing changes at all: the hop is
the identity function for any hub that is not the router.

## The half that mattered more

A ratchet already existed to stop exactly this, and it did not see it. It matched a **literal list of
six node-CRUD request names**, and a teardown is not one of them — so the most router-hostile message
in the mesh was invisible to it, and the only thing that noticed was a runtime detector firing in
production ([#4463](https://github.com/Systemorph/MeshWeaver/issues/4463)). Adding "teardown" to the
list would have bought exactly one more message.

The guard now **derives** what counts as lifecycle from the framework's own handler registrations —
the ones a hub makes for itself at construction, and the ones the node-operation hub makes — and
filters them through the same predicate the runtime detector evaluates. You cannot add a lifecycle
message without registering a handler for it, so a new one joins the guard automatically; a name in
a test can be forgotten, a handler cannot. On its first run the derivation picked up a verb the
hand-written list had already missed.

What this does not change: the older, broader
[#1140](https://github.com/Systemorph/MeshWeaver/issues/1140) family stays open. Its remaining strand
is a stream unsubscribe, which has no equivalent seam and whose origin hub is a correlation question
rather than a routing one — see
[Router Traffic Detection](/Doc/Architecture/RouterTrafficDetection).
30 changes: 29 additions & 1 deletion src/MeshWeaver.Mesh.Contract/HubRecycleExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,17 @@ public static class HubRecycleExtensions
=> hub.WhenNoInstallHoldsRoot(path)
.SelectMany(_ => Observable.Defer(() =>
{
hub.Post(
// 🚨 OFF-ROUTER ISSUING (#4463). Every DOCUMENTED caller already holds a surviving
// non-router hub — that is this class's own rule — but this is a PUBLIC extension
// on `IMessageHub`, so nothing stops a mesh-singleton service from calling it with
// the DI-injected hub, which in the mesh's root container IS the ROUTER. The seam
// is the identity function for every hub whose address type is not the mesh type,
// so the documented callers are byte-for-byte unaffected and the undocumented one
// cannot make the router an end of a teardown delivery. The `hub.Address` in the
// fallback reason below is deliberately the CALLER's address, not the issuing
// hub's: it answers "who asked", which a reader of the target's [QUIESCE-START]
// needs, and `portal/nodeops-{meshId}` would answer nothing.
hub.NodeOperationIssuingHub().Post(
new DisposeRequest
{
Reason = reason
Expand All @@ -105,6 +115,24 @@ public static class HubRecycleExtensions
// and is answered by the reactivated hub (or NACKed ShuttingDown and re-probed until it
// is). Deliberately NOT ReadTimeoutBehavior.EmitNull: "I could not tell" must reach the
// caller as an error, because the caller's next act is to send a user somewhere.
//
// 🚨 THE PRECONDITION OF THAT ORDERING, stated rather than assumed. It holds because
// BOTH deliveries leave the SAME hub: for every caller this class documents — one
// holding a surviving non-router hub — `NodeOperationIssuingHub()` above and the
// `ReadIssuingHub()` inside `GetMeshNode` are each the identity function, so the
// dispose and the read are posted by the caller's own action block in that order.
// A ROOT-HUB caller is the exception, and it is the caller this class's own rule
// already excludes: there the two hop to DIFFERENT off-router hubs
// (`portal/nodeops-{meshId}` and `portal/reads-{meshId}`), whose action blocks are
// independent, so the read can in principle reach the target first and be answered
// by the still-live hub — whereupon this method emits before a fresh activation.
// 🚨 That is NOT introduced by the seam adoption above and must not be read as its
// cost: before it, a root-hub caller posted the dispose from `mesh/{id}` and the
// read from `portal/reads-{meshId}` — two hubs then as well, and with the router on
// an end of the teardown besides. Closing it properly means an explicit
// disposal-completion barrier rather than relying on queue order, which is a
// different change with its own design; it is named here, in the same register as
// the two-installs-one-root residual on this class, rather than papered over.
return hub.GetMeshNode(path, budget ?? DefaultRecycleBudget);
}));

Expand Down
Loading
Loading