A message arriving mid-activation could overtake the queue that ordered it (fixes the kernel CS0103 reorder) by rbuergi · Pull Request #1155 · Systemorph/MeshWeaver

rbuergi · 2026-08-10T15:30:15Z

Root cause

RoutingServiceBase.RouteInMesh preserves FIFO for a not-yet-activated address through the per-address ActivationSerializer (bef2604) — but its hosted-hub short-circuit consults only "is the hub registered?". A hub becomes registered the moment its construction completes (HostedHubsCollection.GetHub publishes it to messageHubs BEFORE the serializer has drained the deliveries queued behind the activating message). A message arriving in that window was handed straight to the hub's mailbox and overtook every delivery still in the backlog.

For MonolithKernelTest.ThreeSubmissions_ShareState: the kernel-session Activity hub is activated concurrently by the test's GetRemoteStream subscriptions racing the three SubmitCodeRequest posts. When the activation completes between the arrival of submission 1 (already queued in the serializer) and submissions 2/3, the later submissions short-circuit past it — "use sharedValue" reaches the REPL before "define sharedValue" → CS0103. Three submissions + two streams put more messages in the backlog and more arrival gaps for the registration to land in than the 2-submission sibling — matching the observed discriminator (2 passes, 3 fails in the same runs).

Everything else on the path was verified order-preserving: client action block → mesh mailbox → RouteInMesh (serial per mailbox), the ActivationSerializer Concat pump, KernelContainer's eager Observe forward, hosted-hub mailboxes, and KernelExecutor's Select(RunSubmission).Concat() REPL queue.

Deterministic repro

ActivationBacklogFifoTest scripts the exact interleaving with a timing-only gate around the real PathResolutionService (no data faked, no sleeps — every step is fenced on an observable condition): A parks in activation (serializer alive), B joins the backlog (fenced), A is released — hub registered, B parked in its serializer turn — and C is posted in exactly that window. Before the fix the probe hub processes A, C, B on every run; after the fix A, B, C. Verified fail-before/pass-after in both orders with the final test shape.

Fix

In RouteInMesh, a still-live ActivationSerializer for the address takes precedence over the hosted-hub short-circuit: the message joins the queue, and Concat preserves total arrival order. The serializer self-retires the moment its backlog drains — TryEnqueue then returns false — so the steady-state direct path is untouched.

Verification

ActivationBacklogFifoTest: deterministic A, C, B fail before, green after (both verified).
MonolithKernelTest (MeshWeaver.Persistence.Test): 8/8 green, incl. ThreeSubmissions_ShareState, MultipleSubmissions_ShareKernelState, and both interactive-showcase pipelines.
MonolithMeshTest + ShutdownRoutingRejectClassificationTest + SubscribeRequestIdentityRoutingTest: 23/23 green; KernelReplChainingTest green.
WhatsNewEntryIntegrityTest green.
dotnet build -c Release -warnaserror clean for the touched chains (MeshWeaver.Hosting via Hosting.Monolith.Test, MeshWeaver.Documentation via Documentation.Test).

Fixes #1145

🤖 Generated with Claude Code

https://claude.ai/code/session_01UYLgoPK1qLtyxZ2ixdFtj5

…ed it Three back-to-back kernel submissions intermittently lost shared state on the monolith path (CS0103: 'sharedValue' does not exist): a later submission reached the REPL before the one defining the variable. RouteInMesh keeps FIFO for a not-yet-activated address through the per-address ActivationSerializer - but its hosted-hub short-circuit consulted only "is the hub registered?". A hub becomes registered the moment its construction completes (HostedHubsCollection.GetHub publishes it BEFORE the serializer has drained the deliveries queued behind the activating message), so a message arriving in that window was handed straight to the hub's mailbox, overtaking every delivery still in the backlog. For the kernel test, the session hub's activation (driven by the GetRemoteStream subscriptions racing the posts) completed between the arrivals of submission 1 and submissions 2/3 - "use sharedValue" executed before "define sharedValue". ActivationBacklogFifoTest pins the leapfrog deterministically with a timing-only gate around the real PathResolutionService: before the fix the probe hub processes A, C, B on every run; after it, A, B, C. Fix: while an activation serializer for the address is still live, every message joins its queue (Concat preserves total arrival order). The serializer self-retires when the backlog drains, so the steady-state direct path is untouched. Fixes #1145 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UYLgoPK1qLtyxZ2ixdFtj5

Copilot

Pull request overview

This PR fixes a routing FIFO violation during hub activation in the monolith path: messages posted while an address’s activation backlog is still draining could bypass the ActivationSerializer once the hub becomes “hosted/registered”, causing later messages to overtake earlier queued deliveries (e.g., triggering kernel CS0103 by running “use variable” before “define variable”).

Changes:

Update RoutingServiceBase.RouteInMesh to prioritize enqueuing into an already-live ActivationSerializer before taking the hosted-hub short-circuit path.
Add ActivationBacklogFifoTest to deterministically reproduce (and pin) the specific mid-activation reordering window.
Add a What’s New entry documenting the user-visible fix (“code cells no longer forget earlier cells”).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
test/MeshWeaver.Hosting.Monolith.Test/ActivationBacklogFifoTest.cs	Adds a deterministic regression test that proves messages A/B/C preserve post order through the activation backlog window.
src/MeshWeaver.Hosting/RoutingServiceBase.cs	Fixes the ordering hole by enqueuing into an existing, still-draining activation serializer before routing directly to a newly hosted hub.
src/MeshWeaver.Documentation/Data/WhatsNew/2026-08-10-code-cells-no-longer-forget-earlier-cells.md	Documents the user-facing symptom and the behavioral guarantee restored by the routing fix.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-08-10T15:44:47Z