Repository navigation
fix(hub): a BuildupAction never starts after its hub has begun shutting down - #3109
Merged
meshweaver-cloud[bot] merged 2 commits intoSep 2, 2026
Merged
Conversation
…ng down The init turn is queued at Build and runs whenever the action block reaches it — for a transient probe (ContentTypeRegistration.ProbeRegister, the schema probes) that is after the probe's own Dispose(). The Thread NodeType's AddThreadExecution chain then created its _Exec child on a hub that was already leaving, and HostedHubsCollection refused it with a Warning in 159 of 643 per-test logs of CD run 33619142646; ProbeHubCostTest reds whenever that late creation lands inside its recording window. HandleInitialize now checks IsShuttingDown at every action boundary and skips the rest — the gate still opens, nothing is installed. The init-turn form of the #3026/#3072 watcher rule. Pinned by InitializationStopsAtTeardownStartTest. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4xPnGYEbdu8AVvR5jytyj
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The new test’s SpinWait park ignores the timeout result, which can make failures less diagnosable (and risks not exercising the intended interleaving under extreme slowness).
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Ensures hub initialization (BuildupActions in InitializeHubRequest) does not start additional actions once the hub has begun shutting down, preventing late control-plane installation (e.g., hosted-child creation) during disposal and eliminating the observed disposal-time warning/flaky teardown faulting.
Changes:
- Gate each
BuildupActionboundary inMessageHub.HandleInitializeonIsShuttingDown, skipping remaining actions (with a single debug log) while still opening the Initialize gate. - Add a deterministic regression test that disposes the hub mid-initialization and asserts later actions never start, plus a control-arm test for the live-hub case.
- Document the initialization rule in the hub disposal model and add a user-facing “What’s New” Fix entry.
File summaries
| File | Description |
|---|---|
| src/MeshWeaver.Messaging.Hub/MessageHub.cs | Adds IsShuttingDown boundary checks around each initialization action so late actions don’t start during teardown while keeping the init gate semantics intact. |
| test/MeshWeaver.Messaging.Hub.Test/InitializationStopsAtTeardownStartTest.cs | New regression + control-arm tests that pin the “no action starts after shutdown begins” rule. |
| src/MeshWeaver.Documentation/Data/Architecture/HubDisposalModel.md | Documents the initialization-specific shutdown rule alongside existing teardown model guidance. |
| src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-02-a-host-that-is-leaving-installs-nothing.md | Adds a user-visible Fix entry describing the behavioral change and the eliminated warning. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
Contributor
Contributor
Contributor
Contributor
Contributor
Contributor
…from an init action the rule now skips ContentTypeProbeControlPlaneTest waited for its two swept probes to REACH the Activity Control Plane install point — an AsyncSubject completed from inside the observable WithInitialization action. That is exactly the action HandleInitialize now skips once IsShuttingDown is true, and ContentTypeRegistration.ProbeRegister disposes the probe on the line after building it, so on CI (run 33624577907, shard 4) and locally the Dispose() lands before the init turn every time: the signal never fires and both #2990 tests time out after TestTimeouts.Quick having asserted nothing. The signal was obsolete, not the assertions. Registration itself is untouched by the skip: AddMeshDataSource composes WithContentType into AddData, whose RegisterWorkspaceTypes is a SYNCHRONOUS buildup action inside Build (DataContext.Initialize -> IMeshContentTypeRegistry .Register), so the sweep's build-and-dispose shape registers every static type whether or not the probe's init turn ever runs. The test now captures each probe's DisposalCompleted from the synchronous WithInitialization overload (runs inside Build, provably before the sweep's Dispose()) and waits on that: the ShutdownRequest is queued behind the init turn, so disposal completing means the init turn ended (run or skipped) and the teardown wrote whatever it writes. The ACP install stays in the observable overload, so whenever the init turn does win the race the #2990 guards are still exercised; which way the race went is written to the test output. Also records a timed-out park in InitializationStopsAtTeardownStartTest (Copilot review: the SpinUntil result was ignored) so a budget-ended park is reported as such and not as the rule under test failing, and documents in HubDisposalModel what the skip does NOT touch and how a test observes a probe. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G4xPnGYEbdu8AVvR5jytyj
This was referenced Sep 2, 2026
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A BuildupAction never starts after its hub has begun shutting down
Red: core CD run 33619142646 (head
eb172af0) —Module bundle (MeshWeaver.AI)→ProbeHubCostTest.ValidateContentWithSchema_OnInvalidContent_BuildsOneProbeNotTwofailed on itslast assertion,
recorder.Records.Where(IsTeardownFault).Should().BeEmpty()→ 1 item:Root cause
ContentTypeRegistration.ProbeRegister(the boot-time sweep, since #2973/2026-09-01) builds atransient probe hub for every static NodeType with a HubConfiguration and disposes it in the same
breath.
Dispose()closes hosted-hub creation synchronously; the probe's init turn(
InitializeHubRequest, queued at Build) then runs whenever the action block reaches it — for theThread NodeType that is
AddThreadExecution's sixWithInitializationactions, the fourth of whicheagerly creates the
_Execchild hub (HostedHubCreation.Always). On a hub that is alreadyleaving, that creation is refused with the Warning above. It is not a regression of one commit in
470cdd58e..eb172af0: none of the involved code moved in the range (sweep,HostedHubsCollection,the immediate
CloseCreationinDispose()), and the run's own evidence artifact shows the Warningin 159 of 643 per-test logs, at a random offset from each boot —
ProbeHubCostTestis simply theone test asserting a fault-free probe teardown, and it reds whenever the late creation lands inside
its recording window. The interleaving does not occur on an idle developer machine (both
eb172af0and #3098's tree pass locally with no Warning anywhere in the class lifetime), so it cannot be
bisected locally.
Fix (core seam, the init-turn form of #3026/#3072)
MessageHub.HandleInitializechecksIsShuttingDownat every BuildupAction boundary: once teardownhas begun the remaining actions are skipped (one Debug line), the Initialize gate still opens so the
disposal state machine flows, and nothing is installed on a hub that is leaving. This is the same
policy the existing
.Catchapplies one step later ("teardown ENDED an action" is a recognisedshutdown, not a failure).
src/MeshWeaver.Messaging.Hub/MessageHub.cs— the boundary check.test/MeshWeaver.Messaging.Hub.Test/InitializationStopsAtTeardownStartTest.cs— parks the firstaction on the action block, calls
Dispose(), releases the park: the second action (which creates achild hub, the
_Execshape) must not have run byDisposalCompleted. Fails before / passes after.A control arm on a live hub proves the chain runs to the end and the child exists.
Doc/Architecture/HubDisposalModel.md— "The rule for initialization" under "The first instant ofteardown". What's New entry (Fix): the Warning appeared on most portal starts.
Verified
dotnet build test/MeshWeaver.Messaging.Hub.Test -c Release -warnaserrorclean; the whole testproject green; the new test red with the
MessageHub.cschange reverted.MeshWeaver.AI.Test --filter ProbeHubCostTestgreen against this tree.Left out, deliberately
The Thread NodeType's
AddThreadExecutionstill installs its full control plane on any transientprobe that lives long enough to answer a request (a
_schema_validation/probe validating Threadcontent). That is wasted work, not a fault, and it lives in MeshWeaver.Plugins — a follow-up there
(skip the chain when the configuration carries
TransientNodeProbe).🤖 Generated with Claude Code
Claude-Session: https://claude.ai/code/session_01G4xPnGYEbdu8AVvR5jytyj