Repository navigation
feat(recycle): a NodeType recycle cascade is ONE ActivationRecycle request (#6520) - #6810
Conversation
…quest (#6520) The definition hub's cascade no longer asks every partition for the stored instances of each type in its network. It reads the NodeType definitions once (dependents, shared per wave), disposes the dependent definitions' own hubs with CascadedFrom, and writes one Admin/_Recycle request naming the type and its dependents with the dispose's reason. Every process disposes only the live activations it hosts; cold addresses are never touched. ActivationRecycle's disposals now carry CascadedFrom = the request, so none re-cascades. DependencyNetwork (used by control-lane plans and Plugins RecycleRunner) is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
Query-dependent issuance and incomplete request projections can prevent the new cascade from recycling live activations.
2 open findings
What changed in this PR
Replaces NodeType cascade instance queries with one activation-recycle request, reducing query load while targeting live hubs across processes.
Changes:
- Publishes one request for the type and its dependents.
- Marks wave disposals to prevent recursive cascades.
- Adds monolith and two-silo regression tests and updates documentation.
| File | Description |
|---|---|
test/MeshWeaver.Hosting.Orleans.Test/TwoSiloNodeTypeRecycleTest.cs |
Tests recycling across two silos. |
test/MeshWeaver.Graph.Test/NodeTypeRecycleCascadeTest.cs |
Tests request composition. |
test/MeshWeaver.Graph.Test/ANodeTypeRecycleIsOneActivationRecycleTest.cs |
Tests one-wave recycling without instance queries. |
src/MeshWeaver.Graph/Configuration/NodeTypeRecycleCascade.cs |
Publishes a shared activation-recycle request. |
src/MeshWeaver.Graph/Configuration/ActivationRecycle.cs |
Prevents wave disposals from cascading again. |
src/MeshWeaver.Documentation/Data/Architecture/StaleStateUntilRecycle.md |
Documents definition-triggered requests. |
src/MeshWeaver.Documentation/Data/Architecture/HubDisposalModel.md |
Explains the revised cascade and failure behavior. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| : ActivationRecycle.Request(mesh, recycle) | ||
| .Select(ticket => (Network: network, Request: recycle, Ticket: (ActivationRecycleTicket?)ticket)); |
There was a problem hiding this comment.
Checked against the provider itself rather than the doc, and the premise no longer holds, so no change here.
The Postgres query projection lives in MeshWeaver.Plugins, src/MeshWeaver.Hosting.PostgreSql/PostgreSqlSqlGenerator.cs. On its current main, GenerateSelectQuery selects {AuthorColumns}, which is n.created_by, n.last_modified_by, n.created_date. The UNION arms carry typed NULLs in the same positions (SyntheticAuthorColumns), but those arms are synthetic content-chunk rows. A request node at Admin/_Recycle/{id} is a real mesh_nodes row. So a query row for it carries CreatedBy, and the System-only check in ActivationRecycle.Handle sees the real creator.
The sentence you quoted from CqrsAndContentAccess.md:349 is stale: it describes the projection before the author columns were added. That doc correction belongs to a separate change and does not alter this PR's behaviour.
The general point is right: a query row is not the node, and the point read is authoritative. Moving Handle to GetMeshNodeStream(path) would be a separate change to the existing activation-recycle agent, and it would apply equally to the direct-recycle path that already uses it. It is not a defect this PR introduces.
| : ActivationRecycle.Request(mesh, recycle) | ||
| .Select(ticket => (Network: network, Request: recycle, Ticket: (ActivationRecycleTicket?)ticket)); |
There was a problem hiding this comment.
Both scenarios are real edges, but neither is a regression this PR introduces. Both fail loudly, never silently, so I'm answering here and not pushing.
Stalled provider. Before this change, the cascade also reached every live activation through the index: DependencyNetwork enumerated the instances of the type and of each dependent with Query(nodeType:…) under the same EnumerationBudget. With a stalled provider, those legs returned INCOMPLETE and no instance was recycled. Now ActivationRecycle.Request's TypeExists/Exists probes time out, and the cascade logs at Error: "the recycle faulted — NOTHING in its dependency network was recycled". The outcome is the same, and it is still reported. You are right that the catch message in DependentTypes ("its own live activations still are") is optimistic in that one case. That is wording, not behaviour.
Deletion during the read. If the space's teardown wins, the deleted type's own activations are in the partition being torn down, so they go with it. What is left is the case you name: dependents OUTSIDE that partition, whose type embeds sources from a type that no longer exists. Their next compile fails regardless, so the build they would be re-bound to is not a newer one. The refusal is logged at Error naming every type it covered, so it is not a silent skip.
An internal issuance path that skips the persisted-type probe for types the cascade has already established is a reasonable hardening. I'm leaving it as a follow-up, not widening this PR.
Test Results 15 files 15 suites 48m 54s ⏱️ Results for commit 6a46b40. |


Refs #6520 — this supersedes the cascade's cross-partition instance query (one
nodeType:{type}read per network type, plus oneDisposeRequestper stored instance).What changes
NodeTypeRecycleCascade.Cascade(the seamMessageHub.HandleDisposeruns on a direct, non-cascaded recycle of a NodeType definition hub) now:NodeTypeRecycleCascade.DependentTypes, shared per wave byRecycleCascadeReadsand issued from the read-issuing survivor (NodeType recycle cascade gives up on instance enumeration when its captured DI scope is already disposed, leaving instance hubs on stale builds #5099).DisposeRequest { CascadedFrom = type }to each dependent definition's own hub. There are only a handful of these, and a cold one is a no-op at the router. They are disposed by address because an activation recycle refuses to nameNodeType.ActivationRecyclerequest (Admin/_Recycle/{id}, as System) throughActivationRecycle.Request(meshHub, …). It names the type plus every dependent, carries the dispose's reason unchanged, and setsRequestedBy: RecycleCascade from '<type>'. The request is built by the pureNodeTypeRecycleCascade.RecycleRequest. Every process then disposes only the per-node hubs it hosts whose bound NodeType is named, paced and off the router, and writes its audit row. No index of stored instances is read, so cold addresses are never touched.ActivationRecycle.RecycleLocalnow setsCascadedFrom = <request path>on every disposal, so nothing a wave disposes fans out again. This is the same rule #6800 applies to the module-swap wave.Unchanged:
CascadedFromsemantics: a cascaded dispose never re-cascades.recycleverb: theUpdatecheck, and RequestedRelease stamping on NodeType nodes. Nothing in that path was touched.NodeTypeRecycleCascade.DependencyNetwork,EnumerationLegandDependencyNetworkResult. These are the address enumeration that control-lane plans and PluginsRecycleRunnerbind a blast radius to. The cascade no longer runs it.A failed definitions read still recycles the type's own live activations and logs
ErrorINCOMPLETE, naming the dependents as not reached. A refused request (for example, an unknown type) is logged atError.Compatible with #6800 (still open). That PR's swap-wave disposals already carry
CascadedFrom, and this one does not touchModuleLiveUpdater.Tests (local, Release)
ANodeTypeRecycleIsOneActivationRecycleTest(Graph.Test, monolith). The setup uses real NodeType declarations whose own hubs carry the real seam, and a routedDisposeRequestto the definition. The instance enumerations of T and D are held in flight and never answered, and the definitions read is held and answered (T ← D).NodeTypes == [T, D]and the reason unchanged.Expected the observable to emit a value within 36s because T1 is a live activation of the recycled type, but it did not.CascadedFromdropped from the dependent's dispose: failed,Expected value to be "…/CascadeT2,TestData/CascadeD1,TestData/CascadeT1" … but found "…/CascadeT2,TestData/CascadeT1".D re-cascaded and its own request took D1.TwoSiloNodeTypeRecycleTest(Hosting.Orleans.Test, two silos sharing the change feed). The same holds apply on the silo that runs the cascade.Expected the observable to emit a value within 72s because TwoSiloCascade/T1 on silo 0 is a live activation of the network, but it did not.NodeTypeRecycleCascadeTest.TheRecycleRequest_NamesTheTypeAndItsDependents_Once_WithTheReason(pure).FullyQualifiedName~Recycle: 80/80.~Recycle: 4/4.~RecycleCascade: 3/3.-c Release -warnaserror, one project per invocation, all with 0 warnings and 0 errors: Graph, Documentation, Graph.Test, Hosting.Orleans.Test, Messaging.Hub.Test, Updates.Test.Docs
Doc/Architecture/HubDisposalModel, section "A recycle of the main bit": the network, the one request, why there is no instance index, the failed-read rule, and the repros. The section now also says that a NodeType recycle reaches every replica.Doc/Architecture/StaleStateUntilRecycle, section "Recycling a type": a new bullet saying that a definition recycle is one of these requests.Not established
nodeType:{type}cross-partition reads on a recycle wave) should be checked in the nextLogswindow after roll: no[RecycleCascade] … INCOMPLETE … instances oflines, and oneAdmin/_Recyclerequest per recycled type.Admin/_Recyclerequest plus one report per process. Nothing prunes these yet.Pairs-with: none — no public type or member removed (
git diff origin/main -- src | grep '^-\s*public 'is empty); only additions (NetworkTypes,DependentTypes,RecycleRequest).Implementers: none — no member added to a public interface or abstract class.
🤖 Generated with Claude Code