Repository navigation
RecycleCascade instance enumeration uses a disposed DI scope, leaving instance hubs unreycled #5100
Description
What is failing
When NodeTypeRecycleCascade recycles a NodeType, the leg that enumerates the live instances of the type (here Hosting/TriageItem) via MeshService.Query throws ObjectDisposedException: the Autofac LifetimeScope behind the AutofacServiceProvider was already disposed. The cascade marks that leg INCOMPLETE, and per the message the type's live instance hubs keep their activations until recycled by hand.
Probable cause
Use-after-dispose of a captured DI scope (high confidence, ~80%): the exception is thrown by AutofacServiceProvider.GetService from MeshService.Query (MeshService.cs:307), so the DI container itself is dead, not the data layer. NodeTypeRecycleCascade.DependencyNetwork (NodeTypeRecycleCascade.cs, ~line 223) resolves MeshService lazily inside SubscribeScopedObservable (ImpersonationScopeExtensions.cs:210); by the time the enumeration runs, an earlier leg of the same cascade or the hub teardown the cascade triggered has disposed that scope.
Impact
Low so far: 2 occurrences today, 2 minutes apart, on a single pod. But the failure mode is functional, not merely noisy — instance enumeration is skipped, so live hubs of the affected NodeType keep stale activations and require manual recycling. If this fires on every large recycle, stale hubs accumulate.
Where to look
MeshWeaver.Graph/Configuration/NodeTypeRecycleCascade.cs—DependencyNetwork, the lambda at ~line 223 callingMeshService.Query.MeshWeaver.Hosting/MeshService.cs—Query<T>(~line 307) resolving from a possibly-disposedAutofacServiceProvider.MeshWeaver.Messaging.Hub/ImpersonationScopeExtensions.cs(~line 210), where the scoped observable defers resolution.
Fix direction: resolve the instance list eagerly before starting the recycle, or catch ObjectDisposedException and re-resolve against a live scope instead of marking the leg INCOMPLETE.
Possibly related
Same ObjectDisposedException-during-teardown family as incidents 0857ac25 (HostedHubsCollection), fcdeab75 (MeshNodeStreamCache UpdateQueue), 47bf50c9 / 4b361ba (MeshNode DeleteNode — the latter seen today too). Worth checking whether they share one disposed-scope root.
Evidence
| Fingerprint | 8bd12828ac02105b |
| Category | MeshWeaver.Graph.Configuration.NodeTypeRecycleCascade |
| Severity | Error |
| Exception | System.ObjectDisposedException |
| Top frame | Autofac.Core.Lifetime.LifetimeScope.ThrowDisposedException() |
| Namespace | memex |
| Pods | memex-portal-deployment-75b545bbd-gwrc6 |
| Occurrences | 2 |
| First seen | 2026-09-21 08:05:16Z |
| Last seen | 2026-09-21 08:07:20Z |
| Routing | not determined — no configured route matches the category MeshWeaver.Graph.Configuration.NodeTypeRecycleCascade. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject. |
Recent log lines
2026-09-21 08:05:16Z memex-portal-deployment-75b545bbd-gwrc6 fail: MeshWeaver.Graph.Configuration.NodeTypeRecycleCascade[0]
[RecycleCascade] Enumerating the instances of Hosting/TriageItem failed — its live instance hubs keep their activations until recycled by hand. This leg is reported as INCOMPLETE rather than as zero instances
System.ObjectDisposedException: Instances cannot be resolved and nested lifetimes cannot be created from this LifetimeScope as it (or one of its parent scopes) has already been disposed.
at Autofac.Core.Lifetime.LifetimeScope.ThrowDisposedException()
at Autofac.Extensions.DependencyInjection.AutofacServiceProvider.GetService(Type serviceType)
at MeshWeaver.Hosting.MeshService.Query[T](MeshQueryRequest request) in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Hosting/MeshService.cs:line 307
at MeshWeaver.Graph.Configuration.NodeTypeRecycleCascade.<>c__DisplayClass5_1.<DependencyNetwork>b__6() in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Graph/Configuration/NodeTypeRecycleCascade.cs:line 223
at MeshWeaver.Messaging.ImpersonationScopeExtensions.SubscribeScopedObservable`1.Subscribe(IObserver`1 observer) in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Messaging.Hub/ImpersonationScopeExtensions.cs:line 210
2026-09-21 08:07:20Z memex-portal-deployment-75b545bbd-gwrc6 fail: MeshWeaver.Graph.Configuration.NodeTypeRecycleCascade[0]
[RecycleCascade] Enumerating the instances of Hosting/TriageItem failed — its live instance hubs keep their activations until recycled by hand. This leg is reported as INCOMPLETE rather than as zero instances
System.ObjectDisposedException: Instances cannot be resolved and nested lifetimes cannot be created from this LifetimeScope as it (or one of its parent scopes) has already been disposed.
at Autofac.Core.Lifetime.LifetimeScope.ThrowDisposedException()
at Autofac.Extensions.DependencyInjection.AutofacServiceProvider.GetService(Type serviceType)
at MeshWeaver.Hosting.MeshService.Query[T](MeshQueryRequest request) in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Hosting/MeshService.cs:line 307
at MeshWeaver.Graph.Configuration.NodeTypeRecycleCascade.<>c__DisplayClass5_1.<DependencyNetwork>b__6() in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Graph/Configuration/NodeTypeRecycleCascade.cs:line 223
at MeshWeaver.Messaging.ImpersonationScopeExtensions.SubscribeScopedObservable`1.Subscribe(IObserver`1 observer) in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Messaging.Hub/ImpersonationScopeExtensions.cs:line 210
Opened automatically from Admin/_LogIncident/8bd12828ac02105b. Recurrences are folded into this issue rather than opening new ones.
Activity
systemorph-com commentedon Sep 21, 2026
Still occurring: 1 more since the last update (3 total).
memex-portal-deployment-75b545bbd-gwrc6Recent log lines
systemorph-com commentedon Sep 21, 2026
Still occurring: 1 more since the last update (4 total).
memex-portal-deployment-75b545bbd-gwrc6Recent log lines
systemorph-com commentedon Sep 21, 2026
Still occurring: 1 more since the last update (5 total).
memex-portal-deployment-75b545bbd-gwrc6Recent log lines
systemorph-com commentedon Sep 21, 2026
Still occurring: 4 more since the last update (9 total).
memex-portal-deployment-75b545bbd-gwrc6Recent log lines
rbuergi commentedon Sep 21, 2026
Closes as duplicate of #5099.
Reason: same defect — the NodeType recycle cascade's captured Autofac
LifetimeScopeis disposed by the cascade's own wave, so the deferred instance-enumeration lambda inNodeTypeRecycleCascade.DependencyNetworkthrowsObjectDisposedExceptionfromAutofacServiceProvider.GetServiceand that leg recycles nothing.Evidence migrated to #5099: this incident (
8bd12828ac02105b) is the one that captured the full stack, and #5099 already cites it for the source locations:MeshWeaver.Graph/Configuration/NodeTypeRecycleCascade.cs~line 223 (the lazy enumeration), resolving throughMeshWeaver.Hosting/MeshService.cs~line 307 (MeshService.Query<T>) viaMeshWeaver.Messaging.Hub/ImpersonationScopeExtensions.cs~line 210 (SubscribeScopedObservable). 2 occurrences 2026-09-21 08:05–08:07Z. Fingerprint8bd12828ac02.