Repository navigation
Pod memory exhaustion throws OutOfMemoryException during MessageHub.Dispose; disposal join falls back to immediate settle #5554
Description
What is failing
During disposal of the hub sync/8oofJD7XRk-aqyAgMuMbNg, the Rx disposal pipeline in MeshWeaver.Messaging.MessageHub.Dispose() (enumerating disposables via ToObservableRecursive/Merge) threw System.OutOfMemoryException. HostedHubsCollection caught it and settled the disposal join for that hub immediately instead of waiting on its completion signal — the designed fallback, so disposal did not hang.
Probable cause
Cause unclear; evidence points at pod-level memory exhaustion rather than a MeshWeaver defect. Confidence: low-medium. The OOM surfaces at the deepest Rx allocation site during an allocation-heavy dispose phase, which is consistent with the process already being near its container memory limit (e.g. under GC pressure during a shutdown/deployment). The dispose logic itself behaved correctly per its own error handling. Recommend checking pod memory metrics/limits for memex-portal-deployment-777677694-cc6zz around the timestamp and whether the pod was terminating.
Impact
Minimal as a standalone event: 1 occurrence, 1 pod, single window. The hub's disposal completed via the fallback join, so no hang or leak was observed at this site. However, an OOM anywhere is a symptom of memory pressure — if it recurs, escalate to investigating the portal pods' memory budget.
Where to look
MeshWeaver.Messaging.MessageHub.Dispose() and the HostedHubsCollection disposal join (log category MeshWeaver.Messaging.HostedHubsCollection); the stack runs through System.Reactive ToObservableRecursive/Merge, driven by the hub's disposable list.
Evidence
| Fingerprint | 619edb4566176ac8 |
| Category | MeshWeaver.Messaging.HostedHubsCollection |
| Severity | Error |
| Exception | System.OutOfMemoryException |
| Top frame | MeshWeaver.Messaging.MessageHub.Dispose() |
| Namespace | memex-cloud |
| Pods | memex-portal-deployment-777677694-cc6zz |
| Occurrences | 1 |
| First seen | 2026-09-23 14:22:38Z |
| Last seen | 2026-09-23 14:22:38Z |
| Routing | not determined — no configured route matches the category MeshWeaver.Messaging.HostedHubsCollection. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject. |
Recent log lines
2026-09-23 14:22:38Z memex-portal-deployment-777677694-cc6zz fail: MeshWeaver.Messaging.HostedHubsCollection[0]
Error during disposal of hub sync/8oofJD7XRk-aqyAgMuMbNg — its disposal can no longer complete, so the join settles this hub now instead of waiting on a signal it will never send
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Reactive.Concurrency.ImmediateScheduler.Schedule[TState](TState state, Func`3 action)
at System.Reactive.Linq.ObservableImpl.ToObservableRecursive`1._.Run(IEnumerable`1 source, IScheduler scheduler)
at System.Reactive.Producer`2.SubscribeRaw(IObserver`1 observer, Boolean enableSafeguard)
at System.Reactive.Linq.ObservableImpl.Merge`1.Observables.Run(_ sink)
at System.Reactive.Producer`2.SubscribeRaw(IObserver`1 observer, Boolean enableSafeguard)
at System.Reactive.Linq.ObservableImpl.Defer`1._.Run()
at System.Reactive.Producer`2.SubscribeRaw(IObserver`1 observer, Boolean enableSafeguard)
at System.Reactive.Sink`2.Run(IObservable`1 source)
at System.Reactive.Producer`2.SubscribeRaw(IObserver`1 observer, Boolean enableSafeguard)
at System.Reactive.Linq.ObservableImpl.Catch`2.Run(_ sink)
at System.Reactive.Producer`2.SubscribeRaw(IObserver`1 observer, Boolean enableSafeguard)
at System.Reactive.Linq.ObservableImpl.Merge`1.Observables._.OnNext(IObservable`1 value)
at System.Reactive.Linq.ObservableImpl.ToObservableRecursive`1._.LoopRec(IScheduler scheduler)
at System.Reactive.Concurrency.AsyncLock.Wait(Object state, Delegate delegate, Action`2 action)
at System.Reactive.Concurrency.ImmediateScheduler.AsyncLockScheduler.Schedule[TState](TState state, Func`3 action)
at System.Reactive.Linq.ObservableImpl.ToObservableRecursive`1._.LoopRec(IScheduler scheduler)
at System.Reactive.Linq.ObservableImpl.ToObservableRecursive`1._.Run(IEnumerable`1 source, IScheduler scheduler)
at Syste…[truncated]
Opened automatically from Admin/_LogIncident/619edb4566176ac8. Recurrences are folded into this issue rather than opening new ones.
Activity
rbuergi commentedon Sep 23, 2026
Duplicate of #5555 — same log site/fingerprint; consolidated. (the same OutOfMemoryException event on pod memex-portal-deployment-777677694-cc6zz, 14:22:25-14:22:38Z — the dispose/drain sites are where the one heap exhaustion surfaced)