Repository navigation
SynchronizationStream.OnCompleted throws ObjectDisposedException during hub disposal #1171
Description
What is failing
When a hub is being torn down, HostedHubsCollection.Dispose calls into SynchronizationStream<T>.Dispose(), which triggers OnCompleted() on an inner ReplaySubject that has already been disposed. The ObjectDisposedException propagates out and is logged as a faulted hub disposal.
Probable cause
A dispose-ordering race in the SynchronizationStream<T> teardown sequence. The stream's Dispose() (at line 1449 of SynchronizationStream.cs) calls OnCompleted() on its inner reactive pipeline, but the downstream ReplaySubject that receives that completion has already been disposed — possibly by a sibling subscription or by an earlier phase of the same disposal. The fix is likely to guard the OnCompleted() call in SynchronizationStream<T> with a null/state check, or to ensure the ReplaySubject outlives the completion signal.
Confidence: High — the stack trace is unambiguous: Dispose() → OnCompleted() → ReplaySubject.CheckDisposed() throws.
Impact
Low. 1 occurrence on 1 pod, never repeated. The hub disposal was already in progress, so this is a harmless secondary fault — no data loss, no user impact. But the pattern could manifest under higher churn.
Where to look
MeshWeaver.Data/Serialization/SynchronizationStream.cs— lines 1449 (Dispose()) and 680 (OnCompleted())- The
RegisterForDispandDisposelogic around line 1449 that initiates the completion chain MeshWeaver.Messaging.HostedHubsCollection— the caller that logs the faulted disposal
Evidence
| Fingerprint | 0857ac252fa6c3e9 |
| Category | MeshWeaver.Messaging.HostedHubsCollection |
| Severity | Error |
| Exception | System.ObjectDisposedException |
| Top frame | MeshWeaver.Data.Serialization.SynchronizationStream1.OnCompleted()` |
| Namespace | memex-cloud |
| Pods | memex-portal-deployment-7d68b474b4-qbmb4 |
| Occurrences | 1 |
| First seen | 2026-08-10 12:55:52Z |
| Last seen | 2026-08-10 12:55:52Z |
Recent log lines
2026-08-10 12:55:52Z memex-portal-deployment-7d68b474b4-qbmb4 fail: MeshWeaver.Messaging.HostedHubsCollection[0]
Hub sync/KVk1hB2Tw02ESgrSpL3Cgg disposal faulted
System.ObjectDisposedException: Cannot access a disposed object.
at System.Reactive.Subjects.ReplaySubject`1.ReplayBase.CheckDisposed()
at System.Reactive.Subjects.ReplaySubject`1.ReplayBase.OnCompleted()
at System.Reactive.Subjects.ReplaySubject`1.OnCompleted()
at MeshWeaver.Data.Serialization.SynchronizationStream`1.OnCompleted() in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Data/Serialization/SynchronizationStream.cs:line 680
at System.Reactive.SafeObserver`1.WrappingSafeObserver.OnCompleted()
at System.Reactive.Sink`1.ForwardOnCompleted()
at System.Reactive.Concurrency.Synchronize`1._.OnCompleted()
at System.Reactive.Sink`2.OnCompleted()
at System.Reactive.Sink`1.ForwardOnCompleted()
at System.Reactive.Concurrency.Synchronize`1._.OnCompleted()
at System.Reactive.Sink`2.OnCompleted()
at System.Reactive.Sink`2.OnCompleted()
at System.Reactive.Sink`1.ForwardOnCompleted()
at System.Reactive.Concurrency.Synchronize`1._.OnCompleted()
at System.Reactive.SafeObserver`1.WrappingSafeObserver.OnCompleted()
at System.Reactive.Sink`1.ForwardOnCompleted()
at System.Reactive.Concurrency.Synchronize`1._.OnCompleted()
at System.Reactive.Subjects.FastImmediateObserver`1.EnsureActive(Int32 count)
at System.Reactive.Subjects.FastImmediateObserver`1.EnsureActive()
at System.Reactive.Subjects.ReplaySubject`1.ReplayBase.OnCompleted()
at System.Reactive.Subjects.ReplaySubject`1.OnCompleted()
at MeshWeaver.Data.Serialization.SynchronizationStream`1.Dispose() in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Data/Serialization/SynchronizationStream.cs:line 1449
at MeshWeaver.Data.Serialization.SynchronizationStream`1.<>c__DisplayClass61_0.<RegisterForDisp…[truncated]
Opened automatically from Admin/_LogIncident/0857ac252fa6c3e9. Recurrences are folded into this issue rather than opening new ones.
Activity
rbuergi commentedon Aug 10, 2026
Duplicate of #1170 — same file, same line (
SynchronizationStream.cs:1449), same stack (Dispose()→ReplaySubject.OnCompleted()→CheckDisposed()). The two were filed separately because the fingerprint keyed on the calling frame (HostedHubsCollection.Disposehere,MessageHubshutdown there), so one defect reached the board twice. Consolidating the analysis on #1170.