CritterWatch telemetry caught by Wolverine tracked-session waits

[uniquelau]

Summary

With AddCritterWatchMonitoring enabled (Wolverine.CritterWatch 1.0.0-beta.3), the monitored service's outbound telemetry envelopes — everything in Wolverine.CritterWatch.Messages.Outbound.* (e.g. ServiceUpdates, EventAppendDiscovered, ShardStatesChanged), emitted on the heartbeat / EventProgressionPoller cadence — are recorded by a Wolverine tracked session
(IHost.TrackActivity()…ExecuteAndWaitAsync / InvokeMessageAndWaitAsync). That background traffic never "completes" within the tracked window, so IsCompleted() stays false and the wait blocks until the tracked-session timeout, failing otherwise-correct integration tests.

Environment

• WolverineFx 6.17.2
• Wolverine.CritterWatch 1.0.0-beta.3
• Marten 9.15.0
• .NET 10

Mechanism

TrackedSession (src/Wolverine/Tracking/TrackedSession.cs) seeds exactly one default ignore rule:

private readonly List<Func<Type, bool>> _ignoreMessageRules = [t => t.CanBeCastTo<IAgentCommand>()];

MaybeRecord / Record drop only messages matching a rule; IsCompleted() requires _envelopes.All(x => x.IsComplete()). Framework agent traffic is excluded by that default, but CritterWatch telemetry is not IAgentCommand, so it is tracked. Every outbound telemetry type implements the framework markers ICritterWatchMessage and INotToBeRouted (verified by reflection over the beta.3 assembly), and AddCritterWatchMonitoring(…, heartbeatInterval) plus the EventProgressionPoller background service publish them continuously — so there is always an in-flight, uncompleted telemetry envelope in the window.

Repro steps

1. Enable AddCritterWatchMonitoring(...) on a Wolverine host.
2. Open any host.TrackActivity().ExecuteAndWaitAsync(...) window spanning ≥1 heartbeat.
3. The wait times out; the tracked-session log shows Wolverine.CritterWatch.Messages.Outbound.*
   envelopes recorded and not completed.

Current workaround

config.IgnoreMessagesMatchingType(t => t.Namespace?.StartsWith("Wolverine.CritterWatch") == true);

Suggested direction

CritterWatch's monitoring envelopes are framework/infra traffic, exactly like the IAgentCommand case already excluded by default. In order of preference:

• (a) Have Wolverine's default tracked-session ignore rule also exclude ICritterWatchMessage
  (and/or INotToBeRouted), so no per-test filtering is needed. The messages already carry
  unambiguous marker interfaces, so this is low-risk.
• (b) Failing that, document the IgnoreMessagesMatchingType filter as the supported pattern for
  monitored hosts.

[jeremydmiller]

Fixed and shipped in Wolverine 6.17.3 (just published) via JasperFx/wolverine#3390.

TrackedSession's default ignore rule was only IAgentCommand. It now covers all of INotToBeRouted, which includes the telemetry a CritterWatch-monitored host publishes — so a monitored host no longer needs any hand-written filter, and a tracked session will not sit waiting on messages your test never sent.

Acknowledgement and FailureAcknowledgement are deliberately carved out of that rule and still tracked, because the session's own acknowledgement APIs (IgnoreFailureAcks, the ack assertions) depend on seeing them.

For anyone on 6.17.2 or earlier, the workaround remains:

.IgnoreMessagesMatchingType(type => type.CanBeCastTo<INotToBeRouted>())

The testing guide now documents both (JasperFx/wolverine#3395). Thanks for the clear report — closing.