Archiver Observability#3353
Conversation
… first, ahead of oberservability
…the service bus. Presumably **your** archiver would have to do the same.
| CommandProcessorSpanOperation.Publish => "publish", | ||
| CommandProcessorSpanOperation.Send => "send", | ||
| CommandProcessorSpanOperation.Clear => "clear", | ||
| CommandProcessorSpanOperation.Archive => "archive", |
There was a problem hiding this comment.
❌ New issue: Complex Method
ToSpanName has a cyclomatic complexity of 9, threshold = 9
|
|
||
| var span = Tracer?.CreateDbSpan( | ||
| new OutboxSpanInfo(DbSystem.Brighter, InMemoryAttributes.DbName, OutboxDbOperation.DispatchedMessages, InMemoryAttributes.DbTable), | ||
| requestContext?.Span, | ||
| options: _instrumentationOptions | ||
| ); |
There was a problem hiding this comment.
❌ New issue: Code Duplication
The module contains 4 functions with similar structure: DispatchedMessages,Get,MarkDispatched,OutstandingMessages
| IAmABoxTransactionProvider<CommittableTransaction>? transactionProvider = null, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| //NOTE: As we call Add, don't create telemetry here |
There was a problem hiding this comment.
✅ Getting better: Missing Arguments Abstractions
The average number of function arguments decreases from 4.63 to 4.47, threshold = 4.00
| IAmABoxTransactionProvider<CommittableTransaction>? transactionProvider = null, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| //NOTE: As we call Add, don't create telemetry here |
There was a problem hiding this comment.
❌ New issue: Primitive Obsession
In this module, 30.3% of all function arguments are primitive types, threshold = 30.0%
|
|
||
| //assert | ||
| _exportedActivities.Count.Should().Be(7); | ||
| _exportedActivities.Count.Should().Be(8); |
There was a problem hiding this comment.
❌ Getting worse: Complex Method
When_Clearing_A_Message_A_Span_Is_Exported increases in cyclomatic complexity from 45 to 49, threshold = 9
|
|
||
| //assert | ||
| _exportedActivities.Count.Should().Be(7); | ||
| _exportedActivities.Count.Should().Be(8); |
There was a problem hiding this comment.
❌ Getting worse: Complex Method
When_Clearing_A_Message_A_Span_Is_Exported increases in cyclomatic complexity from 45 to 49, threshold = 9
| [Fact] | ||
| public void When_archiving_from_the_outbox() | ||
| { | ||
| var parentActivity = new ActivitySource("Paramore.Brighter.Tests").StartActivity("BrighterTracerSpanTests"); | ||
|
|
||
| var context = new RequestContext(); | ||
| context.Span = parentActivity; | ||
|
|
||
| //add and clear message | ||
| var myEvent = new MyEvent(); | ||
| var myMessage = new MyEventMessageMapper().MapToMessage(myEvent, _publication); | ||
| _bus.AddToOutbox(myMessage, context); | ||
| _bus.ClearOutbox([myMessage.Id], context); | ||
|
|
||
| //se should have an entry in the outbox | ||
| _outbox.EntryCount.Should().Be(1); | ||
|
|
||
| //allow time to pass | ||
| _timeProvider.Advance(TimeSpan.FromSeconds(300)); | ||
|
|
||
| //archive | ||
| var dispatchedSince = TimeSpan.FromSeconds(100); | ||
| _archiver.Archive(dispatchedSince, context); | ||
|
|
||
| //should be no messages in the outbox | ||
| _outbox.EntryCount.Should().Be(0); | ||
|
|
||
| parentActivity?.Stop(); | ||
|
|
||
| _traceProvider.ForceFlush(); | ||
|
|
||
| //We should have exported matching activities | ||
| _exportedActivities.Count.Should().Be(9); | ||
|
|
||
| _exportedActivities.Any(a => a.Source.Name == "Paramore.Brighter").Should().BeTrue(); | ||
|
|
||
| //there should be a n archive create span for the batch | ||
| var createActivity = _exportedActivities.Single(a => a.DisplayName == $"{BrighterSemanticConventions.ArchiveMessages} {CommandProcessorSpanOperation.Archive.ToSpanName()}"); | ||
| createActivity.Should().NotBeNull(); | ||
| createActivity.ParentId.Should().Be(parentActivity?.Id); | ||
|
|
||
| //check for outstanding messages span | ||
| var osCheckActivity = _exportedActivities.SingleOrDefault(a => | ||
| a.DisplayName == $"{OutboxDbOperation.DispatchedMessages.ToSpanName()} {InMemoryAttributes.DbName} {InMemoryAttributes.DbTable}"); | ||
| osCheckActivity.Should().NotBeNull(); | ||
| osCheckActivity?.ParentId.Should().Be(createActivity.Id); | ||
|
|
||
| //check for delete messages span | ||
| var deleteActivity = _exportedActivities.SingleOrDefault(a => | ||
| a.DisplayName == $"{OutboxDbOperation.Delete.ToSpanName()} {InMemoryAttributes.DbName} {InMemoryAttributes.DbTable}"); | ||
| deleteActivity?.Should().NotBeNull(); | ||
| deleteActivity?.ParentId.Should().Be(createActivity.Id); | ||
|
|
||
| //check the tags for the create span | ||
| createActivity.TagObjects.Should().Contain(t => t.Key == BrighterSemanticConventions.ArchiveAge && Math.Abs(Convert.ToDouble(t.Value) - dispatchedSince.TotalMilliseconds) < TOLERANCE); | ||
|
|
||
| //check the tags for the outstanding messages span | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbOperation && t.Value == OutboxDbOperation.DispatchedMessages.ToSpanName()).Should().BeTrue(); | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbTable && t.Value == InMemoryAttributes.DbTable).Should().BeTrue(); | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbSystem && t.Value == DbSystem.Brighter.ToDbName()).Should().BeTrue(); | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbName && t.Value == InMemoryAttributes.DbName).Should().BeTrue(); | ||
|
|
||
| //check the tages for the delete messages span | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbOperation && t.Value == OutboxDbOperation.Delete.ToSpanName()).Should().BeTrue(); | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbTable && t.Value == InMemoryAttributes.DbTable).Should().BeTrue(); | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbSystem && t.Value == DbSystem.Brighter.ToDbName()).Should().BeTrue(); | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbName && t.Value == InMemoryAttributes.DbName).Should().BeTrue(); | ||
|
|
||
| } |
There was a problem hiding this comment.
❌ New issue: Complex Method
When_archiving_from_the_outbox has a cyclomatic complexity of 10, threshold = 9
| public OutboxArchiver( | ||
| IAmAnOutbox outbox, | ||
| IAmAnArchiveProvider archiveProvider, | ||
| IAmARequestContextFactory? requestContextFactory = null, | ||
| int archiveBatchSize = 100, | ||
| IAmABrighterTracer? tracer = null, | ||
| InstrumentationOptions instrumentationOptions = InstrumentationOptions.All) | ||
| { | ||
| _archiveProvider = archiveProvider; | ||
| _archiveBatchSize = archiveBatchSize; | ||
| _tracer = tracer; | ||
| _instrumentationOptions = instrumentationOptions; | ||
| _requestContextFactory = requestContextFactory ?? new InMemoryRequestContextFactory(); | ||
|
|
||
| if (outbox is IAmAnOutboxSync<TMessage, TTransaction> syncOutbox) _outBox = syncOutbox; | ||
| if (outbox is IAmAnOutboxAsync<TMessage, TTransaction> asyncOutbox) _asyncOutbox = asyncOutbox; | ||
| } |
There was a problem hiding this comment.
❌ New issue: Constructor Over-Injection
OutboxArchiver has 6 arguments, threshold = 5
| var subscription = _replySubscriptions?.FirstOrDefault(s => s.DataType == typeof(TResponse)); | ||
|
|
||
| if (subscription is null) | ||
| throw new ArgumentOutOfRangeException($"No Subscription registered fpr replies of type {typeof(T)}"); |
There was a problem hiding this comment.
❌ Getting worse: Code Duplication
introduced similar code in: ClearOutstandingFromOutbox
| [Fact] | ||
| public async Task When_archiving_from_the_outbox() | ||
| { | ||
| var parentActivity = new ActivitySource("Paramore.Brighter.Tests").StartActivity("BrighterTracerSpanTests"); | ||
|
|
||
| var context = new RequestContext(); | ||
| context.Span = parentActivity; | ||
|
|
||
| //add and clear message | ||
| var myEvent = new MyEvent(); | ||
| var myMessage = new MyEventMessageMapper().MapToMessage(myEvent, _publication); | ||
| await _bus.AddToOutboxAsync(myMessage, context); | ||
| await _bus.ClearOutboxAsync([myMessage.Id], context); | ||
|
|
||
| //se should have an entry in the outbox | ||
| _outbox.EntryCount.Should().Be(1); | ||
|
|
||
| //allow time to pass | ||
| _timeProvider.Advance(TimeSpan.FromSeconds(300)); | ||
|
|
||
| //archive | ||
| var dispatchedSince = TimeSpan.FromSeconds(100); | ||
| await _archiver.ArchiveAsync(dispatchedSince, context); | ||
|
|
||
| //should be no messages in the outbox | ||
| _outbox.EntryCount.Should().Be(0); | ||
|
|
||
| parentActivity?.Stop(); | ||
|
|
||
| _traceProvider.ForceFlush(); | ||
|
|
||
| //We should have exported matching activities | ||
| _exportedActivities.Count.Should().Be(9); | ||
|
|
||
| _exportedActivities.Any(a => a.Source.Name == "Paramore.Brighter").Should().BeTrue(); | ||
|
|
||
| //there should be a n archive create span for the batch | ||
| var createActivity = _exportedActivities.Single(a => a.DisplayName == $"{BrighterSemanticConventions.ArchiveMessages} {CommandProcessorSpanOperation.Archive.ToSpanName()}"); | ||
| createActivity.Should().NotBeNull(); | ||
| createActivity.ParentId.Should().Be(parentActivity?.Id); | ||
|
|
||
| //check for outstanding messages span | ||
| var osCheckActivity = _exportedActivities.SingleOrDefault(a => | ||
| a.DisplayName == $"{OutboxDbOperation.DispatchedMessages.ToSpanName()} {InMemoryAttributes.DbName} {InMemoryAttributes.DbTable}"); | ||
| osCheckActivity.Should().NotBeNull(); | ||
| osCheckActivity?.ParentId.Should().Be(createActivity.Id); | ||
|
|
||
| //check for delete messages span | ||
| var deleteActivity = _exportedActivities.SingleOrDefault(a => | ||
| a.DisplayName == $"{OutboxDbOperation.Delete.ToSpanName()} {InMemoryAttributes.DbName} {InMemoryAttributes.DbTable}"); | ||
| deleteActivity.Should().NotBeNull(); | ||
| deleteActivity?.ParentId.Should().Be(createActivity.Id); | ||
|
|
||
| //check the tags for the create span | ||
| createActivity.TagObjects.Should().Contain(t => t.Key == BrighterSemanticConventions.ArchiveAge && Math.Abs(Convert.ToDouble(t.Value) - dispatchedSince.TotalMilliseconds) < TOLERANCE); | ||
|
|
||
| //check the tags for the outstanding messages span | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbOperation && t.Value == OutboxDbOperation.DispatchedMessages.ToSpanName()).Should().BeTrue(); | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbTable && t.Value == InMemoryAttributes.DbTable).Should().BeTrue(); | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbSystem && t.Value == DbSystem.Brighter.ToDbName()).Should().BeTrue(); | ||
| osCheckActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbName && t.Value == InMemoryAttributes.DbName).Should().BeTrue(); | ||
|
|
||
| //check the tages for the delete messages span | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbOperation && t.Value == OutboxDbOperation.Delete.ToSpanName()).Should().BeTrue(); | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbTable && t.Value == InMemoryAttributes.DbTable).Should().BeTrue(); | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbSystem && t.Value == DbSystem.Brighter.ToDbName()).Should().BeTrue(); | ||
| deleteActivity?.Tags.Any(t => t.Key == BrighterSemanticConventions.DbName && t.Value == InMemoryAttributes.DbName).Should().BeTrue(); | ||
|
|
||
| } |
There was a problem hiding this comment.
❌ New issue: Complex Method
When_archiving_from_the_outbox has a cyclomatic complexity of 10, threshold = 9
| public OutboxProducerMediator( | ||
| IAmAProducerRegistry producerRegistry, |
There was a problem hiding this comment.
ℹ New issue: Complex Method
OutboxProducerMediator has a cyclomatic complexity of 14, threshold = 9
| public OutboxProducerMediator( | ||
| IAmAProducerRegistry producerRegistry, |
There was a problem hiding this comment.
ℹ New issue: Constructor Over-Injection
OutboxProducerMediator has 14 arguments, threshold = 5
…due to additional parameters
| return commandProcessor; | ||
| } | ||
|
|
||
| private static IAmAnExternalBusService BuildExternalBus(IServiceProvider serviceProvider, |
There was a problem hiding this comment.
✅ No longer an issue: Excess Number of Function Arguments
BuildExternalBus is no longer above the threshold for number of arguments
| } | ||
|
|
||
| private static IAmAnExternalBusService BuildExternalBus(IServiceProvider serviceProvider, | ||
| private static IAmAnOutboxProducerMediator BuildOutBoxProducerMediator(IServiceProvider serviceProvider, |
There was a problem hiding this comment.
ℹ New issue: Excess Number of Function Arguments
BuildOutBoxProducerMediator has 5 arguments, threshold = 4
* feat: add an archive test without observablity, to ensure test passes first, ahead of oberservability * chore: switch branch; commit to save wip * chore: switch to another device * feat: we need a create span for the archive batch, if issued through the service bus. Presumably **your** archiver would have to do the same. * fix: missing mark dispatched in clear otel checks * feat: add tagobjects for archive batch * feat: opentelemetry for the archive operation * feat: add async archive operation opentelemetry * feat: cleanup old archiver telemetry * feat: add archive async telemetry * feat: move archive functionality to OutboxArchiver.cs * feat: rename of ExternalBusService.cs to OutboxProducerMediator.cs * fix: name the mediator correctly * fix: issue with PackageReference not PackageVersion * fix: failing reflection based construction of OutBoxProducerMediator due to additional parameters
We want to add observability to the archiver.