Universal DLQ#4015
Conversation
Requirement specs for AWS SQS, Redis, MsSql, PostgreSQL, RocketMQ, and MQTT dead letter queue support. Numbered from 0010 to leave 0003-0009 free for parallel workstreams on other branches. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…design SQS-specific decisions for replacing ChangeMessageVisibility(0) with direct SendMessage to DLQ. References ADR 0034 (strategy) and ADR 0036 (routing logic) for shared concerns. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…38 producer creation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…MessageConsumer Remove the inverted _hasDlq boolean from SqsMessageConsumer and replace with RoutingKey? deadLetterRoutingKey and RoutingKey? invalidMessageRoutingKey parameters, preparing for Brighter-managed DLQ support (ADR 0038). - Add connection and makeChannels parameters to SqsMessageConsumer for later use by lazy DLQ producers - Update SqsMessageConsumerFactory to pass new parameters - Update reject tests: reject without DLQ now deletes (no longer uses ChangeMessageVisibility(0) which was the inverted _hasDlq behavior) - Applied identically to both AWSSQS and AWSSQS.V4 packages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SqsSubscription implements IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport interfaces - Add DeadLetterRoutingKey and InvalidMessageRoutingKey properties - Add optional constructor parameters, null by default (backward compatible) - Applied identically to both AWSSQS and AWSSQS.V4 packages - Test: When_creating_sqs_subscription_with_dlq_routing_keys_should_expose_properties - Test: When_creating_sqs_subscription_without_dlq_routing_keys_should_default_to_null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…onsumer SqsMessageConsumerFactory.CreateImpl() now extracts routing keys from the subscription via IUseBrighterDeadLetterSupport/IUseBrighterInvalidMessageSupport interface checks and passes them to the SqsMessageConsumer constructor, matching the pattern established by KafkaMessageConsumerFactory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace ChangeMessageVisibility(0) with direct SendMessage to DLQ per ADR 0038. SqsMessageConsumer now creates lazy DLQ/invalid-message producers and routes rejected messages with metadata (originalTopic, rejectionReason, rejectionTimestamp, originalMessageType) before deleting the original from the source queue. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integration test confirms DetermineRejectionRoute correctly sends messages rejected with Unacceptable reason to the invalid message queue (not DLQ) when both queues are configured. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… channel Integration test confirms that when only deadLetterRoutingKey is configured (no invalidMessageRoutingKey), rejecting with Unacceptable reason falls back to the DLQ per ADR 0036 routing decision tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integration test confirms that when no deadLetterRoutingKey or invalidMessageRoutingKey is configured, rejecting with a reason still deletes the original message from the source queue. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integration test confirms the async/Proactor channel path correctly sends messages rejected with DeliveryError to the DLQ with metadata, exercising RejectAsync directly without the sync-over-async BrighterAsyncContext wrapper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…equeue The SNS requeue tests were still testing the old ChangeMessageVisibility(0) behavior. Updated to match the SQS reject test pattern: reject deletes the message from the queue (MT_NONE) rather than requeuing it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ADR, and tasks Redis has no native DLQ support, so ADR 0039 adopts the same Brighter-managed pattern proven in Kafka (spec 0001) and SQS (spec 0010). Also marks spec 0010 tasks as complete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RedisSubscription now implements IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport with optional constructor parameters, following the same pattern as SQS. Backward compatible — defaults to null. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…onsumer RedisMessageConsumerFactory.Create() and CreateAsync() now extract deadLetterRoutingKey and invalidMessageRoutingKey from subscriptions via interface checks and pass them to the RedisMessageConsumer constructor. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the core DLQ routing logic in RedisMessageConsumer.Reject() and RejectAsync() following the SQS pattern: lazy producer creation, metadata enrichment, and ADR 0036 rejection routing decision tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When both DLQ and invalid message channels are configured, Unacceptable rejections route to the invalid channel per ADR 0036. DLQ stays empty. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… channel configured - Test: When_rejecting_message_with_unacceptable_and_no_invalid_channel_should_fallback_to_dlq - Confirms ADR 0036 fallback: Unacceptable → DLQ when invalidMessageRoutingKey is null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…om inflight - Test: When_rejecting_message_with_no_channels_configured_should_remove_from_inflight - Confirms consumer can receive next message after rejecting without DLQ/invalid channels Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test: When_rejecting_message_with_delivery_error_should_send_to_dlq_async - Confirms RejectAsync path sends to DLQ with rejection metadata Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 7 implementation tasks and verification complete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch active spec to 0012-mssql-dead-letter-queue and add design for Brighter-managed DLQ support in the MsSql transport, following the established pattern from Kafka, SQS, and Redis. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pull Request Review: Universal DLQSummaryThis PR extends dead letter queue (DLQ) support across multiple messaging providers (AWS SQS, Redis, and foundation for MS SQL, Postgres, RocketMQ, and MQTT). The implementation follows the pattern established in the Kafka DLQ work (spec 0001) and ensures consistent behavior for Strengths1. Excellent Architecture & Documentation
2. Implementation Quality
3. Test Coverage
4. AWS SQS Improvements
Issues & RecommendationsCritical IssuesNone identified. The implementation is solid. Code Quality Observations1. Duplicate Code Between SDK Versions (Minor)Location: Both packages have identical Recommendation: Low priority - acceptable as-is given the packages target different SDK versions. Consider for future refactoring if the duplication becomes a maintenance issue. 2. Test File Naming ConsistencyLocation: Test files like Test class names are very descriptive but quite long. This is consistent with existing patterns in the codebase, so no change needed. 3. Error Handling in Reject MethodsLocation: The error handling logs errors but continues to delete the original message even if DLQ production fails. This is documented behavior per ADR 0036, but could lead to message loss. From ADR 0038:
Recommendation: Acceptable as designed. The alternative (blocking the consumer) would be worse. Consider documenting this in user-facing documentation so operators understand the trade-off and can set up appropriate monitoring. Documentation Suggestions1. Add Migration Guide (Enhancement)For users currently relying on native SQS redrive policy with the
2. Observability Guidance (Enhancement)The PR adds structured logging (message ID, rejection reason). Consider documenting:
Performance ConsiderationsLazy Producer CreationThe lazy producer pattern is well-implemented: if (_deadLetterRoutingKey != null)
{
_deadLetterProducer = new Lazy<SqsMessageProducer?>(CreateDeadLetterProducer);
}Concern: If DLQ production is rare, the first rejection will pay the cost of producer initialization. This is acceptable, but worth noting for high-throughput scenarios. Recommendation: No change needed - the trade-off is correct (zero overhead for the common path when DLQ is not used). Security ConsiderationsMetadata in Message HeadersRejection metadata (original topic, timestamps, reasons) is added to Recommendation: Acceptable for DLQ messages (they're error scenarios). If DLQ messages are ever forwarded to external systems, review what metadata should be included. Test Coverage AssessmentCovered Scenarios ✅
Potential Gaps
Recommendation: These are edge cases. Current test coverage is good. Consider adding failure injection tests in a future PR if flaky DLQ behavior is observed in production. Specification & Planning QualityThe
This demonstrates excellent engineering discipline and makes the codebase maintainable. Consistency with Codebase StandardsFollowing CLAUDE.md Guidelines ✅
Code Style ✅
Final RecommendationApprove and Merge ✅ This is high-quality work that:
Merge Checklist
Post-Merge Suggestions
Excellent work on this PR! The consistent pattern across transports and thorough documentation will make this feature maintainable and understandable for future contributors. |
PR Review: Universal DLQThis PR extends Brighter-managed DLQ support to SQS (v3 + v4), Redis, MSSQL, PostgreSQL, RocketMQ, and MQTT. The architecture is consistent and the ADRs are thorough. The self-review in Potential Bug:
|
Resolves conflicts in MQTT, MsSql, and Redis transports where both the scheduler (delay requeue) and DLQ (rejection routing) features modified the same consumer and factory files. Combined both sets of constructor parameters, fields, and methods so both features coexist. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR Review: Universal DLQThis is a large, well-structured PR that extends Brighter-managed DLQ support consistently across Redis, MsSql, Postgres, RocketMQ, MQTT, and SQS (v3 + v4). The ADRs are thorough, the Bug:
|
| Category | Count |
|---|---|
| Bugs | 2 (DisposeAsync sync Dispose, null-forgiving on Description) |
| Behavioural inconsistency | 1 (MQTT None reason routing) |
| Test gaps | 2 (FIFO DLQ, producer creation failure) |
| Design/maintenance | 2 (HeaderNames constants, async nesting risk) |
| Minor/nit | 2 |
The core pattern is sound and consistently applied. The main items to address before merging are the DisposeAsync disposal bug, the MQTT/other inconsistency in None reason handling, and verifying FIFO queue DLQ behaviour.
SqsMessageConsumer.DisposeAsync() was calling sync Dispose() on producers that implement IAsyncDisposable, bypassing async cleanup. Now checks for IAsyncDisposable and awaits it, matching the pattern used by Kafka and Redis consumers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… RefreshMetadata reason.Description is nullable; the `!` operator suppressed the compiler warning but could silently insert null into the message bag. Use `?? string.Empty` to provide a safe runtime fallback across all 8 transport consumers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FIFO queues require MessageGroupId (from PartitionKey) and optionally MessageDeduplicationId. These tests verify that rejected FIFO messages arrive at a FIFO DLQ with FIFO attributes preserved, covering both Reactor (sync) and Proactor (async) paths for V3 and V4 SDK variants. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The DLQ and invalid message producers were created without queue attributes, so FIFO queues were created as standard queues (missing .fifo suffix). This caused DLQ sends to fail silently for FIFO messages because MessageGroupId is invalid on standard queues. Pass SqsAttributes from the subscription through the consumer to the DLQ/invalid message producer publications, ensuring FIFO DLQ queues are created correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR Review: Universal DLQGreat scope and execution on this PR. Extending DLQ support consistently across all transports is a significant reliability improvement, and the spec-driven approach with ADRs per transport is well-structured. The comprehensive integration test coverage (reactor + proactor paths, all four rejection scenarios per transport) is particularly valuable. A few things worth examining before merge: Issues1. MsSql "no channels" leaves the message in the queueThe Postgres no-channels path calls 2.
|
| Transport | No-channels behaviour |
|---|---|
| SQS | Acknowledge + return true |
| RocketMQ | Ack + log warning + return true |
| Redis | Remove from inflight + return true |
| Postgres | Delete + log warning + return true |
| MsSql | Log warning + return true (no delete?) |
| MQTT | Return true (fire-and-forget) |
The differences are mostly justified by transport semantics, but the MsSql/Postgres divergence (both SQL-based, poll-based) is likely a defect rather than an intentional design difference (see issue #1 above).
Positives worth calling out
- The
_hasDlq: boolto_deadLetterRoutingKey: RoutingKey?refactor is a clear improvement — the routing key carries meaning that the boolean flag did not. - Adding structured logging to
RejectMessageOnErrorHandler/RejectMessageOnErrorHandlerAsyncvia[LoggerMessage]is the right approach and fills an observability gap. - The fix to await
DisposeAsyncon SQS producers (commit1ad490e) and to passSqsAttributesto DLQ producers for FIFO support (commit41db952) are both correct and would have been subtle production bugs. - Lazy producer initialisation avoids creating AWS/Redis/SQL connections at consumer startup for queues that may never see a rejection.
Review generated with Claude Code
… exceptions The default LazyThreadSafetyMode.ExecutionAndPublication caches thrown exceptions permanently. A single transient failure during DLQ producer creation would disable DLQ routing for the lifetime of the consumer. LazyThreadSafetyMode.None does not cache exceptions, allowing retry on the next .Value access. This is safe because message pumps are single-threaded per consumer. Affects all 7 Brighter-managed DLQ transports: SQS (V3+V4), Kafka, MQTT, Redis, PostgreSQL, MsSql. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Gates Failed
Prevent hotspot decline
(2 hotspots with Complex Method, Bumpy Road Ahead, Code Duplication, Overall Code Complexity, Constructor Over-Injection)
Enforce critical code health rules
(6 files with Bumpy Road Ahead)
Enforce advisory code health rules
(8 files with Complex Method, Code Duplication, Overall Code Complexity, Constructor Over-Injection)
Gates Passed
1 Quality Gates Passed
See analysis details in CodeScene
Reason for failure
| Prevent hotspot decline | Violations | Code Health Impact | |
|---|---|---|---|
| RedisMessageConsumer.cs | 5 rules in this hotspot | 8.57 → 6.72 | Suppress |
| SqsMessageConsumer.cs | 2 rules in this hotspot | 9.69 → 9.14 | Suppress |
| Enforce critical code health rules | Violations | Code Health Impact | |
|---|---|---|---|
| RedisMessageConsumer.cs | 1 critical rule | 8.57 → 6.72 | Suppress |
| PostgresMessageConsumer.cs | 1 critical rule | 8.28 → 6.86 | Suppress |
| MsSqlMessageConsumer.cs | 1 critical rule | 9.39 → 8.04 | Suppress |
| SqsMessageConsumer.cs | 1 critical rule | 9.69 → 9.14 | Suppress |
| SqsMessageConsumer.cs | 1 critical rule | 9.69 → 9.14 | Suppress |
| RocketMessageConsumer.cs | 1 critical rule | 10.00 → 9.54 | Suppress |
| Enforce advisory code health rules | Violations | Code Health Impact | |
|---|---|---|---|
| RedisMessageConsumer.cs | 4 advisory rules | 8.57 → 6.72 | Suppress |
| PostgresMessageConsumer.cs | 3 advisory rules | 8.28 → 6.86 | Suppress |
| MsSqlMessageConsumer.cs | 3 advisory rules | 9.39 → 8.04 | Suppress |
| SqsMessageConsumer.cs | 1 advisory rule | 9.69 → 9.14 | Suppress |
| SqsMessageConsumer.cs | 1 advisory rule | 9.69 → 9.14 | Suppress |
| RocketMessageConsumer.cs | 1 advisory rule | 10.00 → 9.54 | Suppress |
| MqttSubscription.cs | 1 advisory rule | 9.69 | Suppress |
| MQTTMessageConsumer.cs | 1 advisory rule | 9.39 → 9.10 | Suppress |
Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
| _sqlMessageQueue.Send(message, topic); | ||
| return true; | ||
| } | ||
There was a problem hiding this comment.
❌ Getting worse: Code Duplication
introduced similar code in: CreateDeadLetterProducer,CreateInvalidMessageProducer,Reject,RejectAsync
Re: Issue #1 & #8 — MsSql "no channels" behaviourThe MsSql implementation is correct as-is, and the apparent divergence from PostgreSQL is intentional, driven by fundamentally different queue semantics. MsSql uses atomic read-and-delete. The DELETE FROM cte OUTPUT deleted.Payload, deleted.MessageType, deleted.Topic, deleted.IdThe message is permanently removed from the queue table in the same operation that reads it. By the time PostgreSQL uses a visibility timeout model. So the outcome is identical in both transports: the message is removed from the source queue and will not be redelivered. They differ only in when the removal happens (receive-time for MsSql vs. reject/ack-time for PostgreSQL), which is dictated by their underlying queue mechanics. The
No change needed here. |
Code Review: Universal DLQ (#4015)This is a substantial and well-structured PR that brings consistent DLQ behaviour across all Brighter transport providers. The ADRs, test coverage per rejection scenario, and the use of Breaking Changes / Correctness1. SQS: native DLQ users will silently lose messages The old Any existing user who configured a native SQS redrive policy but does not set 2. MsSql consumer does not delete the source row on rejection The Postgres consumer correctly calls Design / DRY3. Both methods are copied with near-identical bodies into Redis, MsSql, Postgres, MQTT, RocketMQ, and both SQS variants (at least seven copies). A shared internal static helper class or a base class would eliminate the duplication and make future routing logic changes a single-site edit. Even a 4. RocketMQ does not use All other providers use 5. SQS factory uses // SqsMessageConsumerFactory (both V3 and V4)
if (sqsSubscription is IUseBrighterDeadLetterSupport dlqSupport)
deadLetterRoutingKey = dlqSupport.DeadLetterRoutingKey;
if (sqsSubscription is IUseBrighterInvalidMessageSupport invalidSupport)
invalidMessageRoutingKey = invalidSupport.InvalidMessageRoutingKey;
// Every other factory
var deadLetterRoutingKey = (subscription as IUseBrighterDeadLetterSupport)?.DeadLetterRoutingKey;
var invalidMessageRoutingKey = (subscription as IUseBrighterInvalidMessageSupport)?.InvalidMessageRoutingKey;The SQS pattern is more verbose for no gain. Align with the other factories for consistency. Minor Issues6.
7. Previously 8. Fragile Several new tests use 9. Kafka The change from 10. Copyright year in The copyright header was changed from 2017 to 2026. Confirm this is intentional and consistent with the project's policy on copyright year updates. Summary
Items 1 and 2 should be addressed before merge. The DRY concern (item 3) is worth at least a follow-up issue given the maintenance cost of keeping seven copies of the same routing logic in sync. Overall the approach is sound and the test coverage is thorough — good work bringing consistency across all providers. |
* docs: add DLQ requirement specs for remaining transports (0010-0015) Requirement specs for AWS SQS, Redis, MsSql, PostgreSQL, RocketMQ, and MQTT dead letter queue support. Numbered from 0010 to leave 0003-0009 free for parallel workstreams on other branches. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add ADR 0038 for SQS DLQ direct send, approve requirements and design SQS-specific decisions for replacing ChangeMessageVisibility(0) with direct SendMessage to DLQ. References ADR 0034 (strategy) and ADR 0036 (routing logic) for shared concerns. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add implementation tasks for SQS DLQ (spec 0010), update ADR 0038 producer creation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: replace _hasDlq flag with DLQ routing key parameters in SqsMessageConsumer Remove the inverted _hasDlq boolean from SqsMessageConsumer and replace with RoutingKey? deadLetterRoutingKey and RoutingKey? invalidMessageRoutingKey parameters, preparing for Brighter-managed DLQ support (ADR 0038). - Add connection and makeChannels parameters to SqsMessageConsumer for later use by lazy DLQ producers - Update SqsMessageConsumerFactory to pass new parameters - Update reject tests: reject without DLQ now deletes (no longer uses ChangeMessageVisibility(0) which was the inverted _hasDlq behavior) - Applied identically to both AWSSQS and AWSSQS.V4 packages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: SqsSubscription exposes DLQ and invalid message routing keys - SqsSubscription implements IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport interfaces - Add DeadLetterRoutingKey and InvalidMessageRoutingKey properties - Add optional constructor parameters, null by default (backward compatible) - Applied identically to both AWSSQS and AWSSQS.V4 packages - Test: When_creating_sqs_subscription_with_dlq_routing_keys_should_expose_properties - Test: When_creating_sqs_subscription_without_dlq_routing_keys_should_default_to_null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: consumer factory passes DLQ routing keys from subscription to consumer SqsMessageConsumerFactory.CreateImpl() now extracts routing keys from the subscription via IUseBrighterDeadLetterSupport/IUseBrighterInvalidMessageSupport interface checks and passes them to the SqsMessageConsumer constructor, matching the pattern established by KafkaMessageConsumerFactory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: reject SQS messages with DeliveryError by sending to DLQ queue Replace ChangeMessageVisibility(0) with direct SendMessage to DLQ per ADR 0038. SqsMessageConsumer now creates lazy DLQ/invalid-message producers and routes rejected messages with metadata (originalTopic, rejectionReason, rejectionTimestamp, originalMessageType) before deleting the original from the source queue. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify Unacceptable rejection routes to invalid message queue Integration test confirms DetermineRejectionRoute correctly sends messages rejected with Unacceptable reason to the invalid message queue (not DLQ) when both queues are configured. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify Unacceptable rejection falls back to DLQ when no invalid channel Integration test confirms that when only deadLetterRoutingKey is configured (no invalidMessageRoutingKey), rejecting with Unacceptable reason falls back to the DLQ per ADR 0036 routing decision tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify rejection with no channels configured acknowledges message Integration test confirms that when no deadLetterRoutingKey or invalidMessageRoutingKey is configured, rejecting with a reason still deletes the original message from the source queue. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify async Proactor path sends rejected messages to DLQ Integration test confirms the async/Proactor channel path correctly sends messages rejected with DeliveryError to the DLQ with metadata, exercising RejectAsync directly without the sync-over-async BrighterAsyncContext wrapper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: update SNS reject tests to assert message deletion instead of requeue The SNS requeue tests were still testing the old ChangeMessageVisibility(0) behavior. Updated to match the SQS reject test pattern: reject deletes the message from the queue (MT_NONE) rather than requeuing it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * spec: add Redis DLQ specification (0011) with approved requirements, ADR, and tasks Redis has no native DLQ support, so ADR 0039 adopts the same Brighter-managed pattern proven in Kafka (spec 0001) and SQS (spec 0010). Also marks spec 0010 tasks as complete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: RedisSubscription exposes DLQ and invalid message routing keys RedisSubscription now implements IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport with optional constructor parameters, following the same pattern as SQS. Backward compatible — defaults to null. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: consumer factory passes DLQ routing keys from subscription to consumer RedisMessageConsumerFactory.Create() and CreateAsync() now extract deadLetterRoutingKey and invalidMessageRoutingKey from subscriptions via interface checks and pass them to the RedisMessageConsumer constructor. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify DeliveryError rejection routes message to DLQ Implements the core DLQ routing logic in RedisMessageConsumer.Reject() and RejectAsync() following the SQS pattern: lazy producer creation, metadata enrichment, and ADR 0036 rejection routing decision tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify Unacceptable rejection routes to invalid message channel When both DLQ and invalid message channels are configured, Unacceptable rejections route to the invalid channel per ADR 0036. DLQ stays empty. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify Unacceptable rejection falls back to DLQ when no invalid channel configured - Test: When_rejecting_message_with_unacceptable_and_no_invalid_channel_should_fallback_to_dlq - Confirms ADR 0036 fallback: Unacceptable → DLQ when invalidMessageRoutingKey is null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify rejection with no channels configured removes message from inflight - Test: When_rejecting_message_with_no_channels_configured_should_remove_from_inflight - Confirms consumer can receive next message after rejecting without DLQ/invalid channels Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: verify async DeliveryError rejection routes message to DLQ - Test: When_rejecting_message_with_delivery_error_should_send_to_dlq_async - Confirms RejectAsync path sends to DLQ with rejection metadata Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: mark all Redis DLQ tasks complete in spec 0011 All 7 implementation tasks and verification complete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add ADR 0040 for MsSql DLQ brighter-managed Switch active spec to 0012-mssql-dead-letter-queue and add design for Brighter-managed DLQ support in the MsSql transport, following the established pattern from Kafka, SQS, and Redis. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * approve mssal specs * fix: use stored receipt handle in SQS RejectAsync to delete source message RefreshMetadata removes ReceiptHandle from the message bag before the subsequent AcknowledgeAsync calls, which silently no-op because the handle is gone. Extract DeleteSourceMessageAsync helper that takes the receipt handle directly, and use it in RejectAsync with the locally stored variable captured before RefreshMetadata runs. Applied to both V3 and V4 SQS consumers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: MsSqlSubscription exposes DLQ and invalid message routing keys MsSqlSubscription now implements IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport with optional deadLetterRoutingKey and invalidMessageRoutingKey constructor parameters. Both the base class and generic MsSqlSubscription<T> support the new parameters. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: MsSql consumer factory passes DLQ routing keys to consumer MsSqlMessageConsumer now accepts optional deadLetterRoutingKey and invalidMessageRoutingKey parameters. MsSqlMessageConsumerFactory extracts routing keys from subscription via IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport interface casts, passing them through to the consumer constructor. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: MsSql consumer rejects messages to DLQ on delivery error Implement Reject()/RejectAsync() with full DLQ routing in MsSqlMessageConsumer following the same pattern as Redis/SQS. Adds lazy MsSqlMessageProducer creation, RefreshMetadata() for rejection metadata enrichment, and DetermineRejectionRoute() for ADR 0036 routing decision tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: MsSql rejects unacceptable messages to invalid message channel Verifies that when both DLQ and invalid message routing keys are configured, rejecting with Unacceptable routes to the invalid message channel (not the DLQ). Confirms DetermineRejectionRoute() correctly prioritizes the invalid message producer per ADR 0036. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: MsSql unacceptable rejection falls back to DLQ when no invalid channel Verifies that when only deadLetterRoutingKey is configured (no invalidMessageRoutingKey), rejecting with Unacceptable falls back to the DLQ per ADR 0036 routing decision tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: MsSql reject with no channels configured returns true Verifies that when neither deadLetterRoutingKey nor invalidMessageRoutingKey is configured, Reject() returns true and the consumer can continue receiving subsequent messages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: MsSql async reject with delivery error sends to DLQ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add ADR 0041 and tasks for PostgreSQL DLQ (spec 0013) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: PostgresSubscription exposes DLQ and invalid message routing keys Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Postgres consumer factory passes DLQ routing keys to consumer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Postgres Reject() forwards messages to DLQ with metadata and deletes source Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: Postgres rejects unacceptable messages to invalid message channel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: Postgres unacceptable rejection falls back to DLQ when no invalid channel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: Postgres rejection with no channels configured deletes source and continues Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: Postgres async DeliveryError rejection sends message to DLQ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: ADR 0042 and tasks for RocketMQ DLQ (Spec 0014) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: RocketSubscription exposes DLQ and invalid message routing keys Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: RocketMQ consumer factory passes DLQ routing keys to consumer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: RocketMQ Reject() forwards messages to DLQ with metadata and Acks source Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ensure we can run rocketmq on a mac * test: RocketMQ DLQ rejection routing tests and docker-compose fixes Add integration tests for remaining rejection routing paths: - Unacceptable reason routes to invalid message channel - Unacceptable falls back to DLQ when no invalid channel configured - No channels configured Acks source and breaks requeue loop - Async DeliveryError routes to DLQ via Proactor path Fix docker-compose for RocketMQ 5.4.0: add platform linux/amd64 for Rosetta compatibility, update mqadmin path, reduce JVM heap sizes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add ADR 0043 for MQTT dead letter queue (Brighter-managed) Approve requirements and design for spec 0015. MQTT DLQ follows the Redis pattern — Lazy<T> producers, no source message cleanup (fire-and- forget), new MqttSubscription and MqttMessageConsumerFactory classes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: MqttSubscription exposes DLQ and invalid message routing keys - MqttSubscription implements IUseBrighterDeadLetterSupport and IUseBrighterInvalidMessageSupport - Generic MqttSubscription<T> variant for typed subscriptions - Test: When_creating_mqtt_subscription_with_dlq_routing_keys_should_expose_properties - Test: When_creating_mqtt_subscription_without_dlq_routing_keys_should_default_to_null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: MqttMessageConsumerFactory passes DLQ routing keys to consumer - MqttMessageConsumerFactory implements IAmAMessageConsumerFactory - Create()/CreateAsync() extract routing keys via IUseBrighterDeadLetterSupport/IUseBrighterInvalidMessageSupport - MqttMessageConsumer constructor accepts optional deadLetterRoutingKey and invalidMessageRoutingKey - MqttSubscription.ChannelFactoryType overridden to MqttMessageConsumerFactory Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: MQTT Reject() forwards messages to DLQ with metadata - Reject() and RejectAsync() forward rejected messages to configurable DLQ topic - RefreshMetadata() adds originalTopic, rejectionReason, rejectionTimestamp, originalMessageType - DetermineRejectionRoute() implements ADR 0036 routing (DeliveryError/Unacceptable/fallback) - Lazy<MqttMessageProducer> for DLQ and invalid message producers - Separate sync/async Reject implementations to avoid nested sync-over-async deadlock - Added .slopwatch/config.json to suppress SW004 in test projects Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: MQTT Reject with Unacceptable routes to invalid message channel - Verifies Unacceptable reason routes to invalid channel (not DLQ) - Confirms DLQ does not receive the message when invalid channel is configured Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: MQTT DLQ rejection routing tests and async variant - Unacceptable fallback to DLQ when no invalid channel configured - No channels configured returns true (breaking change from false) - Async DeliveryError rejection routes to DLQ with metadata Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: mark all MQTT DLQ tasks complete (spec 0015) All 7 tasks + regression verification complete: - 11/11 tests pass (existing flaky 1000-msg async test is pre-existing) - Backward compatible: existing MqttMessageConsumer constructor still works Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: switch current spec to 0015-mqtt-dead-letter-queue - Update .current-spec to MQTT DLQ spec - Update RocketMQ tasks.md with completed checkboxes - Add .tasks-approved for MQTT spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address DLQ cross-spec review findings (H1, H2, M1-M3, L6-L7) - Redis: dispose DLQ producers in Dispose/DisposeAsync to prevent connection leaks - MQTT: add ClientID with -dlq/-invalid suffix to DLQ producer config - MQTT: fix misleading log in ResolveRejectionProducer, return tuple for null safety - Backstop: add source-generated logging per ADR 0037 in both sync/async handlers - RocketMQ: replace throw-in-finally with AckSourceMessageSafeAsync to avoid masking DLQ result - RocketMQ: add thread-safety comment on producer caching fields - Remove stale RELIABILITY-IMPROVEMENTS-SUMMARY.md - Add cross-spec review findings document with checklist Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update collection type * fix: await DisposeAsync on DLQ producers in SQS consumers SqsMessageConsumer.DisposeAsync() was calling sync Dispose() on producers that implement IAsyncDisposable, bypassing async cleanup. Now checks for IAsyncDisposable and awaits it, matching the pattern used by Kafka and Redis consumers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace null-forgiving operator with null-coalescing fallback in RefreshMetadata reason.Description is nullable; the `!` operator suppressed the compiler warning but could silently insert null into the message bag. Use `?? string.Empty` to provide a safe runtime fallback across all 8 transport consumers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add FIFO DLQ routing integration tests for SQS (V3 and V4) FIFO queues require MessageGroupId (from PartitionKey) and optionally MessageDeduplicationId. These tests verify that rejected FIFO messages arrive at a FIFO DLQ with FIFO attributes preserved, covering both Reactor (sync) and Proactor (async) paths for V3 and V4 SDK variants. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: pass SqsAttributes to DLQ producers for FIFO queue support The DLQ and invalid message producers were created without queue attributes, so FIFO queues were created as standard queues (missing .fifo suffix). This caused DLQ sends to fail silently for FIFO messages because MessageGroupId is invalid on standard queues. Pass SqsAttributes from the subscription through the consumer to the DLQ/invalid message producer publications, ensuring FIFO DLQ queues are created correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use LazyThreadSafetyMode.None for DLQ producers to avoid caching exceptions The default LazyThreadSafetyMode.ExecutionAndPublication caches thrown exceptions permanently. A single transient failure during DLQ producer creation would disable DLQ routing for the lifetime of the consumer. LazyThreadSafetyMode.None does not cache exceptions, allowing retry on the next .Value access. This is safe because message pumps are single-threaded per consumer. Affects all 7 Brighter-managed DLQ transports: SQS (V3+V4), Kafka, MQTT, Redis, PostgreSQL, MsSql. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Whilst we have worked on Kafka and InMemory, this PR extends the work out to all of our providers, to ensure that we have consistent behaviour around RejectMessageAction