Skip to content

bug(ingest): unsafe/empty table-name rejections silently drop instead of routing to DLQ #151

Description

@EricAndrechek

Problem

internal/ingest/bento.go:108:

// Validate table name to prevent SQL injection.
if !safeIdentifierRe.MatchString(raw.TableName) {
    slog.WarnContext(msgCtx, "rejecting message with unsafe table name", "table", raw.TableName)
    // TODO: manually push to a DLQ subject with metadata for later analysis instead of silently dropping?
    if doubleAckErr := m.DoubleAck(msgCtx); doubleAckErr != nil {
        slog.WarnContext(msgCtx, "double ack failed for dropped message", "error", doubleAckErr)
    }
    continue
}

When the table name fails the safeIdentifierRe check, the message is logged at WARN and DoubleAck'd — i.e. permanently removed from JetStream — with no DLQ entry and no payload retained anywhere. Per AGENTS.md design decision #6 the DLQ exists precisely "to prevent silent data loss." This path violates that invariant.

The adjacent empty-table_name branch (bento.go:97-103) has the same shape and the same gap.

Proposed Solution

Route both rejection branches through the existing DLQ output (stream WAVEHOUSE_DLQ, subject dlq.<table> — falling back to dlq.__rejected__ when the table name itself is the reason for rejection). Include the rejection reason in headers (x-wh-drop-reason: unsafe-table-name / empty-table-name) so operators can grep the DLQ for these specifically.

Acceptance criteria

  • No code path in bento.go silently drops a message — every drop either lands in the DLQ or surfaces with full payload at ERROR.
  • Tests cover both the unsafe-table-name and empty-table-name DLQ paths.
  • AGENTS.md design decision ci: bump goreleaser/goreleaser-action from 6 to 7 #6 wording matches the new behavior (or stays as-is if the new behavior matches the stated invariant).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/ingestIngest pipeline (Bento, batching, DLQ)bugSomething isn't workingdocumentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions