Skip to content

OtlpHttpFanOut::gc has no callers, so pipelines for deleted exporters are never stopped #954

Description

@membphis

OtlpHttpFanOut::gc (crates/aisix-obs/src/otlp_http_sink.rs:263)
documents itself as "Called periodically by the server to GC pipelines for
deleted / disabled exporters", but git grep '\.gc(' returns no callers
anywhere in the tree. Nothing ever reaps them.

Consequence: when an operator deletes or disables an observability
exporter, the fan-out stops handing it events (the exporter is gone from
the snapshot), but its pipeline — task, client, queue — stays alive for the
life of the process. A deployment that churns exporters accumulates them.

Found while auditing #953, which tightened the adjacent
behaviour: the fan-out now re-reads the live snapshot rather than the
request's frozen one, so a deleted exporter stops receiving events
immediately. That closes the delivery side; this issue is the resource side,
and it predates that PR — gc has never been wired.

Worth deciding between two shapes rather than just adding a caller:

  • call gc from the existing snapshot-apply path, so reaping is driven by
    the config change that caused it, or
  • call it from a periodic task, which is what the doc comment assumes.

The first is more precise and has no idle cost; the second is more forgiving
if a pipeline can outlive its exporter for reasons other than deletion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions