Skip to content

feat(telemetry): adopt OTel GenAI semantic convention names for spans and metrics #1209

Closed

Description

@planetf1

Mellea's OpenTelemetry instrumentation uses `gen_ai.*` span attributes, which is correct and ecosystem-aligned. However, the span name and all metric names deviate from the OTel GenAI semantic conventions, which means off-the-shelf GenAI observability tooling — Grafana GenAI dashboards, OTel Collector semantic processors, vendor-provided GenAI monitors — does not recognise mellea's signals out of the box. Users get data into their backend but none of the pre-built interpretability tooling lights up, and they cannot use community-maintained dashboards without rewriting every metric reference.

This issue tracks the full alignment checklist. Items are additive on the existing cpex hook/plugin architecture; none require structural changes to the backend or session APIs. M1/M2/M3 are breaking changes for existing dashboards and need a migration note.

Spans

  • S1 Span name "chat" → "chat {gen_ai.request.model}" (SHOULD per semconv — enables model-level trace filtering without attribute filters)
  • S2 Span kind should be CLIENT for remote-backend calls, not INTERNAL (SHOULD)
  • S3 server.address / server.port not set on any span (Conditionally Required)
  • S5 gen_ai.request.stream not set; mellea.streaming used instead (Conditionally Required)
  • S6 gen_ai.provider.name enum values must match semconv (ibm.watsonx.ai, not "watsonx") (MUST where set)
  • S8 gen_ai.output.type not set when structured output is requested; mellea.has_format used instead (Conditionally Required)
  • S11 gen_ai.response.time_to_first_chunk missing as a span attribute; currently recorded only as a metric (Recommended)
  • S9 gen_ai.usage.total_tokens is not in the spec; remove or document as a mellea extension
  • S10 mellea.streaming / mellea.format_type duplicate information now in semconv attributes; clean up

Metrics

  • M1 Token usage: two Counter instruments mellea.llm.tokens.input / mellea.llm.tokens.output → single Histogram gen_ai.client.token.usage with gen_ai.token.type dimension ("input" / "output") and semconv-specified bucket boundaries ⚠️ breaking change
  • M2 Duration: mellea.llm.request.duration → gen_ai.client.operation.duration ⚠️ breaking change
  • M3 TTFB: mellea.llm.ttfb → gen_ai.client.operation.time_to_first_chunk ⚠️ breaking change
  • M4 gen_ai.operation.name dimension missing from all client metrics (Required dimension per spec)
  • M5 server.address / server.port missing from all metrics (Recommended / Conditionally Required)
  • M6 error.type missing on the duration metric for failed operations (Conditionally Required)
  • M8 streaming dimension on duration is non-standard; superseded by gen_ai.operation.name + semconv approach
  • M9 gen_ai.client.operation.time_per_output_chunk not implemented (Recommended)
  • M10 error_type vs error.type key inconsistency on the error counter (cleanup)

Migration note (M1/M2/M3)

Renaming the three primary metrics is a breaking change for any existing dashboard or alert rule targeting the mellea.llm.* names. A migration plan (dual-emit period, deprecation notice, version bump) should accompany those three items.

Already tracked elsewhere (not duplicated here)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestp2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions