Repository navigation
Description
Mellea's OpenTelemetry instrumentation uses `gen_ai.*` span attributes, which is correct and ecosystem-aligned. However, the span name and all metric names deviate from the OTel GenAI semantic conventions, which means off-the-shelf GenAI observability tooling — Grafana GenAI dashboards, OTel Collector semantic processors, vendor-provided GenAI monitors — does not recognise mellea's signals out of the box. Users get data into their backend but none of the pre-built interpretability tooling lights up, and they cannot use community-maintained dashboards without rewriting every metric reference.
This issue tracks the full alignment checklist. Items are additive on the existing cpex hook/plugin architecture; none require structural changes to the backend or session APIs. M1/M2/M3 are breaking changes for existing dashboards and need a migration note.
Spans
- S1 Span name
"chat"→"chat {gen_ai.request.model}"(SHOULD per semconv — enables model-level trace filtering without attribute filters) - S2 Span kind should be
CLIENTfor remote-backend calls, notINTERNAL(SHOULD) - S3
server.address/server.portnot set on any span (Conditionally Required) - S5
gen_ai.request.streamnot set;mellea.streamingused instead (Conditionally Required) - S6
gen_ai.provider.nameenum values must match semconv (ibm.watsonx.ai, not"watsonx") (MUST where set) - S8
gen_ai.output.typenot set when structured output is requested;mellea.has_formatused instead (Conditionally Required) - S11
gen_ai.response.time_to_first_chunkmissing as a span attribute; currently recorded only as a metric (Recommended) - S9
gen_ai.usage.total_tokensis not in the spec; remove or document as a mellea extension - S10
mellea.streaming/mellea.format_typeduplicate information now in semconv attributes; clean up
Metrics
- M1 Token usage: two
Counterinstrumentsmellea.llm.tokens.input/mellea.llm.tokens.output→ singleHistogramgen_ai.client.token.usagewithgen_ai.token.typedimension ("input"/"output") and semconv-specified bucket boundaries⚠️ breaking change - M2 Duration:
mellea.llm.request.duration→gen_ai.client.operation.duration⚠️ breaking change - M3 TTFB:
mellea.llm.ttfb→gen_ai.client.operation.time_to_first_chunk⚠️ breaking change - M4
gen_ai.operation.namedimension missing from all client metrics (Required dimension per spec) - M5
server.address/server.portmissing from all metrics (Recommended / Conditionally Required) - M6
error.typemissing on the duration metric for failed operations (Conditionally Required) - M8
streamingdimension on duration is non-standard; superseded bygen_ai.operation.name+ semconv approach - M9
gen_ai.client.operation.time_per_output_chunknot implemented (Recommended) - M10
error_typevserror.typekey inconsistency on the error counter (cleanup)
Migration note (M1/M2/M3)
Renaming the three primary metrics is a breaking change for any existing dashboard or alert rule targeting the mellea.llm.* names. A migration plan (dual-emit period, deprecation notice, version bump) should accompany those three items.
Already tracked elsewhere (not duplicated here)
gen_ai.request.*parameters (temperature, max_tokens, etc.) → feat(telemetry): emit gen_ai.request.* from tracing plugin (post-#1045) #1068- Chat content events (
gen_ai.content.prompt/gen_ai.content.completion) → feat(telemetry): emit gen_ai.input/output.messages content events from tracing plugin (post-#1045) #1073 - Prompt template attributes → feat(telemetry): emit llm.prompt_template.* from tracing plugin (post-#1045) #1067
execute_toolspan → feat: tool call spans via tool_post_invoke hook #1049- Metrics plugins on
generation_batch_*hooks → refactor: subscribe metrics plugins to generation_batch_* hooks #1182
Activity