You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
refactor: subscribe metrics plugins to generation_batch_* hooks #1182
PR #1181 (Phase 1 of #444) introduces three new hooks for raw-path generations:
GENERATION_BATCH_PRE_CALL
GENERATION_BATCH_POST_CALL
GENERATION_BATCH_ERROR
The tracing plugin subscribes to all three so that generate_from_raw emits one span per call (per OTel GenAI semconv). The metrics plugins do not — mellea/telemetry/metrics_plugins.py only subscribes to per-MOT hooks:
Plugin
Subscribed hooks
TokenMetricsPlugin
generation_post_call
LatencyMetricsPlugin
generation_post_call
CostMetricsPlugin
generation_post_call
ErrorMetricsPlugin
generation_error
As a result, generate_from_raw calls produce zero metrics today. Tracing covers it; metrics don't.
Scope
Pure additive. For each metrics plugin, add a parallel hook handler that subscribes to the corresponding generation_batch_* hook and records the same metrics from the batch payload. Estimated ~30–40 LOC.
Acceptance criteria
All four metrics plugins subscribe to the matching generation_batch_* hook.
Token, latency, cost, and error metrics are recorded for generate_from_raw calls in addition to generate_from_chat_context.
Test added that calls generate_from_raw with metrics enabled and asserts each counter/histogram is incremented.
Reference
PR #1181 (the tracing plugin in mellea/telemetry/tracing_plugins.py shows the batch-hook subscription shape to mirror).
Activity
feat(telemetry): wire metrics plugins to generation_batch_* hooks
feat(telemetry): wire metrics plugins to generation_batch_* hooks (ge…