[token-consumption] Daily AIC Consumption Report - 2026-08-03

[github-actions]

Executive Summary

• Total AIC: not computable from either telemetry backend today. Sentry span search over the last 24h shows 0 of 25,912 spans expose the gh-aw.aic attribute (has:gh-aw.aic → count()=0), despite the emitter (actions/setup/js/send_otlp_span.cjs) explicitly always encoding gh-aw.aic as a doubleValue for token-owning jobs (agent, detection, engine job). Grafana/Tempo could not be queried at all in this session (see below) — this is an observability gap, not a "0 usage" day.
• Workflow attribution itself is healthy: 133 unique workflows are identifiable via gh-aw.workflow.name across 6,014 tagged spans, and one representative trace (Test Quality Sentinel, run 30815062964) was confirmed to correctly link 5 job-level spans (pre_activation → activation → agent → push_experiments_state) under a single trace ID.
• Errors and logs datasets returned zero events in the 24h window (both checked explicitly, not skipped).
• Sentry AIC: not queryable. Grafana AIC: not verified this session (Tempo query tools unavailable).

Key Metrics

Metric	Value
Events analyzed	25,912 (all spans, Sentry, 24h)
Events with AIC data	0 confirmed (Grafana unverified — see gaps)
Events with AIC data (Sentry)	0
Events with AIC data (Grafana)	unknown — Tempo query tools unavailable this session
Total AIC	not computable (attribute unqueryable in Sentry EAP)
Unique workflows	133
Avg AIC/event	not computable
P95 AIC/event	not computable

Top 10 Workflows by Event Volume (AIC unavailable — event count used as proxy)

Workflow	Events	Total AIC	Avg AIC/Event
Test Quality Sentinel	693	N/A	N/A
PR Code Quality Reviewer	608	N/A	N/A
Design Decision Gate 🏗️	568	N/A	N/A
PR Sous Chef	514	N/A	N/A
Matt Pocock Skills Reviewer	449	N/A	N/A
Impeccable Skills Reviewer	436	N/A	N/A
Issue Monster	285	N/A	N/A
Avenger	180	N/A	N/A
Contribution Check	90	N/A	N/A
Code Scanning Fixer	76	N/A	N/A

Grafana AIC Findings

Grafana AIC: not queryable this session — tool access gap.

• list_datasources confirmed a Tempo datasource is configured (grafanacloud-ghaw-traces, uid grafanacloud-traces, backend tempo-prod-10-prod-eu-west-2.grafana.net), and get_datasource returned its config successfully.
• However, the Tempo query tools required by the runbook (tempo_get-attribute-names, tempo_traceql-search, tempo_get-trace) were not present in this session's available toolset — only datasource-metadata tools (list_datasources, get_datasource) were reachable. No TraceQL search or attribute enumeration could be executed, so it is unknown whether gh-aw.aic is present, numeric, or indexed on Tempo spans.
• Evidence line: mcp__grafana__get_datasource(uid="grafanacloud-traces") → succeeded (datasource exists); no tempo_* tool was callable to go further.
• This is a tooling availability gap for this run, not a statement that Grafana lacks the data — it should be re-checked in an environment where the Tempo query tools are exposed.

Data Quality and Gaps

• Events missing workflow identifiers: 19,898 of 25,912 spans (76.8%) have no gh-aw.workflow.name attribute — these are likely spans emitted by job types that don't currently attach workflow-identifying attributes (e.g. lower-level internal spans), not necessarily workflow runs without a name.
• Events missing AIC attributes: 25,912 of 25,912 (100%) in Sentry — has:gh-aw.aic, gh-aw.aic:>=0, and gh-aw.aic:0 all return count()=0.
• Root-cause narrowing performed: another numeric attribute emitted in the same code block, gh-aw.turns, is also unqueryable (has:gh-aw.turns → no results), while string attributes emitted in the same span (gh-aw.workflow.name, gh-aw.run.id) are queryable and render correctly. This points at a Sentry EAP schema-inference gap for double/numeric custom attributes on this project, not a code-side emission bug — the source comment in send_otlp_span.cjs explicitly says the double-encoding is meant to make gh-aw.aic "queryable immediately," which is not what's observed.
• Assumptions/fallbacks used: none needed for Sentry workflow attribution (data was clean); AIC values were never zero-filled or invented — reported as unknown per guardrails.
• Sentry-specific caveat: list_events's markdown renderer only displays a fixed subset of fields (id, timestamp, project.name) in its default (non-aggregate) mode regardless of requested fields — aggregate-mode queries (fields: ["x", "count()"]) were required to actually surface attribute values and made this investigation possible.
• Grafana-specific caveat: no span-level query capability in this session; datasource existence confirmed only.

Recommendations

1. Fix the Sentry EAP ingestion/indexing gap for gh-aw.aic (and gh-aw.turns) — file this as a P1 observability bug. Since gh-aw.workflow.name (string) is queryable but gh-aw.aic (double) is not, despite identical emission path, check whether Sentry EAP needs a first non-zero doubleValue emission to infer schema, or whether the attribute name needs to avoid the hyphen (gh-aw.aic) for indexing — compare against gen_ai.usage.* conventions which may already be indexed.
2. Re-run this report in a session where Grafana Tempo query tools (tempo_get-attribute-names, tempo_traceql-search) are exposed, to cross-validate whether AIC is queryable there even while Sentry's schema gap persists — this would unblock daily AIC reporting immediately via Grafana as an interim source.
3. Investigate the 76.8% of spans missing gh-aw.workflow.name — even without AIC, ensuring all job-type spans (not just agent/activation/pre_activation/etc.) carry workflow attribution will make future volume-based cost proxying more accurate.
4. Until AIC is queryable, use event/span volume as a directional proxy: Test Quality Sentinel, PR Code Quality Reviewer, Design Decision Gate, PR Sous Chef, and the two Skills Reviewer workflows account for the top 6 span-volume slots (693–436 spans each) — worth a manual spot-check of their per-run turn counts/model usage while the numeric-attribute gap is fixed.

References

• Sentry aggregate query: workflow event counts (24h)
• Sentry trace continuity check: Test Quality Sentinel run 30815062964
• §30815094466

Generated by 📊 Daily AIC Consumption Report (Sentry + Grafana OTel) · agent · 120.8 AIC · ⌖ 41.3 AIC · ⊞ 7.8K · ◷

• expires on Aug 4, 2026, 4:58 AM UTC-08:00

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #50254

View newer issue

---

This action was performed automatically by the Daily AIC Consumption Report (Sentry + Grafana OTel) workflow.