ai(observability): every claude-code call records exactly 2 input tokens — the cache_read/cache_creation keys are never counted

[JSONbored]

Part of epic #8286 (Phase 3 — AI observability). Sibling of #10207/#10211, found while verifying those.

Problem

The ORB records exactly ~2 input tokens for every claude-code call, forever. Measured on edge-nl-01 over 48h:

provider	calls	input total	input avg	output total	output avg	cost
claude-code	1000	2048	2.0	1,051,346	1051.3	$225.06

An average of 2.0 across a thousand calls is not a measurement, it is a constant. The output side is healthy (1051/call) and the dollar cost is real, so the event is arriving and being parsed — the input figure specifically is meaningless.

Cause

mergeUsage (src/selfhost/ai.ts:821) reads the input side through:

const INPUT_TOKEN_KEYS = ["input_tokens", "inputTokens", "prompt_tokens", "promptTokens"] as const;

Claude Code's stream-json result frame reports usage.input_tokens as only the uncached portion of the prompt. With prompt caching active — which it is for every review, since the system prompt and tool definitions are stable — essentially the entire prompt is served from cache, and the real volume lands in two keys this list does not contain:

• cache_read_input_tokens
• cache_creation_input_tokens

So input_tokens degenerates to the handful of genuinely-new tokens per call, which is the flat ~2 observed. Nothing is malformed and nothing is being dropped defensively — the extractor is reading a real field that simply does not mean what the name suggests under caching.

Impact

• Cost-per-token and input:output ratio are wrong for the provider carrying the majority of ORB spend ($225 of $250 in the window).
• loopover_ai_input_tokens_total (src/selfhost/ai.ts:876) has the same defect, so the Prometheus/Grafana view is wrong in the same direction.
• This compounds ai(observability): PostHog holds no token counts for any ORB AI call, though $498 of spend lands correctly #10207: that issue stops a fabricated 0 from being reported. This one is a real number that is off by roughly three orders of magnitude, which is harder to notice precisely because it looks like data.
• Anything downstream that reasons about prompt size from this figure — budget backstops, neuron estimates — is reasoning from ~2.

Suggested fix

Add the two cache keys to the input-token accounting. They are genuine input tokens: the model processed them, and they are billed (at a reduced rate for cache reads).

Note this is not a matter of appending them to INPUT_TOKEN_KEYS. maxNumber takes the maximum across the key list, which is correct for aliases of one value (input_tokens vs prompt_tokens are two names for the same number) but wrong for these — the three are additive components of one prompt, so they must be summed and then combined with the alias-max result. Getting that distinction wrong would silently under-report again, just less severely.

Worth deciding explicitly whether to keep the cached portion separately as well, since PostHog has a recognised $ai_cache_read_input_tokens property and reporting it separately preserves the cache-effectiveness signal instead of flattening it into one number.

Deliverables

• cache_read_input_tokens / cache_creation_input_tokens are counted as input tokens, summed with the uncached portion rather than max'd against it.
• A regression test built from a real claude-code result frame carrying all three, pinning the summed total — and one with only input_tokens present, pinning that the un-cached path is unchanged.
• loopover_ai_input_tokens_total reports the same corrected figure as the PostHog event, verified against the same fixture.
• Decide whether the cache-read portion is also reported separately ($ai_cache_read_input_tokens).

[linear-code]

JSO-7141

[JSONbored]

Scope clarification — two usage paths, only ONE should change

While confirming the fixture shape I found a deliberate decision that a careless fix would trample.

There are two independent usage-normalisation paths:

1. CLI subprocess path — mergeUsage / INPUT_TOKEN_KEYS (src/selfhost/ai.ts:799). This is the claude-code and codex providers, and this is where the flat ~2 comes from. Claude Code the CLI uses prompt caching internally, so its result frame reports only the uncached remainder in usage.input_tokens. This is the path to fix.

2. BYOK API path — coerceByokUsage (src/services/ai-review.ts:2006). Its doc comment already addresses the same two keys and deliberately does not read them:

   "Anthropic's usage can also carry cache_creation_input_tokens/cache_read_input_tokens, priced differently than input_tokens — intentionally not read here, since callAiProvider never sends cache_control, so Anthropic never populates them on this path."

   That reasoning is sound and still holds. Do not change this one. Adding the keys here would be dead code at best, and if cache_control is ever introduced it would silently mis-price, because coerceByokUsage feeds BYOK_MODEL_PRICING_USD_PER_MTOK where cache-read tokens bill at a different rate than fresh input tokens.

The two paths agreeing on the key names but disagreeing on whether to read them is correct, not drift — a fix that "unifies" them is the wrong fix.

Added deliverable

• coerceByokUsage is left untouched, and its existing comment is extended to say why the CLI path now differs — so the next reader sees an intentional divergence rather than an oversight.