Extract and persist real coding-agent token usage (Agent SDK usage.input_tokens/output_tokens, CLI JSON output)

[JSONbored]

Context

AttemptUsage.tokens/AttemptMeterTotals.tokens (packages/gittensory-engine/src/miner/attempt-metering.ts, iterate-loop.ts) is hardcoded to 0 on every iteration — an intentional, documented placeholder (#5395's own comment: "tokens stays an honest 0, no driver reports a real token count today"). This is a real gap, not a design decision to keep: real token data already exists and is simply not being read.

• The Agent SDK's own SDKResultMessage (both SDKResultSuccess and SDKResultError variants, node_modules/@anthropic-ai/claude-agent-sdk/sdk.d.ts) carries a real usage: NonNullableUsage field with non-nullable input_tokens/output_tokens. agent-sdk-driver.ts already reads resultMessage.total_cost_usd/resultMessage.num_turns off this SAME object — it just never reads resultMessage.usage.
• src/selfhost/ai.ts's extractCliUsage (used by ORB's own self-host AI review path) already has a proven, tested, best-effort token extractor for Claude/Codex CLI JSON/JSONL stdout (INPUT_TOKEN_KEYS/OUTPUT_TOKEN_KEYS/TOTAL_TOKEN_KEYS, scanning record.usage/record.token_usage/record.tokenUsage/record.usage_metadata). packages/gittensory-engine/src/miner/cli-subprocess-driver.ts's own comment explicitly says it "mirrors src/selfhost/ai.ts's extractCliUsage/COST_KEYS... but narrowed to just the cost field this driver's CodingAgentDriverResult surfaces -- tokens and model aren't part of that shape" — the narrowing was a deliberate scope cut at the time, not because the data doesn't exist.
• feat(miner): persist coding-agent provider + real cost on the attempt log #5637 already added tokensUsed as a first-class, nullable column on the attempt-log's attempt_outcome_summary event and the redacted AMS reporting export (Grafana-visible via Add a Grafana dashboard for coding-agent-driver usage, cost, and attempt outcomes #5185's miner-usage.json) — it's currently always written as undefined/null because nothing upstream produces a real value.
• This gap is directly load-bearing for Miner Wave 5 (Rent-a-Loop): Stake/payment → compute-time pricing #4790 (Stake/payment → compute-time pricing) and Per-tenant rate limits & quotas #4796 (Per-tenant rate limits & quotas) will need real usage data, not an honest zero, to price and throttle a paying tenant's actual consumption.

Dependencies

None — independently shippable. Builds on #5637 (merged) purely as a downstream consumer (the tokensUsed field it already persists).

Requirements

1. Add a usage?: { inputTokens?: number; outputTokens?: number; totalTokens?: number } | undefined field (or equivalent) to CodingAgentDriverResult (packages/gittensory-engine/src/miner/coding-agent-driver.ts), mirroring the existing costUsd field's own "absent (not zero) when unavailable, never fabricated" convention.
2. agent-sdk-driver.ts: read resultMessage.usage.input_tokens/.output_tokens (both the SDKResultSuccess and SDKResultError variants) alongside the existing total_cost_usd/num_turns reads, and populate the new usage field.
3. cli-subprocess-driver.ts: port (not reinvent) src/selfhost/ai.ts's extractCliUsage token-extraction logic (the INPUT_TOKEN_KEYS/OUTPUT_TOKEN_KEYS/TOTAL_TOKEN_KEYS constants and the nested-record scan) to extract real tokens from Claude/Codex CLI JSON/JSONL stdout, alongside the existing cost extraction. A missing/malformed field means "no token signal," exactly like the existing cost extraction — never a fabricated 0.
4. iterate-loop.ts: replace the hardcoded tokens: 0 in the per-iteration AttemptUsage increment with the real value from driverResult.usage?.totalTokens ?? 0 (mirroring exactly how costUsd/turns/wallClockMs are already threaded through per Decide fate of attempt-metering.ts: wire real per-attempt budget enforcement, or remove it #5395) — a pure data-accuracy change, no change to the loop's continue/handoff/abandon decision logic, thresholds, or the Governor chokepoint's own control flow.
5. attempt-cli.js: pass the real accumulated token total (from the loop result) into the attempt_outcome_summary event's tokensUsed field instead of always omitting it.
6. maxTokens/tokens-axis budget enforcement (attempt-metering.ts's evaluateAttemptBudget) stays OUT of scope for this issue — wiring a real number into that axis is a separate, deliberate policy decision (whether/how to enforce a token ceiling) and not implied by simply reporting the number accurately.

Deliverables / Acceptance Criteria

• CodingAgentDriverResult carries real, driver-reported token usage when available
• agent-sdk-driver.ts extracts real input_tokens/output_tokens from the SDK's own result message
• cli-subprocess-driver.ts extracts real tokens from Claude/Codex CLI JSON output, mirroring src/selfhost/ai.ts's proven extraction logic
• iterate-loop.ts's per-iteration meter totals report real tokens, not a hardcoded 0, with zero change to any continue/handoff/abandon decision
• attempt_outcome_summary events persist the real token total
• Every driver that genuinely reports no token signal continues to report an honest absence (null/undefined), never a fabricated number

Test Coverage Requirements

This PR must ship with full test coverage for every changed line and branch — the repo's Codecov patch gate requires 99%+ coverage and the house standard is to aim for 100%, including both sides of every conditional/nullish-coalescing branch introduced. Required: (1) agent-sdk-driver.ts tests for both the success and error result-message variants reporting real tokens, and the case where usage is absent/malformed; (2) cli-subprocess-driver.ts tests mirroring src/selfhost/ai.ts's own extractCliUsage test matrix (multiple key spellings, nested usage/token_usage objects, malformed/non-JSON stdout); (3) an iterate-loop.ts regression test asserting the accumulated finalMeterTotals.tokens reflects real per-iteration values across multiple iterations, AND that the loop's continue/handoff/abandon decision is byte-identical to before this change for the same inputs (proves this is data-accuracy-only, no control-flow drift); (4) an attempt-cli.js test asserting the real token total reaches the persisted attempt_outcome_summary event.

Expected Outcome

packages/gittensory-miner/lib/attempt-log.js's tokens_used column, and the grafana/dashboards/miner-usage.json "Total tokens" panel it feeds (#5185), report real per-provider token consumption instead of an honest but uninformative 0 — and Wave 5's pricing/rate-limit work has real usage data to build on instead of needing to invent its own extraction from scratch.

Links & Resources

• packages/gittensory-engine/src/miner/coding-agent-driver.ts (CodingAgentDriverResult)
• packages/gittensory-engine/src/miner/agent-sdk-driver.ts (Agent SDK result-message reading)
• packages/gittensory-engine/src/miner/cli-subprocess-driver.ts (CLI stdout parsing)
• packages/gittensory-engine/src/miner/iterate-loop.ts / attempt-metering.ts (per-iteration accumulation)
• src/selfhost/ai.ts's extractCliUsage (the extraction logic being ported)
• node_modules/@anthropic-ai/claude-agent-sdk/sdk.d.ts's SDKResultMessage/NonNullableUsage types
• feat(miner): persist coding-agent provider + real cost on the attempt log #5637 (attempt_outcome_summary event this feeds), Add a Grafana dashboard for coding-agent-driver usage, cost, and attempt outcomes #5185 (the dashboard this makes real)
• Theme: AMS hardening / Wave 5 prerequisite

[linear-code]

JSO-3878