Repository navigation
feat: add Grok CLI as a usage provider - #133
Merged
Merged
Conversation
The official xAI Grok CLI keeps per-turn token usage in ~/.grok/sessions/<id>/updates.jsonl: each turn appends a durable `sessionUpdate:"turn_completed"` line whose `usage` object is that prompt's ledger — input/output/cached-read/reasoning tokens plus the server's own cost in USD ticks. Nothing else in the session directory carries usage: `chat_history.jsonl` items have no usage field, `events.jsonl` turn records only an outcome, and the metadata that does carry tokens is an upload artifact for remote telemetry. Token mapping preserves the identity the other providers keep (Entry.total == the source's own total). `inputTokens` is the full prompt *including* cache reads, so input is inputTokens − cachedReadTokens and cacheRead carries the rest; reasoning already sits inside outputTokens; cache writes are folded into the prompt server-side, so cacheWrite stays 0. The headless projection spells the same fields in snake_case with `input_tokens` *excluding* the cache, so the parser branches on the spelling rather than treating them as aliases — reading them as one would subtract the cache twice. Cost uses `costUsdTicks` only, and only when neither `costIsPartial` nor `usageIsIncomplete` is set. There is no rate table for Grok models, so an untrustworthy bill reads as $0 instead of an estimate. Two double-counting paths are closed. Turn ids are session-independent (`grok|<prompt_id>`, a v4 uuid) because forking copies the parent's updates file, and replayed appends are marked `_meta.isReplay`. Subagent sessions are skipped: their tokens are already folded into the parent turn's ledger, which is also why the CLI hides them from its own session list. No official-limit row. The CLI exposes no local quota numbers — only a subscription tier name and an error string when the balance runs out — and status.x.ai serves no public statuspage JSON, so there is no status endpoint either. Aggregates, burn tier and the companion rhythm stay provider-agnostic; the only registration points are the new provider and the UsageStore default list. Scan cost is bounded by a string prefilter before JSON parsing: updates.jsonl persists every streaming chunk, so a session is tens of thousands of lines of which only the turn endings matter. The blob cache reads updates.jsonl alone, never the sibling files.
Covers the intro line, the how-it-works step, the per-service tab note, the requirements sentence, the coverage table (today · 5h block · week · month, no official limits) and a data-source row for ~/.grok/sessions/**/updates.jsonl noting that subagent sessions are excluded because the parent turn already counts them. No new screenshot: the change adds no surface of its own — the popover's existing provider tab bar simply gains one more capsule when Grok data is present — and this machine has no Grok session data to photograph.
Most users have no ~/.grok at all, so that is the path the provider takes on almost every machine: it has to stay silent rather than throw (which would paint the whole refresh as an error) or report zero (which would add a tab for a service the user never touched).
The replay case only ever used a duplicate turn id, which global dedup already collapses — so the isReplay branch was never actually exercised and would have kept passing if that check were deleted. It now also feeds a replayed line with a distinct id, which only the isReplay skip can stop. The session-root case recomputed the implementation's own expression and then compared it to itself. It now asserts the invariant instead: the default is ~/.grok/sessions, and when $GROK_HOME is set the root lives under it.
…n dates
Verifying the parser against the upstream writer (xai-org/grok-build) instead of
against its own fixtures turned up two defects that would have shipped a Grok tab
stuck at zero.
The persisted line is `{"timestamp": <unix seconds>, "method":
"_x.ai/session/update", "params": {sessionId, update, _meta}}` — the notification
sits under `params`, not `update`, and the timestamp is an integer, not ISO 8601.
Reading `update` found nothing on every real line, and even had it matched, the
date parse would have failed on a number.
Turn dates now prefer `_meta.agentTimestampMs` over the envelope timestamp.
Forking a session copies the parent's updates file and re-stamps each envelope
with the copy time, so bucketing by envelope time re-dates a forked session's
entire history to the fork moment — today/week/month all wrong. The `_meta`
block survives the copy untouched (only `sessionId` is rewritten), so the turn's
own timestamp does too.
Also from review:
- The subagent skip is decided when files are selected, not inside the cached
parse. The blob cache keys on `updates.jsonl`'s mtime and size, but the verdict
comes from `summary.json` next to it: parse a subagent's turns before its
summary records `session_kind` and the blob freezes that verdict for as long
as the file stops changing, so the double count the skip exists to prevent
becomes permanent. Deciding at selection re-evaluates every refresh.
- `prompt_id` is the only turn id. `_meta.eventId` was a fallback with no
guarantee of global uniqueness, and the dedup is global, so two sessions
sharing an id would have collapsed into one turn — losing real spend.
`turn_completed` always carries `prompt_id`, so the fallback bought nothing.
- JSON `null` no longer reads as "field present". `usage["inputTokens"]` being
`NSNull` passed the old `if let`, yielded 0, and then subtracted the cache from
it, dropping the turn's input tokens instead of falling back to the snake_case
spelling.
- Cache reads are clamped to the prompt total and the parts are reconciled
against the reported `totalTokens`, so `Entry.total` equals the source's own
total instead of silently diverging when the two disagree.
- Grok model names short-circuit to $0 in ModelPricing before the family
fallbacks. `grok-codex-*` or `grok-4o-*` would otherwise match the `codex`/`o4`
substrings and print a fabricated GPT-priced figure.
Tests: real-format fixtures throughout, plus the trigger cases for each fix
above — the late-written `session_kind`, the fork re-dating, null token fields,
the cache-read clamp, the totals residual, and a cache test that proves reuse by
swapping file content behind an unchanged mtime/size (the old one could not tell
a hit from a reparse). Parser-version invalidation and the missing-`grok`-key
snapshot path are covered too; the wall-clock prefilter assertion is gone,
since measurement showed it passes with or without the prefilter.
chattymin
added a commit
that referenced
this pull request
Jul 29, 2026
…134) The defect protocol in this file asks for a mechanism, not a memory, whenever a class of mistake surfaces. #133 surfaced four: The parser was written from an assumed envelope shape and the fixtures were built from the same assumption, so 27 tests passed against a format the CLI never writes — the notification sits under `params`, not `update`, and the timestamp is an integer, not ISO 8601. Fixtures cannot verify a format; only the upstream writer can. Copy paths are where usage sources double count and re-date: forks copy the parent's records and re-stamp the envelope, subagents fold into the parent, and replays re-emit. Each has a rule now — global turn ids, turn time over write time, skip the folded child. A verdict that depends on a neighbouring file must not be decided inside the parse, because the blob cache only invalidates on the parsed file's own stamp. And `obj["x"] != nil` is true for JSON `null`, which then reads as 0 and quietly subtracts the cache from nothing.
Open
10 tasks
6 of 10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the official xAI Grok CLI as the seventh usage provider, following the extension contract in
CLAUDE.md: oneUsageProviderimplementation plus one registration inUsageStore.init. No provider-specific branch is added to any shared path.Where the usage lives
~/.grok/sessions/<encoded-cwd>/<session-id>/updates.jsonl— every turn appends a durable line{"timestamp":<unix seconds>,"method":"_x.ai/session/update","params":{"sessionId":…,"update":{"sessionUpdate":"turn_completed",…,"usage":{…}},"_meta":{…}}}whose
usageis that prompt's ledger.$GROK_HOMEis honoured the same way the CLI resolves its home.Turn dates come from
_meta.agentTimestampMs, not the envelope timestamp: forking copies the parent's updates file and re-stamps every envelope with the copy time, which would re-date a forked session's whole history to the fork moment._metasurvives the copy (onlysessionIdis rewritten).Other candidates were ruled out by reading the upstream source (
xai-org/grok-build, SOURCE_REV6372e41):chat_history.jsonlusagelives on the runtime response type, not the persisted itemevents.jsonlturn_endedrecords an outcome and cancellation reason onlyUsageLedgerToken mapping
Keeps the identity every other provider keeps —
Entry.totalequals the source's own total.inputTokens − cachedReadTokensinputcachedReadTokenscacheReadcacheWrite = 0(cache writes are folded into the prompt server-side)outputTokensoutput(reasoning already included)modelUsagekeymodel(numbers always come from totals)costUsdTicks / 1e10explicitCost, only when trustworthyinputTokens(camelCase, the durable wire) includes cache reads;input_tokens(snake_case, the headless projection) already excludes them. The parser branches on the spelling — treating them as aliases would subtract the cache twice.Cost is dropped when
costIsPartialorusageIsIncompleteis set. There is no rate table for Grok models, so an untrustworthy bill reads as $0 rather than an estimate.Double counting
Two paths would have inflated totals; both are closed and both have a regression test that reproduces the trigger:
grok|<prompt_id>) and dedup globally.prompt_idis a uuid — v4 for interactive prompts, a prefixed v7 for synthetic ones (task-completed-…) — so it is globally unique but must not be parsed as a v4; it is only ever used as an opaque key.turn_completedlines would count it again. Those sessions are skipped by the samesession_kindprefix test the CLI uses to hide them. Known limitation, recorded in the code: a subagent that finishes after its parent turn ends folds into the session ledger only, so those tokens are missed — smaller than counting every subagent twice, and the CLI marks that parent billusageIsIncomplete, which also drops its cost.Replayed appends (
_meta.isReplay) are skipped as a second layer.Not included
/usageis an interactive, tier-gated TUI command, not a headless one.status.x.ai/api/v2/summary.jsonreturns HTTP 403, so there is no statuspage component to read.@vibe-kit/grok-cliand friends) are out of scope. The maintenance forks of the official CLI share this layout and are covered as-is.Tests
GrokUsageTests(27 cases), all on real-format fixtures:Entry.total == usage.totalTokensidentity, the snake_case (headless) semantics branch, the cache-read clamp whencachedReadTokensexceeds the prompt, the residual reconciliation against the reported total, JSONnullfields not zeroing a turn, zero-usage turns producing nothing, the model fallback._meta.agentTimestampMsbeating a fork's re-stamped envelope time, and the integer-seconds envelope fallback when the meta timestamp is absent.session_kindtaking effect on the next refresh instead of freezing in the blob cache.grokkeys still loading, andupdates.jsonl-only scoping._meta.totalTokens, a context-window number that must never leak into spend).Updated: the provider tab overflow guard (now 7 tabs, trigger still reproduced) and the registry assertion.
swift build(debug and release) andswift testpass — 415 tests, 0 failures, 5 skipped (thePTB_PARITY=1smoke tests).Verification status
Grok CLI is not installed on this machine, so the format was established from the upstream writer and reader rather than from a capture:
SessionUpdateEnvelope(session/storage/mod.rs) for the envelope and integer timestamp, theturn_completedserde contract tests for the tag and field spellings,record_subagent_usagefor the parent fold, andcopy_session_data_syncfor the fork copy and its re-stamped envelope. An independent verification pass over those sources refuted three of my initial assumptions — the envelope wrapper key, the timestamp type, and thatprompt_idis always a v4 UUID — and each is fixed or accounted for in the last commit.Still worth doing before this is advertised: run one real Grok session and diff the popover numbers against it.
Follow-ups
gh-pages) already carries the corresponding chip and i18n updates as a local commit; it is intentionally unpushed so it publishes with the release that ships this.