Skip to content

feat(usage): parse real Claude JSON receipts - #299

Merged
mergify[bot] merged 3 commits into
mainfrom
codex/hq2-05b-p2-claude-probe
Oct 4, 2026
Merged

mergify[bot] merged 3 commits into
mainfrom
codex/hq2-05b-p2-claude-probe

Conversation

@Cuarroc

@Cuarroc Cuarroc commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Claude Code liefert bei einem kleinen echten Lauf getrennte Token-Zähler und nennt das verwendete Modell. Der neue Parser übernimmt diese Zähler nur aus einem vollständigen erfolgreichen JSON-Ergebnis und erfindet bei Fehlern keine Null. Die Laufzeit-Anbindung an Route und Store bleibt bewusst einem späteren seriellen Paket vorbehalten.

Report

What changed

  • Added a scrubbed fixture from one real Claude Code --output-format json run. Session/result IDs and monetary values are redacted; no personal paths or account data are present.
  • Added a pure Claude JSON parser that returns the existing UsageReceipt type without route, capture, or store settlement wiring.
  • Measured receipts sum the four independently reported counters: input, cache creation input, cache read input, and output. The observed fixture maps 2 + 1857 + 535 + 4 to 2398 tokens.
  • Missing usage becomes explicit not_reported; missing, negative, non-integer, oversized, malformed, error, and nonzero-exit captures become rejected, never zero.
  • Generalized receipt provenance so the existing Codex collector remains codex-exec-json-v1 while this parser reports claude-print-json-v1.
  • Observed fields that cannot be mapped to the existing receipt: total_cost_usd, per-model costUSD, and modelUsage. They are not guessed. The cost fields were present, but their values are redacted and they are not treated as subscription billing.

Live probe

  • Date: 2026-10-04 (Europe/Berlin).
  • Exact command: claude -p --output-format json --tools "" --strict-mcp-config --mcp-config '{"mcpServers":{}}' --no-session-persistence --safe-mode --model sonnet 'Reply with exactly OK.'
  • Exit: 0.
  • CLI: Claude Code 2.1.287 (claude --version, exit 0).
  • Observed model: claude-sonnet-5-5 (modelUsage key and canonicalModel).
  • Observed usage fields: input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens; cost fields were also present.
  • stream-json was not probed: this parser consumes the single final result produced by --output-format json, and no Claude stream transport is wired in this package.

Evidence

  • Base fetched at start: 70308347998146bfa87478c03715041d575d409c (git fetch origin main; branch HEAD and origin/main matched, exit 0).
  • Start check before the probe: npm run dev:start-check — exit 0; 14.2 GB free RAM, 0 cargo builds. Provider-limit file was not supplied, as reported by the command.
  • Red proof: cargo test --manifest-path src-tauri/Cargo.toml store::development_budget::claude_usage::tests:: -- --nocapture at ee4a010 — exit 101; four compiling tests failed against the explicit stub.
  • Focused parser/receipt tests: three cargo test --manifest-path src-tauri/Cargo.toml --bin projecta ... invocations for claude_usage, usage_receipt, and codex_usage — exit 0; 4 + 8 + 5 tests passed.
  • Full gate: bash scripts/ci/gates.sh lane prepush at c914b586d007e0f03cefa2ec6d0bdfa0bb642826 — exit 0; log /tmp/prepush-hq2-05b-p2-claude-probe-final.log. Frontend 460/460, HQ 510/510, Rust 1709/1709; Clippy and red-first plan green.
  • Push: git push origin codex/hq2-05b-p2-claude-probe — exit 0; its pre-push gate also exited 0.
  • Remote verification: git ls-remote origin refs/heads/codex/hq2-05b-p2-claude-probe — exit 0, remote SHA c914b586d007e0f03cefa2ec6d0bdfa0bb642826.
  • Final SHA: c914b586d007e0f03cefa2ec6d0bdfa0bb642826.
  • Diff from package base: 225 insertions, 6 deletions (231 changed lines; limit 300).
  • Secret scan: bash scripts/ci/secret-scan.sh before each commit — exit 0.

NICHT ABGEDECKT

  • Linux does not compile the #[cfg(windows)] tests or Windows Clippy arms; the real Windows lane first runs in the Mergify merge queue.
  • No route, native capture, or store settlement wiring is included; that is a later serial package.
  • No GUI or desktop app was launched or inspected.
  • stream-json event parsing is not covered because it is not the transport under test.
  • The CLI's cost fields and model metadata have no fields in UsageReceipt; they remain intentionally unmapped. No subscription invoice or billed amount was verified.

Review

Review: done, Tier A review requested by coordinator (candidate c914b586d007e0f03cefa2ec6d0bdfa0bb642826).

Review disposition

  • Reviewer 1: Claude (Sonnet 5.5), read git diff origin/main...HEAD in full. Findings: none survived. Checked and cleared: every non-success path (exit code, empty, >1 MiB, malformed/trailing JSON, is_error, missing type/subtype) returns Rejected, never zero (development_claude_usage.rs:15-39); counters are separate Claude categories so summing cache tokens is correct (unlike Codex, where cached is a subset, development_codex_usage.rs:108); checked_add plus 1e9 cap (:56-66).
  • Reviewer 2: GLM 5.2 (glm-5.2:cloud, via .pa/review_transport.py, exit 0). Findings: none; verdict approve.
  • Architecture drift: none. The parser reuses the UsageReceipt type and mirrors the existing codex_usage module layout; no second pattern for store/lock access, IPC or events.
  • Code changes from review: none.

Open points

  • The later serial package must bind a trusted process-owned Claude JSON capture to this parser and ledger settlement.
  • If product requirements need cost or model persistence, the receipt schema needs a separately reviewed extension rather than overloading token totals.

Cuarroc and others added 3 commits October 4, 2026 16:16
Test-First: src-tauri/src/store/development_claude_usage.rs::real_claude_2_1_287_json_maps_observed_token_counters

Test-First: src-tauri/src/store/development_claude_usage.rs::claude_json_without_usage_is_not_reported_instead_of_zero

Test-First: src-tauri/src/store/development_claude_usage.rs::negative_or_missing_claude_counters_are_rejected

Test-First: src-tauri/src/store/development_claude_usage.rs::claude_error_run_is_rejected_instead_of_zero

Co-Authored-By: Codex GPT-5 <noreply@openai.com>
Test-First: src-tauri/src/store/development_claude_usage.rs::real_claude_2_1_287_json_maps_observed_token_counters

Test-First: src-tauri/src/store/development_claude_usage.rs::claude_json_without_usage_is_not_reported_instead_of_zero

Test-First: src-tauri/src/store/development_claude_usage.rs::negative_or_missing_claude_counters_are_rejected

Test-First: src-tauri/src/store/development_claude_usage.rs::claude_error_run_is_rejected_instead_of_zero

Co-Authored-By: Codex GPT-5 <noreply@openai.com>
No-Test: append-only session handoff

Co-Authored-By: Codex GPT-5 <noreply@openai.com>
@Cuarroc
Cuarroc marked this pull request as ready for review October 4, 2026 14:41
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-04T14:46:38.407132Z c914b58 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c914b586d0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +34 to +35
let Some(usage) = result.get("usage").filter(|value| value.is_object()) else {
return UsageReceipt::NotReported {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject malformed usage payloads

When a successful Claude response contains usage: null, an array, or a string because of schema drift or capture corruption, this filter follows the same branch as an absent field and returns NotReported. That state omits the capture digest and claims the result simply lacked usage, making malformed provider data indistinguishable from a legitimate missing report; distinguish an absent key from a present non-object value and reject the latter, with regression cases for these sibling forms.

AGENTS.md reference: AGENTS.md:L87-L89

Useful? React with 👍 / 👎.

Comment thread .pa/ACTIVITY.md
- Uncommitted: keine

## 2026-10-04 16:36 — codex (codex/hq2-05b-p2-claude-probe, srv-hq2-05b-p2-claude-probe)
- Zusammenfassung: HQ2-05b package 2: real Claude 2.1.287 JSON probe, scrubbed fixture and pure fail-closed UsageReceipt parser; red-first commits ee4a010/5a3a94c; prepush and push hook green; draft PR #299; Tier B review and later route/store wiring pending.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Classify the store change as Tier A

This package changes files under src-tauri/src/store/, which the repository rules explicitly include in a serial seam, but this journal entry schedules only a Tier B review. That classification obtains one reviewer instead of the two cross-vendor reviewers required for seam changes, so reclassify the package as Tier A and record both review dispositions before accepting it.

AGENTS.md reference: AGENTS.md:L26-L32

Useful? React with 👍 / 👎.

Comment on lines +44 to +48
for field in [
"input_tokens",
"cache_creation_input_tokens",
"cache_read_input_tokens",
"output_tokens",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Account for non-executor usage iterations

When a Claude run invokes the Advisor tool, the top-level usage counters cover executor iterations only, while the advisor's tokens are reported separately in usage.iterations[] entries of type advisor_message, as documented in the Anthropic Advisor usage contract. This loop always sums only the four top-level counters and marks the receipt measured, so such a run silently settles fewer tokens than it consumed and can incorrectly leave budget available; either account for the complete iteration breakdown without double-counting or reject unsupported iteration types until they can be measured.

Useful? React with 👍 / 👎.

@mergify

mergify Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@Cuarroc

Cuarroc commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify

mergify Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 1 hour 1 minute 47 seconds in the queue, including 11 minutes 20 seconds running CI.

Required conditions to merge

@mergify
mergify Bot deleted the codex/hq2-05b-p2-claude-probe branch October 4, 2026 15:54
@mergify mergify Bot removed the queued label Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant