Repository navigation
fix(agent): record each Claude turn's own cost, not the session's running total - #2360
Conversation
…ning total The Agent SDK's total_cost_usd is a running total for the query, so every turn after the first recorded the spend of all turns before it, and sums over turns grew quadratically. Record the difference from the previous result. A resumed thread's first total includes turns already recorded, so that turn is priced from its tokens instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zoolsher
left a comment
There was a problem hiding this comment.
Design & Alternatives review: ✅ APPROVE
Problem: Rome keeps one Agent SDK query() open across a Claude session's turns. The SDK's total_cost_usd is the running total for that whole query, so every turn's rome_agent_messages.cost_usd row repeated the spend of the turns before it, and session totals came out far too high (the PR's test run: $7.24 recorded against about $0.07 of tokens).
Goals: each recorded row is the cost of that turn alone, so summing a session's rows gives its true cost. This should hold across new, resumed and branched threads, /clear, and results that carry no total. Non-goals: rewriting old rows, and the Codex provider (codex/app-server-provider.ts:1129 already prices each turn with calculateTurnCostUsd).
PR's approach: a small stateful RunningCost object for each openSession. On a new thread it starts at 0 and returns the difference between successive totals. On resume or branch it starts as unknown, so the first result returns undefined. buildAgentAccounting (provider-accounting.ts:313, reportedCostUsd ?? impliedCostUsd) then prices that result from its tokens. When the total drops, the count starts over from the new total. I checked the provider: every SDK result goes through the single runningCost.next(...) call at provider.ts:935, including results the SDK starts itself (sdkOwned). reopening-session.ts opens a new session with kind: "resume" each time, so it falls back to token pricing correctly.
Alternatives considered:
- Do nothing / fix it at the source. Rome can't change the SDK's meaning of
total_cost_usd. Opening a newquery()for every turn would undo the streaming-session design. - Always price from tokens using the existing
calculateImpliedCostUsd(provider-accounting.ts), ignoringtotal_cost_usd. This matches the Codex path and needs no state. Butresult.usageonly covers the main loop, so it misses spend the SDK does count: subagents and side calls on other models, server-side tool fees, and pricing tiers missing from Rome's rate table. It also returns nothing for models the table doesn't know. - Per-model differences from
result.modelUsage[*].costUSD. More detail per model, but it is still a cumulative figure with the same resume problem, and it adds more code than the single scalar. - Data-model change: store the raw running total plus a session marker, and compute differences when reading. This changes the schema (hard to undo) and every consumer of
cost_usd, and old rows still can't be fixed without knowing where each query started.
| Approach | Added complexity | Correctness/completeness | Blast radius | Reversibility | Maintenance | Effort |
|---|---|---|---|---|---|---|
| PR: running-total difference + token fallback on resume | ~30 LOC, one class | Per-turn cost including subagent and tool spend; first resumed turn is token-only | Anthropic provider only | Easy to undo, no schema or wire change | Low | Low |
| Do nothing | None | Overcounts more as a session goes on | n/a | n/a | n/a | None |
| Always price from tokens | Less (delete the SDK cost) | Misses spend outside the main loop and pricing tiers; nothing for unknown models | Anthropic provider only | Easy to undo | Rate table must be kept up to date | Very low |
Per-model modelUsage difference |
More (a map of state) | Similar to the PR, a bit more detail | Anthropic provider only | Easy to undo | Medium | Low–medium |
| Store raw total, compute differences when reading | High | Exact if markers are right; old rows still wrong | DB schema, all cost readers | Hard to undo (schema) | High | High |
Recommendation: the PR's approach is the best fit. It fixes the overcount at the provider boundary where the SDK's meaning is known, and it reuses the existing reportedCostUsd ?? impliedCostUsd fallback instead of adding a new pricing path. It keeps the SDK's more complete cost for every turn except the first of a resumed thread, and it makes no schema or wire-format commitment. The one caveat is the rule that a drop means a restart, which is a heuristic (see the low-severity note).
Verdict: APPROVE — Taking the difference between successive SDK running totals, with the existing token-pricing fallback for the first result of a resumed thread, is the best fit. It is small and easy to reverse, and it keeps charges that token pricing alone would miss.
1 finding(s) posted as inline comments below.
| Severity | Category | File | Title |
|---|---|---|---|
| P3 | design | packages/core/src/agent/infrastructure/model/anthropic/running-cost.ts |
The reset heuristic undercounts when the first turn after a reset ends above the previous total |
Automated review by RomeOS Code Review · commit 20c8458 · Full review history
There was a problem hiding this comment.
🔁 This review has been superseded. See the latest review.
There was a problem hiding this comment.
🔁 This review has been superseded. See the latest review.
A result with no total was priced from its tokens, but the next difference still started from the total before it, so that turn was counted twice. A zeroed placeholder result also set a resumed thread's starting total to zero, so the next turn recorded the whole saved total. Both now clear the starting total, and the next total only sets it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔁 This review has been superseded. See the latest review.
There was a problem hiding this comment.
🔁 This review has been superseded. See the latest review.
An SDK-owned result ends no Rome turn, so its accounting is never recorded. Moving the cost tracker past it lost that spend. The tracker now skips such results, so the next Rome turn's difference includes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zoolsher
left a comment
There was a problem hiding this comment.
Code Review: ✅ APPROVE
Goal: The Agent SDK's total_cost_usd is a running total for the whole query() call. This PR turns it into the cost of each Claude turn, so summing a session's rome_agent_messages.cost_usd rows gives the session's real cost.
Changes since the last review (a356eab):
- An SDK-owned result no longer moves the tracker: line 938 has
sdkOwned ? undefined : runningCost.next(...). Background spend now lands in the next Rome turn's difference. The new provider test runs totals $0.50 (Rome), $0.75 (background), $1.00 (Rome) and checks that each Rome turn records $0.50. This fixes the P2 that both earlier reviews raised. - A zeroed placeholder can't move the starting point either, because placeholders come in as SDK-owned
task-notificationresults. Thetotal === 0 && last !== 0check still covers zeroed results that Rome owns.
What I checked beyond the diff:
buildAgentAccountingusesreportedCostUsd ?? impliedCostUsd, so a real difference of 0 is kept and not replaced by the token price.undefinedfalls back to the token price, as intended.readPositiveNumberaccepts 0, so the placeholder branch can actually be reached.total_cost_usdhas no other readers inpackages/, so there's no second path that still sums running totals.RunningCostis created once for eachopenSession.params.thread.kind === "new"is the only case where the SDK starts its total at zero.resumeandbranch(fork) both start with no known starting point.- For SDK-owned results,
resultData.costUsd(logging only) now shows the token price instead of the running total. That's harmless. - I traced every case in
running-cost.test.tsby hand and they match the code. I couldn't run the test suites because the clone has no dependencies installed.
Design: A small stateful object for each session, falling back to token pricing when the starting point is unknown, is the simplest fix that works. The alternative, saving SDK-owned results as their own rows, would mean adding a new event or persistence path just for background turns. The author chose to fold that spend into the next Rome turn, which keeps the change small. The trade-off is the trailing-spend case in the single P3 finding.
The earlier /clear notes are now explained in the code comment, so I'm not raising them again.
Verdict: APPROVE — I found no P0–P2 problems. The fix records each Claude turn's own cost, and both P2 findings from earlier reviews are fixed. The one note left is a low-priority (P3) edge case.
1 finding(s) posted as inline comments below.
| Severity | Category | File | Title |
|---|---|---|---|
| P3 | correctness | packages/core/src/agent/infrastructure/model/anthropic/provider.ts |
Background spend is still lost when no later Rome turn can take a difference |
Automated review by RomeOS Code Review · commit a356eab · Full review history
| // An SDK-owned result ends no Rome turn, so nothing records its | ||
| // accounting. Leaving the tracker where it was puts its spend in | ||
| // the next Rome turn's difference. | ||
| sdkOwned ? undefined : runningCost.next(readPositiveNumber(message.total_cost_usd)), |
There was a problem hiding this comment.
[P3] correctness — Background spend is still lost when no later Rome turn can take a difference
An SDK-owned turn's spend only gets recorded when a later Rome result takes a difference from a known starting total. In three cases it is lost:
- The session closes after a background turn and before the next Rome turn.
- On a resumed or forked thread, the background turn comes before the first Rome result. That result is priced from its own
usagetokens only. - The next Rome result has no total.
This isn't a regression, since trailing background spend was never recorded before either, and the amounts are small. If you want it to be exact, you could add the token price of skipped SDK-owned results to the next Rome turn whenever that turn falls back to token pricing. Otherwise, mention the gap in the comment.
zhangfand
left a comment
There was a problem hiding this comment.
Code Review: ✅ APPROVE
Reviewed all four changed files and traced session lifecycle, pricing fallback, and accounting consumers. The implementation is appropriately contained within the Anthropic adapter.
All six tracker scenarios passed direct Node assertions. The provider tests could not run because dependencies, including rstest, are not installed.
Verdict: APPROVE — The query-scoped cost tracker correctly implements the intended per-turn accounting without actionable regressions.
No issues found. Code looks good!
Requested by zhangfan · project thread
Why
Rome records a cost per Claude turn in
rome_agent_messages.cost_usdand sums those rows for a session's usage. The Agent SDK'stotal_cost_usdis not a turn's cost: it is the running total of the wholequery()call, and Rome keeps one query open across a session's turns. So each turn recorded the spend of every turn before it too, and the sums overcounted. The chief of staff's test run showed it: the recorded turns summed to $7.24, while pricing their tokens gave about $0.07.Invariants
/clear, the count starts over from that total. This is best effort: a/clearfollowed by a turn that ends above the old total undercounts that turn. Rome never sends/clearitself.Scope
Merge danger
Low. Usage and cost figures for Claude sessions go down to their true values from here on. Older rows still overcount.
Evidence
running-cost.test.tscovers a new thread, a resumed thread, a dropping total, a missing total and a zeroed placeholder.pnpm preflightpasses.🤖 Generated with Claude Code