💵 feat: Show Cost per Record, Step and Response in the Trace Ledger - #16120
Conversation
45fbb1a to
55609bc
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55609bc8a6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| summary.cost = total; | ||
| } | ||
| for (const turn of turns) { | ||
| turn.cost = costOf(spendByTurn.get(turn.messageId) ?? noSpend()); |
There was a problem hiding this comment.
Withhold response cost until a split turn is complete
When the record limit cuts a response across pages, this assigns a total using only the currently loaded records. The model already marks the common missing-parent case as turn.split, but if every loaded generation is priced, costOf returns a subtotal and the ledger presents it as the response's cost, under-reporting spend until the older page is loaded. Leave turn.cost unset for split or otherwise known-partial turns.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Valid: it contradicts the rule the column is built on, that a total never under-reports. Fixed in cfc6915: a response the record limit cut (turn.split) has no cost until its older records load; a step of it that is fully loaded keeps its own. Coverage: model.test.ts 'gives no cost to a response the record limit cut…' (fails with the condition removed).
| continue; | ||
| } | ||
| node.stepKey = key; | ||
| spendOn(spend, node.record); |
There was a problem hiding this comment.
Include hidden records when totaling a step
In simple mode this call runs only for nodes reached through viewChildIds, while resolveViewTree deliberately omits ordinary spans and events from that projection. Because the trace contract and Langfuse reader permit a cost on any record, a priced hidden child is included in the response and trace totals but omitted from its step, making the displayed hierarchy inconsistent. Aggregate step spend from the underlying record tree rather than only the visible projection.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Valid. Fixed in cfc6915: a step's spend is summed over the whole subtree beneath its roots (childIds), not the listed projection, so a priced span the simple mode rolls out of sight counts toward its step exactly as it does toward its response and the trace. Coverage: model.test.ts 'counts what a step spent in records the simple mode rolls out of sight', in both modes (fails when the walk follows viewChildIds).
| <span className="hidden truncate text-right text-xs font-normal tabular-nums text-text-secondary md:block"> | ||
| {cost != null ? formatCost(cost, currency) : ''} | ||
| </span> |
There was a problem hiding this comment.
Expose ledger costs in tree-item accessible names
When context cost is enabled, these values are visible but unavailable during screen-reader tree navigation: every row has an explicit aria-label that omits cost, so descendant text does not contribute to the tree item's accessible name, and the column header is also aria-hidden. Include the localized cost label and value in each applicable row's accessible name or expose equivalent semantic table/grid information.
AGENTS.md reference: AGENTS.md:L31-L34
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Valid. Fixed in cfc6915: every row that shows a cost says it in its accessible name (record, step and response rows), and says nothing when contextCost is off or the row has no cost. Coverage: Viewer.test.tsx asserts the record's and step's accessible names with the setting on, and the absence with it off (fails with the name left as it was).
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cfc69158a2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
| /** A response the record limit cut holds only its newest records, and their sum is not its cost. */ | ||
| for (const turn of turns) { | ||
| turn.cost = turn.split ? undefined : costOf(spendByTurn.get(turn.messageId) ?? noSpend()); |
There was a problem hiding this comment.
Withhold cost when pagination leaves a turn incomplete
When maxRecords lands between two root observations, or between a turn's title and run traces, the first page can contain only part of the turn while every loaded record has a null or loaded parent. In that case turn.split remains false and this line publishes the loaded subtotal as the response cost until the next page arrives. The fresh evidence beyond the earlier missing-parent finding is that the reader returns continuation cursors at arbitrary record/trace boundaries, while split detects only missing parents; pass page/turn completeness into this decision rather than relying solely on turn.split.
AGENTS.md reference: AGENTS.md:L31-L34
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Valid. A missing parent proves a cut, but a page can also end between a response's traces (its title run loaded, its own run not) with every parent in place. Fixed in 91f38fe: buildTraceModel takes whether older records remain, and while they do the oldest loaded response has no cost, split or not; every newer response is whole by construction, since pages are read newest first. Coverage: model.test.ts 'gives the oldest loaded response no cost while older records remain, parents intact or not' (fails with the page state ignored). The summary's trace-wide total is unchanged: it has always been the total of what is loaded, and the bar says so beside it ('Showing the newest N records').
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
91f38fe to
161ab82
Compare
161ab82 to
b13eacc
Compare
Pull Request
Summary
When a deployment shows cost in the context usage gauge (
interface.contextCost), the trace viewer shows it in two places only: one total in the summary bar and one line in a selected record's details. The ledger, which is where the run is actually read, has durations and tokens per row and no cost, so there is no way to see which response, step or model call the money went to without opening records one at a time.This adds a Cost column to the ledger beside Tokens, under the same
contextCostgate and the deployment'scurrency: each priced record shows its cost, and each step and each response shows its total. WithcontextCostoff nothing changes.A total follows the rule the summary already uses: it is shown only when every model call it covers has a price, because a total that silently skips an unpriced call under-reports. A response's total includes its title run and its activity-label calls, which are spend of that response; a step's total covers the step's own records.
How it works
The summary's total is computed by the same
Spendaccumulator, so the three levels cannot disagree about what counts as priced.Type of change
Testing
Tested environments/configuration:
Viewerwith the real stylesheets in Chromium against a fixture run (dark, 1440x900) withcontextCost: true: the column sits between Tokens and the timeline, with per-record, per-step and per-response values.Automated tests:
model.test.ts: per-step and per-response totals; one unpriced model call leaves its step, its response and the trace without a total while a fully priced step beside it keeps its own; a response's total includes its title run.Viewer.test.tsx: withcontextCoston, the summary, the column header, the model call, its step and its response all show the cost; with it off, no cost appears anywhere.src/components/Chat/Traceand theChatViewspecs;npx tsc --noEmitinclient(trace files);npm run static-checksagainst the base.Screenshots / recordings
To be attached.
Risk / compatibility
Checklist