Skip to content

💵 feat: Show Cost per Record, Step and Response in the Trace Ledger - #16120

Merged
danny-avila merged 3 commits into
devfrom
danny-avila/trace-cost-column
Sep 20, 2026
Merged

danny-avila merged 3 commits into
devfrom
danny-avila/trace-cost-column

Conversation

@danny-avila

Copy link
Copy Markdown
Collaborator

Pull Request

Stacked on #16113. The base is that pull request's branch; retarget to dev once it merges.

Summary

When a deployment shows cost in the context usage gauge (interface.contextCost), the trace viewer shows it in two places only: one total in the summary bar and one line in a selected record's details. The ledger, which is where the run is actually read, has durations and tokens per row and no cost, so there is no way to see which response, step or model call the money went to without opening records one at a time.

This adds a Cost column to the ledger beside Tokens, under the same contextCost gate and the deployment's currency: each priced record shows its cost, and each step and each response shows its total. With contextCost off nothing changes.

A total follows the rule the summary already uses: it is shown only when every model call it covers has a price, because a total that silently skips an unpriced call under-reports. A response's total includes its title run and its activity-label calls, which are spend of that response; a step's total covers the step's own records.

How it works

client/src/components/Chat/Trace/
├── model.ts    # Spend { cost, priced, unpriced } + costOf(): one rule for the trace, a response and a step;
│               # TraceTurn.cost, TraceStep.cost
├── Ledger.tsx  # GRID_WITH_COST when showCost; a cost cell on record, step and response rows; column header
└── Viewer.tsx  # passes the showCost / currency it already resolves for the summary and the inspector

The summary's total is computed by the same Spend accumulator, so the three levels cannot disagree about what counts as priced.

Type of change

  • Feature

Testing

Tested environments/configuration:

  • Rendered the real Viewer with the real stylesheets in Chromium against a fixture run (dark, 1440x900) with contextCost: true: the column sits between Tokens and the timeline, with per-record, per-step and per-response values.
  • Not run against a live backend.

Automated tests:

  • model.test.ts: per-step and per-response totals; one unpriced model call leaves its step, its response and the trace without a total while a fully priced step beside it keeps its own; a response's total includes its title run.
  • Viewer.test.tsx: with contextCost on, the summary, the column header, the model call, its step and its response all show the cost; with it off, no cost appears anywhere.
  • 192 tests across src/components/Chat/Trace and the ChatView specs; npx tsc --noEmit in client (trace files); npm run static-checks against the base.

Screenshots / recordings

To be attached.

Risk / compatibility

  • No new configuration and no server change: the gate and currency are the ones the gauge, the summary and the inspector already use.
  • The column is a wide-screen column, like Tokens; narrow layouts are unchanged.

Checklist

  • I reviewed my own changes
  • Relevant tests have been added or updated
  • Existing relevant tests pass
  • The change does not introduce new warnings or errors
  • User-facing or complex behavior is documented where necessary
  • Required dependency changes have been merged/published
  • Required documentation PR: N/A

@danny-avila
danny-avila force-pushed the danny-avila/trace-cost-column branch from 45fbb1a to 55609bc Compare September 20, 2026 10:35
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review commit 55609bc.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-20T11:15:59.920538Z 91f38fe Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55609bc8a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

summary.cost = total;
}
for (const turn of turns) {
turn.cost = costOf(spendByTurn.get(turn.messageId) ?? noSpend());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Withhold response cost until a split turn is complete

When the record limit cuts a response across pages, this assigns a total using only the currently loaded records. The model already marks the common missing-parent case as turn.split, but if every loaded generation is priced, costOf returns a subtotal and the ledger presents it as the response's cost, under-reporting spend until the older page is loaded. Leave turn.cost unset for split or otherwise known-partial turns.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid: it contradicts the rule the column is built on, that a total never under-reports. Fixed in cfc6915: a response the record limit cut (turn.split) has no cost until its older records load; a step of it that is fully loaded keeps its own. Coverage: model.test.ts 'gives no cost to a response the record limit cut…' (fails with the condition removed).

continue;
}
node.stepKey = key;
spendOn(spend, node.record);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include hidden records when totaling a step

In simple mode this call runs only for nodes reached through viewChildIds, while resolveViewTree deliberately omits ordinary spans and events from that projection. Because the trace contract and Langfuse reader permit a cost on any record, a priced hidden child is included in the response and trace totals but omitted from its step, making the displayed hierarchy inconsistent. Aggregate step spend from the underlying record tree rather than only the visible projection.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Fixed in cfc6915: a step's spend is summed over the whole subtree beneath its roots (childIds), not the listed projection, so a priced span the simple mode rolls out of sight counts toward its step exactly as it does toward its response and the trace. Coverage: model.test.ts 'counts what a step spent in records the simple mode rolls out of sight', in both modes (fails when the walk follows viewChildIds).

Comment on lines +172 to +174
<span className="hidden truncate text-right text-xs font-normal tabular-nums text-text-secondary md:block">
{cost != null ? formatCost(cost, currency) : ''}
</span>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Expose ledger costs in tree-item accessible names

When context cost is enabled, these values are visible but unavailable during screen-reader tree navigation: every row has an explicit aria-label that omits cost, so descendant text does not contribute to the tree item's accessible name, and the column header is also aria-hidden. Include the localized cost label and value in each applicable row's accessible name or expose equivalent semantic table/grid information.

AGENTS.md reference: AGENTS.md:L31-L34

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Fixed in cfc6915: every row that shows a cost says it in its accessible name (record, step and response rows), and says nothing when contextCost is off or the row has no cost. Coverage: Viewer.test.tsx asserts the record's and step's accessible names with the setting on, and the absence with it off (fails with the name left as it was).

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review commit cfc6915.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cfc69158a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}
/** A response the record limit cut holds only its newest records, and their sum is not its cost. */
for (const turn of turns) {
turn.cost = turn.split ? undefined : costOf(spendByTurn.get(turn.messageId) ?? noSpend());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Withhold cost when pagination leaves a turn incomplete

When maxRecords lands between two root observations, or between a turn's title and run traces, the first page can contain only part of the turn while every loaded record has a null or loaded parent. In that case turn.split remains false and this line publishes the loaded subtotal as the response cost until the next page arrives. The fresh evidence beyond the earlier missing-parent finding is that the reader returns continuation cursors at arbitrary record/trace boundaries, while split detects only missing parents; pass page/turn completeness into this decision rather than relying solely on turn.split.

AGENTS.md reference: AGENTS.md:L31-L34

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. A missing parent proves a cut, but a page can also end between a response's traces (its title run loaded, its own run not) with every parent in place. Fixed in 91f38fe: buildTraceModel takes whether older records remain, and while they do the oldest loaded response has no cost, split or not; every newer response is whole by construction, since pages are read newest first. Coverage: model.test.ts 'gives the oldest loaded response no cost while older records remain, parents intact or not' (fails with the page state ignored). The summary's trace-wide total is unchanged: it has always been the total of what is loaded, and the bar says so beside it ('Showing the newest N records').

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review commit 91f38fe.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 91f38fe871

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila added this pull request to stack #16125 September 20, 2026 11:43
Base automatically changed from danny-avila/trace-partial-turn-tools to dev September 20, 2026 12:24
@danny-avila
danny-avila force-pushed the danny-avila/trace-cost-column branch from 91f38fe to 161ab82 Compare September 20, 2026 12:24
@danny-avila
danny-avila force-pushed the danny-avila/trace-cost-column branch from 161ab82 to b13eacc Compare September 20, 2026 12:28
@danny-avila
danny-avila marked this pull request as ready for review September 20, 2026 12:28
@danny-avila
danny-avila merged commit 2551a05 into dev Sep 20, 2026
23 checks passed
@danny-avila
danny-avila deleted the danny-avila/trace-cost-column branch September 20, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant