Skip to content

fix(usage): improve prompt savings estimates - #618

Merged
SantiagoDePolonia merged 4 commits into
mainfrom
fix/counting-tokens
Jul 30, 2026
Merged

fix(usage): improve prompt savings estimates#618
SantiagoDePolonia merged 4 commits into
mainfrom
fix/counting-tokens

Conversation

@SantiagoDePolonia

@SantiagoDePolonia SantiagoDePolonia commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • calculate prompt-savings percentages against input tokens rather than input plus output
  • clarify that saved tokens are cumulative token transmissions estimated at four characters per token
  • estimate saved cost from each request’s observed blended input/cache rate, with static pricing as a fallback
  • label the value as estimated gross input cost avoided and disclose that rewrite-induced cache changes are excluded

Testing

  • pre-commit hooks, including race tests, dashboard unit tests, lint, and performance checks
  • go test ./...
  • 386 dashboard tests
  • git diff --check

Summary by CodeRabbit

  • Bug Fixes

    • Improved “Pro Saved” cost calculations by using an observed blended input-rate when it can be derived safely, including better handling of prompt-cache reads and fixed/non-token charges.
    • When an observed rate can’t be applied (or pricing tiers would be crossed), it now falls back to pricing-based estimates.
  • Usage Dashboard

    • Updated “Pro Saved” tokens and cost card text with standardized estimator disclosure, clarified caveats (prompt-cache changes from rewriting excluded), and refined saved-token percentage baselines.
  • Documentation

    • Clarified the meaning of RewriteCostSaved, including how the rate is derived and when it’s unavailable.

Copilot AI review requested due to automatic review settings July 30, 2026 14:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 1bd3b24c-8874-44ff-b02e-d655fc5417b0

📥 Commits

Reviewing files that changed from the base of the PR and between 512da51 and a96360f.

📒 Files selected for processing (2)
  • internal/usage/savings.go
  • internal/usage/savings_test.go

📝 Walkthrough

Walkthrough

Rewrite savings now derive avoided input costs from observed blended rates, exclude separable fixed charges, and fall back to pricing when needed. Recalculation, tests, usage documentation, dashboard calculations and messaging, and settings help are updated.

Changes

Rewrite savings accounting and dashboard

Layer / File(s) Summary
Observed blended cost calculation
internal/usage/savings.go, internal/usage/recalculate_pricing.go, internal/usage/savings_test.go, internal/usage/usage.go
Rewrite savings use cache-aware observed input costs, exclude auxiliary charges, handle pricing tiers, and validate fallback and recalculation behavior.
Saved metrics dashboard presentation
web/dashboard/src/pages/usage/usage-helpers.js, web/dashboard/tests/usage.test.js
Token percentages use total input tokens, while Pro Saved titles disclose estimation details and prompt-cache caveats.
Pricing recalculation guidance
web/dashboard/src/pages/settings/PricingRecalculation.svelte
Settings help describes recalculated and overwritten historical cost fields.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant UsageEntry
  participant ApplyRewriteSavings
  participant EntryInputSegments
  participant rewriteCostSaved
  UsageEntry->>ApplyRewriteSavings: provide input cost and rewrite tokens saved
  ApplyRewriteSavings->>EntryInputSegments: derive cache-aware input segments
  EntryInputSegments-->>rewriteCostSaved: return input parts
  ApplyRewriteSavings->>rewriteCostSaved: calculate avoided input cost
  rewriteCostSaved-->>UsageEntry: store RewriteCostSaved
Loading

Possibly related PRs

Suggested reviewers: copilot

Poem

I’m a rabbit counting tokens with care,
Blended cache costs float through the air.
Saved cards now speak more clearly too,
With caveats hopping through.
Rewrites leave fewer costs in their wake!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the main change: improving usage prompt savings estimates.
Description check ✅ Passed The description covers the key changes and testing, though it uses Summary instead of the template's Description heading.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/counting-tokens

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov-commenter

codecov-commenter commented Jul 30, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 86.66667% with 10 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/usage/savings.go 83.05% 5 Missing and 5 partials ⚠️

📢 Thoughts on this report? Let us know!

Comment thread internal/usage/savings.go Outdated
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

Safe to merge based on the exercised dashboard persistence and usage-pricing flows.

The reviewed changes have no remaining defects: direct runtime reproductions and focused regression tests exercised the changed date persistence, rollover, cache-rate, fixed-charge, tier-pricing, and historical recalculation behavior successfully.

Files Needing Attention: No files require follow-up attention.

T-Rex T-Rex Logs

What T-Rex did

  • The Vite harness was run against the dashboard store before and after the change using a saved custom reporting range, and the test sequence showed the date-range rollovers and a safe fallback to the default range, supporting proof 0.
  • A Go reproduction compared the pre-change baseline and the changed code, showing that the baseline overstated fixed-charge savings while the updated code recalculated to $0.0084, and the focused internal usage regression suite for savings and SQLite repricing passed, supporting proof 1.
  • Evidence showed how a stored 2026-07-25..2026-07-29 range moved through subsequent rollovers to 2026-07-29..2026-08-02 and then 2026-07-30..2026-08-03, with each rollover advancing syncTick and a safe fallback for an impossible date, supporting proof 2.
  • The final baseline comparison demonstrated that HEAD passes all authored scenarios and re-prices the stored stale saving to the expected blended value ($0.0084), with no further commands executable due to maximum steps reached, supporting proof 3.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (2): Last reviewed commit: "test(usage): consolidate fixed charge ca..." | Re-trigger Greptile

Copilot AI review requested due to automatic review settings July 30, 2026 15:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
internal/usage/savings_test.go (1)

95-199: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Make the new fixed-charge scenarios table-driven.

These cases vary inputs and expectations but repeat the same setup, invocation, and assertion flow. Put them in a case table so provider-specific pricing behavior remains easy to extend consistently.

As per coding guidelines, “Add or update table-driven tests for behavior changes.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/usage/savings_test.go` around lines 95 - 199, Convert the three
fixed-charge tests—image, audio, and per-character cases—into a table-driven
subtest using shared setup, ApplyRewriteSavings invocation, and savings
assertions. Keep each case’s provider, pricing, raw-data key, input cost, and
expected savings distinct, while leaving the unseparated fixed-charge and
tier-crossing tests unchanged.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/usage/savings.go`:
- Around line 89-93: The per-character input branch in the savings calculation
validates pricing but does not apply its charge. Update the logic around
extractInt and pricing.PerCharacterInput to add the character-based input cost
to fixedCost, while preserving the existing missing-pricing failure behavior and
observed-rate calculation.

---

Outside diff comments:
In `@internal/usage/savings_test.go`:
- Around line 95-199: Convert the three fixed-charge tests—image, audio, and
per-character cases—into a table-driven subtest using shared setup,
ApplyRewriteSavings invocation, and savings assertions. Keep each case’s
provider, pricing, raw-data key, input cost, and expected savings distinct,
while leaving the unseparated fixed-charge and tier-crossing tests unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 029e911d-f36c-4246-9921-2281916c0a57

📥 Commits

Reviewing files that changed from the base of the PR and between 2c01a9e and 512da51.

⛔ Files ignored due to path filters (2)
  • internal/admin/dashboard/static/dist/assets/index-AqKCz9Vp.js is excluded by !**/dist/**
  • internal/admin/dashboard/static/dist/index.html is excluded by !**/dist/**
📒 Files selected for processing (6)
  • internal/usage/savings.go
  • internal/usage/savings_test.go
  • internal/usage/usage.go
  • web/dashboard/src/pages/settings/PricingRecalculation.svelte
  • web/dashboard/src/pages/usage/usage-helpers.js
  • web/dashboard/tests/usage.test.js

Comment thread internal/usage/savings.go
Copilot AI review requested due to automatic review settings July 30, 2026 15:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@SantiagoDePolonia
SantiagoDePolonia merged commit 56fb531 into main Jul 30, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants