fix: route volcengine (Doubao) tiered-pricing models to the tiered cost handler

[vineethsaivs]

Relevant issues

Fixes #30346

Type

🐛 Bug Fix

Changes

Volcengine doubao models (for example volcengine/doubao-seed-2-0-pro-260215) define tiered_pricing in the model map but carry no flat input_cost_per_token/output_cost_per_token. The cost_per_token dispatcher had no volcengine branch, so these models fell through to generic_cost_per_token, which only reads flat per-token costs. The result was that every volcengine doubao request was tracked as $0

The dashscope cost calculator already implements graduated tiered pricing and reads tiered_pricing generically from the model map; the only provider-specific part was the hardcoded provider passed to get_model_info. This change makes that provider an argument (defaulting to dashscope, so existing behavior is unchanged) and routes volcengine to the same handler, mirroring how dashscope is dispatched

Added a regression test that asserts the graduated cost across two tiers for a doubao model; it fails on the current code (cost is $0) and passes with the fix

Screenshots / Proof of Fix

This path is network-free: cost is computed from the bundled model_prices_and_context_window_backup.json, so it does not require a live volcengine key. Reproduction on the local model map:

Before (current behavior):

volcengine/doubao-seed-2-0-pro-260215, prompt_tokens=10000, completion_tokens=2000
-> prompt_cost=$0.000000, completion_cost=$0.000000  (tracked as $0)

After this change:

volcengine/doubao-seed-2-0-pro-260215, prompt_tokens=10000, completion_tokens=2000
-> prompt_cost > 0, completion_cost > 0, matching the graduated tiered_pricing in the model map

The regression test test_volcengine_tiered_pricing_graduated_cost in tests/test_litellm/test_cost_calculator.py encodes the expected graduated cost across the first two tiers and verifies it end to end through cost_per_token(..., custom_llm_provider="volcengine")

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
6 out of 7 committers have signed the CLA.

✅ ryan-crabbe-berri
✅ Sameerlite
✅ yuneng-berri
✅ shivamrawat1
✅ mateo-berri
✅ vineethsaivs
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes volcengine (Doubao) models being billed at $0 by adding a volcengine branch to the cost_per_token dispatcher that routes to the existing tiered-pricing handler, and parameterising that handler with a custom_llm_provider argument so it resolves model info for any provider.

• litellm/cost_calculator.py: A new elif custom_llm_provider == "volcengine" branch imports and calls the shared tiered-pricing handler with custom_llm_provider="volcengine", exactly mirroring the existing dashscope branch.
• litellm/llms/dashscope/cost_calculator.py: The cost_per_token function gains an optional custom_llm_provider parameter (default "dashscope") passed straight through to get_model_info; all existing callers are unaffected.
• tests/test_litellm/test_cost_calculator.py: A network-free regression test (test_volcengine_tiered_pricing_graduated_cost) reads the bundled model map, constructs a cross-tier token count for doubao-seed-2-0-pro-260215, and asserts the exact graduated cost; uses monkeypatch.setattr for proper state cleanup.

Confidence Score: 5/5

Safe to merge — the change adds a new provider branch and a backward-compatible parameter; no existing behaviour is altered.

The dispatcher change is additive: it inserts a new elif that was previously missing, so no existing provider path is touched. The shared tiered-pricing function change is fully backward-compatible (new parameter defaults to dashscope). The regression test exercises the full cost-calculation path using the bundled model map, requires no live credentials, and uses monkeypatch for clean teardown. There are no data-path mutations, no auth changes, and no schema modifications.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/cost_calculator.py	Adds a volcengine branch to the cost_per_token dispatcher that routes to the shared tiered-pricing handler, mirroring the existing dashscope branch; the lazy import avoids circular-import risk and the branch placement is consistent with all other provider branches in the file.
litellm/llms/dashscope/cost_calculator.py	Adds an optional custom_llm_provider parameter (default "dashscope") to cost_per_token so the same tiered-pricing logic can resolve model info for any provider; the change is fully backward-compatible and the docstring is updated to reflect the broader purpose.
tests/test_litellm/test_cost_calculator.py	Adds test_volcengine_tiered_pricing_graduated_cost, a network-free regression test that reads tier data from the bundled model map, constructs a cross-tier token count, and verifies the graduated cost exactly; correctly uses monkeypatch.setattr (unlike many existing tests in the file that directly mutate litellm.model_cost).

_{Reviews (3): Last reviewed commit: "fix: route volcengine (Doubao) tiered-pr..." | Re-trigger Greptile}

[greptile-apps]

Volcengine coupled to dashscope's cost calculator file

The volcengine branch imports from litellm.llms.dashscope.cost_calculator, which ties an unrelated provider to dashscope's internal module. Any future dashscope-specific changes to that file (e.g., dashscope-flavoured token handling) could silently affect volcengine cost calculations. Consider either creating a thin litellm/llms/volcengine/cost_calculator.py that re-exports the shared logic, or extracting the reusable tiered-pricing function into a provider-neutral location (e.g., litellm/llms/utils/tiered_cost_calculator.py) that both dashscope and volcengine import from.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[vineethsaivs]

Good catches, thanks. I've switched the test to monkeypatch.setattr(litellm, "model_cost", ...) so it no longer leaks global state into later tests, matching the other tests in this file.

On the cross-provider coupling: I kept the change minimal by reusing the existing tiered-pricing calculator via a custom_llm_provider argument (defaulting to dashscope, so existing behavior is unchanged) rather than duplicating the graduated-tier logic. The function only reads tiered_pricing from the model map, so it's already provider-agnostic. Happy to relocate it to a provider-neutral module (or add a thin volcengine/cost_calculator.py wrapper) if you'd prefer that boundary; just let me know which you'd like.

[Sameerlite]

Thanks for the fix — the before/after cost output in the description is great proof! The Greptile review is a bit stale (commits landed after the last review). Triggering a fresh pass:

@greptileai

[vineethsaivs]

Retargeted this PR onto litellm_oss_branch per the contribution guard and rebased it, so the diff is just the three intended files (the volcengine dispatch branch in cost_calculator.py, the custom_llm_provider argument on the shared tiered-pricing handler, and the regression test). @greptileai

[Sameerlite]

Thanks for the contribution! A few things to get this ready for review:

• Wrong base branch: This PR targets litellm_oss_branch but community PRs should target litellm_internal_staging. Could you rebase?

  git fetch origin
  git rebase --onto origin/litellm_internal_staging origin/litellm_oss_branch <your-branch>
  git push --force-with-lease
  

  Then update the base in GitHub's UI (Edit → Base: litellm_internal_staging).

• Unresolved Greptile review threads — there are open threads from Greptile's review that haven't been resolved yet. Could you address those?

Once those are addressed, we'll take a closer look — thanks again!

[vineethsaivs]

Done. Rebased onto litellm_internal_staging and updated the base.

On the Greptile threads: the global-state one is resolved, the test now uses monkeypatch.setattr(litellm, "model_cost", ...) so it no longer leaks state into later tests. On the dashscope coupling: the shared cost_per_token only reads tiered_pricing from the model map and takes custom_llm_provider as a parameter, so it is already provider-agnostic; routing volcengine through it avoids duplicating the graduated-tier logic. I am happy to extract it into a provider-neutral module (e.g. litellm/llms/utils/tiered_cost_calculator.py) that both dashscope and volcengine import from if you prefer that boundary; I kept it in place for now since moving it would also relocate the private tier helpers and widen the diff. @greptileai

[greptile-apps]

Thanks for addressing both threads and the clear explanation.

The global-state fix is correct — monkeypatch.setattr is the right approach and matches the pattern used elsewhere in the file.

On the dashscope coupling: your reasoning holds. The function is already provider-agnostic in behavior — get_model_info is parameterized, the tier math is generic, and the dashscope module path is purely a naming artifact. Extracting to litellm/llms/utils/tiered_cost_calculator.py would be the cleaner long-term home if a third provider ever needs to share it, but it's not necessary to block this fix. The diff is already minimal and the regression test gives a clear signal if the shared function ever diverges.

LGTM — the fix is correct, the test is solid, and the approach is reasonable given the scope of the change.

[vineethsaivs]

I noticed this was merged into litellm_oss_170626_1 and then reverted there. If the revert was due to something on my end (a failing test or an integration conflict in the batch), I am happy to dig in and re-submit against the right branch; just let me know what you saw and I will turn it around quickly.