feat(pricing): per-provider effective-price multiplier knob (C)#60
Conversation
PR C of the pricing roadmap. On top of the base list prices (B), an operator often pays an EFFECTIVE price that differs from list — a negotiated discount, prepaid credits, or a risk premium for a less-trusted provider. This adds a deterministic per-provider multiplier so ranking reflects the effective cost. - `<provider>.price_multiplier` (float, default 1.0) is auto-added to the knob schema for every provider that contributes a price (has a source), so it shows in the Config tab and persists via the host store like every other knob. No dead knob for a sourceless provider (act over potency). - Applied centrally in `sources.push_prices`: price_in/out are scaled by the multiplier on the way into the core's ranking metrics. The raw list price stays untouched at the source and in the provider_prices table — the multiplier is "effective vs list", deterministic, and lives only in what ranking sees (so the measured correction in PR D can still compare against the raw list price). Marketplace/offer prices (antseed, openrouter_market) ride the live market and are intentionally not scaled. Suite 423/0.
📝 WalkthroughWalkthroughAdds a per-provider ChangesPrice Multiplier Knob and Billing Correction
Sequence Diagram(s)sequenceDiagram
participant Source as push_prices
participant Settings as settings
participant Metrics as host metrics
participant Shim as _executed_cost_usd
Source->>Settings: read provider.price_multiplier
Source->>Metrics: write price_in/price_out * mult
Shim->>Settings: read provider.price_multiplier
Shim->>Shim: divide price_in/price_out by mult
Shim->>Shim: compute cost_usd from list price
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…billing (Axis 7) Review caught it (correctly): the multiplier scaled chosen.price_in/out, and shim._executed_cost_usd step 3 computes billed cost_usd from exactly those fields whenever the provider reports no cost of its own — i.e. the direct openai/anthropic/google providers (#59), bedrock, and codex-scarcity. So the "ranking-only" claim was false: the lever leaked into cost_usd, x_router, the session meter and stats, a "risk premium > 1" over-reported real spend, and the effect was inconsistent (reported-cost providers ignored it, computed ones did not). Make it genuinely ranking-only: the multiplier is a FICTITIOUS routing lever. - push_prices still scales the price RANKING sees. - _executed_cost_usd divides the same multiplier back out before computing cost, so billing settles at the raw list price (or the provider-reported cost in step 2). cost_usd is now invariant to the lever, uniformly across providers — pinned by a new test (0.5 multiplier, cost still bills the list price, not half). - Knob reframed to "ranking price multiplier" with min 0.1 (no zero-price / divide-by-zero footgun); help states it does not change billing. The divide-back is exact because every multiplier-bearing provider is priced via push_prices (no catalog-static price to mis-scale). Suite 435/0.
|
You're right, and verified in the tree — the multiplier leaked into
The missing test you asked for: The divide-back is exact because every multiplier-bearing provider is priced through |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
sources/__init__.py (1)
81-85: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCentralize the multiplier lookup; same pattern is duplicated in
shim.py.
key in settings.SCHEMA/settings.get(key)/ default-to-1.0is repeated verbatim in_executed_cost_usd(shim.py). Also noteshim.pyguardsmult > 0before dividing, but this scaling path has no equivalent guard before multiplying — if an override ever bypasses schema bounds (tests already write directly intosettings._overrides, skipping validation), this could write a zero/negative ranking price unguarded. Extracting a singleproviders.get_price_multiplier(provider_id)helper would remove the duplication and let both call sites share the same validation/guard behavior.♻️ Proposed helper (providers.py) and call-site update
+def get_price_multiplier(provider_id: str) -> float: + import settings + key = f"{provider_id}.price_multiplier" + mult = settings.get(key) if key in settings.SCHEMA else 1.0 + return mult if mult and mult > 0 else 1.0- key = f"{provider}.price_multiplier" - mult = settings.get(key) if key in settings.SCHEMA else 1.0 + mult = providers.get_price_multiplier(provider)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sources/__init__.py` around lines 81 - 85, Centralize the provider price multiplier lookup and share its validation across both `sources/__init__.py` and `shim.py`; the `key in settings.SCHEMA` / `settings.get(key)` / default `1.0` pattern is duplicated and the `host.update_metrics` scaling path currently has no guard against zero or negative multipliers. Add a single helper such as `providers.get_price_multiplier(provider_id)` and update `_executed_cost_usd` and the metrics-writing code to use it so both call sites apply the same schema/default handling and reject invalid multiplier values consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@shim.py`:
- Around line 1421-1431: The request-time adjustment in shim.py’s ranking/cost
path is re-reading provider.price_multiplier instead of using the value that was
applied when sources.push_prices refreshed the price, so cost_usd can drift if
the setting changes later. Update the ranked price data flow to persist the
applied multiplier (or raw list price) at refresh time in the relevant
pricing/ranking structure, then have the _executed_cost_usd logic use that
captured value instead of settings.get when computing pin/pout for the chosen
provider.
In `@sources/__init__.py`:
- Around line 81-86: `push_prices` is still applying
`<provider>.price_multiplier` to marketplace offers, so AntSeed provider ids
like `antseed_free` and `antseed_cheap` are being scaled unexpectedly. Update
the pricing path in `push_prices` to bypass the multiplier for
marketplace-derived providers, or ensure those provider ids are excluded from
the `price_multiplier` schema lookup before calling `host.update_metrics`.
---
Nitpick comments:
In `@sources/__init__.py`:
- Around line 81-85: Centralize the provider price multiplier lookup and share
its validation across both `sources/__init__.py` and `shim.py`; the `key in
settings.SCHEMA` / `settings.get(key)` / default `1.0` pattern is duplicated and
the `host.update_metrics` scaling path currently has no guard against zero or
negative multipliers. Add a single helper such as
`providers.get_price_multiplier(provider_id)` and update `_executed_cost_usd`
and the metrics-writing code to use it so both call sites apply the same
schema/default handling and reject invalid multiplier values consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9894bc40-b6f3-4f64-9cbc-3714b73f5c14
📒 Files selected for processing (6)
providers.pyshim.pysources/__init__.pytests/test_providers.pytests/test_shim.pytests/test_sources.py
| # The ranking price carries the provider's effective-price multiplier — a | ||
| # FICTITIOUS routing lever (providers.price_multiplier, applied in | ||
| # sources.push_prices). Billing must use the RAW list price, so divide it back | ||
| # out: cost_usd never moves with the multiplier. Reported-cost providers settle | ||
| # in step 2 and never reach here, so this keeps billing uniform across them. | ||
| import settings | ||
| mkey = f"{chosen.get('provider_id')}.price_multiplier" | ||
| mult = settings.get(mkey) if mkey in settings.SCHEMA else 1.0 | ||
| if mult and mult > 0: | ||
| pin = pin / mult if pin is not None else pin | ||
| pout = pout / mult if pout is not None else pout |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check the cadence at which push_prices is invoked relative to request serving
rg -n "push_prices" -C 4 --type=pyRepository: genlayerlabs/unhardcoded
Length of output: 162
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the relevant files and inspect the surrounding code
git ls-files | rg '(^|/)(shim\.py|.*test.*\.py|.*price.*\.py)$' || true
printf '\n--- shim.py outline ---\n'
ast-grep outline shim.py --view expanded || true
printf '\n--- search for price_multiplier / push_prices / cost_usd ---\n'
rg -n "price_multiplier|push_prices|cost_usd|executed_cost" -C 3 shim.py . || trueRepository: genlayerlabs/unhardcoded
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the exact region around the reported lines in shim.py
line_count=$(wc -l < shim.py)
start=1390
end=1465
if [ "$line_count" -lt "$end" ]; then end="$line_count"; fi
sed -n "${start},${end}p" shim.py | cat -nRepository: genlayerlabs/unhardcoded
Length of output: 4847
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find tests or call sites that show whether the multiplier is expected to be stable
rg -n "price_multiplier|settings\.get\(mkey\)|push_prices|executed_cost_usd|billing" -C 3 --glob '!shim.py' .Repository: genlayerlabs/unhardcoded
Length of output: 19140
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect how prices are written and whether a refresh can happen independently of requests
sed -n '62,110p' sources/__init__.py | cat -n
printf '\n---\n'
rg -n "prices_pushed|refresh.*price|push_prices\(|price_refreshed_at|scheduler|refresh_interval" -C 3 sources tests shim.py llm_router_host.py providers.pyRepository: genlayerlabs/unhardcoded
Length of output: 12524
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
# Simulate the exact formula used by sources.push_prices and shim._executed_cost_usd.
def billed(list_price_in, list_price_out, push_mult, compute_mult, tokens_in=1_000_000, tokens_out=100_000):
ranked_in = list_price_in * push_mult
ranked_out = list_price_out * push_mult
raw_in = ranked_in / compute_mult if compute_mult else ranked_in
raw_out = ranked_out / compute_mult if compute_mult else ranked_out
return round((tokens_in/1e6)*raw_in + (tokens_out/1e6)*raw_out, 6)
cases = [
(2.0, 10.0, 0.5, 0.5), # same knob
(2.0, 10.0, 0.5, 1.0), # knob changed after push
(2.0, 10.0, 1.0, 0.5), # knob changed after push
]
for c in cases:
print(c, "=>", billed(*c))
PYRepository: genlayerlabs/unhardcoded
Length of output: 249
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the exact refresh path around push_prices usage, if any, in the source package
rg -n "push_prices\(host|push_prices\(" -C 2 sources . --glob '!tests/**' --glob '!shim.py'Repository: genlayerlabs/unhardcoded
Length of output: 1521
Persist the multiplier used at price refresh
shim.py:1402-1431 re-reads <provider>.price_multiplier at request time, but sources.push_prices applies it during refresh. If that knob changes between refresh ticks and a later request, _executed_cost_usd divides by the wrong value and cost_usd drifts from the raw list price. Store the applied multiplier (or raw price) with the ranked price and use that captured value here.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@shim.py` around lines 1421 - 1431, The request-time adjustment in shim.py’s
ranking/cost path is re-reading provider.price_multiplier instead of using the
value that was applied when sources.push_prices refreshed the price, so cost_usd
can drift if the setting changes later. Update the ranked price data flow to
persist the applied multiplier (or raw list price) at refresh time in the
relevant pricing/ranking structure, then have the _executed_cost_usd logic use
that captured value instead of settings.get when computing pin/pout for the
chosen provider.
| key = f"{provider}.price_multiplier" | ||
| mult = settings.get(key) if key in settings.SCHEMA else 1.0 | ||
| host.update_metrics(provider, family, { | ||
| "price_in": p["price_in_usd_per_mtok"], | ||
| "price_out": p["price_out_usd_per_mtok"], | ||
| "price_in": p["price_in_usd_per_mtok"] * mult, | ||
| "price_out": p["price_out_usd_per_mtok"] * mult, | ||
| "price_refreshed_at": now, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect how antseed Price records are constructed and what provider_id they carry
rg -n "_antseed_source" -C 8 --type=py
rg -n "discovery.*marketplace" -C 3 --type=pyRepository: genlayerlabs/unhardcoded
Length of output: 162
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'FILES\n'
git ls-files | rg '(^sources/__init__\.py$|providers\.py$|antseed|marketplace|price)'
printf '\nOUTLINE sources/__init__.py\n'
ast-grep outline sources/__init__.py --view expanded || true
printf '\nOUTLINE providers.py\n'
ast-grep outline providers.py --view expanded || true
printf '\nANTSEED SEARCH\n'
rg -n "antseed|marketplace|price_multiplier|provider_knob_schema|Price\(" sources providers.py . -g '*.py' -g '*.md' || trueRepository: genlayerlabs/unhardcoded
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'FILES\n'
git ls-files | rg '(^sources/__init__\.py$|providers\.py$|antseed|marketplace|price)'
printf '\nOUTLINE sources/__init__.py\n'
ast-grep outline sources/__init__.py --view expanded || true
printf '\nOUTLINE providers.py\n'
ast-grep outline providers.py --view expanded || true
printf '\nANTSEED SEARCH\n'
rg -n "antseed|marketplace|price_multiplier|provider_knob_schema|Price\(" sources providers.py . -g '*.py' -g '*.md' || trueRepository: genlayerlabs/unhardcoded
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,140p' sources/__init__.py
printf '\n---\n'
sed -n '100,340p' sources/antseed.py
printf '\n---\n'
sed -n '250,290p' providers.py
printf '\n---\n'
sed -n '880,890p' tests/test_sources.pyRepository: genlayerlabs/unhardcoded
Length of output: 19615
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'sources/antseed.py pricing section\n'
sed -n '340,430p' sources/antseed.py
printf '\n---\ntests/test_sources.py antseed pricing tests\n'
sed -n '880,920p' tests/test_sources.py
printf '\n---\nsearch for marketplace price multiplier expectations\n'
rg -n "Marketplace/offer prices|price_multiplier|not scaled|antseed_free|antseed_cheap" providers.py sources tests -g '*.py'Repository: genlayerlabs/unhardcoded
Length of output: 11221
Skip price_multiplier for marketplace offers (sources/__init__.py:81-86) push_prices still scales every price by <provider>.price_multiplier, and AntSeed marketplace rows are emitted under provider ids like antseed_free / antseed_cheap, so they still pick up that knob. Add a marketplace bypass here or keep those providers out of the multiplier schema.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/__init__.py` around lines 81 - 86, `push_prices` is still applying
`<provider>.price_multiplier` to marketplace offers, so AntSeed provider ids
like `antseed_free` and `antseed_cheap` are being scaled unexpectedly. Update
the pricing path in `push_prices` to bypass the multiplier for
marketplace-derived providers, or ensure those provider ids are excluded from
the `price_multiplier` schema lookup before calling `host.update_metrics`.
1 participant
What
PR C of the pricing roadmap. On top of the base list prices (B / #59), an operator often pays an effective price that differs from list — a negotiated discount, prepaid credits, or a risk premium for a less-trusted provider. This adds a deterministic per-provider multiplier so ranking reflects effective cost.
How
<provider>.price_multiplier(float, default1.0) is auto-added to the schema for every provider that contributes a price (source is not None). It shows in the Config tab and persists inhost_storelike every other knob. No dead knob for a sourceless provider (act over potency).sources.push_prices—price_in/price_outare scaled by the multiplier on the way into the core's ranking metrics:Design
provider_pricestable (B). The multiplier is "effective vs list", deterministic, and lives only in what ranking sees — so PR D's measured correction can still compare against the raw list price.push_prices), so it covers everypricing()-path provider uniformly (openrouter, bedrock, the direct openai/anthropic/google from B, codex's scarcity price…). Marketplace/offer prices (antseed, openrouter_market) ride the live market and are intentionally not scaled.push_priceslazy-importssettings(which →providers, never back tosources).Verification
push_priceswith a0.5multiplier override → ranking sees2.5/15.0for a5/30list price; default1.0is a no-op (existing test unchanged).Builds on #57 (knob system) and #59 (base prices). Next: D — measured effective-cost correction from
calls.cost_usd.Summary by CodeRabbit
New Features
Bug Fixes
Tests