Skip to content

Per-registry price overrides for cost (models.dev default) #505

Closed

Description

@vgmartinez

v1 shape locked (Victor)

Absolute per model/glob + optional registry % discount.

Config on registry

{
  "pricing": {
    "discount": 0.2,
    "overrides": {
      "gpt-4o": { "input": 0.0000015, "output": 0.000006 },
      "gpt-4o-mini*": { "input": 0.0000001, "output": 0.0000004 }
    }
  }
}
  • overrides: USD per token (same unit as catalog / llmcost.CustomPrice); keys are exact slugs or globs (BestMatch / most-specific wins).
  • discount: optional fraction off list (e.g. 0.2 = 20% off). Applied only when no absolute override matches that model.

Precedence

  1. Absolute override (exact > best glob)
  2. Registry discount × models.dev catalog price
  3. models.dev catalog as-is
  4. Unpriced

No separate “family” entity — families are globs.

Acceptance (refresh)

  • Registry admin API/UI: set overrides + optional discount
  • Metrics cost + monthly rollups use this resolution
  • LLM Budget dollar mode uses the same resolved rates for the request’s registry
  • Same slug on two registries can differ
  • Tests: absolute wins over discount; discount over list; glob specificity; fallthrough
  • Docs: per-token units, precedence, monthly estimate intent
  • PR only

Activity

  1. changed the title [-]Per-client price overrides for cost (models.dev default)[/-] [+]Per-registry price overrides for cost (models.dev default)[/+] on Aug 22, 2026
  2. vgmartinez commented on Aug 22, 2026

    @vgmartinez
    ContributorAuthor

    Clarification (enterprise contract shapes)

    OpenAI public docs: list prices and Scale/Reserved capacity are per model. Committed/Guaranteed Capacity language is often a portfolio % discount off list, not a published absolute rate sheet. Procurement deals may use either absolute $/model rates or a blanket multiplier.

    v1 still: registry pricing_overrides per model slug (input/output USD/token).
    Nice-to-have: optional registry-level multiplier (e.g. 0.8 = 20% off models.dev) applied when no absolute override matches. Precedence: absolute override → multiplier → models.dev.

  3. vgmartinez commented on Aug 22, 2026

    @vgmartinez
    ContributorAuthor

    Product intent (Victor)

    This is required to estimate real monthly cost, not just decorate events.

    Without registry overrides, monthly rollups use models.dev list prices and diverge from the customer’s actual invoice / enterprise rates. With overrides (absolute per-model and/or optional flat multiplier), TrustGate can sum usage × resolved_price over the month and produce a credible cost estimate.

    Extra acceptance

    • Monthly / period cost aggregation uses the same resolved prices as per-request Cost (override → multiplier → models.dev)
    • Docs call out that accurate monthly estimates depend on configuring registry pricing to match the contract
  4. added a commit that references this issue on Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions