Skip to content

feat(usage): user-configurable per-model cost overlay (providers.<name>.modelCosts) #1009

Description

@harryzhou2000

Why

The Logs ~$ estimate is priced from two compiled sources: the jawcode catalog
and the expected-price overlay. Both match exact native provider/model ids, so
internal and custom providers often fall through:

  • provider/model ids from a proxy or internal gateway can be mangled or
    namespaced differently than the catalog rows (there is deliberately no
    fuzzy/case-fold matching), leaving ~$ as an em dash;
  • even when an id matches, the operator's actual cost may vary from the catalog
    list price (reseller rates, negotiated pricing, pooled/subscription billing).

There is currently no way to set a price without editing source code.

Proposal

Add providers.<name>.modelCosts — a per-model display price overlay (USD per
1M tokens), following ocx's existing flat per-model config convention
(modelContextWindows, modelMaxInputTokens, ...):

"providers": {
  "blsc": {
    "adapter": "openai-chat",
    "baseUrl": "https://llmapi.blsc.cn",
    "modelCosts": {
      "deepseek-v4-flash": { "input": 0.14, "output": 0.28, "cacheRead": 0.0028, "cacheWrite": 0 }
    }
  }
}

Resolution order becomes: user-configured modelCosts → jawcode catalog →
expected-price overlay → model-level vendor fallback. All-zero entries fall
through ("zero = not billable here", consistent with catalog semantics).

Semantics

  • Display-time estimation only; never billing.
  • Rows are lifted from config at load and every persist path into a versioned
    registry; the estimator memo keys on that version, so edits apply without a
    restart and stale rows are never served.
  • New provider_cost_overlay estimate reason surfaced in the Logs detail view
    (with i18n labels).
  • Strict config validation (non-negative finite 4-tuples); the management API
    rejects malformed entries; safeConfigDTO exposes the field to the dashboard.

Implemented on feat/provider-cost-overlay (PR pending).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions