Skip to content

FR: one routes list in config.json for per-request model/effort routing #117

Description

@ItsAlbertZhang

Summary

The mapping from what Claude Code sends to what reaches a provider is spread over several independent knobs, and some combinations have no knob at all. This proposes one routes list in config.json that can express all of them, with an empty list meaning "no change from today".

Today

What Claude Code sends is (model, effort)effort arrives as output_config.effort (src/providers/translate_shared.rs:69-91) — plus two request kinds the proxy already detects: compaction (is_compact_messages_request, src/providers/codex/translate/request.rs:343) and the auto-review classifier (is_claude_auto_review_request, src/server.rs:62). The knobs that shape where such a request goes:

Knob Scope Where
aliasProvider / CCP_ALIAS_PROVIDER (#12) all Anthropic-style aliases → codex or kimi, as one set src/config.rs:229, src/registry.rs:171-173
built-in alias tables Codex: haiku → gpt-5.6-luna, sonnet → gpt-5.6-terra, opus/fablegpt-5.6-sol; Kimi: every alias → kimi-for-coding codex/translate/model_allowlist.rs:19-32, kimi/translate/model_allowlist.rs:5-24
codex.model / CCP_CODEX_MODEL every Codex request; forces one model regardless of what was requested src/config.rs:821
codex.effort / CCP_CODEX_EFFORT every Codex request src/config.rs:789
autoReviewModel / CCP_AUTO_REVIEW_MODEL (#72) auto-review → model; on Codex defaults to gpt-5.6-luna src/server.rs:54, src/config.rs:835
CCP_COMPACT_EFFORT (#67) compaction → effort cap, default low translate/request.rs:355-366
(none)#63 proposes compaction → model, gated on a PreCompact hook marker
(none)#93 proposes auto-review → effort

The gaps, concretely:

  • Aliases move between providers only as one set, so "haiku → Grok, sonnet/opus → Codex" has no proxy-side expression; it requires pinning ANTHROPIC_MODEL and ANTHROPIC_SMALL_FAST_MODEL to concrete IDs on the Claude Code side.
  • The alias → model mapping is fixed in code; the only override, codex.model, collapses every request to one model.
  • Effort can only be passed through or overridden globally; there is no per-model or per-kind effort.
  • Per-kind overrides arrive one knob at a time (autoReviewModel, CCP_COMPACT_EFFORT, Route compaction requests to a dedicated model #63, feat(auto-review): add opt-in classifier effort override #93).

Proposal

"routes": [
  { "match": { "kind": "compaction" },                "to": { "model": "gpt-5.6-luna" } },
  { "match": { "kind": "autoReview" },                "to": { "effort": "low" } },
  { "match": { "model": "haiku" },                    "to": { "provider": "codex", "model": "gpt-5.6-luna", "effort": "low" } },
  { "match": { "model": "sonnet" },                   "to": { "provider": "grok",  "model": "grok-4.6" } },
  { "match": { "model": "opus", "effort": "max" },    "to": { "provider": "codex", "model": "gpt-5.6-sol",  "effort": "xhigh" } },
  { "match": { "model": "opus" },                     "to": { "provider": "codex", "model": "gpt-5.6-sol" } }
]
  • First match wins. match: any of kind (compaction | autoReview), model (a bare alias such as haiku, covering every spelling the alias table lists, or an exact name), effort. Omitted fields match anything.
  • to: any of provider, model, effort. Omitted fields keep today's behavior; effort is validated against the target provider.
  • Empty or absent list → no behavior change. Existing settings keep working; a matched route takes precedence over them for that request.
  • config.json is already re-read by the per-request getters (src/config.rs:789, 821, 835), so edits apply without a restart.

The three open requests above are rows of this table: #63 (compaction → faster model), #93 (auto-review → lower effort), and #4 / #11 (sending some aliases elsewhere — native Anthropic passthrough would be a later to.provider value, not part of this).

Scope

If this direction is acceptable, I'll draft the PR: the matcher in src/server.rs, where the provider is selected and the auto-review rewrite already happens, plus docs, as one PR or cell by cell. Anthropic passthrough stays out of it.

Related: #93 is mine and adds the auto-review effort cell of this. If this direction doesn't fit the project, I'll close #93. If it does, I'll rebase it or fold it into the routes implementation, whichever you prefer.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions