[codex] Add dynamic OpenRouter families and native provider adapters#24
[codex] Add dynamic OpenRouter families and native provider adapters#24MuncleUscles wants to merge 6 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
MuncleUscles
changed the title
|
Stopgap native pricing update pushed in 53f769d. This parent PR now points the core submodule at genlayerlabs/unhardcoded-engine#21, which adds static served_by catalog prices to the engine candidate builder. Native provider rows in config.live.lua carry operator_config effective prices; OpenRouter and Codex remain source/scarcity driven. |
MuncleUscles
force-pushed
the
codex/openrouter-dynamic-family-map
branch
from
53f769d to
15d4261
Compare
|
Correction after review: the inline served_by prices and engine submodule dependency were removed. The PR now keeps pricing out of shared catalog/metrics for native providers and adds a parent-side operator_prices source instead. Configure it with LLM_ROUTER_OPERATOR_PRICES_JSON or LLM_ROUTER_OPERATOR_PRICES_PATH; it pushes prices only for existing static catalog pairs and skips OpenRouter/Codex-style source/scarcity-priced providers. genlayerlabs/unhardcoded-engine#21 has been closed as obsolete. |
MuncleUscles
force-pushed
the
codex/openrouter-dynamic-family-map
branch
from
15d4261 to
c9b4ac1
Compare
|
Added the concrete Studio starter overlay as operator-prices.studio.example.json plus operator-prices.schema.json. This file is not auto-enabled by the shared image; workloads should copy/mount it per environment as operator-prices.json and set LLM_ROUTER_OPERATOR_PRICES_PATH to the mounted path. The current starter covers: openai gpt-5.5/gpt-5.4, anthropic claude-opus-4-8/claude-sonnet-4-6, gemini gemini-3.1-pro-preview, and bedrock_mantle qwen3-235b-a22b. |
MuncleUscles
force-pushed
the
codex/openrouter-dynamic-family-map
branch
from
c9b4ac1 to
d19d7bb
Compare
|
Updated the pricing shape again after design review: this is now a price_catalog source, not env JSON. Production/workloads should keep the canonical prices in DB and expose a read-only internal endpoint via LLM_ROUTER_PRICE_CATALOG_URL. LLM_ROUTER_PRICE_CATALOG_PATH remains only a local/dev file fallback. The Studio seed file is now price-catalog.studio.example.json with schema price-catalog.schema.json. |
MuncleUscles
force-pushed
the
codex/openrouter-dynamic-family-map
branch
from
d19d7bb to
f1d2675
Compare
Two correctness/security defects in the native provider adapters added on
this branch, found in review:
1. Gemini API key leaked into error messages (and thus logs/traces).
google.py built the request URL with `?key=<secret>` and embedded that
URL verbatim in `_err("timeout", ..., f"POST {url} ...")` and via the
httpx exception text. Object §3: keys are never logged or echoed. The
key now rides the `x-goog-api-key` header and never the URL query, so no
error path can surface it. (Behavior change named: Gemini auth method
moves from query param to header — same credential, safer channel.)
2. Multi-turn tool loops were broken in both native adapters — the
ecosystem's primary client is an agentic tool loop. `text_from_content`
dropped an assistant turn that was tool-calls-only (no text), and a
`role:"tool"` result was flattened to plain user text, losing the call
linkage. The next turn then referenced a tool_use that was never sent
(Anthropic/Gemini 400). Now:
- assistant `tool_calls` -> Anthropic `tool_use` blocks / Gemini
`functionCall` parts (the turn is kept even with empty text);
- `role:"tool"` -> Anthropic `tool_result` (keyed by `tool_use_id`) /
Gemini `functionResponse` (keyed by name, recovered from the call id);
- shared `json_args` parses the OpenAI arguments string, degrading
malformed args to `{}` rather than aborting the turn.
Tests: the guards are verified by what they must REJECT — a leaked key
(`test_google_native_never_leaks_key_in_url_or_errors`) and a dropped
tool turn (`test_{anthropic,google}_native_tool_roundtrip`). Existing
text-only/user/assistant shapes are preserved (the prior 75 affected-suite
tests stay green). Full suite: 343 passed, 2 skipped (was 340/2 + 3 new),
zero failures, run before and after under nix-shell + timeout.
|
Superseded — split into three focused PRs so the family-naming change lands at its proper locus (the engine's candidate contract) instead of being worked around in every adapter. @MuncleUscles's work is carried forward with authorship credited in each commit body:
Closing in favor of those. Thanks for the work, @MuncleUscles — all of it survives across the three. |
…adapters (#25) Reconstructed from Edgars Nemše's #24 (codex/openrouter-dynamic-family-map), carrying only the provider-adapter half. The provider-neutral OpenRouter family change from #24 is split into its own PR (it touches the core's candidate contract); this PR leaves family naming exactly as on main. Edgars's work, carried here: - Isolate the provider-call code out of llm_router_host.py into provider_adapters/ (openai_compatible, anthropic, google, common, dispatcher); llm_router_host re-exports the old symbols for compatibility. - Native non-OpenAI-compatible backends: Anthropic Messages and Gemini generateContent (non-streaming), with a pre-delta streaming guard (stream_unsupported_api_kind) so a native route falls through to the next candidate instead of being sent down the OpenAI-compatible SSE path. - config.live.lua native examples (openai / anthropic / gemini / bedrock_mantle) and the openai vs openai_codex provider-id split; native provider prices kept out of the committed metrics seed. Folded in on top (review repairs, originally pushed to #24): - Gemini API key now rides the x-goog-api-key header, never the URL query, so it can no longer leak into error_message/logs/traces (§3: keys are never logged or echoed). - Multi-turn tool loops: assistant tool_calls -> Anthropic tool_use / Gemini functionCall (a tool-call-only turn is kept), and role:"tool" -> Anthropic tool_result (by tool_use_id) / Gemini functionResponse (by name). A shared json_args parses the OpenAI arguments string. The adapters prefer offer.wire_model_id when an offer carries it, else served_model_id — harmless on main (no offer sets wire_model_id yet) and ready for the family PR. The marketplace-wiring tests inject offers directly via set_discover_hook; the OpenRouter production side lands in the family PR. Verification: nix-shell --run 'python -m pytest tests -q' -> 342 passed, 2 skipped, 0 failed (main was 340; +native-provider and +tool/key tests, the openrouter family tests move to the family PR). No new dependency.
…26) Reconstructed from Edgars Nemše's #24, carrying only the family half. The provider-adapter half is in its sibling PR; this one makes discovered OpenRouter marketplace families provider-neutral. What it does: - sources/openrouter.py: a discovered marketplace model's policy-facing family is the provider-neutral name (`openai/gpt-5-mini` -> `gpt-5-mini`), while `wire_model_id` keeps the exact OpenRouter slug for the wire. `service_aliases` (config) handle the canonicalization exceptions where stripping the vendor isn't the right family (dated/suffixed slugs). Curated families served by the static `openrouter` provider are deduped out, so a marketplace row never shadows a curated family. - This pairs with the engine's `provider_eq` (#18): family is the model, provider is a separate axis the algebra filters. It lets a single provider-agnostic policy span curated + marketplace routes for one model, which is the point of the router ("stop hardcoding models"). Core bump: `core` -> 97d0333 (unhardcoded-engine #22), which makes `served_model_id` the offer's wire id (`offer.wire_model_id or model_family`). With a neutral family + a distinct wire slug, the engine now wires the slug on both the curated and discovered paths without every adapter special-casing `wire_model_id`, and the replayable trace records the real wire id. The §3 determinism boundary holds: `model_meta.lua` stays curated-keyed (refresh_model_meta untouched), so an on-chain/genvm host lacking a discovered offer still fails closed. Pinned by test_openrouter_model_meta_still_keyed_by _curated_family and the dedup assertion in test_openrouter_discovery_derives_policy_families_from_raw_model_ids. Verification: nix-shell --run 'python -m pytest tests -q' -> 333 passed, 2 skipped, 0 failed, against the bumped core. No new dependency.
…policy selection) (#48) Pulls unhardcoded-engine #24: the `served_by_eq` predicate, which selects by the executed route (a marketplace peer, or the provider for a direct route) — the same notion the host already records as calls.served_by. A policy can now pin or exclude a specific peer (or(served_by_eq …) / not(…), sugar served_by_in / served_by_not_in), not just a whole provider via provider_eq. The host passes policy_ir straight through to the engine, so adoption is the submodule bump plus the SKILL.md guidance: peer targeting IS now expressible — the old "not a policy concern" note is corrected — but prefer gating on properties (reputation_score, success_rate) over pinning peer ids, which rot as peers churn.
2 participants
Summary
vendor/modelto provider-neutralmodelfamilies while preserving raw slugs aswire_model_idservice_aliasesonly for provider-local canonicalization exceptions, such as dated Qwen/Gemma slugsconfig.live.luaopenaifor the official OpenAI API route andopenai_codexfor the ChatGPT/Codex OAuth routellm_router_host.pyintoprovider_adapters/modules for OpenAI-compatible, Anthropic, Gemini, shared primitives, and API-kind dispatchNotes
api_kind = "openai_compatible"because those endpoints expose OpenAI-compatible APIs.api_kindplus AWS request signing behavior; this PR keeps Bedrock as a Mantle OpenAI-compatible example.tier = "partner"enum is legacy policy vocabulary for highest-preference static providers, not a business relationship. Renaming that enum should be a separate schema/docs migration.llm_router_host.pystill re-exports the old adapter symbols for script/test compatibility, but provider-specific implementation now lives underprovider_adapters/.Validation
nix-shell --run 'python -m pytest tests/test_native_providers.py tests/test_sources.py tests/test_streaming.py tests/test_live_wiring.py -q'(62 passed)nix-shell --run 'python -m pytest tests -q'(343 passed, 2 skipped)git diff --check