feat(providers): native Anthropic/Gemini adapters + isolate provider_adapters

[jmlago]

Reconstructed from @MuncleUscles's #24, carrying only the provider-adapter half. The provider-neutral OpenRouter family change from #24 is split into its own PR (it belongs in the core's candidate contract — see unhardcoded-engine#22); this PR leaves family naming exactly as on main.

Edgars's work, carried here

• Refactor: provider-call code moved out of llm_router_host.py into provider_adapters/ (openai_compatible, anthropic, google, common, dispatcher); llm_router_host re-exports the old symbols for compat.
• Native non-OpenAI-compatible backends: Anthropic Messages + Gemini generateContent (non-streaming), with a pre-delta streaming guard (stream_unsupported_api_kind) so a native route falls through to the next candidate instead of being sent down the OpenAI-compatible SSE path.
• config.live.lua native examples (openai / anthropic / gemini / bedrock_mantle), the openai vs openai_codex provider-id split, and native prices kept out of the committed metrics seed.

Review repairs folded in (originally pushed to #24)

• Gemini key leak fixed: the API key rides the x-goog-api-key header, never the URL query — it can no longer leak into error_message/logs/traces (§3: keys never logged/echoed).
• Multi-turn tool loops fixed: assistant tool_calls → Anthropic tool_use / Gemini functionCall (a tool-call-only turn is kept); role:"tool" → Anthropic tool_result (by tool_use_id) / Gemini functionResponse (by name). Shared json_args parses the OpenAI arguments string. Both verified by must-reject tests (key absent on error; tool turn preserved).

Notes

• The adapters prefer offer.wire_model_id when present, else served_model_id — harmless on main (no offer sets it yet), ready for the family PR.
• Marketplace-wiring tests inject offers directly via set_discover_hook; the OpenRouter production side lands in the family PR.
• No new dependency.

Verification

nix-shell --run 'python -m pytest tests -q' → 342 passed, 2 skipped, 0 failed (main was 340). No CI, so re-run before merge.

Split

Part of superseding #24: this PR (native adapters) + the family PR (provider-neutral OpenRouter families) + unhardcoded-engine#22 (served_model_id = wire id). #24 to be closed once these land.

Summary by CodeRabbit

• New Features

  • Added native routing support for additional provider options, including OpenAI, Anthropic, Gemini, and Bedrock Mantle.
  • Expanded request handling so supported models can be served through the most appropriate provider path.
  • Added support for additional API kinds in routing and streaming, with clear fallback behavior for unsupported streaming cases.

• Bug Fixes

  • Improved routing consistency for Codex-related models by separating them from the standard OpenAI path.
  • Updated live configuration and metrics so model availability and usage tracking reflect the new provider names.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Provider adapters were split into shared helpers and native Anthropic/Gemini modules, dispatch and streaming wiring were updated to use them, and live config, metrics, docs, and tests were changed to use openai_codex plus new native provider IDs.

Changes

Provider adapter and live routing updates

Layer / File(s)	Summary
Shared adapter core provider_adapters/__init__.py, provider_adapters/common.py, provider_adapters/dispatcher.py, provider_adapters/openai_compatible.py, llm_router_host.py	Common hook/auth/error helpers, the API-kind dispatcher, and OpenAI-compatible call handling were moved into provider_adapters, and llm_router_host.py now re-exports them while folding route outcomes from request fields.
Native adapters and dispatch provider_adapters/anthropic.py, provider_adapters/google.py, serve.py, streaming.py, tests/test_native_providers.py, tests/test_streaming.py	Anthropic and Gemini adapters translate OpenAI-shaped requests and responses, serve.py registers their api_kind handlers, and streaming.py adds a bad_request fallback for unsupported native kinds.
Live catalog updates config.live.lua, metrics.live.lua, docs/DEPLOY.md, tests/test_codex_scarcity.py, tests/test_live_wiring.py, tests/test_sources.py	The live catalog, metrics seeds, and deploy mode labels switch codex traffic to openai_codex and add native openai, anthropic, gemini, and bedrock_mantle routes.
Live routing assertions tests/test_live_wiring.py	Live wiring tests update provider attribution for pinned OpenAI requests and verify OpenRouter alias-family policy routing.

Sequence Diagram(s)

sequenceDiagram
  participant Serve as serve.py
  participant Dispatcher as make_api_kind_dispatcher
  participant Anthropic as make_anthropic_async_call_provider
  participant Google as make_google_async_call_provider
  participant HTTPX as httpx.AsyncClient
  participant AnthropicAPI as Anthropic Messages API
  participant GeminiAPI as Gemini generateContent API
  Serve->>Dispatcher: request with api_kind
  alt api_kind = anthropic
    Dispatcher->>Anthropic: await handler(request)
    Anthropic->>HTTPX: POST /v1/messages
    HTTPX->>AnthropicAPI: request JSON
    AnthropicAPI-->>HTTPX: response JSON
    HTTPX-->>Anthropic: parsed response
    Anthropic-->>Dispatcher: normalized result
  else api_kind = google
    Dispatcher->>Google: await handler(request)
    Google->>HTTPX: POST /v1beta/models/...:generateContent
    HTTPX->>GeminiAPI: request JSON
    GeminiAPI-->>HTTPX: response JSON
    HTTPX-->>Google: parsed response
    Google-->>Dispatcher: normalized result
  end
  Dispatcher-->>Serve: result

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

I hopped through openai_codex lanes tonight,
With carrots of routing all glowing bright.
Anthropic hummed, Gemini sang,
The server went zippy with a happy clang.
🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 26.76% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: new native Anthropic/Gemini adapters and extraction of provider logic into provider_adapters.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/native-provider-adapters

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}