Repository navigation
feat(provider-xai): xAI (Grok) provider for llm-router - #379
Merged
Merged
Conversation
xAI's API is OpenAI-compatible, so this mirrors provider-openai's wire handling and adapts the provider-specific surface to xAI: - provider::xai::stream / refresh_models / on_router_ready, registered with llm-router; credential env var XAI_API_KEY, endpoint https://api.x.ai/v1. - curated.rs: grok model catalog + capability metadata (grok-4.3, grok-4, grok-4-fast, grok-3, grok-3-mini, grok-code-fast, grok-build), legacy filter for grok-1/grok-2, MMDD snapshot folding. - reasoning.rs: xAI reasoning_effort rules — only grok-3-mini takes the param (low/high); grok-4.x reasons but rejects it; others non-reasoning. - discovery.rs: grok-* family gate, image/video (grok-imagine) excluded, /v1/models discovery. Verified end-to-end against live api.x.ai: provider registers with the router (shows as xAI, supports_model_listing), refresh_models discovered and reconciled 5 real models with correct metadata, and api.x.ai chat/completions confirmed OpenAI-shape. fmt + clippy -D warnings clean, 74 tests pass (incl. live engine integration suite).
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 49 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (36)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
skill-check — worker0 verified, 27 skipped (no docs/).
Four for four. Nicely done. |
…rflow
Two issues found testing grok models in the console:
1. Reasoning deltas dropped. xAI reasoning models stream chain-of-thought as
`delta.reasoning_content` ahead of the answer `content`; the OpenAI-cloned
SSE parser only read `content`, so the console showed "thinking…" with no
body. Now parsed into a thinking block (ThinkingStart/Delta/End +
ContentBlock::Thinking), emitted before the text block.
2. Prompt-overflow misclassified. grok-build returns a gRPC-style envelope
`{"code":"invalid-argument","error":"This model's maximum prompt length is
256000 but the request contains N tokens."}` — a bare-string `error` and
wording the sniffer missed, so it classified Permanent and failed the turn
instead of ContextOverflow (which the harness compacts + retries on, like
provider-anthropic). Now handles the string envelope and the "maximum
prompt length" / "prompt is too long" phrasings.
fmt + clippy -D warnings clean, 76 tests pass.
Every grok-4.x snapshot collapsed to "Grok 4" in the console picker because display_name was the family label. Derive a per-id display instead (display_for): drop the 4-digit MMDD snapshot segment, title-case the rest. grok-4.20-0309-non-reasoning -> Grok 4.20 Non Reasoning grok-4.20-0309-reasoning -> Grok 4.20 Reasoning grok-4.20-multi-agent-0309 -> Grok 4.20 Multi Agent grok-4.3 -> Grok 4.3 grok-build-0.1 -> Grok Build 0.1 family_meta keeps capability/pricing per family; only the label is per-id. fmt + clippy -D warnings clean, 77 tests pass.
Per docs/sops/new-worker.md §6: add provider-xai to create-tag.yml worker options and the release.yml tag patterns so the first release tag triggers the build/publish pipeline.
rohitg00
marked this pull request as ready for review
June 30, 2026 16:20
CodeRabbit review + empirical verification against api.x.ai: - reasoning.rs: the effort gating was inverted. Verified live: grok-4.3 accepts the full ladder none/low/medium/high/xhigh, grok-3-mini takes low/high, and every other current model (grok-build, the grok-4.20-* snapshots incl. multi-agent) returns 400 "does not support parameter reasoningEffort". The old code dropped the user's thinking_level on grok-4.3 (the flagship). Fixed the ladder + tests to match the API. - curated.rs: supports_xhigh was a blanket false; grok-4.3 accepts xhigh (verified), so the router's thinking:xhigh route now matches it. Others false. - curated.rs: guard base_id's byte split with is_char_boundary so a non-ASCII model id from a custom OpenAI-compatible server can't panic the refresh task. - README: rewrote the reasoning bullet — it was copied verbatim from provider-openai (o1/gpt-5 families, "no reasoning streamed"), both wrong for xAI; the worker DOES surface reasoning_content as thinking blocks. code-simplifier pass: no changes (faithful clone of provider-openai, already clean). fmt + clippy -D warnings clean, 77 tests pass.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
provider-xai: an llm-router provider for xAI's Grok models, so the harness chat loop can route togrok-*the same way it routes to OpenAI/Anthropic. xAI's API is OpenAI-compatible (https://api.x.ai/v1/chat/completions, OpenAI-shape streaming + tools), so this mirrorsprovider-openai's wire handling and swaps the provider-specific surface.This is distinct from the
grokworker (#378): that wraps the Grok agent CLI; this exposes raw Grok models to the harness.Verified against live api.x.ai
xAI,available,supports_model_listing: true.provider::xai::refresh_modelsdiscovered and reconciled 5 real models fromapi.x.ai/v1/modelswith correct metadata (e.g. grok-4.3 → 1M context).api.x.ai/v1/chat/completionsconfirmed OpenAI-shape (choices[].message.content,usage,finish_reason).router::chat→provider::xai::stream→ SSE → done frame + cost fill) is covered by the live-engine integration suite.Provider-specific surface
provider::xai::stream/refresh_models/on_router_ready; credential env varXAI_API_KEY; endpointhttps://api.x.ai/v1.curated.rs: grok catalog + capability metadata (grok-4.3, grok-4, grok-4-fast, grok-3, grok-3-mini, grok-code-fast, grok-build); legacy filter for grok-1/grok-2;-MMDDsnapshot folding. Pricing for grok-4.3/grok-build from docs.x.ai; other rows best-effort (display only, never routing).reasoning.rs: xAIreasoning_effortrules — onlygrok-3-minitakes the param (low/high);grok-4.xreasons but rejects it; others non-reasoning.discovery.rs:grok-*family gate,grok-imagine/image/video excluded.Configure
Checks
cargo fmt --all -- --checkclean ·cargo clippy --all-targets --all-features -- -D warningsclean · 74 tests pass (incl. live-engine integration). Draft pending maintainer review + release wiring (create-tag.yml/release.yml, perdocs/sops/new-worker.md§6).