Skip to content

Add more LLM providers to the catalog #502

Open

Description

@vgmartinez

Goal

Expand TrustGate’s LLM provider catalog so more teams can route/govern traffic without swapping gateways.

Already supported (pkg/infra/providers)

OpenAI, OpenAI-compatible, Anthropic, Google, Vertex, Azure, Bedrock, Mistral, Groq, DeepSeek, xAI, Cerebras, OpenRouter, Cohere.

Proposed next providers (priority candidates)

Ship as separate PRs when possible; start with OpenAI-compatible wrappers where the upstream is chat-completions shaped.

High leverage (Victor + OpenRouter gap scan)

  1. Moonshot / Kimi (Kimi K3) — first priority
  2. Fireworks AI
  3. Together AI
  4. DeepInfra — big OpenRouter traffic for open-weights; OpenAI-compatible
  5. Perplexity
  6. Novita
  7. Nebius
  8. SiliconFlow
  9. MiniMax
  10. NVIDIA NIM / NGC
  11. SambaNova
  12. Alibaba DashScope (Qwen) — OpenRouter slug alibaba
  13. Z.AI / Zhipu (GLM) — OpenRouter slug z-ai
  14. Databricks Model Serving

Also consider (OpenRouter has; lower urgency)

  • AI21, Replicate, Friendli, Baseten, Upstage (Solar), BytePlus / Seed (Doubao), Baidu, Tencent, Xiaomi, StepFun, Featherless, Parasail, Inference.net
  • Hugging Face Inference Endpoints only as optional openaicompat + custom base URL

Gaps vs OpenRouter (method)

Compared OpenRouter /api/v1/providers (~100 slugs) to TrustGate first-class clients. Many OR “providers” are niche GPU hosts or media APIs — we only promote chat LLM vendors users would BYOK directly.

Explicitly not / deprioritized

  • Cloudflare — competing AI gateway positioning
  • Hugging Face Inference Providers router — multi-provider router; overlaps OpenRouter
  • Other gateways / proxies as “providers”
  • Non-LLM OpenRouter entries (Deepgram, Fish Audio, HeyGen, Runway, Recraft, Voyage embeddings, etc.)

Acceptance criteria (per provider)

  • Registered in pkg/infra/providers/factory locator
  • Client under pkg/infra/providers/<name> following existing patterns (chat + stream where applicable)
  • Auth/credentials wired like peer providers
  • Connection tester when the interface is supported
  • Catalog / UI listing updated if required for the provider to appear in Admin
  • Docs / README provider list updated
  • Basic unit or integration coverage consistent with nearby providers
  • Opens as PR only (never merge from automation)

Kimi K3 slice (first)

  • Moonshot/Kimi client supports Kimi K3 models (chat + stream)
  • Documented base URL / auth for Moonshot API
  • Appears in Admin catalog as a selectable provider

Notes

  • Prefer reusing openaicompat when the vendor is truly OpenAI-compatible; only add a dedicated package when auth, headers, streaming, or error mapping diverge.
  • One provider per PR when possible.
  • Focus on model/inference vendors, not other AI gateways.

Out of scope

  • Changing core routing/governance semantics
  • Linear ticket execution by agents (tracking lives here on GitHub)

Activity

  1. vgmartinez commented on Aug 22, 2026

    @vgmartinez
    ContributorAuthor

    Follow-up (after this): embeddings endpoint expansion → #503. Route /v1/embeddings already exists; #503 is broader provider coverage + docs/catalog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions