Repository navigation
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Goal
Expand TrustGate’s LLM provider catalog so more teams can route/govern traffic without swapping gateways.
Already supported (pkg/infra/providers)
OpenAI, OpenAI-compatible, Anthropic, Google, Vertex, Azure, Bedrock, Mistral, Groq, DeepSeek, xAI, Cerebras, OpenRouter, Cohere.
Proposed next providers (priority candidates)
Ship as separate PRs when possible; start with OpenAI-compatible wrappers where the upstream is chat-completions shaped.
High leverage (Victor + OpenRouter gap scan)
- Moonshot / Kimi (Kimi K3) — first priority
- Fireworks AI
- Together AI
- DeepInfra — big OpenRouter traffic for open-weights; OpenAI-compatible
- Perplexity
- Novita
- Nebius
- SiliconFlow
- MiniMax
- NVIDIA NIM / NGC
- SambaNova
- Alibaba DashScope (Qwen) — OpenRouter slug
alibaba - Z.AI / Zhipu (GLM) — OpenRouter slug
z-ai - Databricks Model Serving
Also consider (OpenRouter has; lower urgency)
- AI21, Replicate, Friendli, Baseten, Upstage (Solar), BytePlus / Seed (Doubao), Baidu, Tencent, Xiaomi, StepFun, Featherless, Parasail, Inference.net
- Hugging Face Inference Endpoints only as optional
openaicompat+ custom base URL
Gaps vs OpenRouter (method)
Compared OpenRouter /api/v1/providers (~100 slugs) to TrustGate first-class clients. Many OR “providers” are niche GPU hosts or media APIs — we only promote chat LLM vendors users would BYOK directly.
Explicitly not / deprioritized
- Cloudflare — competing AI gateway positioning
- Hugging Face Inference Providers router — multi-provider router; overlaps OpenRouter
- Other gateways / proxies as “providers”
- Non-LLM OpenRouter entries (Deepgram, Fish Audio, HeyGen, Runway, Recraft, Voyage embeddings, etc.)
Acceptance criteria (per provider)
- Registered in
pkg/infra/providers/factorylocator - Client under
pkg/infra/providers/<name>following existing patterns (chat + stream where applicable) - Auth/credentials wired like peer providers
- Connection tester when the interface is supported
- Catalog / UI listing updated if required for the provider to appear in Admin
- Docs / README provider list updated
- Basic unit or integration coverage consistent with nearby providers
- Opens as PR only (never merge from automation)
Kimi K3 slice (first)
- Moonshot/Kimi client supports Kimi K3 models (chat + stream)
- Documented base URL / auth for Moonshot API
- Appears in Admin catalog as a selectable provider
Notes
- Prefer reusing
openaicompatwhen the vendor is truly OpenAI-compatible; only add a dedicated package when auth, headers, streaming, or error mapping diverge. - One provider per PR when possible.
- Focus on model/inference vendors, not other AI gateways.
Out of scope
- Changing core routing/governance semantics
- Linear ticket execution by agents (tracking lives here on GitHub)
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
vgmartinez commentedon Aug 22, 2026
Follow-up (after this): embeddings endpoint expansion → #503. Route
/v1/embeddingsalready exists; #503 is broader provider coverage + docs/catalog.