Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions images/registry/providers/watsonx.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions trustgate/concepts/prompt-caching.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ the request marks.
| **OpenRouter** | Markers for `anthropic/*`, `google/gemini*`, `qwen/*` and `openai/` GPT-5.6 and later. Other models cache as their upstream does. |
| **Mistral** | `prompt_cache_key`. |
| **DeepSeek, Groq, xAI, Google AI Studio, Vertex AI** | Nothing. Whatever caching the provider does happens on its own. |
| **Cohere, Cerebras, OpenAI Compatible** | Nothing the gateway can pass on. Cached tokens are reported when the provider returns them. |
| **Cohere, Cerebras, IBM watsonx.ai, OpenAI Compatible** | Nothing the gateway can pass on. Cached tokens are reported when the provider returns them. |

A provider decides for itself how long a prefix has to be before it caches it.
A marker on a shorter prefix is accepted and ignored.
Expand Down Expand Up @@ -66,7 +66,7 @@ On a cross-format request, the target gets the part of that intent it accepts:
| **OpenRouter**, `google/gemini*`, `qwen/*`, `openai/` GPT-5.6 and later | `cache_control` on system and message parts, 5m only. |
| **OpenRouter**, other models | No cache fields. |
| **Mistral** | `prompt_cache_key`. |
| **Google AI Studio, Vertex AI, Groq, DeepSeek, xAI, Cohere, Cerebras, OpenAI Compatible** | No cache fields. |
| **Google AI Studio, Vertex AI, Groq, DeepSeek, xAI, Cohere, Cerebras, IBM watsonx.ai, OpenAI Compatible** | No cache fields. |

Markers on assistant turns and images do not reach OpenAI Responses, and no
target in the OpenRouter rows takes tool markers.
Expand Down
54 changes: 54 additions & 0 deletions trustgate/registry/models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Some providers do not authenticate with a single key:
| **Azure Foundry** | The Azure endpoint, an **API surface**, and an API key, a service principal (tenant, client ID, client secret), or **Managed identity**. See the note below — **API version** applies to only one of the four surfaces. |
| **Amazon Bedrock** | **Access key** — access key ID, secret access key and region, plus a session token for temporary credentials — or **Assume role** with a role ARN. An application with a Bedrock entry also answers native [Bedrock Runtime](/trustgate/endpoints/bedrock-runtime) calls from AWS SDKs. |
| **Vertex AI** | A GCP service-account JSON, uploaded or pasted. Project ID is read from the file. |
| **IBM watsonx.ai** | An IBM Cloud API key, the region of the watsonx.ai Runtime and a project ID. The gateway trades the key for IAM tokens and renews them itself. See [Setting up IBM watsonx.ai](#setting-up-ibm-watsonx-ai). |
| **Custom Model** | A base URL, and whatever that endpoint expects, including nothing. **Custom header** takes a Header Name and a Value. |

<Note>
Expand Down Expand Up @@ -100,6 +101,58 @@ a warning naming each missing permission. A change in IAM takes effect within
about a minute, or immediately after **Test connection**.
</Note>

### Setting up IBM watsonx.ai

An IBM watsonx.ai entry calls the watsonx.ai Runtime in one IBM Cloud region.
It authenticates with an IBM Cloud API key: the gateway exchanges the key for
an IAM bearer token, reuses that token across calls and fetches a new one
before it expires, so nothing needs rotating every hour.

Set up three things in IBM Cloud first:

1. **A watsonx.ai Runtime instance** in the region you will call. The free Lite
plan is enough to try the integration.
2. **A project with that instance associated.** In the project, open
**Manage → Services & integrations → Associate service**. Without the
association, calls fail even with a valid key. Copy the project ID from
**Manage → General**.
3. **An API key**, from **IBM Cloud → Manage → Access (IAM) → API keys**. The
key acts as the user or service ID that owns it, so that identity must be a
collaborator on the project.

| Field | What it is |
|---|---|
| **IBM Cloud API key** | The key from step 3. The gateway sends it only to IBM Cloud IAM (`iam.cloud.ibm.com`); calls to watsonx.ai carry the token. |
| **Region** | Where the Runtime instance lives: Dallas (`us-south`), Frankfurt (`eu-de`), London (`eu-gb`), Tokyo (`jp-tok`), Sydney (`au-syd`) or Toronto (`ca-tor`). |
| **Project ID** | The project from step 2. Every call runs, and is billed, in this project. A project named in a request body is ignored. |
| **API version** (Advanced) | The watsonx.ai API version, as a `YYYY-MM-DD` date. Defaults to `2025-10-01`. |
| **Base URL** (Advanced) | Replaces the regional endpoint, for a private or dedicated one. |

Clients call the entry in the dialect they already speak, as described in
[Connect](/trustgate/llm/connect), and TrustGate translates to the watsonx.ai
chat API. Streaming and tool calls work. Embeddings, images and audio are not
served. Model names are watsonx.ai's own, such as `ibm/granite-4-h-small` or
`meta-llama/llama-3-3-70b-instruct`.

**Models.** The list is the region's own. Regions serve different models, so a
Frankfurt entry lists fewer than a Dallas one. Withdrawn models are left out and
deprecated ones are labelled.

**Pricing.** List prices come from models.dev, which publishes them for only
some watsonx.ai models. The others are listed without a price, and their
traffic is recorded without a cost until you add a
[model override](#contract-pricing): `ibm/*` covers the Granite family.

<Note>
**Test connection** checks the API key and the region: it obtains an IAM token
and lists the region's models with it. It does not check the project ID. A
wrong project, or one without the Runtime associated, shows up as IBM's error on
the first call.
</Note>

Not supported yet: deployment spaces (`space_id`), the IBM Cloud private IAM
endpoint, and the Mumbai region, which runs on AWS with its own sign-in.

## Contract pricing

Cost in Analytics, on every trace, and in any budget policy is only as true as
Expand Down Expand Up @@ -161,3 +214,4 @@ routing change and nothing else.
| <img src="/images/registry/providers/openrouter.svg" alt="" width="18" height="18" /> **OpenRouter** | `openrouter` | openai | API key. |
| <img src="/images/registry/providers/cohere.svg" alt="" width="18" height="18" /> **Cohere** | `cohere` | cohere | API key. |
| **Moonshot AI** | `moonshot` | openai | API key. |
| <img src="/images/registry/providers/watsonx.svg" alt="" width="18" height="18" /> **IBM watsonx.ai** | `watsonx` | openai | IBM Cloud API key, region and project ID; optional API version and base URL. See [Setting up IBM watsonx.ai](#setting-up-ibm-watsonx-ai). |
Loading