docs: add inference providers page and improve configure model index

fern/versions/latest/pages/model-server/azure-openai.mdx

@@ -0,0 +1,72 @@
+---
+title: "Azure OpenAI"
+description: "Use models hosted on Azure OpenAI deployments"
+position: 3
+---
+
+The `azure_openai_model` server connects NeMo Gym to models hosted on [Azure OpenAI](https://learn.microsoft.com/azure/ai-services/openai/). It calls Azure's Chat Completions endpoint and converts to and from NeMo Gym's native [Responses API](https://developers.openai.com/api/reference/resources/responses/methods/create) — so, like [Inference Providers](/model-server/inference-providers), it translates between the two formats (unlike [OpenAI](/model-server/openai), which forwards Responses requests natively). Azure also requires an explicit `api-version`.
+
+<Info>
+For **training** workloads that require token IDs and log probabilities, use [vLLM](/model-server/vllm) instead. Hosted endpoints do not expose the token-level information needed for RL training.
+
+</Info>
+
+## Supported APIs
+
+This server exposes both endpoints, calling Azure's Chat Completions under the hood:
+
+- **OpenAI Responses** — `/v1/responses`
+- **OpenAI Chat Completions** — `/v1/chat/completions`
+
+## Set Your Credentials
+
+Store your values in `env.yaml` in the project root (gitignored). For Azure, `policy_model_name` is your **deployment name** and `policy_base_url` is your Azure endpoint:
+
+```yaml
+policy_base_url: https://your-resource.openai.azure.com
+policy_api_key: your-api-key
+policy_model_name: your-deployment-name
+```
+
+Azure requires an API version (for example `2024-10-21`). Set it on the `default_query.api-version` config field — see the usage example below.
+
+## Configuration Reference
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `openai_base_url` | `str` | — | **Required.** Azure OpenAI endpoint. |
+| `openai_api_key` | `str` | — | **Required.** Azure API key. |
+| `openai_model` | `str` | — | **Required.** Azure deployment name. |
+| `default_query` | `dict` | — | **Required.** Must include `api-version` (for example, `2024-10-21`). |
+| `num_concurrent_requests` | `int` | — | **Required.** Maximum concurrent requests to Azure. |
+
+<Note>
+**The model is fixed by configuration.** This server always sends the configured `openai_model` (from `policy_model_name`) to Azure. If an incoming request carries its own `model` field — as standard OpenAI-compatible clients and SDKs do — that value is **overwritten**, so you cannot switch models on a per-request basis. To run a different model, change the config and start a new server.
+
+</Note>
+
+## Usage Example
+
+### 1. Set model and environment config
+
+```bash
+environment_config="resources_servers/mcqa/configs/mcqa.yaml"
+model_config="responses_api_models/azure_openai_model/configs/azure_openai_model.yaml"
+```
+
+### 2. Start servers
+
+Pass your Azure API version on the command line:
+
+```bash
+ng_run "+config_paths=[${environment_config},${model_config}]" \
+    +policy_model.responses_api_models.azure_openai_model.default_query.api-version=2024-10-21
+```
+
+### 3. Evaluate your agent
+
+```bash
+ng_collect_rollouts +agent_name=mcqa_simple_agent \
+    +input_jsonl_fpath=resources_servers/mcqa/data/example.jsonl \
+    +output_jsonl_fpath=results/mcqa_rollouts.jsonl
+```

fern/versions/latest/pages/model-server/index.mdx

@@ -1,34 +1,41 @@
 ---
-title: "Model Server"
+title: "Configure Model"
 description: ""
 position: 1
 ---
-Model servers are stateless LLM inference endpoints. They receive a conversation and return the model's next output (text, tool calls, or code) with no memory or orchestration logic. During training, you will typically have at least one active Model server: the "policy" model being trained.
 
-Any OpenAI-compatible inference backend can serve as a Model server. NeMo Gym provides middleware to bridge format differences (e.g., converting between Chat Completions and Responses API schemas).
+Configure your model with the inference backend of your choice.
 
-Model servers implement `ResponsesAPIModel` and expose two endpoints:
+NeMo Gym uses the [Responses API](https://developers.openai.com/api/reference/resources/responses/methods/create) as its native schema because it natively represents tool calls, multi-turn conversations, and structured outputs. NeMo Gym provides middleware to automatically convert other protocols, for example, Chat Completions, into Responses format.
 
-- **`/v1/responses`** — [OpenAI Responses API](https://developers.openai.com/api/reference/resources/responses/methods/create)
-  - This is the default input/output schema for all NeMo Gym rollouts.
-- **`/v1/chat/completions`** — [OpenAI Chat Completions API](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)
+For training, use a backend that returns token IDs and log probabilities (marked with <Badge minimal outlined>training</Badge> below).
 
-## Backend Guides
-
-Guides for OpenAI and Azure OpenAI Responses API models and more are coming soon!
+## Available Backends
 
 <Cards>
 
+<Card title="OpenAI" href="/model-server/openai">
+Use OpenAI models.
+
+<Badge minimal outlined>cloud</Badge>
+</Card>
+
+<Card title="Azure OpenAI" href="/model-server/azure-openai">
+Use models hosted on Azure OpenAI deployments.
+
+<Badge minimal outlined>cloud</Badge>
+</Card>
+
+<Card title="Inference Providers" href="/model-server/inference-providers">
+Use models through a cloud API.
+
+<Badge minimal outlined>cloud</Badge>
+</Card>
+
 <Card title="vLLM" href="/model-server/vllm">
 Self-hosted inference with vLLM for maximum control.
 
-<Badge minimal outlined>self-hosted</Badge> <Badge minimal outlined>open-source</Badge>
+<Badge minimal outlined>self-hosted</Badge> <Badge minimal outlined>training</Badge>
 </Card>
 
 </Cards>
-
-## Server Configuration
-<Note>
-[Model Server Fields](/reference/configuration#model-server-fields) for server configuration syntax and fields.
-
-</Note>

fern/versions/latest/pages/model-server/inference-providers.mdx

@@ -0,0 +1,94 @@
+---
+title: "Inference Providers"
+description: "Use hosted inference providers like Fireworks, Together.ai, OpenRouter, and more for eval workloads"
+position: 4
+---
+
+The `inference_provider` server connects NeMo Gym to any hosted inference provider. The server manages the conversion to and from the Responses API: it translates incoming Responses requests to Chat Completions for the provider and converts the reply back into a Responses object — so your agent code stays the same across backends.
+
+<Info>
+For **training** workloads that require token IDs and log probabilities, use [vLLM](/model-server/vllm) instead. Hosted providers do not expose the token-level information needed for RL training.
+
+</Info>
+
+## Supported APIs
+
+- **OpenAI Responses** — `/v1/responses`
+- **OpenAI Chat Completions** — `/v1/chat/completions`
+
+## Supported Providers
+
+All provider configs live in [`responses_api_models/inference_provider/configs/`](https://github.com/NVIDIA-NeMo/Gym/tree/main/responses_api_models/inference_provider/configs).
+
+| Provider | Config | Base URL |
+|----------|--------|----------|
+| Baseten | `baseten.yaml` | `https://bridge.baseten.co/v1` |
+| DeepInfra | `deepinfra.yaml` | `https://api.deepinfra.com/v1/openai` |
+| Fireworks | `fireworks.yaml` | `https://api.fireworks.ai/inference/v1` |
+| Friendli | `friendli.yaml` | `https://inference.friendli.ai/v1` |
+| Gemini | `gemini.yaml` | `https://generativelanguage.googleapis.com/v1beta/openai` |
+| HF Inference | `hf_inference.yaml` | `https://router.huggingface.co/v1` |
+| Nebius | `nebius.yaml` | `https://api.studio.nebius.ai/v1` |
+| OpenRouter | `openrouter.yaml` | `https://openrouter.ai/api/v1` |
+| Together.ai | `together.yaml` | `https://api.together.xyz/v1` |
+
+## Set Your Credentials
+
+You need an API key and a model name from your provider; store values in `env.yaml` in the project root (gitignored):
+
+```yaml
+policy_api_key: your-api-key
+policy_model_name: nvidia/Nemotron-3-Nano-30B-A3B
+```
+
+If your provider is not in the table above, also set the base URL:
+
+```yaml
+# env.yaml
+policy_base_url: https://your-provider.com/v1
+```
+
+## Configuration Reference
+
+This table lists the server's own config fields. The `policy_*` keys you set in `env.yaml` are NeMo Gym's shared credential variables, reused across every model server; each provider config maps them onto these fields — `policy_base_url` → `base_url`, `policy_api_key` → `api_key`, and `policy_model_name` → `model`.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `base_url` | `str` | — | **Required.** Provider's OpenAI-compatible API base URL. |
+| `api_key` | `str` | — | **Required.** Provider API key. |
+| `model` | `str` | — | **Required.** Model identifier (provider-specific format). |
+| `uses_reasoning_parser` | `bool` | `false` | Parse `<think>` tags and `reasoning_content` fields from thinking models. |
+| `num_concurrent_requests` | `int` | `1000` | Maximum concurrent requests to the provider. Reduce if your provider has a lower rate limit. |
+| `extra_body` | `dict` | `{}` | Additional parameters merged into every request body. |
+
+<Note>
+**The model is fixed by configuration.** This server always sends the configured `model` (from `policy_model_name`) to the provider. If an incoming request carries its own `model` field — as standard OpenAI-compatible clients and SDKs do — that value is **overwritten**, so you cannot switch models on a per-request basis. To run a different model, change the config and start a new server.
+
+</Note>
+
+## Usage Example
+
+### 1. Set model and environment config
+
+In this example, we use Together.ai:
+
+```bash
+environment_config="resources_servers/mcqa/configs/mcqa.yaml"
+model_config="responses_api_models/inference_provider/configs/together.yaml"
+```
+
+For unlisted providers, use the generic config: `responses_api_models/inference_provider/configs/inference_provider.yaml`
+
+### 2. Start servers
+
+```bash
+ng_run "+config_paths=[${environment_config},${model_config}]"
+```
+
+### 3. Evaluate your agent
+
+```bash
+ng_collect_rollouts +agent_name=mcqa_simple_agent \
+    +input_jsonl_fpath=resources_servers/mcqa/data/example.jsonl \
+    +output_jsonl_fpath=results/mcqa_rollouts.jsonl
+```

fern/versions/latest/pages/model-server/openai.mdx

@@ -0,0 +1,72 @@
+---
+title: "OpenAI"
+description: "Use OpenAI (or any endpoint that natively serves the Responses API) with no conversion"
+position: 2
+---
+
+The `openai_model` server connects NeMo Gym to OpenAI. It forwards requests straight through with no conversion in either direction: a Responses request goes to OpenAI's [Responses API](https://developers.openai.com/api/reference/resources/responses/methods/create) and returns a Responses object, and a Chat Completions request goes to OpenAI's Chat Completions and returns a chat completion — OpenAI serves both natively.
+
+Because it is a pass-through, the Responses endpoint also works with any other backend that implements the Responses API natively — point `openai_base_url` at it. In practice that is rare: most hosted backends only expose Chat Completions and instead go through [Inference Providers](/model-server/inference-providers), which translates between the two formats.
+
+<Info>
+For **training** workloads that require token IDs and log probabilities, use [vLLM](/model-server/vllm) instead. Hosted endpoints do not expose the token-level information needed for RL training.
+
+</Info>
+
+## Supported APIs
+
+This server exposes both endpoints and forwards each directly to the upstream endpoint:
+
+- **OpenAI Responses** — `/v1/responses`
+- **OpenAI Chat Completions** — `/v1/chat/completions`
+
+## Set Your Credentials
+
+Store your values in `env.yaml` in the project root (gitignored):
+
+```yaml
+policy_base_url: https://api.openai.com/v1
+policy_api_key: your-api-key
+policy_model_name: gpt-4o-mini
+```
+
+Point `policy_base_url` at any OpenAI-compatible Responses endpoint to reuse this server with a different host.
+
+## Configuration Reference
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `openai_base_url` | `str` | — | **Required.** Base URL of an endpoint that serves the Responses API. |
+| `openai_api_key` | `str` | — | **Required.** API key for the endpoint. |
+| `openai_model` | `str` | — | **Required.** Model identifier (for example, `gpt-4o-mini`). |
+| `openai_default_headers` | `dict` | `{}` | Extra headers sent on every request. |
+| `extra_body` | `dict` | `{}` | Default parameters merged into every request body. Values set on the incoming request take precedence. |
+| `max_concurrent_requests` | `int` | `null` | Cap on in-flight upstream requests (per-process). `null` = unlimited; set it on rate-limited endpoints. |
+
+<Note>
+**The model is fixed by configuration.** This server always sends the configured `openai_model` (from `policy_model_name`) to the upstream endpoint. If an incoming request carries its own `model` field — as standard OpenAI-compatible clients and SDKs do — that value is **overwritten**, so you cannot switch models on a per-request basis. To run a different model, change the config and start a new server.
+
+</Note>
+
+## Usage Example
+
+### 1. Set model and environment config
+
+```bash
+environment_config="resources_servers/mcqa/configs/mcqa.yaml"
+model_config="responses_api_models/openai_model/configs/openai_model.yaml"
+```
+
+### 2. Start servers
+
+```bash
+ng_run "+config_paths=[${environment_config},${model_config}]"
+```
+
+### 3. Evaluate your agent
+
+```bash
+ng_collect_rollouts +agent_name=mcqa_simple_agent \
+    +input_jsonl_fpath=resources_servers/mcqa/data/example.jsonl \
+    +output_jsonl_fpath=results/mcqa_rollouts.jsonl
+```

fern/versions/latest/pages/model-server/vllm.mdx

@@ -1,7 +1,7 @@
 ---
 title: "vLLM"
 description: ""
-position: 2
+position: 5
 ---
 [vLLM](https://docs.vllm.ai/) is a popular LLM inference engine. The NeMo Gym VLLMModel server wraps vLLM's Chat Completions endpoint and converts requests and responses to NeMo Gym's native format, the OpenAI [Responses API](https://platform.openai.com/docs/api-reference/responses) schema.
 

fern/versions/main.yml

@@ -19,7 +19,7 @@ navigation:
         title: "Agent Server"
         title-source: frontmatter
       - folder: ./latest/pages/model-server
-        title: "Model Server"
+        title: "Configure Model"
         title-source: frontmatter
       - folder: ./latest/pages/resources-server
         title: "Resources Server"