Add inference provider server for hosted inference providers #1285

Goal

Enable model inference using popular hosted inference providers: Fireworks, Together.ai, Baseten, DeepInfra, Nebius, Friendli, OpenRouter, HF Inference. This provides more flexibility for users to use the inference providers they are already using today for evaluation or profiling environments.

Background

• Agents internally speak the Responses API. Model servers translate to/from whatever the upstream provider speaks.
• vllm_model already contains a "Responses ↔ Chat Completions" converter (VLLMConverter), but it's bundled with vLLM-specific logic (token IDs, /tokenize endpoint, chat_template_kwargs, vLLM error parsing) that doesn't apply to hosted providers.
• All the target inference providers expose standard OpenAI-compatible /v1/chat/completions endpoints.

Deliverables

1. Extract shared converter (small refactor)

Move VLLMConverter (Responses ↔ Chat Completions translation) to a shared location (e.g., nemo_gym/converters/). Both vllm_model and the new chat_completions_model import from there.

2. New server: chat_completions_model

A clean model server for any provider that speaks /v1/chat/completions. No vLLM-specific logic.

• Config: base_url, api_key, model, uses_reasoning_parser, num_concurrent_requests, extra_body
• responses(): uses shared converter to translate Responses → Chat Completions, calls upstream, converts back
• chat_completions(): straightforward passthrough with semaphore-based rate limiting
• Convenience YAML configs per provider:
  • configs/fireworks.yaml
  • configs/together.yaml
  • configs/openrouter.yaml
  • configs/deepinfra.yaml
  • configs/nebius.yaml
  • configs/friendli.yaml
  • configs/baseten.yaml
  • configs/hf_inference.yaml

3. Testing & docs

• Integration tests per provider (at minimum: confirm a basic chat completion round-trips correctly)
• Docs page: "Using External Inference Providers" — which server to pick, example configs, provider-specific notes

Target state (model servers)

Non-goals

• No changes to vllm_model or openai_model behavior
• Claude/Gemini native API support tracked in separate issues:
  • feat: claude model support #1146
  • feat: gemini model support #1147
• No deprecations in this issue