AzureOpenAIModel implementation

@xinyu-dev

A lot of enterprise users that are on ChatGPT does not use public server, but the Azure OpenAI private deployment. This endpoint requires an "default_query" and "api_version" parameters on the client init. Since it is almost certain that there will be subtle but critical API differences between the normal OpenAI endpoints and Azure OpenAI endpoints, we should separate this into a separate Responses API model called AzureOpenAIModel.

Example env.yaml

# for azure openai only
policy_api_version: preview

Example responses_api_models/openai_model/configs/openai_model.yaml

openai_model:
  responses_api_models:
    openai_model:
      entrypoint: app.py
      openai_base_url: ${policy_base_url}
      openai_api_key: ${policy_api_key}
      openai_model: ${policy_model_name}
      # for azure openai only
      default_query:
        api-version: ${policy_api_version}

Thank you to @xinyu-dev for the raise!

thanks very much Brian for your help

@xinyu-dev

A few additional notes from @xinyu-dev:

If you're using chat completion API on Azure OpenAI, you might need to use the AuzureOpenAI object.
If you're using response API on Azure OpenAI, you can use the OpenAI object.
Not all Azure OpenAi deployment might support response API. But all Azure deployment all support chat completion API at least.

Just need docs for this

@bxyu-nvidia

@bxyu-nvidia can we split docs into a separate ticket and close this if the model server code is complete

tracking docs in this ticket #665

fix: tau2 knowledge deps + repoint tau2-bench pins to stable branches (…

Navigation Menu

Repository navigation

AzureOpenAIModel implementation #17

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

AzureOpenAIModel implementation #17

Description

Activity

xinyu-dev commented on Sep 8, 2025

bxyu-nvidia commented on Sep 8, 2025

bxyu-nvidia commented on Oct 19, 2025

cwing-nvidia commented on Jan 29, 2026

cwing-nvidia commented on Feb 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions