fix(azure): preserve AD token refresh in v1 OpenAI client path#28625
Draft
mateo-berri wants to merge 1 commit into
Draft
fix(azure): preserve AD token refresh in v1 OpenAI client path#28625mateo-berri wants to merge 1 commit into
mateo-berri wants to merge 1 commit into
Conversation
The /openai/v1/ code path (api_version in {"v1", "latest", "preview"})
constructs a plain OpenAI/AsyncOpenAI client, but only forwarded
`api_key` from `azure_client_params`. When `enable_azure_ad_token_refresh`
is set (or any AD-only auth), `api_key` is None and the client
constructor raised "The api_key client option must be set...", breaking
every Azure call with a v1 api_version.
The OpenAI SDK (>=2.20.0) accepts a callable for `api_key` and re-invokes
it on every request via `_refresh_api_key`, so we now forward
`azure_ad_token_provider` directly — preserving the per-request token
refresh behavior of the regular AzureOpenAI client and avoiding the
expiry hole that resolving the token once at client-creation time would
introduce. Static `azure_ad_token` strings fall through to `api_key`.
For the async path we wrap the sync provider returned by azure-identity
in an async function since AsyncOpenAI expects `Callable[[], Awaitable[str]]`.
Fixes #27945
https://claude.ai/code/session_01UnzrDSFUUgp5T2wRoPMxq5
|
|
Contributor
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the Azure AD authentication regression in 1.84.0 reported in #27945. When
api_versionisv1,latest, orpreview, litellm routes Azure calls through a plainOpenAI/AsyncOpenAIclient (to hit/openai/v1/). That path only forwardedapi_keyfromazure_client_params, so any AD-only configuration — includingenable_azure_ad_token_refresh: true, the case in #27945 — hitOpenAIError: The api_key client option must be set…on every request.The OpenAI SDK (>= 2.20.0, our minimum) already accepts a
Callable[[], str]asapi_keyand re-invokes it on every request via_refresh_api_key. We now forwardazure_ad_token_providerdirectly — preserving the per-request refresh behavior of the regularAzureOpenAIclient and avoiding the expiry hole that resolving the token once at client-creation time would introduce.litellm/llms/azure/common_utils.py: in the_is_azure_v1_api_versionbranch, setapi_keytoazure_client_params["api_key"] or azure_ad_token_provider or azure_ad_token. For the async client, wrap the sync provider returned byazure-identityin an async function (AsyncOpenAIrequiresCallable[[], Awaitable[str]]).azure_ad_tokenstrings fall through toapi_keyas before.api_keystill wins (test added).Test plan
tests/test_litellm/llms/azure/test_azure_common_utils.py, parametrized overv1/latest/preview:test_azure_v1_api_with_azure_ad_token_provider— provider forwarded asapi_key; SDK stores it as_api_key_provider(sync) or wrapped async provider (async).test_azure_v1_api_async_token_provider_resolves_to_current_token— async wrapper calls the underlying provider on each invocation (token rotation honored).test_azure_v1_api_with_static_azure_ad_token— static AD token used asapi_key.test_azure_v1_api_key_wins_over_ad_token— explicitapi_keytakes precedence;_api_key_provideris None.Fixes #27945
https://claude.ai/code/session_01UnzrDSFUUgp5T2wRoPMxq5
Generated by Claude Code