Skip to content

fix(tests): unblock local_testing_part1 + litellm_router_testing on openai SDK 2.34+#28087

Open
mateo-berri wants to merge 3 commits into
litellm_internal_stagingfrom
litellm_fix-openai-sdk-2.34-empty-key-tests
Open

fix(tests): unblock local_testing_part1 + litellm_router_testing on openai SDK 2.34+#28087
mateo-berri wants to merge 3 commits into
litellm_internal_stagingfrom
litellm_fix-openai-sdk-2.34-empty-key-tests

Conversation

@mateo-berri

@mateo-berri mateo-berri commented May 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

local_testing_part1 and litellm_router_testing have been consistently red on litellm_internal_staging since #27557 merged. That PR relaxed the openai pin from ==2.33.0 to >=2.20.0,<3.0.0, and uv.lock now resolves to openai==2.36.0. OpenAI Python SDK 2.34 introduced Admin API Key support and rewrote credential validation — the constructor now enforces credentials via an internal _enforce_credentials=True and rejects api_key="" with:

openai.OpenAIError: Missing credentials. Please pass an api_key, workload_identity, admin_api_key, or set the OPENAI_API_KEY or OPENAI_ADMIN_KEY environment variable.

(The workload_identity / admin_api_key / OPENAI_ADMIN_KEY names are the giveaway — none existed in 2.33.0.)

Three tests broke as a result; this PR fixes them in the tests, not in litellm itself, because each test was constructing a throwaway client whose api_key value never mattered.

What broke

  • tests/local_testing/test_exceptions.py::test_exception_with_headers — all 14 parametrizations (openai/azure × sync/async × call_type). The test builds an openai.{Async,}OpenAI / openai.{Async,}AzureOpenAI with api_key="" purely as a placeholder; the actual .create is patched out with a 429 mock.
  • tests/local_testing/test_router.py::test_router_dynamic_cooldown_correct_retry_after_time — same pattern, single test.
  • tests/local_testing/test_exceptions.py::test_completion_perplexity_exception_on_openai_client — different: it deletes PERPLEXITYAI_API_KEY + OPENAI_API_KEY and expects openai.AuthenticationError with specific wording. With 2.34+ the SDK raises OpenAIError at construction (wrapped by LiteLLM as InternalServerError), and the message text changed.

What changed

  • api_key=""api_key="sk-test" in the five constructor sites. The upstream .create is mocked, so the value is ignored.
  • test_completion_perplexity_exception_on_openai_client: catch any exception and assert the env-var name PERPLEXITY_API_KEY is in the message — that substring is stable across both old and new SDKs and is what the user-visible error contract is really about. Wrap the env-var restore in try/finally so the originals are always restored.

Verification

All 16 previously-failing tests now pass locally with AZURE_API_BASE + PERPLEXITYAI_API_KEY + OPENAI_API_KEY set (matching CI env):

============================== 16 passed in 6.29s ==============================

Test plan

  • CI local_testing_part1 is green
  • CI litellm_router_testing is green

Note

Low Risk
Low risk: changes are confined to test code to accommodate stricter OpenAI SDK credential validation; no production logic is modified.

Overview
Fixes failing local tests on OpenAI Python SDK 2.34+ by avoiding api_key="" placeholder clients and by loosening Perplexity missing-credentials assertions.

Tests now construct OpenAI/Azure clients with a dummy non-empty key ("sk-test"), and test_completion_perplexity_exception_on_openai_client uses pytest.raises(openai.OpenAIError) + try/finally env restoration, asserting the stable PERPLEXITY_API_KEY substring instead of exact error type/message.

Reviewed by Cursor Bugbot for commit c6aefe1. Bugbot is set up for automated code reviews on this repo. Configure here.

…penai SDK 2.34+

The pin in pyproject.toml was relaxed from ==2.33.0 to >=2.20.0,<3.0.0
in the componentized-proxy PR (#27557), and uv.lock now resolves to
openai==2.36.0. OpenAI Python SDK 2.34 introduced Admin API Key support
and rewrote credential validation: the client constructor now enforces
credentials via an internal _enforce_credentials=True flag and rejects
api_key="" with `OpenAIError: Missing credentials. Please pass an
api_key, workload_identity, admin_api_key, or set the OPENAI_API_KEY
or OPENAI_ADMIN_KEY environment variable.`

Two tests built throwaway openai clients with `api_key=""` purely as
placeholders to feed into LiteLLM internals (the actual upstream `.create`
is patched out with a mock). With 2.34+ the constructor itself raises
before any assertion runs:

- tests/local_testing/test_exceptions.py::test_exception_with_headers
  (all 14 parametrizations across openai/azure x sync/async x call_type)
- tests/local_testing/test_router.py::test_router_dynamic_cooldown_correct_retry_after_time

Pass api_key="sk-test" — the value never matters because the upstream
call is mocked.

A third test, test_completion_perplexity_exception_on_openai_client,
asserts on the wording of the upstream auth error after deleting the
env vars. The SDK changed both the exception class (raised at
construction time, wrapped by LiteLLM as InternalServerError instead of
re-raising openai.AuthenticationError) and the message text. Rewrite
to catch any exception and assert on the env-var name (PERPLEXITY_API_KEY),
which is the stable substring of both the old and new error messages
and is what users actually look for. Also wrap the env-var restore in
try/finally so the originals are always restored on any failure path.
@greptile-apps

greptile-apps Bot commented May 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Test-only fixes to unblock two CI suites that started failing after the openai dependency was relaxed and the lockfile resolved to a newer version whose 2.34 release introduced stricter credential validation at client construction time.

  • Replaces empty string api keys with a dummy placeholder in five OpenAI/AzureOpenAI client constructors; the actual network calls are patched with mock responses so the placeholder value is never used.
  • Rewrites test_completion_perplexity_exception_on_openai_client to catch openai.OpenAIError (covering both the old AuthenticationError path and the new construction-time error wrapped by LiteLLM as InternalServerError), asserts a stable env-var name substring in the message, and moves env-var restoration to a try/finally block that was previously absent in some exception branches.

Confidence Score: 5/5

All changes are test-only; no production code is touched, and the dummy api key values are never sent over the wire since the underlying client calls are mocked.

Changes are narrowly scoped to three tests: swapping empty api keys for a dummy placeholder (mechanical, safe given mocking), and rewriting one perplexity test to broaden the caught exception type and fix env-var cleanup. No production logic is affected.

No files require special attention.

Important Files Changed

Filename Overview
tests/local_testing/test_exceptions.py Two fixes: replaces api_key="" with "sk-test" in five OpenAI/AzureOpenAI client constructors (calls are mocked); rewrites test_completion_perplexity_exception_on_openai_client to broaden exception to openai.OpenAIError, check stable PERPLEXITY_API_KEY substring, and restore env vars via try/finally.
tests/local_testing/test_router.py Single-line fix: api_key="" replaced with "sk-test" for a throwaway OpenAI client whose .create call is fully mocked with a 429 response.

Reviews (2): Last reviewed commit: "fix(tests): narrow perplexity exception ..." | Re-trigger Greptile

Comment thread tests/local_testing/test_exceptions.py
@codecov

codecov Bot commented May 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Greptile feedback on #28087: catching bare Exception in pytest.raises
also accepts unrelated KeyError/AttributeError from setup bugs. Narrow
to openai.OpenAIError — the root of the openai SDK exception tree.
Every LiteLLM exception class (AuthenticationError, InternalServerError,
APIError, APIConnectionError, ...) subclasses an openai.* exception, so
this covers both the pre-2.34 path (openai.AuthenticationError raised
at request time) and the 2.34+ path (litellm.InternalServerError
wrapping the construction-time OpenAIError) without admitting unrelated
exceptions.
@mateo-berri

Copy link
Copy Markdown
Collaborator Author

@greptileai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant