fix(tests): unblock local_testing_part1 + litellm_router_testing on openai SDK 2.34+#28087
fix(tests): unblock local_testing_part1 + litellm_router_testing on openai SDK 2.34+#28087mateo-berri wants to merge 3 commits into
Conversation
…penai SDK 2.34+ The pin in pyproject.toml was relaxed from ==2.33.0 to >=2.20.0,<3.0.0 in the componentized-proxy PR (#27557), and uv.lock now resolves to openai==2.36.0. OpenAI Python SDK 2.34 introduced Admin API Key support and rewrote credential validation: the client constructor now enforces credentials via an internal _enforce_credentials=True flag and rejects api_key="" with `OpenAIError: Missing credentials. Please pass an api_key, workload_identity, admin_api_key, or set the OPENAI_API_KEY or OPENAI_ADMIN_KEY environment variable.` Two tests built throwaway openai clients with `api_key=""` purely as placeholders to feed into LiteLLM internals (the actual upstream `.create` is patched out with a mock). With 2.34+ the constructor itself raises before any assertion runs: - tests/local_testing/test_exceptions.py::test_exception_with_headers (all 14 parametrizations across openai/azure x sync/async x call_type) - tests/local_testing/test_router.py::test_router_dynamic_cooldown_correct_retry_after_time Pass api_key="sk-test" — the value never matters because the upstream call is mocked. A third test, test_completion_perplexity_exception_on_openai_client, asserts on the wording of the upstream auth error after deleting the env vars. The SDK changed both the exception class (raised at construction time, wrapped by LiteLLM as InternalServerError instead of re-raising openai.AuthenticationError) and the message text. Rewrite to catch any exception and assert on the env-var name (PERPLEXITY_API_KEY), which is the stable substring of both the old and new error messages and is what users actually look for. Also wrap the env-var restore in try/finally so the originals are always restored on any failure path.
Greptile SummaryTest-only fixes to unblock two CI suites that started failing after the openai dependency was relaxed and the lockfile resolved to a newer version whose 2.34 release introduced stricter credential validation at client construction time.
Confidence Score: 5/5All changes are test-only; no production code is touched, and the dummy api key values are never sent over the wire since the underlying client calls are mocked. Changes are narrowly scoped to three tests: swapping empty api keys for a dummy placeholder (mechanical, safe given mocking), and rewriting one perplexity test to broaden the caught exception type and fix env-var cleanup. No production logic is affected. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/local_testing/test_exceptions.py | Two fixes: replaces api_key="" with "sk-test" in five OpenAI/AzureOpenAI client constructors (calls are mocked); rewrites test_completion_perplexity_exception_on_openai_client to broaden exception to openai.OpenAIError, check stable PERPLEXITY_API_KEY substring, and restore env vars via try/finally. |
| tests/local_testing/test_router.py | Single-line fix: api_key="" replaced with "sk-test" for a throwaway OpenAI client whose .create call is fully mocked with a 429 response. |
Reviews (2): Last reviewed commit: "fix(tests): narrow perplexity exception ..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile feedback on #28087: catching bare Exception in pytest.raises also accepts unrelated KeyError/AttributeError from setup bugs. Narrow to openai.OpenAIError — the root of the openai SDK exception tree. Every LiteLLM exception class (AuthenticationError, InternalServerError, APIError, APIConnectionError, ...) subclasses an openai.* exception, so this covers both the pre-2.34 path (openai.AuthenticationError raised at request time) and the 2.34+ path (litellm.InternalServerError wrapping the construction-time OpenAIError) without admitting unrelated exceptions.
…itellm_fix-openai-sdk-2.34-empty-key-tests
Summary
local_testing_part1andlitellm_router_testinghave been consistently red onlitellm_internal_stagingsince #27557 merged. That PR relaxed theopenaipin from==2.33.0to>=2.20.0,<3.0.0, anduv.locknow resolves toopenai==2.36.0. OpenAI Python SDK 2.34 introduced Admin API Key support and rewrote credential validation — the constructor now enforces credentials via an internal_enforce_credentials=Trueand rejectsapi_key=""with:(The
workload_identity/admin_api_key/OPENAI_ADMIN_KEYnames are the giveaway — none existed in 2.33.0.)Three tests broke as a result; this PR fixes them in the tests, not in
litellmitself, because each test was constructing a throwaway client whoseapi_keyvalue never mattered.What broke
tests/local_testing/test_exceptions.py::test_exception_with_headers— all 14 parametrizations (openai/azure × sync/async × call_type). The test builds anopenai.{Async,}OpenAI/openai.{Async,}AzureOpenAIwithapi_key=""purely as a placeholder; the actual.createis patched out with a 429 mock.tests/local_testing/test_router.py::test_router_dynamic_cooldown_correct_retry_after_time— same pattern, single test.tests/local_testing/test_exceptions.py::test_completion_perplexity_exception_on_openai_client— different: it deletesPERPLEXITYAI_API_KEY+OPENAI_API_KEYand expectsopenai.AuthenticationErrorwith specific wording. With 2.34+ the SDK raisesOpenAIErrorat construction (wrapped by LiteLLM asInternalServerError), and the message text changed.What changed
api_key=""→api_key="sk-test"in the five constructor sites. The upstream.createis mocked, so the value is ignored.test_completion_perplexity_exception_on_openai_client: catch any exception and assert the env-var namePERPLEXITY_API_KEYis in the message — that substring is stable across both old and new SDKs and is what the user-visible error contract is really about. Wrap the env-var restore intry/finallyso the originals are always restored.Verification
All 16 previously-failing tests now pass locally with
AZURE_API_BASE+PERPLEXITYAI_API_KEY+OPENAI_API_KEYset (matching CI env):Test plan
local_testing_part1is greenlitellm_router_testingis greenNote
Low Risk
Low risk: changes are confined to test code to accommodate stricter OpenAI SDK credential validation; no production logic is modified.
Overview
Fixes failing local tests on OpenAI Python SDK 2.34+ by avoiding
api_key=""placeholder clients and by loosening Perplexity missing-credentials assertions.Tests now construct OpenAI/Azure clients with a dummy non-empty key (
"sk-test"), andtest_completion_perplexity_exception_on_openai_clientusespytest.raises(openai.OpenAIError)+try/finallyenv restoration, asserting the stablePERPLEXITY_API_KEYsubstring instead of exact error type/message.Reviewed by Cursor Bugbot for commit c6aefe1. Bugbot is set up for automated code reviews on this repo. Configure here.