test(gemini): de-flake test_gemini_image_size_limit_exceeded#28039
Conversation
Mock the image fetch instead of downloading a 50MB+ image from upload.wikimedia.org. The runner was intermittently rate-limited (HTTP 429), so the code raised "Unable to fetch image ... Status code: 429" and the size-limit assertions failed even though pytest.raises(litellm.ImageFetchError) still matched. Mirror the established LargeImageClient pattern in tests/test_litellm/litellm_core_utils/test_image_handling.py: stub litellm.module_level_client with a response whose Content-Length exceeds the 50MB limit and bypass SSRF validation, so the size-limit rejection path is exercised deterministically with no external network dependency.
Greptile SummaryThis PR de-flakes
Confidence Score: 5/5Safe to merge — a single test file is changed, with no modifications to production code. The change removes an external network dependency causing intermittent CI failures and replaces it with a deterministic mock. The tested logic path and assertions are identical to the original. The only minor observation is that the mock allocates 100 MB of memory unnecessarily, but this is a pre-existing pattern mirrored from elsewhere in the test suite and does not affect correctness. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/llm_translation/test_gemini.py | De-flaked test_gemini_image_size_limit_exceeded by replacing an external Wikimedia URL with a fully mocked HTTP client; assertions and the tested code path are unchanged. |
Reviews (1): Last reviewed commit: "test(gemini): de-flake test_gemini_image..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
| def get(self, url, follow_redirects=True): | ||
| size_bytes = int(100 * 1024 * 1024) # 100MB > 50MB default limit | ||
| return Response( | ||
| status_code=200, | ||
| headers={ | ||
| "Content-Type": "image/jpeg", | ||
| "Content-Length": str(size_bytes), | ||
| }, | ||
| content=b"x" * size_bytes, | ||
| request=Request("GET", url), | ||
| ) |
There was a problem hiding this comment.
The
content=b"x" * size_bytes expression eagerly allocates 100 MB of memory, even though _process_image_response raises ImageFetchError on the Content-Length header check before ever calling iter_bytes. Since the body is never read, b"" is sufficient and avoids the allocation entirely.
| def get(self, url, follow_redirects=True): | |
| size_bytes = int(100 * 1024 * 1024) # 100MB > 50MB default limit | |
| return Response( | |
| status_code=200, | |
| headers={ | |
| "Content-Type": "image/jpeg", | |
| "Content-Length": str(size_bytes), | |
| }, | |
| content=b"x" * size_bytes, | |
| request=Request("GET", url), | |
| ) | |
| def get(self, url, follow_redirects=True): | |
| size_bytes = int(100 * 1024 * 1024) # 100MB > 50MB default limit | |
| return Response( | |
| status_code=200, | |
| headers={ | |
| "Content-Type": "image/jpeg", | |
| "Content-Length": str(size_bytes), | |
| }, | |
| content=b"", # Content-Length triggers rejection before body is read | |
| request=Request("GET", url), | |
| ) |
…itellm_/nostalgic-johnson-eeb7c3
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high mode and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 2130bdc. Configure here.
| "safe_get", | ||
| lambda client, url, **kw: client.get(url, follow_redirects=True), | ||
| ) | ||
| monkeypatch.setattr(litellm, "module_level_client", LargeImageClient()) |
There was a problem hiding this comment.
Mocked image download code is never invoked by Gemini path
High Severity
The monkeypatches of image_handling.safe_get and litellm.module_level_client are never exercised because the Gemini transformation path in _process_gemini_media passes https:// URLs with recognized extensions (like .jpg) directly as file_data references without downloading them. The URL https://example.com/large-image.jpg matches the third branch in _process_gemini_media (via _get_image_mime_type_from_url returning "image/jpeg"), so convert_url_to_base64 is never called. The test does not actually exercise the size-limit rejection path it claims to test.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 2130bdc. Configure here.
The Content-Length header check in _process_image_response rejects the image before the body is streamed, so the mock body never needs to be materialized. Use an empty body instead of b"x" * 100MB (addresses greptile/cursor review feedback).
ec2f3aa
into
litellm_internal_staging


Summary
tests/llm_translation/test_gemini.py::test_gemini_image_size_limit_exceededis flaky in CI. It downloaded a 50MB+ image fromupload.wikimedia.orgto assert the image exceedsMAX_IMAGE_URL_DOWNLOAD_SIZE_MB. Wikimedia intermittently rate-limited the runner (HTTP 429), so the code raisedImageFetchError("Unable to fetch image ... Status code: 429").pytest.raises(litellm.ImageFetchError)still matched, butassert "Image size" in error_messagefailed.Fix
Remove the external-network dependency by mirroring the established
LargeImageClientpattern intests/test_litellm/litellm_core_utils/test_image_handling.py:litellm.module_level_clientwith a client returning a response whoseContent-Lengthreports 100MB (> 50MB default limit).image_handling.safe_get) so no DNS resolution / network call occurs.completion(model="gemini/...")call so the gemini path's size-limit rejection is still exercised end-to-end.The test now deterministically hits the size-limit path and is fully offline.
Test plan
Passes in 0.24s with no network access.
Note
Low Risk
Low risk: test-only change that removes an external network dependency by mocking HTTP image download behavior; no production logic is modified.
Overview
Makes
test_gemini_image_size_limit_exceededdeterministic by removing the real Wikimedia download and instead monkeypatching image fetching to return a mockedhttpx.Responsewith an oversizedContent-Length.The test now stubs
image_handling.safe_getandlitellm.module_level_client(via a localLargeImageClient) to force the size-limit rejection path offline, and updates the URL/assertions to validate the expected "exceeds maximum allowed size" error text.Reviewed by Cursor Bugbot for commit 8c537f7. Bugbot is set up for automated code reviews on this repo. Configure here.