merge main

[Sameerlite]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

---

Note

Medium Risk
Medium risk because it changes proxy runtime startup options (new --run_granian path) and expands guardrail telemetry emitted on failures, which could affect production server behavior and observability payloads.

Overview
Proxy runtime: Adds a new litellm proxy launch mode using Granian via --run_granian (plus --granian_threads), wires in SSL handling for Granian, and pins granian==2.5.7 in proxy extras; the non-root Docker image now installs npm alongside nodejs.

Guardrail observability: Extends standard guardrail logging/tracing to surface guardrail_status, provider guardrail_action, and pre-redaction violation_categories, and ensures OpenTelemetry emits guardrail spans even for pre-call guardrail blocks by pulling guardrail info from request_data.metadata.

CI/tests/maintenance: Forwards LITELLM_LICENSE into CircleCI E2E runs to exercise premium flows, adds a large authz behavior matrix suite for /team/* endpoints (plus improved seeding/cleanup helpers), bumps dev black to 26.3.1 with liccheck allowlist updates, and includes minor dependency/formatting churn (e.g. package-lock.json, trailing commas/whitespace).

^{Reviewed by Cursor Bugbot for commit d04373f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Too many files changed for review. (116 files found, 100 file limit)

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
3 out of 4 committers have signed the CLA.

✅ ryan-crabbe-berri
✅ yuneng-berri
✅ harish-berri
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 81.17647% with 16 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_cli.py	69.44%	11 Missing ⚠️
litellm/litellm_core_utils/litellm_logging.py	0.00%	2 Missing ⚠️
litellm/proxy/proxy_server.py	0.00%	1 Missing ⚠️
...proxy/spend_tracking/spend_management_endpoints.py	0.00%	1 Missing ⚠️
litellm/proxy/utils.py	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Unreachable exception handler makes pytest.skip dead code
  • Removed InternalServerError from the first except tuple so the dedicated handler runs and pytest.skip is reachable again for transient Google Maps 500s.
• ✅ Fixed: Granian path unconditionally calls uvicorn init helper
  • Included run_granian in the running_uvicorn check and moved _get_default_unvicorn_init_args (which imports uvicorn) inside the running_uvicorn branch so Granian-only installs no longer hit ImportError.

Preview (d04373f4ce)

diff --git a/litellm/proxy/proxy_cli.py b/litellm/proxy/proxy_cli.py
--- a/litellm/proxy/proxy_cli.py
+++ b/litellm/proxy/proxy_cli.py
@@ -1191,20 +1191,23 @@
             )
             return
 
-        running_uvicorn = run_gunicorn is False and run_hypercorn is False
-        uvicorn_args = ProxyInitializationHelpers._get_default_unvicorn_init_args(
-            host=host,
-            port=port,
-            log_config=log_config,
-            keepalive_timeout=keepalive_timeout,
-            timeout_worker_healthcheck=(
-                timeout_worker_healthcheck if running_uvicorn else None
-            ),
+        running_uvicorn = (
+            run_gunicorn is False and run_hypercorn is False and run_granian is False
         )
-        # Optional: recycle uvicorn workers after N requests
-        if max_requests_before_restart is not None:
-            uvicorn_args["limit_max_requests"] = max_requests_before_restart
-        if run_gunicorn is False and run_hypercorn is False and run_granian is False:
+        if running_uvicorn:
+            # Only build uvicorn args (which imports uvicorn) when actually
+            # running uvicorn — Granian/Hypercorn/Gunicorn users may not have
+            # uvicorn installed.
+            uvicorn_args = ProxyInitializationHelpers._get_default_unvicorn_init_args(
+                host=host,
+                port=port,
+                log_config=log_config,
+                keepalive_timeout=keepalive_timeout,
+                timeout_worker_healthcheck=timeout_worker_healthcheck,
+            )
+            # Optional: recycle uvicorn workers after N requests
+            if max_requests_before_restart is not None:
+                uvicorn_args["limit_max_requests"] = max_requests_before_restart
             if ssl_certfile_path is not None and ssl_keyfile_path is not None:
                 print(  # noqa
                     f"\033[1;32mLiteLLM Proxy: Using SSL with certfile: {ssl_certfile_path} and keyfile: {ssl_keyfile_path}\033[0m\n"  # noqa

diff --git a/tests/local_testing/test_amazing_vertex_completion.py b/tests/local_testing/test_amazing_vertex_completion.py
--- a/tests/local_testing/test_amazing_vertex_completion.py
+++ b/tests/local_testing/test_amazing_vertex_completion.py
@@ -38,7 +38,6 @@
 )
 from litellm.llms.vertex_ai.vertex_llm_base import VertexBase
 
-
 litellm.num_retries = 3
 litellm.cache = None
 user_message = "Write a short poem about the sky"
@@ -1104,9 +1103,7 @@
             {
                 "content": {
                     "role": "model",
-                    "parts": [
-                        {
-                            "text": """{
+                    "parts": [{"text": """{
                             "recipes": [
                                 {"recipe_name": "Chocolate Chip Cookies"},
                                 {"recipe_name": "Oatmeal Raisin Cookies"},
@@ -1114,9 +1111,7 @@
                                 {"recipe_name": "Sugar Cookies"},
                                 {"recipe_name": "Snickerdoodles"}
                             ]
-                            }"""
-                        }
-                    ],
+                            }"""}],
                 },
                 "finishReason": "STOP",
                 "safetyRatings": [
@@ -4223,11 +4218,11 @@
             )
         print(f"Response: {response.model_dump_json(indent=4)}")
         assert response.choices[0].message.content is not None
-    except (litellm.RateLimitError, litellm.InternalServerError):
-        # Transient Vertex-side failures (rate limiting, 500 INTERNAL from the
-        # Google Maps grounding backend) are not LiteLLM bugs — don't fail CI.
+    except litellm.RateLimitError:
         pass
     except litellm.InternalServerError:
+        # Transient 500 INTERNAL from the Google Maps grounding backend is not a
+        # LiteLLM bug — skip (rather than pass) so CI still surfaces upstream flakes.
         pytest.skip(
             "Google Maps Platform returned a transient 500 (upstream flake); skipping."
         )

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit d04373f. Configure here.}

[cursor]

Unreachable exception handler makes pytest.skip dead code

Medium Severity

The except (RateLimitError, InternalServerError) clause silently catches InternalServerError, making the subsequent except InternalServerError block unreachable. This means transient 500s are now silently ignored instead of causing a pytest.skip, reducing CI visibility for upstream flakes.

 

^{Reviewed by Cursor Bugbot for commit d04373f. Configure here.}

[cursor]

Granian path unconditionally calls uvicorn init helper

Medium Severity

running_uvicorn is computed as run_gunicorn is False and run_hypercorn is False without checking run_granian, so it's incorrectly True when using Granian. More importantly, _get_default_unvicorn_init_args() is called unconditionally (line 1195), which internally does import uvicorn. If a user installs only granian (as the error message on line 911 suggests is valid with pip install granian), this crashes with an ImportError before reaching the Granian branch.

 

^{Reviewed by Cursor Bugbot for commit d04373f. Configure here.}