fix(guardrails): return 400 not 500 when AIM blocks a request#30573
Conversation
AIM guardrail blocks raised a bare HTTPException whose type and param serialized as the literal string "None", which broke OpenAI-SDK error parsing for downstream consumers. Switching AIM to raise a ProxyException surfaced a second bug: the shared error funnel re-derived the HTTP status from a nonexistent status_code attribute and downgraded the 400 to a 500. The funnel now honors an already-normalized ProxyException rather than rebuilding it, and ProxyException is excluded from llm_exceptions alerting so a content-policy block no longer pages on-call as an LLM API failure Resolves LIT-3751
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes two compounding bugs that caused AIM guardrail blocks to return HTTP 500 with a malformed OpenAI error body (
Confidence Score: 5/5Safe to merge — the change is a targeted fix to two tightly scoped bugs (wrong exception type in guardrail hooks, wrong status re-derivation in the error funnel) with no impact on the happy path. All three AIM rejection paths are covered by dedicated regression tests, the funnel fix has its own unit test asserting the exact wire body, and the alerting/logging classifier changes are verified by two new test classes. The No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/aim/aim.py | All three rejection paths (input block, output block, multimodal anonymize) now raise ProxyException via the shared _rejection helper instead of bare HTTPException, producing conformant type/param fields on the wire. The previous P2 about the multimodal path is resolved in this PR. |
| litellm/proxy/common_request_processing.py | Adds an early-exit branch in _handle_llm_api_exception that merges response headers into an already-normalized ProxyException and re-raises it verbatim, preventing the status from being re-derived from a nonexistent status_code attribute and defaulted to 500. |
| litellm/proxy/utils.py | Extends both post_call_failure_hook (LLM-exceptions alert guard) and _is_proxy_only_llm_api_error to include ProxyException alongside HTTPException, correctly classifying guardrail blocks as non-infra failures that still drive proxy-only logging. |
| tests/local_testing/test_aim_guardrails.py | Adds three new regression tests (input block, output block, multimodal anonymize) covering all changed rejection paths with proper mocking; updates test_block_callback to assert stronger ProxyException attributes instead of just catching HTTPException. |
| tests/test_litellm/proxy/test_common_request_processing.py | New test test_already_normalized_proxy_exception_is_honored verifies the funnel re-raises a ProxyException with its 400 intact and validates the wire body via to_dict(). |
| tests/test_litellm/proxy/test_proxy_utils.py | Two new test classes verify that ProxyException is excluded from high-severity LLM-exception alerts and is still recorded by proxy-only failure logging, matching the pre-fix HTTPException behavior. |
Reviews (2): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
The block-action fix left two AIM rejection paths raising a bare HTTPException: the multimodal anonymize rejection and the output-side block. Both serialized type and param as the literal string "None", the same malformed shape the block fix removed. Funnel all three through a shared _rejection helper so they return a conformant OpenAI error body. The output block carries content_policy_violation; the multimodal rejection stays a plain invalid_request_error because it is a usage error, not a policy violation Resolves LIT-3751
Switching AIM blocks from HTTPException to ProxyException made _is_proxy_only_llm_api_error return False for them, so _handle_logging_proxy_only_error was skipped and the blocked prompt was dropped from the configured failure loggers. Classify ProxyException as a proxy-only error alongside HTTPException so guardrail blocks are recorded again, matching the prior behavior. The llm_exceptions alert suppression is a separate check and stays in place Resolves LIT-3751
…itellm_lit3751_aim_block_status_passthrough
|
@greptileai re review |
b5fcd85
into
litellm_internal_staging
…I#30573) * fix(guardrails): return 400 not 500 when AIM blocks a request AIM guardrail blocks raised a bare HTTPException whose type and param serialized as the literal string "None", which broke OpenAI-SDK error parsing for downstream consumers. Switching AIM to raise a ProxyException surfaced a second bug: the shared error funnel re-derived the HTTP status from a nonexistent status_code attribute and downgraded the 400 to a 500. The funnel now honors an already-normalized ProxyException rather than rebuilding it, and ProxyException is excluded from llm_exceptions alerting so a content-policy block no longer pages on-call as an LLM API failure Resolves LIT-3751 * fix(guardrails): route all AIM rejection paths through ProxyException The block-action fix left two AIM rejection paths raising a bare HTTPException: the multimodal anonymize rejection and the output-side block. Both serialized type and param as the literal string "None", the same malformed shape the block fix removed. Funnel all three through a shared _rejection helper so they return a conformant OpenAI error body. The output block carries content_policy_violation; the multimodal rejection stays a plain invalid_request_error because it is a usage error, not a policy violation Resolves LIT-3751 * fix(guardrails): record AIM ProxyException blocks in failure logs Switching AIM blocks from HTTPException to ProxyException made _is_proxy_only_llm_api_error return False for them, so _handle_logging_proxy_only_error was skipped and the blocked prompt was dropped from the configured failure loggers. Classify ProxyException as a proxy-only error alongside HTTPException so guardrail blocks are recorded again, matching the prior behavior. The llm_exceptions alert suppression is a separate check and stays in place Resolves LIT-3751 * style(guardrails): use str | None over Optional[str] in AIM _rejection * style(guardrails): collapse AIM _rejection signature per black
Linear ticket
Resolves LIT-3751
Summary
AIM guardrail rejections now return HTTP 400 with a well-formed OpenAI error body. Previously a block raised a bare
HTTPExceptionwhosetypeandparamserialized as the literal string"None", which broke OpenAI-SDK error parsing for downstream consumers (e.g. Google ADK). Making AIM raise aProxyExceptionfixed the body but exposed a second bug: the shared error funnel (_handle_llm_api_exception) re-derived the HTTP status from a nonexistentstatus_codeattribute (ProxyExceptioncarries its status in.code) and downgraded the 400 to a 500. The funnel now honors an already-normalizedProxyExceptioninstead of rebuilding it, every AIM rejection path returns a conformant body, and the error-classification helpers inpost_call_failure_hooktreatProxyExceptioncorrectly: excluded fromllm_exceptionspaging (a policy block is not an infra failure) but still recorded by proxy-only failure logging.Type
🐛 Bug Fix
Changes
litellm/proxy/guardrails/guardrail_hooks/aim/aim.py: all three AIM rejection paths (input block, output block, and the multimodal anonymize rejection) raise aProxyExceptionvia a shared_rejectionhelper instead of a bareHTTPException, sotype/paramare no longer the literal string"None". The two real blocks carryopenai_code="content_policy_violation"; the multimodal rejection stays a plaininvalid_request_errorbecause it is a usage error, not a policy violationlitellm/proxy/common_request_processing.py:_handle_llm_api_exceptionre-raises an already-normalizedProxyException(merging response headers) instead of re-deriving its status fromstatus_codeand clobbering it to 500. This honors the deliberate status for every hook that raises aProxyException, not just AIMlitellm/proxy/utils.py: inpost_call_failure_hook,ProxyExceptionis excluded from the High-severityllm_exceptionsalert alongsideHTTPException(user-facing errors, not infra failures), and_is_proxy_only_llm_api_errornow classifiesProxyExceptionas a proxy-only error so guardrail blocks are still recorded by the configured failure loggers (restoring the pre-changeHTTPExceptionbehavior). These two checks are independentDeferred on purpose: serializing
openai_codeonto the wire soerror.codereadscontent_policy_violationinstead of400.openai_codeis currently write-only, and changingProxyException.to_dict()would shift provider error codes at the re-wrap sites and desync the hand-built streaming error frame, with no consumer reading it today. Worth its own change if a client needs to branch on the semantic code.Proof of Fix
Local proxy with the AIM guardrail pointed at a mock Aim
analyzeendpoint (real Aim is enterprise-gated); the clean request hits real groq.Blocked request (prompt contains the configured trigger), now HTTP 400 with a conformant body:
```
$ curl -s -w "\nHTTP %{http_code}\n" -X POST http://localhost:4000/v1/chat/completions
-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "content-type: application/json"
-d '{"model":"groq-llama","messages":[{"role":"user","content":"My name is Leroy Jenkins"}]}'
{"error":{"message":""Leroy Jenkins" detected as name","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400
```
Clean request still succeeds against the real upstream:
```
$ curl -s -w "\nHTTP %{http_code}\n" -X POST http://localhost:4000/v1/chat/completions
-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "content-type: application/json"
-d '{"model":"groq-llama","messages":[{"role":"user","content":"Say hi in 3 words"}]}'
{"id":"chatcmpl-...","model":"groq-llama","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello to you","role":"assistant"}}],...}
HTTP 200
```
Before this change the same blocked request returned
HTTP 500(status downgraded by the funnel), and onmainthe body carried"type":"None","param":"None".Test plan
ProxyExceptionwith its 400 intact and asserts the wire body viato_dict(); fails (500) without the funnel fixProxyExceptionandHTTPExceptiondo not page; a genuineExceptionstill doesProxyExceptionon an LLM route still drives proxy-only failure logging; a rawExceptiondoes not; fails without the classifier fixProxyExceptionwith the correcttype/param/code/openai_codeProxyExceptionwithcontent_policy_violationinvalid_request_error(400) and is NOT labeled a policy violation