Skip to content

chore(release): backport #30064, #29991, #30009 to stable/1.86.x and cut 1.86.5#30148

Merged
yuneng-berri merged 6 commits into
stable/1.86.xfrom
litellm_fable5_stable_1_86_x
Jun 11, 2026
Merged

chore(release): backport #30064, #29991, #30009 to stable/1.86.x and cut 1.86.5#30148
yuneng-berri merged 6 commits into
stable/1.86.xfrom
litellm_fable5_stable_1_86_x

Conversation

@mateo-berri

@mateo-berri mateo-berri commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Relevant issues

Backports #30064 (Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI), #29991 (CrowdStrike AIDR identity fix) together with its prerequisite identity-capture feature, and #30009 (batch file authorization via upload target_model_names) to stable/1.86.x, and cuts 1.86.5. Tag v1.86.4 and branch release/v1.86.4 both exist and point at this line's tip, so the tip version has shipped and the branch moves to 1.86.5 as the patch-to-be.

The CrowdStrike pair is an upgrade-monotonicity backport: stable/1.84.x and stable/1.85.x already carry both the identity-capture feature and the #29991 fix, so a customer upgrading from those lines to 1.86.x would silently lose CrowdStrike AIDR identity attribution. This replicates the stable/1.85.x treatment exactly: the feature is cherry-picked from the stable/1.84.x commit 6fc715c5bd (its staging counterpart exists only inside aggregator commit 32c88ca74f, content-verified against the aggregator's hunks), and the fix from the staging squash of #29991. The same gap exists on stable/1.87.x and stable/1.88.x.

What is included

In merge order on top of the original #30064 pick (59ae344f9e, cherry-picked from staging e15b37a18e):

Adaptation notes

All three picks carry -x footers. None are byte-verbatim because this line's files drifted; the divergences are:

Identity feature (1038a68f34): the from collections.abc import Mapping line is dropped because this branch already imports Mapping; the ai_guard_payload hunk resolves to ai_guard_payload: dict[str, Any] = {"guard_input": guard_input.model_dump(mode="json"), ...} because this branch carries the multimodal guard-input rework, which matches staging's final shape at that site byte for byte; the new tests are appended at the test file tail (this branch has newer tests mid-file). The appended test block is byte-identical to the upstream block.

#29991 (2d2fb8c131): source hunks are identical to the staging commit; the only divergence is the appended parametrized test landing at the test file tail instead of mid-file.

#30009 (5cea68f3ae): the +/- hunk lines are byte-identical to the staging commit (empty interdiff, same 19/10 numstat); only diff context differs because this line's _enforce_batch_file_model_access predates staging's team-aware access checks. The buggy reverse-mapping block the fix removes (resolve_model_name_from_model_id first-match lookup) was present on this line in the same shape, and get_models_from_unified_file_id is identical on this line and staging.

Known noise on this line

None. The targeted baseline (both touched test files, captured on the line tip before any pick) was 23/23 green.

Screenshots / Proof of Fix

Targeted tests went 23 passed (baseline) to 33 passed (post-pick, zero new failures); the 10 new tests are the ones the picks carry. Ruff, mypy, and Black are clean on the touched modules. Gauntlet adversarial verification (deep): SURVIVED on all four sub-claims (diff fidelity vs source commits, symbol resolution, picked tests passing, no broken on-tree caller), 10 agents, zero refutations

Live proxy on this branch (worktree proxy, port 4002), baseline and post-pick:

$ curl -sf "http://localhost:4002/health/liveliness"
"I'm alive!"
$ curl -s http://localhost:4002/v1/chat/completions -H "Authorization: Bearer $KEY" \
    -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Say POSTPICK-OK and nothing else"}],"max_tokens":20}'
POSTPICK-OK | model: claude-haiku-4-5

CrowdStrike reproducer replay (the #29991 bug case). A local sink stood in for the CrowdStrike endpoint, the guardrail was registered via config, and a real user and key were created so authenticated identity flows through /chat/completions:

$ curl -s http://localhost:4002/user/new -H "Authorization: Bearer $MASTER_KEY" \
    -d '{"user_id":"bp30148-user","user_email":"bp30148@example.com","auto_create_key":false}'
$ KEY=$(curl -s http://localhost:4002/key/generate -H "Authorization: Bearer $MASTER_KEY" -d '{"user_id":"bp30148-user"}' | jq -r .key)
# three variants: litellm_metadata absent, null (the #29991 bug case), and a user-supplied dict
$ for EXTRA in '' ',"litellm_metadata":null' ',"litellm_metadata":{"trace_id":"t-bp30148"}'; do
    curl -s http://localhost:4002/v1/chat/completions -H "Authorization: Bearer $KEY" \
      -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Say OK"}],"max_tokens":10,"guardrails":["crowdstrike-aidr"]'"$EXTRA"'}'
  done
# payloads the proxy POSTed to the CrowdStrike endpoint, all three variants:
{'user_id': 'bp30148-user', 'extra_info': {'user_name': 'bp30148@example.com'}, 'model': 'claude-haiku-4-5'}
{'user_id': 'bp30148-user', 'extra_info': {'user_name': 'bp30148@example.com'}, 'model': 'claude-haiku-4-5'}
{'user_id': 'bp30148-user', 'extra_info': {'user_name': 'bp30148@example.com'}, 'model': 'claude-haiku-4-5'}

Before these picks this line's guardrail had no identity capture at all, so these fields could never reach CrowdStrike; with only the feature and not #29991, the null and dict variants would drop them. #30009 has no HTTP reproducer in its PR body; its behavioral proof is the four batch-authorization tests it carries, which pass on this branch.

Pre-Submission checklist

  • The cherry-picked PRs carry their own tests
  • My PR passes the targeted unit tests for every touched file
  • Scope is limited to backporting already-merged changes plus the version bump

Type

🆕 New Feature (backport)
🐛 Bug Fix (backport)

Changes

Cherry-picks onto stable/1.86.x as listed above, plus the 1.86.5 version bump and uv.lock refresh

…#30064)

* Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI

Adds cost map entries for claude-fable-5 ($10/$50 per MTok, 1M context,
128K output, adaptive thinking only) on the Anthropic API, Bedrock
converse (base, global, and us/eu geo inference profiles at the 10%
regional premium), Vertex AI, and Azure AI (Microsoft Foundry, which
serves Fable 5 with the full 1M context window unlike Opus 4.8).

Registers anthropic.claude-fable-5 in BEDROCK_CONVERSE_MODELS, lists the
model in the setup wizard, and extends the reasoning effort e2e grid.
The Bedrock, Vertex, and Azure grid cells carry fail_reason markers
until the CI accounts are provisioned: Bedrock needs the provider data
sharing opt-in Fable 5 requires, and the Foundry resource needs a
claude-fable-5 deployment.

The first-party entry carries provider_specific_entry {us: 1.1} for the
inference_geo premium and deliberately no fast multiplier since Fable 5
has no fast mode.

https://claude.ai/code/session_01MZarYYT3aS7DxaNjoax6Gm

* Drop removed sampling params for Claude 4.7+ when drop_params is set

Fable 5, Opus 4.7, and Opus 4.8 removed sampling params: the API rejects
top_p, top_k, and any temperature other than 1 with a 400. LiteLLM was
forwarding them even with drop_params enabled because the Anthropic and
Bedrock converse transformations passed temperature/top_p through
unconditionally.

Mirror the GPT-5/o-series handling: temperature=1 still passes through,
other values and any top_p are dropped when drop_params is set, and
without drop_params a clean client-side UnsupportedParamsError tells the
caller how to opt in, instead of surfacing the raw provider error.

https://claude.ai/code/session_01MZarYYT3aS7DxaNjoax6Gm

* Drive sampling param gating from the cost map and cover top_k

Greptile review follow-ups on the sampling param fix: the restriction for
Fable 5 / Opus 4.7 / 4.8 is now declared as supports_sampling_params: false
on every affected cost map entry (perplexity excluded; that route is
OpenAI-compatible and maps sampling params upstream) and read back through
a tri-state map lookup, keeping the name check only as a fallback for
provider-routed ids whose hosted map entries predate the flag, the same
layering supports_adaptive_thinking uses. top_k bypasses map_openai_params
as a provider-specific kwarg, so it is gated at the shared
AnthropicConfig.transform_request boundary (direct, Bedrock invoke, Vertex,
Azure) and in the Bedrock converse _handle_top_k_value path, with
drop_params threaded through the converse transform helpers.

Also updates the reasoning effort grid cell count assertion for the four
Fable 5 rows added on this branch (29 x 11 cells).

https://claude.ai/code/session_01MZarYYT3aS7DxaNjoax6Gm

* Declare supports_sampling_params in the cost map schema

The model map validation schema uses additionalProperties: false, so the
new flag must be declared for the 28 entries that carry it; this was the
one failing job (misc / Run tests) on the previous commit.

https://claude.ai/code/session_01MZarYYT3aS7DxaNjoax6Gm

* fix(bedrock): gate top_k=0 on converse to match Anthropic boundary

Truthiness check let top_k=0 silently disappear on models that removed
sampling params, while AnthropicConfig.transform_request treats 0 as
present and raises UnsupportedParamsError (or drops when drop_params is
set). Switch to 'is not None' so converse, direct Anthropic, invoke,
Vertex, and Azure all behave the same for top_k=0.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

@mateo-berri mateo-berri marked this pull request as ready for review June 10, 2026 22:14
@mateo-berri mateo-berri requested a review from a team June 10, 2026 22:14
@greptile-apps

greptile-apps Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR backports four changes to stable/1.86.x and cuts 1.86.5: Claude Fable 5 support across Anthropic/Bedrock/Vertex/Azure AI (#30064), the CrowdStrike AIDR identity-capture feature (from stable/1.84.x), its follow-up fix for reading identity from either metadata bag (#29991), and the batch-file authorization fix via upload target_model_names (#30009).

  • Fable 5 / sampling-param gating: adds supports_sampling_params: false to the model map for Fable 5, Opus 4.7/4.8, and gates temperature/top_p/top_k in both the Anthropic and Bedrock Converse paths; also fixes a top_k=0 silent-drop bug (if val_top_k:if val_top_k is not None:).
  • CrowdStrike AIDR identity: _merge_metadata_bags merges metadata and litellm_metadata so identity fields (user_api_key_user_id, user_api_key_user_email) reach CrowdStrike regardless of which bag carries them; existing tests are hardened with monkeypatch.delenv for isolation.
  • Batch authorization: replaces the first-match reverse-lookup (resolve_model_name_from_model_id) with the upload-time target_model_names stored in the unified file ID, fixing auth failures on multi-deployment proxies; all new tests are mock-only.

Confidence Score: 5/5

Safe to merge — all three backports carry their own mock-only tests, the touched paths are well-isolated, and the one pre-existing concern (hardcoded name fallback in _supports_sampling_params) was already flagged in a prior review thread.

All changes are cherry-picks of already-merged staging commits with adaptation notes confirmed in the PR description. The Fable 5 sampling-param gating is correctly driven by the model-map flag (supports_sampling_params) with the name fallback only as a secondary guard. The top_k=0 silent-drop fix (if val_top_k:if val_top_k is not None:) is a genuine correctness improvement. The CrowdStrike identity merge order (litellm_metadata wins) is correct for system-injected fields. The batch authorization fix removes the multi-deployment first-match race without regressing non-managed files. Ten new mock-only tests cover all added behavior.

No files require special attention.

Important Files Changed

Filename Overview
litellm/proxy/guardrails/guardrail_hooks/crowdstrike_aidr/crowdstrike_aidr.py Adds _merge_metadata_bags and injects user identity + model into the CrowdStrike payload; Mapping is already imported, merge order (litellm_metadata wins) is correct for system-injected identity
litellm/proxy/hooks/batch_rate_limiter.py Replaces buggy first-match reverse-lookup with upload-time target_model_names from unified file ID; correctly falls back to JSONL scan for non-managed files via target_model_names or None
litellm/llms/anthropic/common_utils.py Adds _supports_sampling_params, _apply_sampling_param, _get_model_capability, and _model_map_lookup_candidates; primary path driven by model-map flag, name fallback handled as documented
litellm/llms/anthropic/chat/transformation.py Routes temperature/top_p through _apply_sampling_param in map_openai_params and gates top_k at the transform_request boundary shared by Anthropic/Bedrock/Vertex/Azure paths
litellm/llms/bedrock/chat/converse_transformation.py Propagates drop_params through _prepare_request_params and _handle_top_k_value; fixes silent top_k=0 drop (if val_top_k:if val_top_k is not None:)
model_prices_and_context_window.json Adds supports_sampling_params: false to existing Opus 4.7/4.8 entries and new Bedrock Fable 5 regional entries (eu/us/global/anthropic prefixes) with full capability flags
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_crowdstrike_aidr.py Adds 4 new identity-injection tests and a parametrized 4-case test for dual-bag reads; existing tests hardened with monkeypatch.delenv to prevent env-var leakage between runs
tests/test_litellm/proxy/hooks/test_batch_file_validation.py Updates existing proxy-alias test to pass target_model_names directly and asserts resolve_model_name_from_model_id is no longer called; adds new parametrized multi-deployment test covering the LIT-3593 bug
litellm/constants.py Adds anthropic.claude-fable-5 to BEDROCK_CONVERSE_MODELS

Reviews (2): Last reviewed commit: "chore: refresh uv.lock for 1.86.5" | Re-trigger Greptile

Comment on lines +289 to +305
return not any(
v in model_lower
for v in (
"fable",
"opus-4-7",
"opus_4_7",
"opus-4.7",
"opus_4.7",
"opus-4-8",
"opus_4_8",
"opus-4.8",
"opus_4.8",
)
)

@staticmethod
def _apply_sampling_param(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Hardcoded model-name fallback in _supports_sampling_params

The team's convention is that model-specific capability flags belong exclusively in model_prices_and_context_window.json, read via get_model_info. The fallback name list here ("fable", "opus-4-7", "opus-4-8", …) diverges from that pattern: a future model that removes sampling params but uses an unfamiliar name would silently pass through until the hardcoded list is extended, while a model whose name accidentally contains "fable" would be incorrectly gated even if it supports sampling params.

The primary map-driven path (_get_model_capability) already handles all the newly-added entries correctly. Removing the name fallback and relying solely on the map would be the safest long-term approach — any caller on an old map that lacks supports_sampling_params would transparently forward params rather than silently dropping them, which is the safe default.

Rule Used: What: Do not hardcode model-specific flags in the ... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

kenany and others added 5 commits June 10, 2026 19:02
…gs (#29991)

Capture user_id and extra_info from metadata or litellm_metadata. The single-bag read dropped identity whenever a request carried a present litellm_metadata field (null or a user-supplied dict), since /chat/completions routes the authenticated identity into metadata while the guardrail read litellm_metadata first

(cherry picked from commit 1bbaf1c)
…T-3593) (#30009)

* fix(proxy): authorize batch files using upload target_model_names (LIT-3593)

After replace_model_in_jsonl, body.model is a stripped provider id. Reverse-mapping it via resolve_model_name_from_model_id is first-match on model_list and caused false 403s when multiple deployments share the same stripped name. Use target_model_names from the unified file id instead.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): restore resolve_model_name_from_model_id for JSONL fallback path (LIT-3593)

Restores the reverse-lookup for the JSONL body.model fallback path so that
legacy/pre-target_model_names managed files still map stripped provider IDs
back to proxy aliases before auth. Also cleans up redundant `or None`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Revert "fix(proxy): restore resolve_model_name_from_model_id for JSONL fallback path (LIT-3593)"

This reverts commit 30d2e96.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit 2cd7e87)
@yuneng-berri yuneng-berri changed the title chore(release): backport #30064 (Claude Fable 5) to stable/1.86.x chore(release): backport #30064, #29991, #30009 to stable/1.86.x and cut 1.86.5 Jun 11, 2026
@yuneng-berri

Copy link
Copy Markdown
Collaborator

@greptileai

@yuneng-berri yuneng-berri enabled auto-merge June 11, 2026 02:46
@yuneng-berri yuneng-berri merged commit 9f445c0 into stable/1.86.x Jun 11, 2026
45 of 63 checks passed
@yuneng-berri yuneng-berri deleted the litellm_fable5_stable_1_86_x branch June 11, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants