test(proxy): behavior-pinning matrix for tier-2/3 key + team management endpoints#28620
Merged
yuneng-berri merged 11 commits intoMay 22, 2026
Merged
Conversation
Adds create_scratch_actor() to the management behavior-suite conftest and extends create_scratch_team() with team_member_permissions / models kwargs, needed by the PR3 team-key-permission and team-model matrices. The new helper mints a scratch-prefixed user + verification token (+ org memberships), all reclaimed by the existing scratch-prefix teardown.
Adds behavior-pinning matrices for POST /key/block, POST /key/unblock, POST /key/health, and GET /key/aliases. Pins that the management-route gate 401s ORG_ADMIN-role callers before _check_key_admin_access runs, the block/unblock round-trip on the blocked column, missing-key 404, and the _apply_non_admin_alias_scope visibility rules for /key/aliases.
Adds behavior-pinning matrices for POST /key/bulk_update (PROXY_ADMIN-only; ORG_ADMIN stopped 401 at the route gate, INTERNAL_USER-role 403 at the handler) and POST /team/key/bulk_update (team-member-permission gate keyed on KEY_UPDATE). Pins batch semantics: empty/over-cap 400, per-key failure isolation into failed_updates, all_keys_in_team broadcast, and no-keys 404. Adds an optional key_alias arg to create_scratch_key for multi-key scenarios.
Adds behavior-pinning matrices for POST /key/service-account/generate
(team-membership + team-member-permission gating; SA keys carry no user_id),
POST /v2/key/info (per-key _can_user_query_key_info silently drops invisible
keys), and POST /key/{key}/reset_spend (PROXY_ADMIN or team admin only;
missing key 404, reset-value 400). Pins that ORG_ADMIN-role callers are
stopped 401 at the management-route gate on the two non-info routes.
Closes the four key-side gaps deferred from PR1/PR2: - 404 on missing key for /key/update and /key/delete (not 401/403) - denied /key/update leaves max_budget/tpm_limit/rpm_limit untouched - /key/regenerate enforces litellm.upperbound_key_generate_params (#26340) - /key/list key_alias substring vs exact (admin-only) + team_id filter, and a non-admin filtering a foreign team is 403
Adds behavior-pinning matrices for POST /team/block + /team/unblock
(management-route gate fronts _verify_team_access; reachable only by
PROXY_ADMIN and an org admin of the team's own org), GET /team/available
(default empty path), GET /team/filter/ui (route-gated PROXY-ADMIN-only
despite the handler having no gate), and GET /team/{team_id}/members/me
(caller resolves its own membership; non-member 404, no-user_id key 400).
Adds behavior-pinning matrices for POST /team/model/add + /team/model/delete (route-gated PROXY-ADMIN-only; missing team 404), GET /team/permissions_list + POST /team/permissions_update (self-managed; proxy/team/org admin pass), and POST /team/permissions_bulk_update (PROXY_ADMIN-only). Pins the deliberate divergence that the available-team self-join grants read access via permissions_list but never write access via permissions_update.
Adds behavior-pinning matrices for POST /team/delete (per-team _verify_team_access; batch aborts whole on a missing id), POST /team/bulk_member_add (route-gated PROXY-ADMIN-only; empty/over-cap 400), GET /v2/team/list (_enforce_list_team_v2_access — bare query 401s regular users, org-scoped for org admins) and GET /team/daily/activity (non-member team_ids filter 404, the VERIA-43 fix).
Adds test_route_coverage.py (PR3.M1): parses every @router route literal from the two management-endpoint source files and asserts each is exercised by >=1 behavior-suite scenario — a permanent regression guard for future routes. Closes the last PR1/PR2 deferred gap: the /team/update org-relocation allowed branch, exercised by a dual-org-admin minted via create_scratch_actor. test_team_model uses literal route URLs so the coverage parser resolves them.
…itellm_/exciting-bassi-ab48f6
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryTests-only PR adding HTTP-boundary behavior-pinning coverage for the remaining tier-2/3 key and team management routes (26 new/expanded files, 334 new scenarios). No production code is touched — all changes are confined to
Confidence Score: 5/5Tests-only change with no production code modifications; safe to merge. All 26 changed files are confined to the test directory. No handler logic, no migrations, no production imports are modified. The new create_scratch_actor helper correctly uses hash_token and is reclaimed by the existing prefix-based teardown. Route coverage guard uses correct single-segment [^/?]+ path-param regex. Monkeypatching of module-level litellm state is properly scoped to function lifetime and safe under sequential execution. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/proxy_behavior/management/test_route_coverage.py | New permanent CI guard; parses @router decorators from source files and verifies each route has a behavior-suite scenario. Uses correct [^/?]+ for single-segment path params and [^?]+ for :path params. |
| tests/proxy_behavior/management/conftest.py | Adds optional key_alias to create_scratch_key, team_member_permissions/models to create_scratch_team, and the new create_scratch_actor helper (user + token + org memberships, reclaimed by existing scratch teardown). Looks correct. |
| tests/proxy_behavior/management/test_scratch_teardown.py | Adds test_a2 to verify create_scratch_actor minting lands correctly; expands test_b to assert tokens, users, and org memberships are all reclaimed. Ordering and teardown logic are correct. |
| tests/proxy_behavior/management/test_key_reset_spend.py | New file covering /key/{key}/reset_spend authz matrix, missing-key 404, and above-current-spend 400 validation with full DB-state assertions. |
| tests/proxy_behavior/management/test_team_key_bulk_update.py | New file covering /team/key/bulk_update authz matrix (admin/member-allowed/member-denied/nonmember), all_keys_in_team broadcast, empty-team 404, and per-key isolation. |
| tests/proxy_behavior/management/test_team_permissions.py | New file covering permissions_list/update authz matrix, available-team self-join divergence (read allowed, write denied), and permissions_bulk_update proxy-admin-only gate. |
| tests/proxy_behavior/management/test_team_update.py | Closes the deferred PR2 org-relocation allowed branch by minting a dual-org admin via create_scratch_actor; correct DB-state assertion. |
Reviews (2): Last reviewed commit: "test(proxy): bound plain route params to..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
… gate
Plain path params ({team_id}) now compile to [^/?]+ instead of [^?]+, so a
parameter cannot span '/'. Starlette ':path' params still match across '/'.
Keeps the route-coverage guard from falsely reporting a future multi-segment
route as covered. All 37 routes remain covered.
Collaborator
Author
|
@greptile |
ryan-crabbe-berri
approved these changes
May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Final phase of the management-endpoint behavior-pinning effort (follows #28321 and #28441). Adds HTTP-boundary behavior-pinning coverage for the remaining tier-2/3 key + team management routes, closes the five coverage gaps deferred from PR1/PR2, and codifies route coverage as a permanent pytest regression guard.
Tests-only — the diff is confined to
tests/proxy_behavior/management/;pyproject.tomlis untouched and no CI workflow changes.What's covered
Key endpoints:
/key/block,/key/unblock,/key/health,/key/aliases,/key/bulk_update,/team/key/bulk_update,/key/service-account/generate,/v2/key/info,/key/{key}/reset_spend.Team endpoints:
/team/block,/team/unblock,/team/available,/team/filter/ui,/team/{team_id}/members/me,/team/model/add,/team/model/delete,/team/permissions_list,/team/permissions_update,/team/permissions_bulk_update,/team/delete,/team/bulk_member_add,/v2/team/list,/team/daily/activity.Deferred PR1/PR2 gaps closed: 404-on-missing-key for
/key/updateand/key/delete; a denied/key/updateleavesmax_budget/tpm_limit/rpm_limituntouched;/key/regenerateupperbound enforcement;/key/listkey_alias/team_idfilter params; the/team/updateorg-relocation allowed branch.Each route's status codes are pinned to observed handler behavior — including where the management-route gate fronts a handler's own authz checks.
Exit gates
PR3.M1 — route coverage, codified.
test_route_coverage.pyparses every@routerroute literal from the two management-endpoint source files (37 routes) and asserts each is exercised by at least one behavior-suite scenario. A future route added without a behavior test fails CI here — the same waytest_no_management_imports.pycodifies the import grep.G1 — full suite green. 620 scenarios pass locally against Postgres (286 from PR1/PR2 + 334 new), no skips or xfails.
G3 — strict-import grep clean.
test_no_management_imports.pypasses —tests/proxy_behavior/imports nothing fromlitellm.proxy.management_endpointsand mocks nouser_api_key_auth.G4 — regression replay. Verified RED→GREEN against three historical fixes: checked out the fix commit's parent revision of the handler file, confirmed the new test fails, restored, confirmed it passes.
db8ef44323(#26340)/key/regenerateupperbound_key_generate_params-> now enforced19efe556cb/key/block,/key/unblock8881c36405/key/listkey_aliasfilter worked -> now exact-match onlyG2 / PR3.M2 — CI wall-time is validated on this PR's
test-unit-proxy-mgmt-behavior.ymlrun (target <= 10 min; the suite is ~15s locally, run sequentially withworkers: 0).Test plan
uv run pytest tests/proxy_behavior/management/— 620 passed locally against Postgres.test-unit-proxy-mgmt-behavior.ymlpicks up the new files automatically viatest-path: tests/proxy_behavior.