Skip to content

ci: drop mypy entirely, standardize type checking on basedpyright#30648

Merged
mateo-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_drop_mypy_use_basedpyright
Jun 17, 2026
Merged

ci: drop mypy entirely, standardize type checking on basedpyright#30648
mateo-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_drop_mypy_use_basedpyright

Conversation

@mateo-berri

@mateo-berri mateo-berri commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Relevant issues

We ran two type checkers, mypy (through the pydantic.mypy plugin) and basedpyright, plus a separate mypy-backed Any gate. The original reason to keep mypy was a belief that pydantic doesn't play well with basedpyright; that stopped being true with pydantic v2, which emits @dataclass_transform so pyright/basedpyright understand models natively with no plugin. Verified empirically: basedpyright infers field types, flags wrong-type constructor args, and flags missing required fields on a plain pydantic v2 model with no mypy plugin in play.

This PR removes mypy from the toolchain entirely and leaves basedpyright as the single type checker.

What changed, in two parts

First commit removes the redundant mypy type-check gate. The four rules it tracked all map to basedpyright rules already gated at higher counts: no-untyped-def -> reportUnknownParameterType / reportMissingParameterType, no-any-return -> reportAny / reportUnknownVariableType, valid-type -> reportInvalidTypeForm, import-not-found -> reportMissingImports. The [tool.mypy] plugins = "pydantic.mypy" block was vestigial anyway: the gating pass ran cd litellm && mypy ., which picks up litellm/mypy.ini (cwd config wins) and never loaded that plugin. Gone: the lint-mypy / lint-mypy-budget-update Makefile targets, the "Run MyPy type checking" CI step, mypy-code-budget.json, and its budget-ratchet entry. type_check_gate.py is specialized to basedpyright.

Second commit removes the Any-discipline gate, which was the last consumer of mypy; it imported mypy as a library to detect values whose inferred type contains Any, gated per-file against any-discipline-budget.json. basedpyright already reports the same class of finding through reportAny / reportExplicitAny, gated tree-wide in basedpyright-code-budget.json. Gone: scripts/check_any_discipline.py and its test, the any-discipline CI job, the lint-any / lint-any-budget-update Makefile targets, any-discipline-budget.json, litellm/mypy.ini, the .mypy_cache_any references, and mypy from the dev dependencies. budget_ratchet_check.py drops the any-discipline entry and the now-unused zero-floor mechanism. check_type_discipline.py drops the any-ok suppression token, and the 134 now-orphaned # any-ok comments across 14 files are stripped (they never affected basedpyright, which uses # pyright: ignore).

One consequence worth calling out for review: the Any gate enforced a per-file zero-floor, so brand-new files had to be Any-free. basedpyright's reportAny is gated tree-wide with a large grandfathered baseline, so new code is held to a looser net than before. If we want strict any-checking back, the path is to ratchet the reportAny / reportExplicitAny ceilings in basedpyright-code-budget.json down over time rather than maintain a second mypy-based gate.

Third commit relocks uv.lock to drop mypy. Worth noting for anyone reproducing locally: CI pins uv 0.10.9, which honors the repo's exclude-newer window and produces a minimal relock (removes only mypy and its transitive librt, no other version changes). An older uv that can't parse exclude-newer = "3 days" silently ignores it and will try to bump the whole graph, so use the pinned version when relocking.

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🧹 Refactoring

Screenshots / Proof of Fix

This is a CI/tooling change with no runtime or proxy behavior to exercise, so the proof is the gates themselves, run against the branch:

  1. make lint-basedpyright gates type errors against basedpyright-code-budget.json (the sole type checker now)
  2. uv run python scripts/ruff_strict_gate.py --base origin/litellm_internal_staging -> "OK: every strict rule is within its codebase ceiling"
  3. uv run python scripts/type_discipline_gate.py --base origin/litellm_internal_staging -> "OK: every LIT rule is within its codebase ceiling"
  4. uv run python scripts/budget_ratchet_check.py --base origin/litellm_internal_staging -> "OK: no budget ceiling increased" across the remaining three budgets
  5. uv run python -m pytest tests/test_litellm/test_type_check_gate.py tests/test_litellm/test_budget_ratchet_check.py tests/test_litellm/test_check_type_discipline.py -> 47 passed
  6. uv lock --check (uv 0.10.9, matching CI) exits 0 with mypy removed from pyproject
  7. git grep -n "any-ok\|lint-mypy\|lint-any\|check_any_discipline" returns nothing outside .venv

Type checking ran both mypy (via the pydantic.mypy plugin) and basedpyright.
pydantic v2 emits dataclass_transform, so basedpyright understands models
natively with no plugin, and its gated rules already cover what the mypy pass
caught (no-untyped-def, no-any-return, valid-type, import-not-found all map to
basedpyright equivalents). Running both meant two checkers, two budgets, and a
plugin only mypy could load.

This removes the mypy type-check gate: the lint-mypy/lint-mypy-budget-update
Makefile targets, the CI MyPy step, mypy-code-budget.json, the budget-ratchet
entry, and the vestigial [tool.mypy] pydantic plugin block (the gating pass used
litellm/mypy.ini, which never loaded the plugin). type_check_gate.py is
specialized to basedpyright since the mypy parsing path is now unused.

mypy stays a dev dependency because the Any-discipline gate
(scripts/check_any_discipline.py) imports it as a library to detect Any-typed
values; it is no longer run as a type checker.
@codecov

codecov Bot commented Jun 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

The Any-discipline gate (scripts/check_any_discipline.py) was the last consumer
of mypy: it imported mypy as a library to detect values whose inferred type
contains Any, gated per-file against any-discipline-budget.json. basedpyright
already reports the same class of finding through reportAny/reportExplicitAny,
which are gated tree-wide in basedpyright-code-budget.json, so the separate gate
(and the mypy dependency behind it) is redundant.

Removes the gate end to end: check_any_discipline.py and its test, the
any-discipline CI job, the lint-any/lint-any-budget-update Makefile targets,
any-discipline-budget.json, litellm/mypy.ini, the .mypy_cache_any references,
and mypy from the dev dependencies. budget_ratchet_check.py drops the
any-discipline entry and the now-unused zero-floor mechanism (rewritten as a
comprehension). check_type_discipline.py drops the any-ok suppression token,
since # any-ok suppressed only the deleted gate; the 134 now-orphaned
# any-ok comments across 14 files are stripped (they never affected
basedpyright, which uses # pyright: ignore).

uv.lock is intentionally left untouched: uv still considers it consistent with
the mypy-removed pyproject (uv lock --check and uv sync --frozen both pass), and
a relock bumps 30+ unrelated packages because of the moving exclude-newer window.
A future intentional relock will prune the now-unreferenced mypy entry.
@mateo-berri mateo-berri changed the title ci: drop redundant mypy type-check gate, standardize on basedpyright ci: drop mypy entirely, standardize type checking on basedpyright Jun 17, 2026
CI's uv 0.10.9 honors the repo's exclude-newer window and correctly flags the
lockfile as out of sync once mypy leaves pyproject; my earlier local uv 0.8.17
could not parse exclude-newer and silently passed --check. Relocking with the
pinned CI version removes only mypy and its transitive librt, with no other
version changes.
@mateo-berri mateo-berri marked this pull request as ready for review June 17, 2026 16:09
@mateo-berri mateo-berri requested review from a team and ryan-crabbe-berri June 17, 2026 16:09
@greptile-apps

greptile-apps Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Drops mypy from the toolchain entirely and removes the mypy-backed per-file Any-discipline gate, leaving basedpyright as the single type checker. No runtime or proxy behavior is affected.

  • Mypy removal: mypy==1.19.0 is removed from dev dependencies; litellm/mypy.ini, mypy-code-budget.json, the [tool.mypy] pyproject section, and all lint-mypy / lint-any Makefile targets are deleted. CI drops both the "Run MyPy type checking" step and the entire any-discipline job.
  • Any-discipline gate removal: scripts/check_any_discipline.py and its test are deleted; any-discipline-budget.json is removed; budget_ratchet_check.py drops the ZERO_FLOOR_BUDGETS special-case and simplifies regressions_for to an equivalent list comprehension. The 134 # any-ok suppression comments across 14 files are stripped (they were noops for basedpyright).
  • Acknowledged trade-off: the former gate enforced a per-file zero-floor for new files; basedpyright's reportAny is gated tree-wide against a large grandfathered baseline, so brand-new files now get more slack. The PR notes the path back (ratchet reportAny / reportExplicitAny ceilings down over time).

Confidence Score: 5/5

Pure CI/tooling change with no runtime or proxy behavior modifications; all application code changes are cosmetic comment removals.

Every changed runtime file is a # any-ok comment strip with no logic diff. The scripts are coherently updated — DEFAULT_BUDGETS, the on-disk budget files, and the invariant test all agree on the same 3-file set. The regressions_for rewrite is a semantically equivalent list comprehension. The acknowledged trade-off (looser per-file Any floor for new code) is a deliberate policy choice, not a defect, and a clear remediation path is documented.

No files require special attention; the key invariant test (test_default_budgets_watch_every_budget_file_in_the_repo) still guards against budget files going untracked.

Important Files Changed

Filename Overview
.github/workflows/test-linting.yml Removes the any-discipline CI job entirely and drops --tool mypy from the basedpyright step; all references are consistent with the toolchain removal.
Makefile Drops lint-mypy, lint-any, and related targets; lint and lint-dev are updated consistently; lint-budget-update now only ratchets ruff + basedpyright.
scripts/type_check_gate.py Removes mypy text-parsing path and --tool argument; the module is now basedpyright-only with a hardcoded BUDGET_PATH constant and simplified entry point.
scripts/budget_ratchet_check.py Removes mypy-code-budget.json and any-discipline-budget.json from DEFAULT_BUDGETS, drops the ZERO_FLOOR_BUDGETS special-case, and simplifies regressions_for to an equivalent list comprehension.
scripts/check_type_discipline.py Removes the any-ok suppression token and its regex from LIT005; reclaims LIT000 in the module docstring (was previously attributed to the deleted check_any_discipline.py).
pyproject.toml Removes mypy==1.19.0 from dev dependencies, drops the [tool.mypy] section and .mypy_cache source-exclude patterns; all other config is untouched.
tests/test_litellm/test_budget_ratchet_check.py Drops tests for the removed ZERO_FLOOR_BUDGETS / any-discipline behavior; the key invariant test (test_default_budgets_watch_every_budget_file_in_the_repo) is preserved and now asserts the correct 3-file set.
tests/test_litellm/test_check_any_discipline.py Deleted alongside check_any_discipline.py; removal is appropriate since the module it tested no longer exists.
litellm/proxy/common_request_processing.py Strips 22 # any-ok suppression comments; no logic changes, only cosmetic cleanup.
litellm/proxy/management_endpoints/key_management_endpoints.py Strips # any-ok comments and reformats long expressions split across lines to fit them inline; no semantic changes.

Reviews (1): Last reviewed commit: "build: relock to drop mypy from uv.lock" | Re-trigger Greptile

@mateo-berri mateo-berri merged commit b8d79d1 into litellm_internal_staging Jun 17, 2026
123 of 128 checks passed
@mateo-berri mateo-berri deleted the litellm_drop_mypy_use_basedpyright branch June 17, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants