Skip to content

tools: content-keyed ruff/mypy baselines (line shifts no longer churn them) - #3396

Merged
squid-protocol merged 2 commits into
mainfrom
tool/3384-content-keyed-ruff-baseline
Sep 23, 2026
Merged

squid-protocol merged 2 commits into
mainfrom
tool/3384-content-keyed-ruff-baseline

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Part of #3384 (the ruff-baseline half; a sibling PR splits the golden masters, which this PR does not touch).

Problem

tests/ruff_audit_baseline.json keys were {file}:{line}: {code}. Any unrelated line shift in a touched file changed the baseline and conflicted on rebase. Rebases repeatedly had to hand-fix moved audit_recorder.py entries.

Key scheme

"{file}: {code} @{hash}#{occurrence}"   ->  message
e.g. "gitgalaxy/recorders/gpu_recorder.py: C414 @2c826ccd023d#0"
  • hash: first 12 hex characters of sha256 over the flagged source line with all whitespace removed, so reindenting or respacing the line doesn't change it.
  • occurrence: 0-based index among findings with the same (file, code, hash), in source order (line, then column).
  • The file comes first, so key.split(":", 1)[0] still returns the file. rebase_rebless.verify_baseline_refresh_scope relies on that.
  • Line numbers are not in the baseline. They are still shown in the output, for example gitgalaxy/x.py:15: C414 -- msg [key].

The scheme is implemented once, in the new tests/lint_baseline.py.

Which audits changed

  • ruff_audit.py: switched to content keys. It also gets a --write-baseline flag, which replaces the broken python -c "from tests.ruff_audit import ..." hint because tests/ has no __init__.py.
  • mypy_audit.py: had the same line-number-keyed design, including a documented limitation that several errors of one code on the same line collapsed into a single entry. With the shared helper the change was trivial, so it now uses the same scheme and --write-baseline. The occurrence index also removes that collapse.
  • dead_key_audit.py: not affected. Its baseline is keyed by dict-key name and has no line numbers (and is currently {}), so nothing changed.
  • audit_check.py: a pure line shift no longer creates new keys. The old "same file/code/message, just moved" heuristic now only applies to an edited baselined line. It also matches one-for-one now (a multiset): N+1 identical new findings against N stale entries leaves one genuine finding, so --regenerate can never accept a new duplicate. Before, matching was set-based.

CI strictness (unchanged)

  • A key that is not in the baseline fails --ci: a new finding, a new duplicate of a baselined line (next #occurrence), or an edit to a flagged line.
  • A baseline key that is no longer flagged is printed as the same FYI "no longer flagged, consider removing" list and does not fail.
  • The count is still len(baseline) / number of new keys. Findings that used to collide can no longer do so, so every finding is counted.
  • The ruff format --check zero-tolerance gate is untouched. No workflow change is needed because .github/workflows/ruff-audit.yml and mypy-audit.yml still run python tests/*_audit.py --ci.

Migration

Both baselines were migrated (5 ruff entries, 1 mypy entry), sorted, with a trailing newline. A translation of the old line keys to the new keys (read the old line, hash it) is byte-identical to the --write-baseline output on main.

Interaction with #3392 (tests/tools/rebase_rebless.py, merged)

Its step 4, "refresh baseline entries for moved lines", existed only because line keys moved. With content keys a rebase that only shifts lines leaves the ruff/mypy baselines byte-identical, so that step becomes a no-op in the common case. These baselines should also stop showing up as artifact conflicts except when both sides changed the same findings. The step still calls audit_check.py --regenerate, which now only re-accepts edited baselined lines. It is still useful as the ruff check/format gate, so I left it in place and updated its docstrings, log line and the CLAUDE.md paragraph. verify_baseline_refresh_scope keeps working because the file stays first in the key. The step could be removed or renamed later, but that is not needed here.

Tests: tests/tools/test_content_keyed_baseline.py (13 tests)

Real ruff_audit.parse_ruff_json / compare / run_ci_check code runs against real temp files fed by a small fake linter, plus one test that uses the real ruff binary:

  • Inserting unrelated lines above a baselined violation: the baseline stays valid and regenerating is byte-identical. The reported line numbers do move.
  • Reindenting or respacing the flagged line is stable.
  • A new duplicate of a baselined line is caught as #1.
  • A changed line is caught (new key plus a stale key).
  • A removed violation is reported as stale.
  • Two findings on one line don't collapse.
  • run_ci_check end to end: passes after a shift, fails on a duplicate and prints the current line, and prints the FYI for a stale entry.
  • mypy output parsing uses the same scheme.
  • audit_check._classify pairs one-for-one.
  • The committed baselines are canonical and parse.
  • Two branches that each shift lines in the same file merge conflict-free in a temp git repo, and the merged baseline equals the base baseline.

Test plan

  • python tests/ruff_audit.py --ci: no NEW lint findings beyond the 5-finding baseline (ruff 0.16.0, same pin as CI)
  • python tests/mypy_audit.py --ci: no NEW type errors beyond the 1-error baseline
  • python tests/dead_key_audit.py --ci: no NEW keys beyond the 0-key baseline
  • python tests/tools/audit_check.py: all clear (ruff, mypy, dead-key, ast-accuracy)
  • ruff format is clean on every changed file
  • pytest tests/tools passes (plain pytest, the same invocation CI uses), including the new file and test_rebase_rebless.py

🤖 Generated with Claude Code

…hem (#3384)

Baseline keys were `{file}:{line}: {code}`, so any unrelated line shift in a
touched file changed the baseline and conflicted on rebase (the recurring
hand-fixes of moved audit_recorder.py entries). Keys are now
`{file}: {code} @{sha256[:12] of the whitespace-stripped flagged line}#{occurrence}`
via a shared tests/lint_baseline.py, used by both ruff_audit.py and
mypy_audit.py (same design, same problem).

- --ci strictness unchanged: new key -> fail, baseline key no longer flagged
  -> FYI stale list; a new duplicate of a baselined line gets the next
  occurrence index and fails; editing a flagged line fails.
- Output still shows each finding's current line number.
- New --write-baseline flag writes the canonical sorted form (byte-identical
  on regenerate); replaces the broken `from tests.ruff_audit import` hint.
- Baselines migrated (5 ruff, 1 mypy); old-key translation equals the
  regenerated file.
- audit_check.py: shift detection now pairs edited baselined lines
  one-for-one with stale entries (multiset), never a new duplicate.
- Tests incl. a temp-git-repo merge of two line-shifting branches.

Part of #3384

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread tests/mypy_audit.py Fixed
Comment thread tests/ruff_audit.py Fixed
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

Finding is now aliased from the module instead of a second
'from lint_baseline import Finding', in ruff_audit.py and mypy_audit.py.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants