Repository navigation
tools: content-keyed ruff/mypy baselines (line shifts no longer churn them) - #3396
Merged
Merged
Conversation
…hem (#3384) Baseline keys were `{file}:{line}: {code}`, so any unrelated line shift in a touched file changed the baseline and conflicted on rebase (the recurring hand-fixes of moved audit_recorder.py entries). Keys are now `{file}: {code} @{sha256[:12] of the whitespace-stripped flagged line}#{occurrence}` via a shared tests/lint_baseline.py, used by both ruff_audit.py and mypy_audit.py (same design, same problem). - --ci strictness unchanged: new key -> fail, baseline key no longer flagged -> FYI stale list; a new duplicate of a baselined line gets the next occurrence index and fails; editing a flagged line fails. - Output still shows each finding's current line number. - New --write-baseline flag writes the canonical sorted form (byte-identical on regenerate); replaces the broken `from tests.ruff_audit import` hint. - Baselines migrated (5 ruff, 1 mypy); old-key translation equals the regenerated file. - audit_check.py: shift detection now pairs edited baselined lines one-for-one with stale entries (multiset), never a new duplicate. - Tests incl. a temp-git-repo merge of two line-shifting branches. Part of #3384 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Contributor
Finding is now aliased from the module instead of a second 'from lint_baseline import Finding', in ruff_audit.py and mypy_audit.py. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 23, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #3384 (the ruff-baseline half; a sibling PR splits the golden masters, which this PR does not touch).
Problem
tests/ruff_audit_baseline.jsonkeys were{file}:{line}: {code}. Any unrelated line shift in a touched file changed the baseline and conflicted on rebase. Rebases repeatedly had to hand-fix movedaudit_recorder.pyentries.Key scheme
hash: first 12 hex characters of sha256 over the flagged source line with all whitespace removed, so reindenting or respacing the line doesn't change it.occurrence: 0-based index among findings with the same (file, code, hash), in source order (line, then column).key.split(":", 1)[0]still returns the file.rebase_rebless.verify_baseline_refresh_scoperelies on that.gitgalaxy/x.py:15: C414 -- msg [key].The scheme is implemented once, in the new
tests/lint_baseline.py.Which audits changed
--write-baselineflag, which replaces the brokenpython -c "from tests.ruff_audit import ..."hint becausetests/has no__init__.py.--write-baseline. The occurrence index also removes that collapse.{}), so nothing changed.--regeneratecan never accept a new duplicate. Before, matching was set-based.CI strictness (unchanged)
--ci: a new finding, a new duplicate of a baselined line (next#occurrence), or an edit to a flagged line.len(baseline)/ number of new keys. Findings that used to collide can no longer do so, so every finding is counted.ruff format --checkzero-tolerance gate is untouched. No workflow change is needed because.github/workflows/ruff-audit.ymlandmypy-audit.ymlstill runpython tests/*_audit.py --ci.Migration
Both baselines were migrated (5 ruff entries, 1 mypy entry), sorted, with a trailing newline. A translation of the old line keys to the new keys (read the old line, hash it) is byte-identical to the
--write-baselineoutput on main.Interaction with #3392 (
tests/tools/rebase_rebless.py, merged)Its step 4, "refresh baseline entries for moved lines", existed only because line keys moved. With content keys a rebase that only shifts lines leaves the ruff/mypy baselines byte-identical, so that step becomes a no-op in the common case. These baselines should also stop showing up as artifact conflicts except when both sides changed the same findings. The step still calls
audit_check.py --regenerate, which now only re-accepts edited baselined lines. It is still useful as the ruff check/format gate, so I left it in place and updated its docstrings, log line and the CLAUDE.md paragraph.verify_baseline_refresh_scopekeeps working because the file stays first in the key. The step could be removed or renamed later, but that is not needed here.Tests:
tests/tools/test_content_keyed_baseline.py(13 tests)Real
ruff_audit.parse_ruff_json/compare/run_ci_checkcode runs against real temp files fed by a small fake linter, plus one test that uses the realruffbinary:#1.run_ci_checkend to end: passes after a shift, fails on a duplicate and prints the current line, and prints the FYI for a stale entry.audit_check._classifypairs one-for-one.Test plan
python tests/ruff_audit.py --ci: no NEW lint findings beyond the 5-finding baseline (ruff 0.16.0, same pin as CI)python tests/mypy_audit.py --ci: no NEW type errors beyond the 1-error baselinepython tests/dead_key_audit.py --ci: no NEW keys beyond the 0-key baselinepython tests/tools/audit_check.py: all clear (ruff, mypy, dead-key, ast-accuracy)ruff formatis clean on every changed filepytest tests/toolspasses (plainpytest, the same invocation CI uses), including the new file andtest_rebase_rebless.py🤖 Generated with Claude Code