ci: per-commit fast-check gate (catch broken intermediate commits under rebase-merge)#1190
Open
vringar wants to merge 3 commits into
Open
ci: per-commit fast-check gate (catch broken intermediate commits under rebase-merge)#1190vringar wants to merge 3 commits into
vringar wants to merge 3 commits into
Conversation
f361334 to
1041f4f
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1190 +/- ##
==========================================
- Coverage 62.14% 62.12% -0.03%
==========================================
Files 40 40
Lines 3918 3918
==========================================
- Hits 2435 2434 -1
- Misses 1483 1484 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…er rebase-merge) Add a lightweight CI job that runs the fast static checks on every commit in a PR / merge-queue candidate, not just the tip. Because the project uses rebase-and-merge, every commit lands on master individually, but the existing tests job only validates the tip — a commit that is broken in isolation can reach master. This gate closes that gap cheaply: per commit it runs the pre-commit hooks (black/isort/mypy/actionlint) on the changed files, rebuilds the extension when its sources changed, imports the package, and collects the test suite. It intentionally does not run the browser test suite — runtime and browser-behavior regressions stay covered by the tests job on the tip plus the merge queue.
1041f4f to
24076a0
Compare
There was a problem hiding this comment.
Pull request overview
Adds a new GitHub Actions workflow to gate rebase-and-merge PRs by running a fast static-check suite on every individual commit in the PR (and in merge-queue merge_group candidates), preventing “broken intermediate commits” from landing on master.
Changes:
- Introduces
per-commit-checks.yaml, triggered onpull_requestandmerge_group. - Computes a commit range and iterates commit-by-commit, running
pre-commit(changed-files), conditional Extension build,import openwpm, andpytest --collect-only. - Aggregates failures across commits (doesn’t stop at first failure) and fails the job at the end if any commit failed.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The per-commit gate reused the node_modules installed by the setup action on the tip ref. A commit that changed Extension/package*.json without a matching node_modules would run its build against the wrong dependency set. Run npm ci (lockfile-driven, fast) before the build when this commit touched the manifest or lockfile.
The per-commit gate checks out each PR commit but kept the conda env from the tip, so an intermediate commit that changes deps was validated against the wrong environment (stale-deps false negative). Mirror the existing Extension-lockfile guard: track a sha256 baseline of the environment.yaml the env was built from, and re-run install.sh to recreate the env only when a commit's environment.yaml differs from that baseline, updating the baseline afterward. Conda setup re-runs only on actual dep changes (rare), preserving full correctness at minimal cost.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
OpenWPM uses rebase-and-merge, so every commit in a PR lands on
masterindividually. But CI only tests the PR tip (and the merge-queue candidate
tip). That means a commit which is broken in isolation — e.g. it introduces a
syntax error, an undefined name, a bad import, or a type/lint regression that a
later commit in the same PR happens to fix — can still ship to
masteras anindividual broken commit. Anyone who later bisects, checks out, or builds from
that commit hits the breakage.
This adds a lightweight gate,
per-commit-checks.yaml, that runs the faststatic checks on every commit in the PR / merge-queue range, so an
intermediate commit that's broken on its own can't slip through. It's the cheap
80/20: it reuses the existing
./.github/actions/setupcomposite action (condaenv + built extension) and finishes quickly because it skips the expensive part.
What it catches (per commit)
python -c "import openwpm"plus
pytest --collect-onlyexercise import of the package and the whole testtree (this also covers cross-file breaks that mypy's changed-files-only view
can miss, e.g. a symbol deleted in one module but still referenced by an
untouched one).
pre-commit run --from-ref <c~1> --to-ref <c>runsblack / isort / mypy / actionlint on the files that commit changed. mypy is
the load-bearing check here: it flags
[name-defined](undefined name) andtype errors in changed files.
npm run buildruns only when the committouched
Extension/, catching broken extension sources.What it intentionally does NOT do
browser-behavior regressions stay covered by the existing
testsjob on thePR tip + the merge queue, plus author discipline. Running the full suite on
every commit would be far too expensive for the marginal value.
Notes
in branch protection. Until then it runs informationally.
checks belong in the per-commit tier) and on the merge-queue handling.
pull_requesttrigger).The
merge_grouptrigger only fires inside a real merge queue, so that pathcan't be exercised from the PR itself; the field names
(
merge_group.base_sha/merge_group.head_sha) were validated against thepublished webhook payload schema.
How the commit range is computed
pull_request:base = pull_request.base.sha,head = pull_request.head.sha— the PR's own commits (not the synthetic merge commit).
fetch-depth: 0plusexplicit-SHA
git checkoutin the loop makes this independent of which refactions/checkoutmaterialized.merge_group:base = merge_group.base_sha(the group's parent commit),head = merge_group.head_sha(the group tip) — exactly the candidate commits.The loop uses
set -uo pipefail(notset -e) and|| { ...; fail=1; }so itvisits every commit and reports all failing ones, then fails the job.