fix(scripts): make plugin-test skips loud and run the pytest suite in CI - #2434
Conversation
…strict-skips, run pytest suite in CI A suite whose only coverage skipped (SKIP: pytest not installed) used to be indistinguishable from a pass: exit 0, generic aggregate count, final line 'All plugin tests passed or were skipped.' Now the summary names every suite that skipped and why, the final line distinguishes a clean pass from a pass with skips, and --strict-skips lets a fully-provisioned caller treat any optional skip as failure. pytest is pinned into requirements-ci.txt so the session-flow parse_transcript suite actually runs in CI. Fixes #1313 Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
…n-tests.sh The runner half of #1313 (the pytest CI pin landed in the previous commit; this edit was lost to a concurrent working-tree revert before staging). The summary now names every suite that emitted a SKIP and its first reason, the final line distinguishes a clean pass from a pass with skipped coverage, and --strict-skips turns any optional skip into a failure for fully-provisioned callers. Fixes #1313 Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Claude finished @kyle-sexton's task in 1m 54s —— View job Security Review — PR #2434
Scope: Only the two files changed in this PR — No security issues found. Details of what I checked:
|
|
Claude finished @kyle-sexton's task in 4m 17s —— View job Review complete
Root One finding posted inline on Read through, no defects found:
Could not independently verify: the five new |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
2 participants
Fixes #1313
Summary
run-plugin-tests.shcould exit 0 while the only Python suite silently skipped (SKIP: pytest not installed), making a green run indistinguishable from verified coverage. Skips are now named per suite in the summary, a--strict-skipsmode turns them into failures for fully-provisioned callers, and pytest is pinned into CI so theparse_transcriptsuite actually runs there.Fix
SKIP:line and prints aSuites with skipped coveragesection naming each suite, its skip count, and the first reason; the final line now readsAll executed plugin tests passed; N case(s) skipped (named above).instead of the old undifferentiatedAll plugin tests passed or were skipped.--strict-skipstreats any optional SKIP as a failure, for environments that declare the full toolchain (CI is deliberately not wired to it yet — its toolchain is also missing goimports, typos, PSScriptAnalyzer, and the deliberately-optional shfmt, so blanket strictness is a separate toolchain-completion decision)..github/requirements-ci.txtpins pytest 9.1.1 and its pure-Python dependency closure (iniconfig, packaging, pluggy, pygments), one sha256 hash each, on the same reviewed Dependabot surface as the existing ruff pin — so the session-flow pytest suite runs in CI instead of skipping green.Verification
SKIPPED: plugins/demo/skipper.test.sh — 1 SKIP(s), first: pytest not installed for python3;--strict-skipsexits 1 with the strict message; an all-green tree printsAll plugin tests passed.with no skip section.bash plugins/session-flow/skills/retro/scripts/parse-transcript.test.shwith pytest installed:37 passed.shellcheckandshfmt -dclean on the runner.Related
scripts/check-silent-skips.sh— same anti-silent-skip posture, scoped to hook entry scripts; extending it to test harnesses stays a separate decision per the triage panel.