From 5ad42656bb134c69c45d8c27f13e7c70ba271a55 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Sat, 8 Aug 2026 05:38:03 +0000
Subject: [PATCH] fix(ci): pin Lighthouse's Chrome so the budget gate isn't
 runner-fleet noise
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

lighthouse-budget measured the ambient Chrome the ubuntu-24.04 runner image
ships instead of a pinned binary. On 2026-08-07 the runner fleet served
HeadlessChrome/150 and /151 to jobs minutes apart, so every grading run hit
the "baseline measured by a different browser" fail-closed path (working as
designed — a browser bump is indistinguishable from a real regression, so
the grader refuses to compare across it). Refreshing the baseline for
whichever version showed up (#1690) only bought a few hours before the next
mismatched runner failed it again.

Install Playwright's managed Chromium (same binary/cache the Production UI
and Visual baselines jobs already use, pinned by package-lock.json) and pass
its executablePath via PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH, which
run-lighthouse-budget.mjs already reads into CHROME_PATH for the lighthouse
CLI. No script changes needed.
---
 .github/workflows/ci.yml | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 10e004b5f1..8c524f7cbd 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -716,11 +716,44 @@ jobs:
         with:
           persist-credentials: false
 
-      # Lighthouse drives the Chrome that ships in the ubuntu-24.04 runner image, so
-      # no browser install is needed here (matching live-web-vitals.yml).
       - name: Setup Node and dependencies
         uses: ./.github/actions/setup-node-cached
 
+      # This job used to drive whatever Chrome ships in the ubuntu-24.04 runner image
+      # (matching live-web-vitals.yml, which measures a live domain and isn't graded
+      # against a committed baseline). That ambient browser is NOT pinned per commit —
+      # the runner fleet was observed serving HeadlessChrome/150 and /151 to jobs
+      # minutes apart on 2026-08-07 — so every grading run this job's baseline
+      # comparator treats a version drift as "evidence incomplete" and fails closed
+      # (by design: a browser bump is otherwise indistinguishable from a real
+      # regression). Refreshing the baseline for whichever version showed up bought a
+      # few hours before the next mismatched runner failed it again (#1690).
+      #
+      # Pin Playwright's own managed Chromium instead, the same binary the Production
+      # UI / Visual baselines jobs already use, cached by lockfile hash so it is
+      # identical across every runner for a given commit. run-lighthouse-budget.mjs
+      # already honours CHROME_PATH/PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH.
+      - name: Restore Chromium browser cache
+        id: pw-cache
+        uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
+        with:
+          path: ~/.cache/ms-playwright
+          key: playwright-chromium-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
+          restore-keys: |
+            playwright-chromium-${{ runner.os }}-
+
+      - name: Install Chromium
+        run: |
+          if [ "${{ steps.pw-cache.outputs.cache-hit }}" = "true" ]; then
+            npx playwright install-deps chromium
+            npx playwright install chromium
+          else
+            npx playwright install --with-deps chromium
+          fi
+
+      - name: Pin the Chromium executable path
+        run: echo "PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=$(node -e "console.log(require('playwright').chromium.executablePath())")" >> "$GITHUB_ENV"
+
       - name: Measure routes and grade against the baseline
         run: npm run verify:lighthouse -- --keep --dir lighthouse