Skip to content

fix(scripts): verify actual Chromium binary presence in check:playwright-browser-revision - #1965

Merged
BigSimmo merged 9 commits into
mainfrom
claude/playwright-browser-revision-check
Aug 14, 2026
Merged

fix(scripts): verify actual Chromium binary presence in check:playwright-browser-revision#1965
BigSimmo merged 9 commits into
mainfrom
claude/playwright-browser-revision-check

Conversation

@BigSimmo

Copy link
Copy Markdown
Owner

Summary

  • Fixes check:playwright-browser-revision (ledger #312): the check could report ok: true even when the pinned Chromium revision had no launchable binary on disk anywhere. It now always resolves the effective browsers root (an explicit PLAYWRIGHT_BROWSERS_PATH, or Playwright's own default managed-cache directory when unset) and verifies a real executable exists for the pinned revision — not just a same-named directory — using an executable-layout table mirrored from playwright-core's own EXECUTABLE_PATHS.
  • Two concrete gaps are closed:
    1. The "no root forced" branch previously skipped the disk entirely and unconditionally returned ok: true, status: "managed-or-unconstrained". It now inspects the resolved root and returns not-installed / binary-missing when the pinned revision isn't actually present.
    2. Even the designated-container ("container-aligned") branch only checked directory names. A same-named empty directory (partial/corrupt install) previously passed; it now requires a real, launchable file at the expected path for the current platform/architecture.
  • Reproduced live in this session's own container: PLAYWRIGHT_BROWSERS_PATH=/opt/pw-browsers was set without PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD, the container only had chromium-1194 on disk, and the pinned revision (from playwright-core@1.62.1) is 1234. The old check reported {"ok": true, "status": "managed-or-unconstrained", ...} for that exact state. The fixed check now correctly reports not-installed for that same state, and container-aligned/installed only once a real 1234 binary is verified on disk.

Verification

  • npm run verify:pr-local — completed: check:runtime, check:installed-lock-parity, format:changed, lint, typecheck, test (602 files / 6518 tests passed, 4 skipped), check:rag:fixtures, check:medication-interactions; build and offline RAG production contracts skipped (no build-affecting or RAG-scoped changes). The gate's final step, check:medication-lexicon-report, fails on this branch. This PR does not touch either input the check compares (data/medication-interaction-index.json, docs/medication-interaction-lexicon-review.md) — git status was clean apart from this PR's own two files when the check ran, and both inputs are byte-identical to origin/main (both last touched by fix(medications): reach lithium, retire the dead z-drugs term, make lexicon review flags evidence-bearing #1923, unrelated to Playwright). Since the check is a deterministic function of those two files' content, this is a pre-existing drift on main, not something this diff caused.
  • Focused Vitest: tests/check-playwright-browser-revision.test.ts (10/10 passing, includes new regression tests for the exact false-OK scenario and for a same-named-but-empty revision directory) and tests/playwright-browser-preflight.test.ts (12/12 passing, unaffected — untouched file).
  • Not run: verify:ui / verify:release (no UI, routing, or release-confidence change), eval:retrieval:quality / eval:rag (no retrieval/ranking/answer-generation change), check:production-readiness (no clinical/Supabase/deployment change).

Risk and rollout

  • Risk: Low. Changes are confined to scripts/check-playwright-browser-revision.mjs (a standalone diagnostic script invoked manually via npm run check:playwright-browser-revision, and imported only by scripts/playwright-browser-preflight.mjs for the already-tested container-revision-drift fail path) and its test file. Nothing in CI or verify:* invokes check:playwright-browser-revision directly today (confirmed by repo-wide search), so this makes an under-tested diagnostic accurate rather than changing any enforced gate's pass/fail outcome.
  • Rollback: Revert this commit; the prior (looser) check returns.
  • Provider or production effects: None.

RAG impact: N/A — no path under src/lib/rag/**, retrieval, ranking, or the eval harness is touched.
Clinical Governance Preflight: N/A — no ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output is touched.

Note on solo scope: Running classifyPullRequestFiles(['scripts/check-playwright-browser-revision.mjs', 'tests/check-playwright-browser-revision.test.ts']) from scripts/pr-policy.mjs directly against this diff returns { clinicalRisk: false, operationalRisk: false, ragRanking: false, ui: false } — neither file matches any pattern in operationalRiskPatterns (that list is scoped to .github/workflows, package(-lock).json, next|playwright|vitest.config.*, and Dockerfile/railway/nixpacks files, not arbitrary scripts//tests/ paths). This PR is opened solo because it is a standalone ledger-sweep item from a fresh origin/main branch with no other currently-open, already-assembled PR in this session to bundle it with — not because the classifier flags it as risky.

Notes

  • Ledger #312 (docs/outstanding-issues.md) suggested, as a minimum fix, making the check "say plainly which browsers are present and which the locked Playwright version requires." This PR goes further, per the explicit task instructions: it verifies actual binary presence on disk rather than only directory-name/version-metadata comparison, so a false OK is structurally impossible rather than just less likely to be misread.
  • Scope is intentionally Chromium-only, matching the existing script's scope (Firefox/Webkit revision checks are out of scope for this fix).

Generated by Claude Code

…ry presence (#312)

The check could report ok:true even when the pinned Chromium revision had
no launchable binary anywhere on disk. The "no root forced" branch skipped
the disk entirely and unconditionally passed; the designated-container
branch only compared directory names, so a same-named empty directory also
passed. Both are now real, disk-verified checks against Playwright's own
executable layout, reproduced live against this session's own container
(PLAYWRIGHT_BROWSERS_PATH=/opt/pw-browsers with only chromium-1194 present
while playwright-core@1.62.1 pins 1234 — the old check reported OK for
that exact state).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLbw9qpfjv5CeNz6XpmteN
@supabase

supabase Bot commented Aug 14, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 23 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: eabdbf99-98a7-4e8b-8403-3ccf8dd5c06c

📥 Commits

Reviewing files that changed from the base of the PR and between 885c613 and e795f2d.

📒 Files selected for processing (5)
  • docs/branch-review-records/1cd9399241d3cbac65652b1b8008514d5c4f4b2c2e82254fec86eb5c50ee6784.record.md
  • docs/branch-review-records/7060c1fcf1c15570212c29f7afe5379d101d7e3ada1bb102dcd1631e83d7bf55.record.md
  • docs/branch-review-records/91679c0f8d364c93cfbe6f9c54463ec90d9dcbbfa158d1876e0ed8744f2c0162.record.md
  • scripts/check-playwright-browser-revision.mjs
  • tests/check-playwright-browser-revision.test.ts

Comment @coderabbitai help to get the list of available commands.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLbw9qpfjv5CeNz6XpmteN

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 24e8fd0409

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/check-playwright-browser-revision.mjs Outdated
Comment thread scripts/check-playwright-browser-revision.mjs Outdated
@BigSimmo
BigSimmo enabled auto-merge (squash) August 14, 2026 16:20
@github-actions

Copy link
Copy Markdown
Contributor

CI triage

CI failed on this PR. Automated classification of the 2 failed job(s):

  • Unit coverageneeds investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
  • PR requiredneeds investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.

Compared with main CI run #10953 (success).

Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger.

@BigSimmo
BigSimmo merged commit 3ec6116 into main Aug 14, 2026
26 checks passed
@BigSimmo
BigSimmo deleted the claude/playwright-browser-revision-check branch August 14, 2026 18:43
BigSimmo pushed a commit that referenced this pull request Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants