-
Notifications
You must be signed in to change notification settings - Fork 0
ci: add dispatch-only live Web-Vitals workflow to discharge #017 #1385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
eab2183
ci: add dispatch-only live Web-Vitals workflow to discharge #017
BigSimmo 0f4395b
Merge origin/main into claude/live-web-vitals-workflow
claude 6972f36
fix(web-vitals): count a missing mobile run as a breach
claude 0d88075
Merge origin/main into claude/live-web-vitals-workflow
claude e2e05ea
fix(web-vitals): measure /documents/search, not the non-existent /doc…
claude a626ef0
Merge origin/main into claude/live-web-vitals-workflow
claude d860e63
fix(web-vitals): fail closed on desktop gaps and null metrics too
claude 46c85c4
fix(web-vitals): compute completeness over the whole matrix
claude 07d0fdb
fix(web-vitals): refuse an #017 verdict for a non-production origin
claude 8b3904d
fix(web-vitals): never close #017 on lab metrics; pin Lighthouse exactly
claude 44a4221
fix(web-vitals): reject runs that redirected or errored
claude 93a6955
fix(web-vitals): compare query strings when validating measured routes
claude a286c40
Merge origin/main into claude/live-web-vitals-workflow
claude 75a2ed8
Merge origin/main into claude/live-web-vitals-workflow
claude 2ba5abc
Merge remote-tracking branch 'origin/main' into claude/live-web-vital…
claude 7ed9785
docs(issues): record the #017 dispatch caveat with the instrument
claude b345746
fix(web-vitals): gate all verdict prose on the run being able to give…
claude e2a636c
docs(ledger): record the PR #1391 merge
claude 5516e63
Merge remote-tracking branch 'origin/main' into claude/live-web-vital…
claude 30b1cda
Merge remote-tracking branch 'origin/main' into claude/live-web-vital…
cursoragent 662c332
fix(web-vitals): close open review gaps before merge
cursoragent File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| name: Live Web Vitals baseline | ||
|
|
||
| # Ledger #017 gates every client payload decision (#012/#013/#016) behind | ||
| # reproducible mobile/desktop Web-Vitals evidence, and the 2026-07-28 latency | ||
| # audit added seven more findings behind the same gate. Nothing in the repo | ||
| # could produce that evidence: in-sandbox runtime vitals are blocked because the | ||
| # production server hard-requires Supabase secrets, and local Playwright timings | ||
| # rank routes against each other without discharging #017, which asks for the | ||
| # real origin. | ||
| # | ||
| # This runs Lighthouse against the deployed domain and keeps the JSON, so the | ||
| # baseline is reproducible after payload work rather than a one-off screenshot. | ||
| # | ||
| # Deliberately dispatch-only — no schedule. It is heavier than the plain GETs in | ||
| # live-domain-monitor.yml, and the decision it feeds is made a handful of times a | ||
| # year, not every six hours. Like that monitor it uses no secrets and no | ||
| # providers beyond what any anonymous visitor triggers. | ||
| # | ||
| # The decision rule is written down BEFORE the numbers are read, so the gate | ||
| # cannot be rationalised after the fact: | ||
| # mobile LCP < 2.5s AND CLS < 0.1 AND INP < 200ms on every route | ||
| # -> close #017 "metrics acceptable"; only the explicitly measured payload | ||
| # findings gated by #017 become WONTFIX (e.g. #013 route-chunk weight). | ||
| # #016's motion/CSS/waterfall/caching/dynamic-import items stay open | ||
| # unless they have separate evidence — LCP/CLS do not validate them. | ||
| # any breach | ||
| # -> only the breaching route's findings become actionable, ranked by | ||
| # measured contribution | ||
| # Record the verdict in docs/outstanding-issues.md against #017 either way. | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| routes: | ||
| description: "Comma-separated routes to measure" | ||
| required: false | ||
| # Every entry must be a real page route. `/documents` is not one — the | ||
| # documents segment holds only `search`, `source` and `[id]` with no | ||
| # `page.tsx` (see docs/site-map.md), so it would have measured the 404 | ||
| # document. `/documents/search` is the canonical documents-mode route. | ||
| default: "/,/therapy-compass,/documents/search,/dsm,/forms" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: live-web-vitals | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| measure: | ||
| name: Lighthouse against the live domain | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 30 | ||
| env: | ||
| # Repository variable overrides the default (e.g. a staging cutover), | ||
| # matching live-domain-monitor.yml. | ||
| LIVE_DOMAIN_URL: ${{ vars.LIVE_DOMAIN_URL || 'https://psychiatry.tools' }} | ||
|
BigSimmo marked this conversation as resolved.
|
||
| ROUTES: ${{ inputs.routes }} | ||
| # Pinned exactly, not `lighthouse@12`. That range is >=12.0.0 <13.0.0-0, | ||
| # so a patch published between a baseline run and its follow-up would | ||
| # change metric collection independently of the application and silently | ||
| # decalibrate the comparison this workflow exists to make. The resolved | ||
| # version is recorded in summary.json so a baseline states its own tooling. | ||
| LIGHTHOUSE_VERSION: "12.8.2" | ||
| steps: | ||
| - name: Check out the repository | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
|
BigSimmo marked this conversation as resolved.
|
||
| with: | ||
| # The job only reads files; nothing pushes. Keeping the token in the | ||
| # local git config for the rest of the job buys nothing (zizmor | ||
| # artipacked). | ||
| persist-credentials: false | ||
|
|
||
| - name: Use the repository Node runtime | ||
| uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 | ||
| with: | ||
| node-version: "24" | ||
|
|
||
| - name: Normalize the configured origin | ||
| run: | | ||
| set -euo pipefail | ||
| # LIVE_DOMAIN_URL may be configured with a trailing slash (common for | ||
| # "site root" variables). Appending "/forms" would then yield | ||
| # `https://…//forms`, which the summariser rejects as the wrong page | ||
| # after the server canonicalises the double slash. | ||
| normalized="${LIVE_DOMAIN_URL%/}" | ||
| echo "LIVE_DOMAIN_URL=$normalized" >> "$GITHUB_ENV" | ||
| echo "origin -> $normalized" | ||
|
|
||
| - name: Confirm the target is reachable before spending a Lighthouse run | ||
| run: | | ||
| set -euo pipefail | ||
| code="$(curl -sSL -o /dev/null -w '%{http_code}' --max-time 30 --retry 2 --retry-delay 10 --retry-all-errors "$LIVE_DOMAIN_URL/")" | ||
| echo "root -> $code" | ||
| if [ "$code" != "200" ]; then | ||
| echo "::error::$LIVE_DOMAIN_URL/ returned $code — not measuring" | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Measure each route on mobile and desktop | ||
| run: | | ||
| set -euo pipefail | ||
| mkdir -p web-vitals | ||
| IFS=',' read -ra route_list <<< "$ROUTES" | ||
| for strategy in mobile desktop; do | ||
| for route in "${route_list[@]}"; do | ||
| route="$(echo "$route" | xargs)" | ||
| [ -n "$route" ] || continue | ||
| # Filename-safe slug: "/" -> root, "/a/b" -> a-b | ||
| slug="$(echo "$route" | sed 's|^/||; s|/|-|g')" | ||
| [ -n "$slug" ] || slug="root" | ||
| out="web-vitals/${strategy}-${slug}" | ||
| echo "::group::$strategy $route" | ||
| # One flaky route must not discard the whole run, so a failure is | ||
| # a warning here; the summary step fails if NOTHING was produced. | ||
| npx --yes "lighthouse@$LIGHTHOUSE_VERSION" "$LIVE_DOMAIN_URL$route" \ | ||
|
BigSimmo marked this conversation as resolved.
cursor[bot] marked this conversation as resolved.
|
||
| --output=json --output-path="${out}.json" \ | ||
| --preset="$([ "$strategy" = desktop ] && echo desktop || echo perf)" \ | ||
| --only-categories=performance \ | ||
| --chrome-flags="--headless=new --no-sandbox --disable-dev-shm-usage" \ | ||
| --max-wait-for-load=60000 \ | ||
| --quiet || echo "::warning::lighthouse failed for $strategy $route" | ||
|
BigSimmo marked this conversation as resolved.
|
||
| echo "::endgroup::" | ||
| done | ||
| done | ||
|
|
||
| # ROUTES is passed so a route whose Lighthouse run failed above is counted | ||
| # as a breach. Grading only the reports that happen to exist would let a | ||
| # partially failed run read as "every mobile route passed". | ||
| - name: Summarise LCP / CLS / TBT against the decision rule | ||
| run: node scripts/summarise-web-vitals.mjs web-vitals "$ROUTES" | ||
|
|
||
| - name: Upload the reports | ||
| if: always() | ||
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 | ||
| with: | ||
| name: live-web-vitals | ||
| path: web-vitals/ | ||
| retention-days: 30 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.