Skip to content

feat(ci): diff wire size against the merge base and comment it on the PR - #289

Merged
oddharsh merged 1 commit into
mainfrom
perf/merge-base-diff
Aug 8, 2026
Merged

oddharsh merged 1 commit into
mainfrom
perf/merge-base-diff

Conversation

@oddharsh

@oddharsh oddharsh commented Aug 8, 2026

Copy link
Copy Markdown
Owner

The problem

Every performance number in this repo is absolute: a measurement against a constant somebody typed. perf-budget.mjs's own baseline history is the receipt for what that costs.

baseline outcome
86 KiB pre-consolidation
129.23 KiB sat 58% stale for months. The advisory fired on every run and CI printed "hard checks green" over it
204.24 KiB set 2026-08-04 after a direct measurement. Already over its 255.30 KiB alert by 2026-08-08 (258.34 KiB at 295ee97)

Four days from a fresh baseline to a firing advisory. A constant isn't wrong because someone chose it badly; it's wrong because it describes a moment and the bundle doesn't hold still. And "258.34 against 255.30" says nothing about which PR spent the 54 KiB.

The change

The differential half, modelled on how astral-sh/ruff runs its memory and ecosystem jobs: build the merge base, build HEAD, measure both, post the delta, gate on nothing.

  • scripts/perf-snapshot.mjs records the built tree's wire sizes (11 client assets, 46 pages, the /a/ immutable tier, worker bundle gzip, per-module esbuild attribution, dictionary-delta coverage) and diffs two recordings into markdown. One wrangler deploy --dry-run self-builds, so a snapshot costs ~12s.
  • .github/workflows/perf-diff.yml runs it per PR and updates one marker-keyed comment, matching the pattern already in dependabot-site-review.yml.

It is deliberately not part of validate. validate is the one required check on main, and a perf number that blocks a merge teaches people to widen the threshold rather than to read it.

What the report looks like

A tooling-only PR (this one):

Client assets

No change. 11 files, 56.82 KiB Brotli total.

Pages

No change. 46 files, 465.36 KiB Brotli total.

Worker bundle

No change. 258.34 KiB gzip.

That silence is the feature. A report that always has content stops being read.

A one-line nav.js edit:

Client assets

Total Brotli: 56.82 KiB → 56.87 KiB (+0.05 KiB, +0.1%) across 11 files, 1 changed.

file base head Δ
nav.js 13.41 KiB 13.46 KiB +0.05 KiB (+0.4%)

Pages

Total Brotli: 465.36 KiB → 465.33 KiB (-0.03 KiB, -0.0%) across 46 files, 38 changed.

No single file moved by 0.13 KiB or more.

Plus 38 files under the 0.13 KiB floor, net -0.03 KiB — usually content-hash churn in /a/ references, counted in the total above.

Three design decisions worth reviewing

Both sides are measured with HEAD's copy of the script, stashed in $RUNNER_TEMP before the first git switch. The base doesn't have the script on the PR that adds it, and measuring each side with its own copy would report a change to the measurement as a change in wire size.

The noise floor is asymmetric: pages get 128 bytes, client assets get none. A page carries /a/<name>.<hash8>.<ext> references, so touching one shared asset flips its hash and moves every page's compressed size (measured: 38 of 46 pages, max 41 bytes, net -0.03 KiB). An asset's bytes are its own content, so nothing but editing it can move them and every byte is signal. One floor everywhere would have hidden the 50-byte nav.js change that caused the churn. Sub-floor movers collapse into a counted aggregate line and stay in every total.

Nothing sampled is recorded. wrangler check startup read 9.6, 7.6, 6.4 and now 16.4 ms across identical bytes, a 2.6x spread. Diffing two draws from that manufactures findings in a comment whose credibility rests on every row being real. That check is reframed from "regression tripwire" to diagnostic; the tripwire is the deterministic bundle gzip and module attribution next to it. Same instinct as CodSpeed counting simulated instructions and ty's memory job pinning TY_MAX_PARALLELISM=1 — remove the noise source rather than widen a threshold around it.

Deliberately not done

The worker advisory is left firing rather than re-baselined. Re-baselining an unattributed 54 KiB is indistinguishable from silencing the check, and the diff is the thing that can now name which PR spent it. The observation is recorded in the baseline history instead.

Verification

Run locally against origin/main:

  • npm test — 179 pass
  • npm run perf-budget — hard checks green (1 advisory, the pre-existing bundle one)
  • npm run typecheck, npm run check-wrangler, npm run pages:check — clean
  • All five workflow YAMLs parse
  • End-to-end: recorded origin/main, recorded this branch, diffed — silent, as it should be for a tooling-only change
  • Synthetic 50-byte nav.js edit produces the table above, then reverted

🤖 Generated with Claude Code

Every performance number in this repo is absolute: a measurement against a
constant somebody typed. perf-budget.mjs's own baseline history records what
that costs. The 129.23 KiB worker baseline sat 58% stale for months while the
advisory fired on every single run and CI printed "hard checks green" over it.
The 204.24 KiB replacement, set on 2026-08-04 after a direct measurement, was
already over its alert four days later (258.34 KiB at 295ee97).

A constant is not wrong because someone chose it badly. It is wrong because it
describes a moment and the bundle does not hold still.

So add the differential half, modelled on how astral-sh/ruff runs its memory
and ecosystem jobs: build the merge base, build HEAD, measure both, post the
delta, gate on nothing.

- scripts/perf-snapshot.mjs records the built tree's wire sizes (client assets,
  46 pages, the /a/ immutable tier, worker bundle gzip, per-module esbuild
  attribution, dictionary-delta coverage) and diffs two recordings into
  markdown. One dry-run self-builds, so a snapshot costs ~12s.
- .github/workflows/perf-diff.yml runs it per PR and updates one marker-keyed
  comment. It is deliberately NOT part of `validate`: `validate` is the one
  required check on main, and a perf number that blocks a merge teaches people
  to widen the threshold rather than to read it.

Two things the measurement had to get right. Both sides are measured with
HEAD's copy of the script, stashed outside the tree before the first git
switch, because the base does not have it on the PR that adds it and measuring
each side with its own copy would report a change to the measurement as a
change in wire size. And the noise floor is asymmetric: pages get 128 bytes,
assets get none. A page carries /a/<name>.<hash8>.<ext> references, so touching
one shared asset flips its hash and moves every page a few bytes (measured: a
one-line nav.js edit moved 38 of 46 pages, max 41 bytes, net -0.03 KiB); an
asset's bytes are its own content, so every byte of movement is signal. One
floor everywhere would have hidden the 50-byte nav.js change that caused the
churn. Sub-floor movers collapse into a counted line and stay in the totals.

Nothing sampled is recorded. `wrangler check startup` read 9.6, 7.6, 6.4 and
now 16.4 ms across identical bytes, and diffing two draws from a 2.6x spread
manufactures findings in a comment whose credibility rests on every row being
real. That check is reframed here from "regression tripwire" to diagnostic; the
tripwire is the deterministic bundle gzip and module attribution beside it.

The worker advisory is left firing rather than re-baselined. Re-baselining an
unattributed 54 KiB is indistinguishable from silencing the check, and the
diff is the thing that can now name which PR spent it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
aadhar-sh 00d7579 Aug 08 2026, 08:19 PM

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Wire-size diff vs merge base

2f781a6 → 0d0d39a

Brotli q11, the quality the edge serves. Every number here is deterministic:
an unchanged file produces no row. Advisory — this check fails on nothing.

Client assets

No change. 11 files, 57.99 KiB Brotli total.

Pages

No change. 46 files, 465.79 KiB Brotli total.

Worker bundle

No change. 261.74 KiB gzip.

Dictionary deltas (pd/)

No change. 134 deltas, 373.59 KiB total.

@oddharsh
oddharsh merged commit 01e079b into main Aug 8, 2026
7 of 8 checks passed
@oddharsh
oddharsh deleted the perf/merge-base-diff branch August 8, 2026 20:47
oddharsh added a commit that referenced this pull request Aug 8, 2026
#294)

#289 and #293 shipped and ramped to 100% without a changelog entry, so
/updates and /restore still name v178 as the newest build. This stages the
entry that describes them.

Nothing here reaches D1. bump-version.sh writes the projection only, and
`deploy:promote` records the row once traffic actually reaches 100% — which
is why the log is a follow-up commit rather than part of the work it
describes.

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant