Repository navigation
perf(photos): avifenc --speed 2 for new tiles, and a deliberately mixed library - #653
Merged
Merged
Conversation
…ed library --speed 2 wins on both axes against the 4 the pipeline has been passing. Measured over 6 stems covering both yuv paths through the real ingest geometry (sips to TIFF, then zenc square at 600/400/200), shipping flags verbatim and varying only --speed: the 600 tier goes 153,138 to 150,948 bytes (-1.43%), all three tiers 262,158 to 257,836 (-1.65%), and mean ssimulacra2 on the 600 tier rises 79.439 to 79.601. The bill is +0.21 s per photo serial, so an incremental add of 5 goes 0.7 s to 1.7 s. speed 0 is rejected on its own numbers: +0.09 ssimulacra2 over speed 2 for 3.6x the time, and larger files than speed 2 on 2 of the 6 stems. Five pipeline call sites move (add-photos.sh, reencode-thumbnails.sh x3, add-car-photo.sh). The two /garage/encoding study generators keep their own efforts (6 and 4) and now say why: their byte counts are what that page prints, so aligning them means regenerating public/garage/enc, which is a separate change with a real diff. The library stays MIXED on purpose. Collecting the difference on the committed tiles is worth 118.6 KiB over 495 immutable-1y AVIF files, about 245 bytes each, of which a homepage visit fetches 12. It would cost 495 re-minted /i/ URLs, 495 fingerprint rows, the a/s/x keys of all 165 stems in hashes.json, a hand-edit to src/pages/garage/tooltips.html, and a p-dict roll for that page. Two speeds cost nothing operationally, because a /i/ URL names exact bytes per file and nothing downstream reads encoder settings. Nothing under public/ moves. Verified by building with and without the change: both trees hash to 37aeddb8ab629b16ef733d1c4a411c0ea2abbb238e41970102277462d81532e5.
github-actions
Bot
added
area: docs
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
aadhar-sh | 07c5b8e | Aug 28 2026, 03:25 PM |
Contributor
Wire-size diff vs merge base
Brotli q11, the quality the edge serves. Every number here is deterministic: Client assetsNo change. 25 files, 86.35 KiB Brotli total. PagesNo change. 54 files, 571.62 KiB Brotli total. Worker bundleNo change. 239.10 KiB gzip. Dictionary deltas (
|
oddharsh
added a commit
that referenced
this pull request
Aug 28, 2026
…s that had gone quiet (#660) #653 moved five pipeline call sites from --speed 4 to --speed 2 and shipped it forward-only, leaving the 495 committed AVIF tiles at speed 4. Its own body named the next forced re-encode as where the rest gets collected. This is that run. WHAT IT BOUGHT, and it beat the 6-stem projection. 138,609 bytes over 495 files, which is 135.4 KiB against the 118.6 KiB projected: -1.816% on the 600 tier, -1.968% on 400, -2.007% on 200, -1.882% across all three. Quality rose with it, mean ssimulacra2 +0.327 over 24 tier comparisons on 8 stems, better on 20 of 24 and smaller on 23 of 24. Both candidates are scored against the same lossless reference the encoder was handed, so neither gets one built from its own encode. THE JPG TIER DID NOT MOVE, on any of the 165. That is what said no histogram re-bake was owed, and it was checked three ways rather than assumed: a per-tier hash diff taken before hash-thumbnails.sh ran, that script's own j-hash tripwire staying silent, and derive:check reporting images/histograms fresh. git agrees, with 0 .jpg among the 990 changed files under public/i. TWO CONTROLS RAN FIRST, because a re-encode is interpretable only if the toolchain is the one that baked what it replaces. Rebuilding three stems at the OLD --speed 4 reproduced their committed bytes exactly, grayscale yuv400 included. And build.sh asks for a vendored-versus-brew re-measurement before any full re-encode, so that ran too: byte-identical on all 6 pairs across both speeds and both yuv paths, despite brew reporting libyuv unavailable. So every byte that moved is attributable to --speed and to nothing else. TWO SCANNERS HAD STOPPED MATCHING, and this run is what found them. check-photo-pipeline.ts walked public/garage and public/lwe for hardcoded /i/ refs. Neither has held an HTML file since the src/pages split on 2026-08-18, so it read 0 refs and reported a pass while the 12 real ones sat in src/pages/garage/tooltips.html. That is gotcha 40 from the scanner's side, and a grep for the old path finds nothing because the roots are a list of directories. Roots are src/pages, src/content and src/client now, with a floor of 12 so a collapse fails instead of passing. Control: against the stale refs it names the file and the dead hash, which is what it did here before the refs were fixed. derive:check's census walks the filesystem rather than git, so the vendored libavif tree build.sh clones reported 7 of its own scripts as undeclared generators. Anyone following build.sh got a red derive:check on a clean tree. The shell tier fires hardest and by construction, since it matches any mention of src/ and every C project has one. Skipped by path now, because the basenames are src and build, which are the two least safe names to put in SKIP. WHAT ELSE MOVED. fingerprints.json regenerated by hash-thumbnails.sh, 660 digests. hashes.json carries new a/s/x for all 165 stems and unchanged j. tooltips.html re-points 12 literal refs across 3 stems. config/derivations.json and its lock re-record images/fingerprints and images/hashes; the other four pinned derivations stayed fresh on their own and are untouched. NO DICTIONARY ROLL HERE. dict:roll reads production, so it can only run once this deploys, and the nightly dictionary-roll workflow opens that PR itself. a-dict is untouched because /i/ files are not shell assets. tooltips.html owes a p-dict snapshot on the next roll and is the only page whose bytes moved. Nothing reached R2. A thumbnail re-encode leaves the originals alone. config/tools.json's recorded versions are a real record for the pixel tiers now rather than the observed baseline its own comment admitted to, and it gained the flag half of that provenance. Measured the same day: --speed decides output, and --jobs decides it at 1 against any value above, while 2, 4 and 8 are identical at this tile size. The pipeline pins 4 and this run held it there, so one variable moved. Neither flag is added as a field, on the reasoning min_version already cost that file: a declaration nothing reads is the one nobody keeps true. photos:check, derive:check, tools:check, build, lint, typecheck and 520 tests green. The build reports 3234 internal refs across 54 documents all resolving, which is the backstop under the hardcoded-ref fix. Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
Every avifenc call site in the photo pipeline passed
--speed 4. That wasinherited rather than chosen, and it was leaving both bytes and quality on the
table at a cost of a fifth of a second per photo.
The measurement
Taken against true pipeline intermediates (sips to TIFF, then
zenc squareat600/400/200), 6 stems covering both yuv paths, shipping flags verbatim and
varying only
--speed:Incremental add of 5 photos: 0.7 s to 1.7 s. Full library (165): 21.8 s to 57 s.
So speed 2 is smaller AND better for about +0.21 s per photo.
speed 0 is rejected on its own numbers. It beats speed 2 by 0.09 points of
ssimulacra2 for 3.6x the time, and it produced LARGER files than speed 2 on 2 of
the 6 photos.
My own spot-check
One stem end to end, on this machine, brew avifenc 1.4.2 over aom 3.14.1 (the
binary
build.shrecords as byte-identical to the vendored build at thesesettings).
XT500018.HIF, orientation 8, yuv420, sips to TIFF thenzenc square --orient 8 --filter boxat 600/400/200:ssimulacra2 on the 600 tier against its own source PNG (avifdec to PNG first,
since ssimulacra2 will not decode AVIF): 73.857 at speed 4, 73.902 at speed
2. Wall clock on that tier: 0.248 s to 0.580 s (2.34x), which brackets the
2.09x serial figure above.
Same direction on both axes, so the handed-over numbers reproduce.
The fix
Five pipeline call sites move to
--speed 2:tools/photos/add-photos.sh:297(avif_encode, which all three tiers go through)tools/photos/reencode-thumbnails.sh:253,261,272tools/photos/add-car-photo.sh:99add-car-photo.shinherits the measurement rather than carrying its own: sameencoder at the same
-q 63 -d 10on a 480px tier, one image per invocation, andpublic/carsis not content-addressed so it re-mints no URL. The comment saysso.
The library stays MIXED, and that is the point
Existing tiles are speed 4. New ones are speed 2. Re-encoding to collect the
difference buys 118.6 KiB spread over 495 immutable-1y AVIF files, about 245
bytes per tile, of which a homepage visit fetches 12.
It would cost 495 re-minted
/i/URLs, 495 rewritten rows inpublic/images/fingerprints.json, thea/s/xkeys of all 165 stems inhashes.json, a hand-edit tosrc/pages/garage/tooltips.html(12 literal/i/refs across 3 stems: XT508687 a=5f22fe09, XT508940 a=0707c701, XT508756
a=5b99dd40), and a p-dict roll for that page. That is the trade that has broken
this build twice.
Two speeds cost nothing operationally, and
config/tools.json's own$commentalready makes the argument: "Nothing recorded which encoder made the 632 files
in public/i, and #394 re-encoded 316 of them on 2026-08-14, so claiming these
versions produced them would be inventing provenance." A
/i/URL names exactbytes per file, and nothing downstream reads encoder settings. The remaining
118.6 KiB gets collected whenever something forces a full re-encode anyway: a
LIBAVIF_TAGbump, a geometry change, or anotherreencode-thumbnails.shrun.Written down at the call site in
add-photos.sh, again atreencode-thumbnails.sh(the script that collects it), and in CLAUDE.md.The study generators are left alone, deliberately
gen-encoding-grids.shpasses--speed 6andgen-encoding-samples.shpasses--speed 4. Both stay, with a comment each recording the divergence.Aligning them means regenerating
public/garage/enc, which this PR isexplicitly a no-op for.
gen-encoding-samples.shPRINTS the byte counts thatget transcribed onto /garage/encoding, so changing the effort without re-running
it makes the script disagree with the numbers on the page. On a page whose job
is teaching encoder axes, a script that lies about its own output is worse than
three efforts. Picking one effort there is an editorial decision about what the
page teaches, and it belongs in the change that regenerates the assets.
What records the encoder settings
CLAUDE.mdpipeline diagram-q 63 -d 10 --speed 2, plus a prose block with the measurement and the mixed-library argumentCLAUDE.mdlibavif bullet--speed 4tools/photos/libavif/build.sh:23--speed 4docs/MAINTENANCE.md:1186--speed 4and says it has not been re-measuredconfig/tools.jsonrecordedmust not move because no binary changeddocs/PHOTO-PIPELINE.mddesign/PORTING.md--speed 4design/README.mdfiles it under "History (a record of decisions, NOT a spec to implement)" and says its numbers are stale; editing it would falsify a recordThe byte-identity claim (vendored avifenc == brew avifenc) was measured at
--speed 4on 2026-08-26. It is now flagged as unmeasured at speed 2 in allthree places rather than quietly re-pointed at settings nobody tested. Both are
still libavif 1.4.2 over aom 3.14.1 so identity is likely, and likely is not
evidence. It costs nothing until somebody re-encodes, because a new photo mints
a new URL under either binary.
Gates
bun run lintbun run typecheckbun testbun run buildbun run photos:checkbash -non all 6 edited scriptsThe no-op property, verified rather than asserted
git statusafter all five gates shows 8 files, every one undertools/ordocs. Nothing under
public/,src/, orconfig/.The stronger check, because "tools/ does not ship" is a claim: I hashed the
built tree, stashed the change, rebuilt from the clean tree, and hashed again.
Identical over every file in
.build/public. So no served byte moves, no/a/or
/i/URL is re-minted, and no dictionary roll is owed.