Skip to content

photos: one JPEG encode per thumbnail, not three - #394

Merged
oddharsh merged 1 commit into
mainfrom
photos/single-encode
Aug 15, 2026
Merged

oddharsh merged 1 commit into
mainfrom
photos/single-encode

Conversation

@oddharsh

Copy link
Copy Markdown
Owner

Phase 1 put every tile through four JPEG encodes: sips wrote a q100 JPEG,
jpegtran rotated it losslessly, then sips -Z and sips -c EACH decoded
and re-encoded it, and only then did zenc encode at q84. The two middle
ones existed for no reason beyond the intermediate being a JPEG.

The geometry runs on a lossless intermediate now. TIFF for the resize and
crop, because that is what sips writes fastest (a PNG of the same frame
costs 13x), then one PNG because that is what zenc and avifenc read. The
mobile AVIF gains the most: it was a JPEG resized from a JPEG.

Measured on the shipped output, all 158 re-encoded and scored against a
lossless reference square built by the same resampler:

median ssimulacra2 66.09 -> 80.38 (+13.61)
better on 158 of 158, worse on none
bytes 5448 KB -> 5477 KB (+0.5%)

jpegtran keeps the rotation. sips -r was the obvious replacement and it
disagrees with jpegtran on Orientation 6, which is one photo in this
corpus and would have been silently wrong.

Nine hashed references across tooltips.html and the baseline fixture are
repointed, since new bytes mint new /i/ URLs.

reencode-thumbnails.sh gets the identical change; the two encode paths are
meant to stay in sync and the header says so.

Phase 1 put every tile through four JPEG encodes: sips wrote a q100 JPEG,
jpegtran rotated it losslessly, then `sips -Z` and `sips -c` EACH decoded
and re-encoded it, and only then did zenc encode at q84. The two middle
ones existed for no reason beyond the intermediate being a JPEG.

The geometry runs on a lossless intermediate now. TIFF for the resize and
crop, because that is what sips writes fastest (a PNG of the same frame
costs 13x), then one PNG because that is what zenc and avifenc read. The
mobile AVIF gains the most: it was a JPEG resized from a JPEG.

Measured on the shipped output, all 158 re-encoded and scored against a
lossless reference square built by the same resampler:

  median ssimulacra2  66.09 -> 80.38   (+13.61)
  better on 158 of 158, worse on none
  bytes 5448 KB -> 5477 KB (+0.5%)

jpegtran keeps the rotation. `sips -r` was the obvious replacement and it
disagrees with jpegtran on Orientation 6, which is one photo in this
corpus and would have been silently wrong.

Nine hashed references across tooltips.html and the baseline fixture are
repointed, since new bytes mint new /i/ URLs.

reencode-thumbnails.sh gets the identical change; the two encode paths are
meant to stay in sync and the header says so.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 15, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
aadhar-sh 30d7a05 Aug 15 2026, 03:27 AM

@github-actions

Copy link
Copy Markdown
Contributor

Wire-size diff vs merge base

83f2ada → 4af342e

Brotli q11, the quality the edge serves. Every number here is deterministic:
an unchanged file produces no row. Advisory — this check fails on nothing.

Client assets

No change. 20 files, 73.98 KiB Brotli total.

Pages

Total Brotli: 496.67 KiB → 496.67 KiB (-0.00 KiB, -0.0%) across 48 files, 3 changed.

No single file moved by 0.13 KiB or more.

Plus 3 files under the 0.13 KiB floor, net -0.00 KiB — usually content-hash churn in /a/ references, counted in the total above.

Worker bundle

No change. 284.68 KiB gzip.

Dictionary deltas (pd/)

144 → 144 deltas, 458.09 KiB → 461.76 KiB (+3.67 KiB).

@oddharsh
oddharsh merged commit 6a4c401 into main Aug 15, 2026
7 of 8 checks passed
@oddharsh
oddharsh deleted the photos/single-encode branch August 15, 2026 03:29
oddharsh added a commit that referenced this pull request Aug 24, 2026
…erved

#394 re-encoded 316 JPEG thumbnails on 2026-08-14, minting a new content hash
for each, and re-baked no histograms. images/histograms.json has described
thumbnails nobody is served ever since. All 158 photos move here: 31.2% of the
40,448 bins, mean 0.95 of 100, but 65 photos carry at least one bin past 15 and
the worst is 59, so this is a visible bar rather than decoder noise.

The decoder was the obvious suspect and is innocent. image 0.25.10 and
zune-jpeg 0.5.15 are pinned identically at every commit involved, and
Cargo.lock has not moved since the pipeline was relocated. What changed is the
INPUT, which is the one thing a version pin cannot hold still: the j-tier hash
for L1000069_3 went 0e092e5f -> 2b30aedc in #394, and the bars stayed.

WHY NOTHING CAUGHT IT, which is the more useful half.

photos:check compares histograms.json against images/meta/. Since #441 made
meta/ build OUTPUT that build.mjs derives FROM histograms.json, those two agree
no matter what the JPEGs contain. It is gotcha 24's shape: two values from one
source, so the check proves that copying works.

zenc histogram --check does read the pixels, and pointing it at a built tree is
guaranteed to report drift whether or not any exists. The index packs each bin
into ONE character over 64 levels while zenc computes 0..100, so a build-derived
meta/ has been through a lossy 101 -> 64 -> 101 round trip. Measured: the
derived tree reproduces roundTrip(committed) on 100.00% of 40,448 bins, so the
packing is faithful and that drift was an artifact of where it was pointed. The
real signal was underneath it and needed the pre-#441 committed meta to see.

THE FIX AND THE GUARD.

The data: materialise meta/, re-run zenc against the current j-tier, roll it
back through build-histogram-index. The result satisfies zenc's own check
("histograms match on disk for 158 photos") and round-trips into the built
meta/ at 256 of 256 bins for a spot-checked stem.

The guard: hash-thumbnails.sh snapshots each stem's j hash and warns when one
moves, naming the re-bake. That script is the choke point where new bytes become
served bytes. add-photos.sh re-bakes the whole library every run and was never at
risk; the gap is a standalone reencode-thumbnails.sh, whose own header tells you
to re-run hash-thumbnails.sh and stops there.

Controlled both ways: a no-op run stays silent, and a stem whose JPEG bytes
change prints the warning naming it. The control mutated real files and the
tree was restored to 632 tracked / 632 on disk afterwards.

353 tests, typecheck, lint, build, photos:check and the 142-route oracle green.
oddharsh added a commit that referenced this pull request Aug 24, 2026
…erved (#535)

#394 re-encoded 316 JPEG thumbnails on 2026-08-14, minting a new content hash
for each, and re-baked no histograms. images/histograms.json has described
thumbnails nobody is served ever since. All 158 photos move here: 31.2% of the
40,448 bins, mean 0.95 of 100, but 65 photos carry at least one bin past 15 and
the worst is 59, so this is a visible bar rather than decoder noise.

The decoder was the obvious suspect and is innocent. image 0.25.10 and
zune-jpeg 0.5.15 are pinned identically at every commit involved, and
Cargo.lock has not moved since the pipeline was relocated. What changed is the
INPUT, which is the one thing a version pin cannot hold still: the j-tier hash
for L1000069_3 went 0e092e5f -> 2b30aedc in #394, and the bars stayed.

WHY NOTHING CAUGHT IT, which is the more useful half.

photos:check compares histograms.json against images/meta/. Since #441 made
meta/ build OUTPUT that build.mjs derives FROM histograms.json, those two agree
no matter what the JPEGs contain. It is gotcha 24's shape: two values from one
source, so the check proves that copying works.

zenc histogram --check does read the pixels, and pointing it at a built tree is
guaranteed to report drift whether or not any exists. The index packs each bin
into ONE character over 64 levels while zenc computes 0..100, so a build-derived
meta/ has been through a lossy 101 -> 64 -> 101 round trip. Measured: the
derived tree reproduces roundTrip(committed) on 100.00% of 40,448 bins, so the
packing is faithful and that drift was an artifact of where it was pointed. The
real signal was underneath it and needed the pre-#441 committed meta to see.

THE FIX AND THE GUARD.

The data: materialise meta/, re-run zenc against the current j-tier, roll it
back through build-histogram-index. The result satisfies zenc's own check
("histograms match on disk for 158 photos") and round-trips into the built
meta/ at 256 of 256 bins for a spot-checked stem.

The guard: hash-thumbnails.sh snapshots each stem's j hash and warns when one
moves, naming the re-bake. That script is the choke point where new bytes become
served bytes. add-photos.sh re-bakes the whole library every run and was never at
risk; the gap is a standalone reencode-thumbnails.sh, whose own header tells you
to re-run hash-thumbnails.sh and stops there.

Controlled both ways: a no-op run stays silent, and a stem whose JPEG bytes
change prints the warning naming it. The control mutated real files and the
tree was restored to 632 tracked / 632 on disk afterwards.

353 tests, typecheck, lint, build, photos:check and the 142-route oracle green.

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh added a commit that referenced this pull request Aug 24, 2026
…nobody read (#576)

Third version surface, and the third distinct shape. bun is an exact pin nothing
updates, so it got a bumper. node is a bare major, so it got a support-window
watcher. These are unpinned binaries whose output SHIPS, so what they need is a
record.

Seven of the thirteen declared tools produce committed bytes: exif-sooc, sips,
jpegtran, cjpeg, avifenc, cwebp, ffmpeg. public/i is content-addressed, so
re-encoding under a different encoder mints 632 new URLs, orphans every a-dict
snapshot naming the old hash, and can leave derived data describing pixels
nobody serves. That last one is gotcha 41, where #394 re-encoded 316 thumbnails
and the histograms went on describing the old pixels for nine days.

So tools:check gains a VERSION tier that reads each binary and compares it
against a `recorded` field, and drift is a NOTICE rather than a failure. For an
encoder, newer is not automatically take-it: a bump means re-encoding, re-hashing
and rolling the dictionaries, so the useful signal is the local one. `brew
outdated` already answers whether something newer exists, and this does not
duplicate it.

`recorded` is a BASELINE OBSERVED TODAY rather than a reconstruction. Nothing
recorded which encoder made the current artifacts, so claiming these versions
produced them would be inventing provenance. It becomes a real record on the
next re-encode.

Reading a version has no convention, so each tool declares its own flag and
pattern: `exif-sooc 0.2.0`, `jq-1.7.1-apple`, `sips-316`, `Version: 1.4.2`, a
bare `1.6.0`, and mozjpeg's two answering on stderr. ssimulacra2 and
butteraugli_main report nothing and say so with a reason; both are metrics, so no
shipped byte depends on them. A declared pattern that stops matching FAILS,
because a version tier that silently reads nothing is the rot the floors exist
to catch.

ONE NUMBER LIVED IN SIX PLACES AND THE CANONICAL ONE WAS UNREAD.
`EXIF_SOOC_MIN=0.2.0` is written out in five shell scripts while
config/tools.json carried a `min_version` that nothing consulted, so the file
designed to be the single declaration was the only one nobody read. That is the
failure its own header says it was created to fix, one field further in. The
declaration tier now asserts both directions, and both are proven by control:
drifting one guard fails by name, and a declared floor nothing enforces fails
too. The version tier's three red paths are exercised as well (below minimum,
recorded drift, a pattern that no longer matches).

The file came out cleaner than it went in: 18 type errors to 0, so it leaves the
ratchet baseline entirely and the total drops 574 to 556.

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh added a commit that referenced this pull request Aug 26, 2026
Four incidents here share one shape: a content-addressed input moved and
something derived from it did not. #394 re-encoded 316 thumbnails and re-baked
no histograms, so the bars described pixels nobody had been served for nine
days. An /a/ asset changing hash orphans every a-dict snapshot naming the old
one. The search index froze twice under two unrelated causes. Each bought one
bespoke check, and each was written to catch the thing that had already
happened.

config/derivations.json records what each committed derived artifact was made
FROM as a content hash of its inputs, and `bun run derive:check` re-hashes them.
Adding an artifact is a data edit rather than a new script.

It runs no generator and writes nothing without --lock. Both halves are
requirements: several sessions work in this tree at once, so a check that
rebuilt artifacts in place would be writing over somebody else's uncommitted
work, and the three families that actually go stale cannot be rebuilt from a
checkout anyway (images/meta/ is a local artifact, metadata.json reads the SOOC
originals, the og cards capture production). Hashing inputs works on all of them
for the cost of reading 15 MB, in about a second.

Two tiers, because the alternative to an honest note is a summary line that
covers more than the check does. `pinned` is checkable here with no tool and no
network. `unverifiable` says which inputs are not in this repository and why,
and never counts as a pass. Same standing as the two tools in tools.json that
report no version.

THE BASELINE IS VERIFIED, NOT ASSUMED. Recording a digest nobody had checked
would make this decoration on day one, so the library was re-baked from the
committed JPEG tiers with `zenc histogram` and packed through the exported
packHistogram: 165 of 165 stems byte-identical, 42,240 bins, zero drift. Both
failure shapes were then run as controls. Perturbing one JPEG names it under
`changed`; renaming one to a new hash8, which is the real re-encode shape, names
it under both `added` and `removed`.

Deliberately not covered, so the claim stays honest: alt.json and semantics.json
depend on the SET of stems rather than the pixel bytes, and pinning them to
bytes would invent staleness on every re-encode; the MCP cards already have a
contract test that deep-equals them against each server's live tools/list, which
is stronger than a digest. Both are written down in the declaration.

The kernel is node-only with zero dependencies and knows nothing about this
site, so it lifts into any repository that commits derived artifacts.
oddharsh added a commit that referenced this pull request Aug 26, 2026
Four incidents here share one shape: a content-addressed input moved and
something derived from it did not. #394 re-encoded 316 thumbnails and re-baked
no histograms, so the bars described pixels nobody had been served for nine
days. An /a/ asset changing hash orphans every a-dict snapshot naming the old
one. The search index froze twice under two unrelated causes. Each bought one
bespoke check, and each was written to catch the thing that had already
happened.

config/derivations.json records what each committed derived artifact was made
FROM as a content hash of its inputs, and `bun run derive:check` re-hashes them.
Adding an artifact is a data edit rather than a new script.

It runs no generator and writes nothing without --lock. Both halves are
requirements: several sessions work in this tree at once, so a check that
rebuilt artifacts in place would be writing over somebody else's uncommitted
work, and the three families that actually go stale cannot be rebuilt from a
checkout anyway (images/meta/ is a local artifact, metadata.json reads the SOOC
originals, the og cards capture production). Hashing inputs works on all of them
for the cost of reading 15 MB, in about a second.

Two tiers, because the alternative to an honest note is a summary line that
covers more than the check does. `pinned` is checkable here with no tool and no
network. `unverifiable` says which inputs are not in this repository and why,
and never counts as a pass. Same standing as the two tools in tools.json that
report no version.

THE BASELINE IS VERIFIED, NOT ASSUMED. Recording a digest nobody had checked
would make this decoration on day one, so the library was re-baked from the
committed JPEG tiers with `zenc histogram` and packed through the exported
packHistogram: 165 of 165 stems byte-identical, 42,240 bins, zero drift. Both
failure shapes were then run as controls. Perturbing one JPEG names it under
`changed`; renaming one to a new hash8, which is the real re-encode shape, names
it under both `added` and `removed`.

Deliberately not covered, so the claim stays honest: alt.json and semantics.json
depend on the SET of stems rather than the pixel bytes, and pinning them to
bytes would invent staleness on every re-encode; the MCP cards already have a
contract test that deep-equals them against each server's live tools/list, which
is stronger than a digest. Both are written down in the declaration.

The kernel is node-only with zero dependencies and knows nothing about this
site, so it lifts into any repository that commits derived artifacts.
oddharsh added a commit that referenced this pull request Aug 26, 2026
…#609)

* feat(tools): derive:check, one graph instead of a tripwire per incident

Four incidents here share one shape: a content-addressed input moved and
something derived from it did not. #394 re-encoded 316 thumbnails and re-baked
no histograms, so the bars described pixels nobody had been served for nine
days. An /a/ asset changing hash orphans every a-dict snapshot naming the old
one. The search index froze twice under two unrelated causes. Each bought one
bespoke check, and each was written to catch the thing that had already
happened.

config/derivations.json records what each committed derived artifact was made
FROM as a content hash of its inputs, and `bun run derive:check` re-hashes them.
Adding an artifact is a data edit rather than a new script.

It runs no generator and writes nothing without --lock. Both halves are
requirements: several sessions work in this tree at once, so a check that
rebuilt artifacts in place would be writing over somebody else's uncommitted
work, and the three families that actually go stale cannot be rebuilt from a
checkout anyway (images/meta/ is a local artifact, metadata.json reads the SOOC
originals, the og cards capture production). Hashing inputs works on all of them
for the cost of reading 15 MB, in about a second.

Two tiers, because the alternative to an honest note is a summary line that
covers more than the check does. `pinned` is checkable here with no tool and no
network. `unverifiable` says which inputs are not in this repository and why,
and never counts as a pass. Same standing as the two tools in tools.json that
report no version.

THE BASELINE IS VERIFIED, NOT ASSUMED. Recording a digest nobody had checked
would make this decoration on day one, so the library was re-baked from the
committed JPEG tiers with `zenc histogram` and packed through the exported
packHistogram: 165 of 165 stems byte-identical, 42,240 bins, zero drift. Both
failure shapes were then run as controls. Perturbing one JPEG names it under
`changed`; renaming one to a new hash8, which is the real re-encode shape, names
it under both `added` and `removed`.

Deliberately not covered, so the claim stays honest: alt.json and semantics.json
depend on the SET of stems rather than the pixel bytes, and pinning them to
bytes would invent staleness on every re-encode; the MCP cards already have a
contract test that deep-equals them against each server's live tools/list, which
is stronger than a digest. Both are written down in the declaration.

The kernel is node-only with zero dependencies and knows nothing about this
site, so it lifts into any repository that commits derived artifacts.

* feat(tools): set-valued inputs and live coverage, which found a dead generator

Two artifacts could not join the graph as it stood. alt.json and semantics.json
are keyed by stem, and re-encoding a photo does not change what is in the
picture, so pinning them to pixel bytes would report them stale on every encode
and teach somebody to re-record without looking. `set` projects each input path
to a key and digests the key SET instead, so they go stale for the one reason
they can be: a photo arrived or left.

The digest alone is still too weak for an artifact built one key at a time. A
resumable generator that dies halfway leaves a file covering most of the set,
and --lock afterwards would call it fresh forever. `covers` reads the output and
asserts an entry per input. It is computed live rather than recorded, which
makes it the one verdict --lock cannot clear, and there is a test for exactly
that.

IT FOUND A LIVE BUG ON THE DAY IT WAS WRITTEN. semantics.json covered 158 of
165 photos. The cause was not a missed run: gen-photo-semantics.ts joined
ROOT + "www" five times, and www/ was deleted by the 2026-08-18 split, so the
script had been unrunnable for a week and the seven photos added since carried
no retrieval terms. tools/lib/lens-chips.ts had the same rot pointing at
www/_worker.js/lens.js, taking lens-seed and lens-warm with it. Both repaired;
lens-chips reads 7 chips again, over a floor of 5.

Regenerating semantics is purely additive, verified rather than assumed: all 158
pre-existing records are byte-identical and 7 stems are new, with no term
dropped and no vision tier to lose.

The general lesson is in gotcha 40 alongside the nine sites it already lists. A
derived artifact is a witness for its generator, so checking the OUTPUT found a
broken producer that a sweep of path expressions had already walked past once:
gen-photo-semantics.ts had been audited before, and that audit fixed ROOT while
leaving www in every join beneath it.

* feat(tools): a writer census, so a generator nobody declared cannot exist

The graph answers "is this artifact stale". It could not answer the question
underneath it, which is whether an artifact has a declaration at all, and that
is the gap gotcha 41 actually fell through: histograms.json had no declaration,
so there was nothing for any check to be wrong about.

Every file under tools/ and pipelines/ that writes is now classified exactly
once. It is either named by some derivation's regenerate command or exempt with
a reason. 29 writers, 29 classified. A new generator is covered by EXISTING
rather than by somebody remembering, which is the property worth copying from
tools.json's declaration tier. Controls both ways: adding a throwaway generator
fails the check by name and exits 1, and removing it goes green.

It found two things immediately. og/cards was declared but invisible to the
census, because its regenerate said `bun run og-cards` and named no script, so
regenerate commands now name their file and a test asserts that file exists.
And gen-quadgram-table.ts wrote to ../lwe/quadgrams.txt relative to
tools/photos/, which is tools/lwe/, a directory that has never existed under any
of this repo's three layouts; the artifact lives at public/lwe/quadgrams.txt.
That is the twelfth instance of gotcha 40 and the third in one afternoon, none
of them found by looking for them.

The scanner reads JS, TS and Python and NOT shell, because a shell script writes
with redirects, cp, mv, tee and sed -i, and a loose matcher would spend its life
on false positives while still missing half of them. The limit is written down
at the top of derive-writers.ts rather than left implied.

INCREMENTAL BUILD WAS THE OTHER FOLLOW-UP AND IS DELIBERATELY NOT HERE. The full
build is 2.0-2.15s over 1606 files, dev serves from the symlink farm rather than
the build, and the output is byte-identical across back-to-back runs with its
one date derived from the photo index rather than the wall clock. So skipping
steps would save about two seconds while putting a verified byte-identical
property at risk against a content-addressed deploy, which is the trade gotcha
28 already refused. The census was measured to be worth more and cost less.

* chore(tools): classify relock-dependabot.ts in the writer census

It landed on main while this branch was open and writes docs/DEPENDENCIES.md,
so the census caught it on the rebase. That is the mechanism working on a real
change rather than on the synthetic control: a generator arriving from somewhere
else has to be classified before it can merge.

Exempt rather than declared. It rewrites version pins inside prose that a human
edits, and contract-docs-dependencies-md-states-version-pins-in-prose.test.mjs
already holds those pins to the lockfile, which is the check a digest here would
duplicate.

* feat(tools): close the census's shell hole, and the hole was real

The census shipped reading JS, TS and Python with shell written down as a known
limit. Taking the limit seriously found four shell scripts writing committed
bytes that nothing classified: add-car-photo.sh (public/cars), the two
encoding-grid scripts (public/garage/enc), and reencode-thumbnails.sh, which is
the exact script whose standalone use caused gotcha 41.

Shell gets a DELIBERATELY WIDER question than the exact scan, because shell has
no exact answer: a script writes through redirects, cp, mv, tee, sed -i and
heredocs, and also through the tools it drives (avifenc -o, zenc square,
exif-sooc --merge-into). A loose matcher for those spends its life on false
positives and still misses some. So the shell tier asks whether a script mentions
a committed path at all. Over-approximating fails CLOSED: a script that only
reads still has to be classified, and its exemption says it only reads.

36 of 36 classified now, up from 30, with two new unverifiable derivations
(cars/reference, garage/encoding-samples) and three exemptions. Control: an
undeclared shell script fails by name and exits 1.

Two things worth noting about building it. The first regex excluded `/` before
the path segment, which is exactly the character that precedes every real case
(`../../public/cars`, `$PROJECT_DIR/public/images`), so it found 3 of 6 and the
floor caught it. And a script handed its target as an ARGUMENT stays invisible,
which is deliberate and is the same reason perf-snapshot.ts is exempt: a tool
that writes where it is told has no fixed artifact to pin.

reencode-thumbnails.sh now names derive:check in its closing advice and says why,
with a test that fails if the reference goes away. gotcha 41's residual gap was
that script telling you to re-run hash-thumbnails.sh and stopping there, so the
script is the one place the remedy has to be legible.

* fix(test): stop the bun-pin test writing the live package.json

CI failed with ERR_INVALID_PACKAGE_CONFIG naming the repo root, on
contract-the-deploy-time-page-renderers.test.mjs, which touches none of that.

The cause is a race that has been latent for months.
contract-the-bun-pin-is-declared-once.test.mjs called writePin(root, "9.9.9")
against the REAL package.json and restored it in a finally, which is safe in one
process and is not what happens: `node --test` runs test FILES in parallel
processes, and writeFileSync truncates before it writes. For the width of those
two writes any other file's module resolution could read a half-written
package.json, and Node answers that by refusing the package scope.

It is a race, so it passed locally and failed in CI under load, and it failed on
a file unrelated to the cause while the test responsible reported green. Nothing
about the diff on this branch caused it; more test files in one parallel run is
the likeliest reason it surfaced now.

writePin already took its root as an argument, so the fix is to hand it a copy in
a temp directory. The final assertion now says the live file was never modified,
which is the property the race was about.

A narrow guard comes with it: no test may hand the repo root to a write-shaped
call. It is deliberately narrow, catching that exact shape rather than every way
a test could reach outside its sandbox, because a heuristic that also fired on
temp-dir writes would be muted within a week. Control: reintroducing
writePin(root, ...) fails it by name, and reverting goes green. It has to strip
comment lines first, since the note explaining it names the call it forbids, and
a test that fails on its own explanation can only be fixed by deleting the
explanation. The JSDoc-tag contract learned that the same way.

It also corrects an exemption reason this branch got wrong. tools/lib/bun-pin.ts
was exempted as writing "a temp release zip", which is bump-bun-pin.ts. It writes
the root package.json. The census caught the right file and the reason beside it
was wrong, which is the failure mode an exemption list has, and is why every
entry has to state something checkable rather than something plausible.

---------

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh added a commit that referenced this pull request Aug 28, 2026
…657)

* fix(build): a repository path cited in served bytes must still exist

The sibling of the link-integrity invariant, and the direction it cannot see.
That one asks whether an href resolves to something the site SERVES. This asks
whether a path in the prose resolves to something the repository HAS, which is
what a rename actually breaks here.

Salvaged from the parked bun branch, where it was written the day before the
www/ split. Re-pointed at the current layout and widened to the staged Worker
tree, it finds TEN stale citations on main today, every one created by a move
that landed weeks ago:

  public/_headers                scripts/gen-og-cards.mjs, scripts/gen-feeds.mjs
  src/content/md/terminal.md     tools/photos/radar-sample.mjs
  src/pages/lwe/vigenere.html    scripts/gen-quadgram-table.mjs
  src/pages/garage/workers.html  www/_worker.js/index.js, www/_worker.js/counter.js
  src/worker/photos.ts           scripts/build-histogram-index.mjs, scripts/hash-thumbnails.sh
  src/worker/lib/photo-grid.ts   scripts/build-histogram-index.mjs
  src/worker/updates.ts          scripts/bump-version.sh
  src/worker/lib/assets.ts       scripts/gen-shell-deltas.mjs
  src/worker/webmention.ts       migrations/0001_init.sql

/garage/workers is the worst of them. It is the page ABOUT the Worker, and its
wrangler.jsonc excerpt named a directory deleted ten days earlier while the
page's whole argument is that the config is one checked-in file anybody can
read. Its "main" and "directory" are the real ones now.

Byte-identity discipline makes this MORE likely rather than less: preserving
bytes means reverting served files and leaving their prose behind. That is
gotcha 40 from the other end, where the same rename left eight scripts writing
into directories that do not exist.

WHY THE WORKER TREE IS SCANNED BESIDE public/. A Worker string literal is served
the moment somebody asks for the route that prints it, and /terminal/radar has
been printing `node tools/photos/radar-sample.mjs` since that file became .ts.
The other five are comments, which this repo treats as documentation rather than
as decoration. scripts/gen-shell-deltas.mjs is stale twice over: the script is
gone AND the deltas are build output now, so the sentence says deploy time.

WHAT KEEPS IT FROM BEING A NUISANCE, which is the difference between a tripwire
and a tripwire somebody comments out. Only a token naming a FILE has to resolve,
so a bare directory mention in prose passes. It anchors on a real top-level
entry, so /images/x, an MCP tools/list and somebody else's src/ do not match.
One allowlist entry remains, for a file in another repository cited as
attribution, and /lwe/vigenere now names that repository's URL beside it.

Verified against two controls rather than against its own agreement: reverting
one real fix fails the build naming that file, and planting a citation to
src/worker/definitely-not-here.ts fails it naming all four staged copies the
page produces. It counts what it scanned and throws below 300, because a scanner
that stops matching reports a pass.

541 staged files in the noise of a 1.8s build. lint, typecheck, 511 tests,
photos:check, pages:check, tools:check, perf-budget, 149 routes and a wrangler
dry-run all green. Bundle 708.52 KiB / 239.88 KiB gzip.

* fix(build): a floor on the /*min*/ CSS pass, so its silence cannot ship again

Step 5 minifies the static CSS template literals in the Worker modules. It had
no floor, and every failure it has had was an absence.

The parked bun branch found the live version: the marker scan matched nothing at
all after the Worker moved from .js to .ts, so the step printed "0 literals" and
shipped unminified CSS on every Worker-rendered page for as long as nobody read
the line. The extension list was fixed on main when the Worker landed. The
silence that hid it was not.

Seven literals across seven modules today, so the floor is 7. Same idiom as the
client-edge mirror's `mirrored < 32` thirty lines up, which exists because that
tripwire had been set at 25 against a real count of 33.

Control: rewriting the sentinel to /*MIN*/ fails the build with "found only 0
/*min*/ literals (expected 7+)" instead of passing quietly.

* test(photos): measure whether rustc moves the histograms, and record that it does not

The parked bun branch proposed pinning rustc for the zenc crate, on a measured
31.2% of 40,448 histogram bins moving. Went to salvage it and the claim does not
reproduce.

WHAT THE BRANCH ACTUALLY MEASURED. It compared a fresh bake against the
COMMITTED histograms, ruled out Cargo.lock, the input hashes and
nondeterminism, and concluded the remaining variable was the build. It never ran
an A/B between two compilers, so rustc was the last suspect standing rather than
a measured cause. CLAUDE.md gotcha 41 found the real one a week later: #394
re-encoded 316 thumbnails, so the committed set described bytes nobody was being
served. Drift-from-committed is fully explained with no compiler in it.

MEASURED HERE, 2026-08-28, over all 165 stems: rustc 1.93.0, 1.96.0 and 1.98.0
produce BYTE-IDENTICAL histograms. 0 files differ on either pair, from three
binaries with three distinct sha256s built off one --locked Cargo.lock. So no
rust-toolchain.toml was added, and config/tools.json's cargo entry keeps
bytes: false. Its bytes_why asserted this with nothing behind it and now names
the control.

THE CONTROL IS THE FINDING, and skipping it is how the first answer went wrong.
The first version of this comparison used sips to re-encode a tile in place,
which sips treats as a no-op, so the "perturbed" file was byte-identical and the
diff was measuring nothing. A null result across three compilers looked
excellent and proved nothing. Re-encoding aside and copying back moves exactly
one stem's meta file and no others, which is what makes the null result mean
something. Gotcha 15 and the onestep probe are the same lesson from two other
directions: ask what the instrument is structurally incapable of seeing before
trusting it.

COMMITTED as `bun run zenc:reproducible`, because this question has now produced
one wrong committed conclusion and re-deriving it from scratch is exactly what
CLAUDE.md warns about at onestep:probe. It builds each toolchain, runs the
control, refuses to compare two binaries with the same digest, and prints a
verdict naming what to do in either direction. 4s warm. Workstation-only: it
needs rustup with those toolchains, the same standing as kitesurf:check.

Also here, and independently justified: every cargo build call site passes
--locked. The DECODER version is a real variable per histogram.rs's own header,
and without --locked a build can silently update the committed Cargo.lock, which
would move every histogram with no visible cause. Twelve call sites across the
eight shell scripts, the pixel-peeper hint, config/tools.json, CLAUDE.md and
MAINTENANCE.md, so no copy disagrees.

lint, typecheck, 511 tests, tools:check and photos:check green.

---------

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant