Repository navigation
fix(photos): the ingest regenerates semantics.json, the one derivation it skipped - #662
Merged
Merged
Conversation
…n it skipped add-photos.sh runs five of the six stem-keyed generators and never ran the sixth. hashes and fingerprints go through hash-thumbnails.sh (the second at line 150), histograms through extract-photo-metadata.sh line 274 plus the zenc bake, alt text through the gen-alt-text.py block. images/semantics had no caller at all, which is how seven photos drifted out of it for a week. #609 fixed the producer and not the hole. Its `covers` verdict found semantics.json at 158 of 165, traced it to gen-photo-semantics.ts joining a www/ the 2026-08-18 split deleted (gotcha 40), repaired the joins and re-ran it. Nothing then made the ingest call it, so the next photo add reopens the gap and lands on derive:check as a red PR with a command to go find. It goes after the captioner because the derived tier folds alt[stem] into `terms` (gen-photo-semantics.ts:206). Captioning second leaves a new stem with camera vocabulary and no subject, and that failure is quiet by construction: queryPhotos still finds the photo through the alt field at weight 5, one tier above expansion at 3, so the search result is worse rather than absent. Cheap and mandatory rather than tolerated. The derived tier reads metadata.json, hashes.json and alt.json, so unlike gen-alt-text.py it cannot be rate-limited and needs no credential; `set -e` stops the run instead of degrading it. Measured here: 95ms over 165 stems, and re-running the whole library is byte-identical, so the step adds no diff on an add that changes nothing. --vision stays opt-in and is not passed.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
aadhar-sh | d8ecf5c | Aug 29 2026, 01:38 PM |
Contributor
Wire-size diff vs merge base
Brotli q11, the quality the edge serves. Every number here is deterministic: Client assetsNo change. 25 files, 86.41 KiB Brotli total. PagesNo change. 55 files, 581.17 KiB Brotli total. Worker bundleNo change. 240.38 KiB gzip. Dictionary deltas (
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The reported defect no longer exists, and I checked before changing anything
The task said
public/images/semantics.jsonis stale by 7 stems (XT500084, XT500088, XT500132, XT500138, XT500144, XT500183, XT500189). Re-measured onorigin/mainatf7ed4916:All seven carry records. #609 (
b7526da8) refilled them a week ago, so no backfill is owed and nothing here needs an owner-run step or a credential.What it is, and what a missing stem costs
Live data, read by the Worker.
queryPhotos(src/worker/photos.ts:388) loadsimages/semantics.jsonand ranks itstermsas theexpansionfield at weight 3, underaltat 5 andfilmat 6. It widens what the photo search matches past the one-sentence caption and the EXIF, so a stem missing from it stays findable one tier down. The failure is a search result that is quietly worse, never an error. Not dead data, and not deletable.Three of the four asks were already done, by #609
config/derivations.jsonwith an input digestid: images/semantics,set-mode inputs,coversI am deliberately NOT adding the assertion to
check-photo-pipeline.ts.coversinconfig/derivations.jsonalready asserts exactly it: every input key projected by thesetregex must have an entry in the output, computed live rather than recorded, which the declaration's own$commentcalls "the one verdict--lockcannot clear".derive:checkruns invalidateon every PR atci.yml:107, immediately afterphotos:checkat 97.A second copy would contradict a rule that file states explicitly, in the block listing what is deliberately left out:
Control: the existing check has teeth
Deleted
XT500144's record and ran the gate.Exit 1, the stem by name, the fix command.
photos:checkon the same tree exited 0 and said165 photos … complete EXIF + histogram + alt-text metadata, which is the blindness the task describes, and it is covered one gate over rather than absent. Reverted;semantics.jsondiffs byte-identical toorigin/main.What was actually still broken
add-photos.shruns five of the six stem-keyed generators and never ran the sixth:images/hasheshash-thumbnails.shimages/fingerprintshash-thumbnails.sh:150images/histogramsextract-photo-metadata.sh:274+ the zenc bakeimages/altgen-alt-text.pyblockimages/semantics#609 fixed the producer and left the hole. It found 158 of 165, traced it to
gen-photo-semantics.tsjoining awww/the 2026-08-18 split deleted (gotcha 40), repaired the joins and re-ran it. Nothing then made the ingest call it, so the next photo add reopens the gap. That is caught now instead of silent, and it arrives as a redderive:checkon somebody's PR with a command to go find, which the ingest should have run itself.The change
One
nodecall, after the captioner and before the validator, plus its argument in comments and the phase-6 header line.Ordering is a real constraint rather than tidiness: the derived tier folds
alt[stem]intotermsatgen-photo-semantics.ts:206, so captioning second leaves a new stem carrying camera vocabulary and no subject.It is mandatory rather than tolerated, which is where it differs from the
gen-alt-text.pyblock above it. That one is wrapped in|| echo "captions incomplete"because a 429 against the free neuron budget is expected. This tier readsmetadata.json,hashes.jsonandalt.json, needs no credential and no network, so it cannot be rate-limited andset -eshould stop the run.--visionstays opt-in and is not passed.Control: the wiring refills
Removed
XT500183andXT500189(163 stems), then ran the committed line verbatim with the script's own variable, which also proves the path expression resolves:Separately, running it against the untouched library is a no-op: 165 stems,
git diffempty, byte-identical, 95ms. So the step adds no diff to an add that changes nothing.Gates, from this worktree after
bun install --frozen-lockfilebun run photos:check165 photos, 660 hashed tiers, index in bijectionbun run derive:check6 fresh, 0 stale, 6 unverifiable, 0 undeclared, census36 of 36bun run buildbun run lintbun run typecheck300 source files, all heldbun testbun run tools:checkzencabsent locally, advisory presence tier, predates this branch)bun run pages:checkbash -n tools/photos/add-photos.shNothing lands red. The library is complete on
mainand stays complete.