Repository navigation
refactor: merge scripts/ into tools/ - #453
Merged
Merged
Conversation
44 developer scripts move from scripts/ to tools/, joining tools/photos and tools/oxlint. Nothing in any of them ships; they are the build, the checks, the generators and the release tooling, and having two sibling directories for one job was a split by history rather than by kind. Fourth slice of lifting the parked bun branch's renames onto main (#450, #451, #452). This one could NOT be cherry-picked: that commit sits on top of the TypeScript conversion and brings 714 conflicts, so the move is done by hand against main's layout instead. THE REFERENCE SWEEP IS BY FILENAME, not by prefix. Rewriting `scripts/` blindly would have renamed package.json's "scripts" KEY and any /scripts/ URL in prose, so each of the 44 moved names was replaced exactly: 47 files updated. design/PORTING.md keeps its `scripts/gen-desktop-partial.mjs` mention, since that file is documented history rather than instruction. ONE VISITOR-FACING STRING CHANGES, and it is the reason this is not a pure rename. /lens prints a command for the reader to run: Use `node scripts/lens-webmcp.mjs <url>` on the current machine That script moved, so the instruction had to move with it or it would name a path that no longer exists. The string lives in src/client/lens-browser.js, which is content-addressed, so it cascades exactly three links: lens-browser.js 821aa61e -> 62f8bf05 the string itself lens.js c0c21217 -> a9ee703b imports lens-browser by hashed URL lens-boot.js a8fd9733 -> c7fac8b7 imports lens.js by hashed URL and from there to /lens.html, its .src.html twin, and the dcz deltas for those three assets. 10 staged files leave, 13 arrive. Everything else is byte-identical. Six Worker sources change only in comments naming a moved script, which reaches the bundle and no served asset. Gates: 305 tests, lint, typecheck, build, photos:check, routes:check, pages:check, tools:check, perf-budget, check-wrangler, the cal suite and a wrangler dry-run.
Contributor
Wire-size diff vs merge base
Brotli q11, the quality the edge serves. Every number here is deterministic: Client assetsTotal Brotli: 75.14 KiB → 75.10 KiB (-0.04 KiB, -0.1%) across 21 files, 2 changed.
PagesTotal Brotli: 502.08 KiB → 502.10 KiB (+0.02 KiB, +0.0%) across 48 files, 1 changed. No single file moved by 0.13 KiB or more. Plus 1 file under the 0.13 KiB floor, net +0.02 KiB — usually content-hash churn in Worker bundleNo change. 230.10 KiB gzip. Dictionary deltas (
|
oddharsh
added a commit
that referenced
this pull request
Aug 18, 2026
…rc/dict/ (#455) www/a-dict/ and www/p-dict/ become src/dict/a-dict/ and src/dict/p-dict/. They are the previously shipped bytes of the content-hashed shell and of each page, kept so build.mjs can compute dcz deltas against what a returning visitor still holds. Nothing in either directory is ever served. Fifth slice of lifting the parked bun branch's renames onto main (#450, #451, #452, #453), and the first of three that finish www/. Done by hand rather than cherry-picked, for the reason #453 records: the branch's split commit sits on the TypeScript conversion and brings 714 conflicts. BEING OUTSIDE www/ IS A STRONGER GUARANTEE THAN THE .assetsignore ENTRY IT REPLACES. An ignore line stops an upload; living outside the served tree means the build never stages the files at all. The staged tree loses 130 files that were copied on every build purely so wrangler could then decline to upload them. The contract test that asserted the ignore entry now asserts the directory is absent from www/ instead, which is the property that actually matters. NO SERVED ARTIFACT CHANGES: 1496 compared, all byte-identical. The only staged differences are those 130 snapshots no longer being copied, and .assetsignore losing two entries that can no longer match anything. Also removes a filter in build.mjs's view-transition tripwire that skipped "a-dict/" while walking www/. With a-dict gone from that tree the exclusion could not match, and a filter that cannot fire reads as protection that is not there. Gates: 305 tests, build, lint, typecheck, routes:check, pages:check, tools:check, perf-budget, photos:check, check-wrangler, cal, wrangler dry-run. Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh
added a commit
that referenced
this pull request
Aug 18, 2026
The last of the www/ split. 36 HTML documents become src/pages/; the other 804 files, which are bytes a browser fetches unchanged, become public/. The staged root renames with them: .build/www is .build/public, and wrangler.jsonc's assets.directory follows. Seventh and final slice of lifting the parked bun branch's renames onto main (#450, #451, #452, #453, #455, #456). The mapping needed no judgement: main's www/ held exactly 840 files, the branch's two destinations hold exactly those 840, and every one lands in exactly one of them. Every HTML is a page; nothing else is. EVERY SERVED ARTIFACT IS BYTE-IDENTICAL, 1497 compared with the root name normalised. The four remaining differences are path references inside cal, serendipity and the Worker: three comments and one JSON import specifier. THE BUILD COMPOSES THE SERVED TREE NOW rather than copying one directory. public/ + src/pages/ + src/content/ + src/client/ + src/styles/ all stage into .build/public, so the URL layout is untouched while the source layout says what each file IS. servedFiles() walks the three authored roots in one place, because a tripwire that walks only some of them reports a clean tree it never read. SIX THINGS THE GATES CAUGHT, and the pattern is worth stating: a blanket path sweep cannot tell a source read from a staged path, an asset from a document, or a path from a regex. * staticShellPages() read www/ for the freshness check * the Worker imported ../../www/images/hashes.json * gen-md-twins looked for page HTML under www/, then read the generated tier from the staged www/ root * the page generators in pipelines/ wrote into www/ * the .src.html twin read its AUTHORED source from the wrong root, so it fell into a catch and shipped the STAGED bytes: readable twins carrying rewritten /a/ hashes, which is the whole point of the twin lost silently. The build's own counter is what showed it, 47 from staged where main says 12. * and one false positive: `assert.match(lens, /… 401 && www/)` is a regex whose DELIMITER the sweep read as a path separator, rewriting the assertion out from under a real variable named www. Gates: 305 tests, build, lint, typecheck, routes:check, pages:check, tools:check, perf-budget, photos:check, check-wrangler, cal, wrangler dry-run. Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh
added a commit
that referenced
this pull request
Aug 18, 2026
* docs: the layout table describes the tree that now exists Seven slices moved almost every path in this repository (#450, #451, #452, #453, #455, #456, #457), and the automated sweeps that repointed code also walked CLAUDE.md. That left the map wrong in two different ways. STALE: the table still said public/ holds "pages, client scripts, CSS, photos, dictionaries". It holds none of the first four now. Rows added for src/pages, src/content and src/dict; scripts/ folded into the tools/ row; every row's description checked against the tree rather than against its own history. FALSE: the sweep rewrote HISTORICAL names, so the file claimed the Worker lived at `public/_worker.js/` and the photo pipeline at `public/scripts/`. Neither ever existed. A stale note is a note somebody distrusts; a false one is a note somebody follows. The naming note now records the full sequence, holding -> www -> the split, and says explicitly that the rewrites deliberately left old mentions alone, so reading `www/` in an older note is correct rather than a typo. It also drops the "duplicate path" explanation, since scripts/ no longer exists to be confused with tools/photos/. And it states the test for where a new file goes, which the table never did: if a build step transforms it, it is src/; if it ships byte-for-byte, it is public/. * docs: the intro names the split tree too The bullet under the header still called public/ the live site including the _worker.js/ dispatcher, which is two moves out of date: the Worker left the served tree in #450 and the documents left it in #457. It now names what each root is for, which is the same question the table below answers per entry. --------- Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh
added a commit
that referenced
this pull request
Aug 24, 2026
… from it Eight sites across six photo and asset scripts find their output with `$SCRIPT_DIR/..`. That was the served tree when these lived in www/scripts/; it has been tools/ since #452 and #453, so extract-photo-metadata.sh was writing tools/images/metadata.json, hash-thumbnails.sh was hashing into tools/i/, and add-car-photo.sh was aiming at tools/cars. add-photos.sh passed `--root "$PROJECT_DIR/www"`, and www/ stopped existing on 2026-08-18. A rename sweep greps for the old string, and there is no old string here, only a relative expression whose meaning is a function of where the file sits. Two things then kept it silent: `zenc histogram` prints "cannot read" and exits 0, and both callers pipe it to `tail -1`, so the bake skips without failing. And the two scripts resolving through $PROJECT_DIR were correctly updated by the split, which left the pipeline looking half-migrated rather than broken. No photo has been added since the move, so nobody ran into any of it. The same rename left build.mjs's taste exemption anchored on /^www\/(garage| lwe)\//, matching nothing, so the demo pages it exists to exempt lost the exemption and the build printed 14 warnings on every run since 2026-08-18. A warn block nobody can clear is how a warn block gets ignored. Re-anchored on the SERVED_SOURCES roots. Controls both ways: an injected @font-face url() on a garage page still hard-fails, and a 99px radius there is silent again. lens-seed.mjs still spawned execFileSync("pnpm", ["exec", "wrangler", ...]), which refuses outright on a bun tree. The contract test that bans package manager spawns carried lens-seed.mjs as its one RECORDED exception, justified as driving a package script rather than using a manager to reach another binary. That described no code in the file. So the single genuine offender was the single exempted file and the suite reported green. The spawn goes through wranglerCommand() now and the allowlist is gone; a control confirms the guard catches what it used to exempt. Also dead or wrong, removed or corrected: - explorerChrome(), exported and called by nothing anywhere, including its own file - an isCallable doc comment orphaned fifteen lines above its function - build.mjs told you to run node public/scripts/build-histogram-index.mjs, which is not a path - check-wrangler.mjs told you to run `pnpm install`. Its transitive-copy scan reads node_modules/.pnpm and so reports 0 forever on a bun tree; the logic is unchanged and the blind spot is now named at the code - MAINTENANCE.md documented `zenc histogram --root www` and two .ts modules as .js CLAUDE.md counts re-derived from the tree: 36 to 37 documents, 46 to 49 test files, 117 to 147 relative specifiers, 474 to 632 hashed tiers (the 200px tier from #441 was never written down), 2645/48 to 2921/50 link refs. Gotcha 16 named contract-tests.mjs under `node --test`; both runners were measured, and the rule holds while both of its symptoms changed, so the documented error string now finds nothing and the blast radius is one file rather than the run. Gotcha 39 records the $SCRIPT_DIR/.. class so the next rename has something to check against. 353 tests, typecheck, lint, build, pages, photos and the 142-route oracle all green.
oddharsh
added a commit
that referenced
this pull request
Aug 24, 2026
… from it (#525) Eight sites across six photo and asset scripts find their output with `$SCRIPT_DIR/..`. That was the served tree when these lived in www/scripts/; it has been tools/ since #452 and #453, so extract-photo-metadata.sh was writing tools/images/metadata.json, hash-thumbnails.sh was hashing into tools/i/, and add-car-photo.sh was aiming at tools/cars. add-photos.sh passed `--root "$PROJECT_DIR/www"`, and www/ stopped existing on 2026-08-18. A rename sweep greps for the old string, and there is no old string here, only a relative expression whose meaning is a function of where the file sits. Two things then kept it silent: `zenc histogram` prints "cannot read" and exits 0, and both callers pipe it to `tail -1`, so the bake skips without failing. And the two scripts resolving through $PROJECT_DIR were correctly updated by the split, which left the pipeline looking half-migrated rather than broken. No photo has been added since the move, so nobody ran into any of it. The same rename left build.mjs's taste exemption anchored on /^www\/(garage| lwe)\//, matching nothing, so the demo pages it exists to exempt lost the exemption and the build printed 14 warnings on every run since 2026-08-18. A warn block nobody can clear is how a warn block gets ignored. Re-anchored on the SERVED_SOURCES roots. Controls both ways: an injected @font-face url() on a garage page still hard-fails, and a 99px radius there is silent again. lens-seed.mjs still spawned execFileSync("pnpm", ["exec", "wrangler", ...]), which refuses outright on a bun tree. The contract test that bans package manager spawns carried lens-seed.mjs as its one RECORDED exception, justified as driving a package script rather than using a manager to reach another binary. That described no code in the file. So the single genuine offender was the single exempted file and the suite reported green. The spawn goes through wranglerCommand() now and the allowlist is gone; a control confirms the guard catches what it used to exempt. Also dead or wrong, removed or corrected: - explorerChrome(), exported and called by nothing anywhere, including its own file - an isCallable doc comment orphaned fifteen lines above its function - build.mjs told you to run node public/scripts/build-histogram-index.mjs, which is not a path - check-wrangler.mjs told you to run `pnpm install`. Its transitive-copy scan reads node_modules/.pnpm and so reports 0 forever on a bun tree; the logic is unchanged and the blind spot is now named at the code - MAINTENANCE.md documented `zenc histogram --root www` and two .ts modules as .js CLAUDE.md counts re-derived from the tree: 36 to 37 documents, 46 to 49 test files, 117 to 147 relative specifiers, 474 to 632 hashed tiers (the 200px tier from #441 was never written down), 2645/48 to 2921/50 link refs. Gotcha 16 named contract-tests.mjs under `node --test`; both runners were measured, and the rule holds while both of its symptoms changed, so the documented error string now finds nothing and the blast radius is one file rather than the run. Gotcha 39 records the $SCRIPT_DIR/.. class so the next rename has something to check against. 353 tests, typecheck, lint, build, pages, photos and the 142-route oracle all green. Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh
added a commit
that referenced
this pull request
Aug 24, 2026
…was never wrong (#566) * fix(photos): the two scripts that kept going now stop Gotcha 40 records `zenc histogram ... | tail -1` swallowing a status inside add-photos.sh, and #560 added that of 11 committed shell scripts, 9 carry `set -euo pipefail` and 2 carry a bare `set -e`. Those 2 are the photo pipeline. This closes the gap and adds the check that keeps a 12th from reopening it. The zenc call is no longer piped at all. zenc writes one summary line to stderr on the happy path, so `tail -1` was buying no quiet while costing both the status and the per-stem warnings that name WHICH stem was skipped. Measured on the real binary: exit 2 run bare, 0 through the pipe, 2 through the pipe under pipefail. Adopting -u cost one real fix, and it is the reason not to turn -u on blind. `"${META_MODE[@]}"` on an EMPTY array is an unbound variable under bash 3.2, which is the bash macOS ships and therefore the one `env bash` finds here, and that expansion sits on the DEFAULT path (META_MODE is only non-empty under REMOTE_RENDER_ONLY). Worse than a crash: measured in the real script with -u and pipefail both on, the expansion died inside a `| tail -1` pipeline and the pipeline still reported 0, so `set -e` never fired and the run continued with the metadata regeneration silently skipped. A normal non-zero child exit propagates through that same pipe correctly, which is why the rest of this has teeth. The expansion used is `${META_MODE+"${META_MODE[@]}"}`; `${META_MODE[@]:-}` is not a substitute, since it sends one empty-string argument the callee reads as a path. Exercised rather than reasoned about, on real photos from the source folder: both the default and REMOTE_RENDER_ONLY paths, grayscale and colour sources, and a control run of the pre-change script that produced byte-identical output and the same exit code, so pipefail surfaced no pre-existing failure in these pipes. The two remaining `| tail -1` calls are kept, because those tools print many lines and tail reads all of its input, so there is no SIGPIPE to lose. The contract test discovers scripts rather than listing them, carries a floor so a broken scanner cannot report a pass, and was proven both ways: it names the offending script and its missing options when one reverts to `set -e`, and it fails on its own floor when the scanner is broken. * fix(photos): a HEIC input and every caption run were quietly broken Both found by RUNNING the pipeline while adding pipefail, rather than by reading it, and neither is a pipefail regression: the pre-change scripts fail identically. add-car-photo.sh accepted "any format sips reads (JPG/PNG/HEIC/WEBP/AVIF)" in its own usage line and could only ever process a JPEG. `sips -Z` RESIZES and does not convert, so a .HIF came out still HEIC under a .jpg name and zenc refused it with "Illegal start bytes". Every step there is >/dev/null, so the script just exited 1 with nothing said. add-photos.sh has always passed `-s format jpeg`; this one never did. Verified byte-identical output for a JPEG source, so no committed car photo moves, and exit 0 instead of 1 for a .HIF, checked across grayscale, colour and HEIF inputs. gen-alt-text.py is a NINTH instance of gotcha 40's path class, and the first one in Python. ROOT was `dirname(dirname(__file__))`, which was the served tree at www/scripts/ and became `tools/` after #452/#453, so it read tools/images/metadata.json. Its caller is `python3 ... || echo "captions incomplete"`, which excuses the traceback by design, so every photo add since the move captioned nothing and carried on. It is anchored on the tree now (`../../..` plus `public`), which is what gotcha 40 itself recommends over counting directories. Confirmed by running it: 158 photos, 158 already done, exit 0, alt.json unchanged and no inference calls spent. * docs(gotcha-40): the exit code it blames was always correct Three corrections and one addition to gotcha 40, all measured 2026-08-24. The entry says `zenc histogram` prints its error AND exits 0, so two things made the bake silent. Only one did. It returns 2 there and has done since the day it was written (#373), so the `| tail -1` was the whole mechanism and the repair was entirely in the shell. Kept as a visible correction rather than a quiet edit, because blaming the tool sends the next reader to patch an exit code that is already right, which is the more expensive mistake and the harder one to notice finishing. The 9-versus-2 split #560 recorded is gone: all 11 scripts carry `set -euo pipefail`, and a contract test fails on a 12th that does not. Adopting -u alongside it is written up as the trap it turned out to be. On bash 3.2 an empty `"${arr[@]}"` is an unbound variable, and inside a `| tail -1` pipeline that death was measured NOT propagating: pipefail on, pipeline reports 0, `set -e` never fires, step silently skipped. So -u is not a free consistency win on a script that pipes, and the safe expansion is named along with the `:-` spelling that looks like it and is not. The site table gains a ninth row. `gen-alt-text.py` had the same broken path arithmetic in Python, which is why the shell sweep that caught the other eight could not see it. --------- Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh
added a commit
that referenced
this pull request
Aug 24, 2026
… moves old (#571) /garage/encoding names the photo pipeline four times and every one of them still says `www/scripts/`: the sentence "the live pipeline (www/scripts/add-photos.sh)" plus three cites of `www/scripts/zenc/`, the Rust wrapper the page's whole corpus study is an argument for. Both are `tools/photos/` now, after www/scripts to scripts/ to tools/photos (#452, #453). Present tense, on a public page, pointing at nothing. /garage/horizon's JXL section priced a third format tier as "another stem format in www/images/". That one is NOT a literal rename: `public/images/` holds the photo DATA surfaces (metadata, exif, alt text, hashes) while the pixel tiers have lived in content-addressed `public/i/` since 2026-07-03, so the sentence names the directory a JXL tier would actually land in. The "120 stems" count is left alone deliberately: it is the site's photo count when the study was written, and updating a measured number without re-running the study invents data. /garage/workers is UNTOUCHED on purpose. It quotes `"main": "www/_worker.js/index.js"` and names `www/_worker.js/counter.js`, but that page is about the Pages-to-Workers migration and the old config IS its subject matter. Correcting it would falsify the page. Both files are hand-authored rather than generated: pipelines/garage/pages.json covers six pages (pqc, octane, useragent, typed-config, hidden-flags, ig-prep) and neither of these is one, so editing src/pages directly is the whole edit. Page bytes move, so both p-dict snapshots go stale until the next roll, which is what that roll is for. Build clean: 51 documents, 3015 refs resolve, 33 understanding payloads byte-equal, no tripwires. Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
44 developer scripts move from
scripts/totools/, joiningtools/photosandtools/oxlint. Nothing in any of them ships: they are the build, the checks, the generators and the release tooling. Two sibling directories for one job was a split by history rather than by kind.Fourth slice of lifting the parked bun branch's renames onto
main, after #450, #451 and #452.This one could not be cherry-picked
That commit sits on top of the TypeScript conversion on the parked branch, so replaying it here brings 714 conflicts. The move is done by hand against
main's layout instead, which is also why it is smaller than the ~1980-file figure I quoted: thewww/split is still to come, separately.The sweep is by filename, not by prefix
Rewriting
scripts/blindly would have renamed package.json's"scripts"key and any/scripts/URL in prose. Each of the 44 moved names was replaced exactly instead: 47 files updated.design/PORTING.mdkeeps itsscripts/gen-desktop-partial.mjsmention, because that file is documented history rather than instruction.One visitor-facing string changes, and it is the interesting part
/lensprints a command for the reader to run:That script moved, so the instruction had to move with it or it would name a path that no longer exists. The string lives in
src/client/lens-browser.js, which is content-addressed, so it cascades exactly three links:lens-browser.js821aa61e62f8bf05lens.jsc0c21217a9ee703blens-boot.jsa8fd9733c7fac8b7From there it reaches
/lens.html, its.src.htmltwin, and the dcz deltas for those three assets. 10 staged files leave, 13 arrive. Every other served artifact is byte-identical, counted rather than sampled.Six Worker sources change only in comments naming a moved script. That reaches the bundle and no served asset.
Gates
305 tests, lint, typecheck, build,
photos:check,routes:check,pages:check,tools:check,perf-budget,check-wrangler, the cal suite, and awrangler deploy --dry-run.Note for the open PRs
#445 (the deploy bridge, draft) edits
scripts/contract-tests.mjsand will need a rebase after this lands. #449 adds a new workflow file and is unaffected.