Repository navigation
refactor: reorganize the repo so the layout explains itself - #340
Conversation
The root held 38 entries and the name of the one holding every served byte
was `holding/`. Four dev scripts sat at the root beside a `scripts/` directory
holding 22 more, and three `*-pipeline/` directories sat beside both. None of
that told a newcomer, human or agent, where anything lives.
holding/ -> www/
build.mjs, contract-tests.mjs,
perf-budget.mjs,
verify-routes.mjs -> scripts/
infra.json, site-manifest.json,
tsconfig.json -> config/
MAINTENANCE.md, PHOTO-PIPELINE.md,
DEPENDENCIES.md,
UNDERSTANDING-REVIEW.md -> docs/
{content,garage,lwe}-pipeline -> pipelines/{content,garage,lwe}
Root is 21 entries now, and CLAUDE.md opens with a table naming every one of
them plus the four things that stay at the root because their tooling demands
it. Both wrangler configs are in that last group: Workers Builds runs from the
repo root, wrangler resolves `main` and `assets.directory` relative to its own
config file, and the deploy command is mirrored in config/infra.json, so moving
them would mean editing the Cloudflare dashboard first.
Rewriting the references was the risk, since this repo's prose uses "holding"
in its ordinary sense ("slot-holding", "paint-holding", and a page description
about "the 1,765 lines of statement holding up OpenAI's ten proofs" that ships
to the wire). So the rewrite ran path-shaped rules and then audited every
remaining occurrence by hand. Four blind spots turned up that way, each one a
shape no rule matched:
1. Regex literals with escaped slashes, `/holding\/p-dict/`.
2. A template literal, `` `holding${p}.html` ``, in gen-md-twins.mjs. This one
was live: it silently dropped the twins for /access, /garage, /lwe and
/pixel-peeper and made the other 32 fall back to the STAGED tree, which is
exactly the tree that file's own comment says not to read.
3. Dynamic `import()` specifiers, which the static-import rule missed.
4. `join(ROOT, "lwe-pipeline", "specs")`, a path assembled from segments.
contract-tests.mjs resolved 57 paths against `import.meta.url`, so it now
carries a named ROOT anchor instead. That is what those calls always meant, and
it costs one line the next time the file moves.
Verified by diffing the whole 1430-file build against a pre-change baseline.
Every difference is accounted for: 51 files are the cascade from icons.svg,
whose provenance comment names the generator script that moved, and the rest
are files whose source text changed. `git diff` over www/, cal/ and serendipity/
reports zero changed lines that do not involve holding or www.
238 contract tests, 59 cal tests, 130 routes, typecheck, page contracts, infra
offline and the wrangler dry-run all pass, matching the baseline exactly.
One consequence worth knowing: the icons.svg re-hash changes every page's bytes,
so every committed p-dict snapshot is stale until `pnpm run pages:roll` runs
after this deploys. That is the normal post-release step, just triggered for all
46 pages at once rather than the handful a content change usually touches.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
aadhar-sh | cf7923b | Aug 12 2026, 01:45 AM |
Wire-size diff vs merge base
Brotli q11, the quality the edge serves. Every number here is deterministic: Client assetsTotal Brotli: 0.00 KiB → 67.49 KiB (+67.49 KiB, n/a) across 16 files, 16 changed.
PagesTotal Brotli: 0.00 KiB → 484.87 KiB (+484.87 KiB, n/a) across 46 files, 46 changed.
Worker bundle276.19 KiB → 276.18 KiB gzip (-0.01 KiB, -0.0%) Largest module deltas (raw bytes in bundle):
Dictionary deltas (
|
|
Two bot results on this PR need reading with the rename in mind. Neither is a
That is the outage CLAUDE.md gotcha 27 documents, now its seventh occurrence,
The comparison the job was reaching for is already in the PR body and is I left |
* fix: close the tag-stripping gaps CodeQL keeps finding All 15 open code-scanning alerts are the same 15 that were triaged and dismissed months ago. They came back because #340 renamed holding/ to www/, lwe-pipeline/ to pipelines/lwe/, and moved build.mjs and contract-tests.mjs into scripts/. CodeQL tracks an alert by file path, so every dismissal whose file moved reset to open. The alerts on files that stayed put, in scripts/lib/html-to-md.mjs and check-page-contracts.mjs, are still dismissed, which is what confirms the cause. So this fixes the 11 where fixing is cheaper than re-dismissing, and leaves 4 that no code change should touch. The security half of every old dismissal still holds: lensMarkdown returns JSON that reaches the DOM through esc(), an excerpt is stored escaped, and contentOf feeds link extraction rather than rendering. What those dismissals conceded was a FIDELITY cost, and that part is real and now fixed. `</script >` is a legal end tag and `--!>` legally closes a comment, so a stripper spelled `</script>` or `-->` hands the body through: script source lands in a markdown render, and minified JS lands in a webmention excerpt a human reads. www/_worker.js/lens.js \s* on the closer, --!> on the comment, and \b so <head> stops swallowing a page that opens with <header> and runs to the next literal </head> www/_worker.js/webmention.js same closers in excerptAround www/_worker.js/webmention-send.js same closers in contentOf pipelines/lwe/generate.mjs replaceAll on the accent's #, and --!> on the console.log marker strip scripts/build.mjs escape the backslash and the rest of the metacharacters before turning * into .* lensStripTags loops, and the comment at it says plainly that one pass of /<[^>]+>/g is already a fixpoint: the match runs from a < to the next >, so a surviving < has no > after it and a second pass cannot match. The loop is for CodeQL, which cannot see that argument, and for whoever narrows the pattern later, where the proof stops holding. Better to write that down than to claim a fix the code does not make. The contract test fails if either closer regex goes back, verified by reverting one. Left open deliberately, because the right resolution is a dismissal: #51 www/garage/horizon.html the unsafe parse IS the demo, setHTMLUnsafe beside setHTML on the visitor's own textarea #48 #49 #50 contract-tests two of these are Array.prototype.includes on a list of extracted citations, so they are not string operations at all, let alone URL checks Suite 240 pass, build, pages:check, typecheck, and 130 routes all green. * fix: end tags carry attributes, and one strip pass can re-form what it removes CodeQL re-scanned the changed lines and found two things the first pass got wrong. Both are real, and the second is the interesting one. An end tag may carry attributes. `</script bar>` closes a script element as surely as `</script>` does, so `<\/script\s*>` was still handing the body through, just for a narrower set of inputs than before. Every closer in lensMarkdown, excerptAround and contentOf is `<\/tag\b[^>]*>` now. Removing a span can RE-FORM the thing being removed out of what is left on either side. `<!-` + `<!--x-->` + `-` collapses to `<!--` the moment the middle goes, so one pass leaves a live comment opener behind. The contract test asserts that collapse directly before asserting the fix, so it demonstrates the bug rather than asserting around it. What explains the split in the alerts is the REPLACEMENT, not the pattern. The webmention strippers replace with a space, which keeps the two sides from touching and makes them immune for free; that is why CodeQL never flagged them for this and flagged lens.js twice. lensMarkdown replaces with "" to preserve markdown spacing, so it pays for that with a fixpoint loop instead. The old lensStripTags is now a general untilStable(s, re, repl) covering all five strips, and the comment at it says which of the two problems each caller has. pipelines/lwe/generate.mjs builds its console.log label by allowlisting the wordy part of the marker rather than denylisting delimiters it has to remember. Output is byte-identical on all four call sites, checked against the old form. Suite 242 pass, build, pages:check, typecheck, 130 routes all green. * fix: keep the end-tag fix, drop the fixpoint loops that fixed nothing The loops in the previous commit were wrong and the measurement is why. Keeping the reasoning in the file so it is not re-argued every time this moves. CodeQL wants a fixpoint loop on each strip in lensMarkdown, because removing a span can re-form what was removed out of what is left either side. That really happens here: one pass of the comment strip over `<p>a<!-<!--x-->-</p>` leaves `<p>a<!--</p>`, since the match starts at the INNER opener. It is unreachable anyway, twice over. The re-formed opener is eaten by the generic `<[^>]+>` strip further down, because `<!--</p>` matches whole. And that strip is itself already a fixpoint: its match runs from a `<` to the next `>`, so any `<` surviving a pass has no `>` after it and a second pass cannot match. The bill for looping all three, measured on one 256KB fixture, node, same V8: 6.29ms against 5.94ms median. Roughly 10% of the CPU-bound half of a scan, on the exact path LENS_PARSE_CAP exists to keep under the 10ms Workers ceiling. Real money for a bug nothing can reach, so the loops are out and the four lens.js multi-character alerts want a dismissal instead. The contract test I wrote for the loops is out with them. It passed with the loops reverted, which made it decoration, and this repo has the scar tissue to know better. The two end-tag tests stay and both fail when their fix is reverted, verified. What survives from that commit is the part that was real. An end tag may carry attributes, so `</script bar>` closes a script element as surely as `</script>` does, and `<\/script\s*>` was still handing the body through. Every closer in lensMarkdown, excerptAround and contentOf is `<\/tag\b[^>]*>` now. Suite 241 pass, build, pages:check, typecheck, 130 routes all green. --------- Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
…#561) The tour still opened on `www/` and `scripts/`, neither of which has existed since the 2026-08-18 split: served bytes are `public/`, everything authored is `src/`, and the dev tools merged into `tools/`. The one command a newcomer runs also still started with `pnpm install`, which now fails outright, since `packageManager` is bun@1.4.0, the lockfile is `bun.lock`, and there is no `pnpm-lock.yaml` or `pnpm-workspace.yaml` left in the tree. All three drifted the same way. This section was written in #340 and the only edits since were #483 swapping `pnpm run` for `bun run` and #538 adding the cf-garage note, so neither the layout move nor the package-manager move came back through here. Nothing asserts on this file, which is why it could go stale without failing anything. Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
The root held 38 entries and the one holding every served byte was called
holding/. Four dev scripts sat at the root beside ascripts/directoryholding 22 more, and three
*-pipeline/directories sat beside both. None ofthat told a newcomer, human or agent, where anything lives.
holding/(1063 files)www/build.mjs,contract-tests.mjs,perf-budget.mjs,verify-routes.mjsscripts/infra.json,site-manifest.json,tsconfig.jsonconfig/MAINTENANCE.md,PHOTO-PIPELINE.md,DEPENDENCIES.md,UNDERSTANDING-REVIEW.mddocs/{content,garage,lwe}-pipeline/pipelines/{content,garage,lwe}/Root is 21 entries now. CLAUDE.md opens with a table naming every one of them
and the four groups that stay at the root because their tooling demands it.
README.md gets a shorter version in its own voice.
Both wrangler configs deliberately stay put. Workers Builds runs from the
repo root, wrangler resolves
mainandassets.directoryrelative to its ownconfig file, and the deploy command is mirrored in
config/infra.json, somoving them would mean editing the Cloudflare dashboard first. That is the one
surface CLAUDE.md calls silently undoable, and a legibility win is not worth
spending it.
How the rewrite was done
Path-shaped rules, then a hand audit of every remaining occurrence, because
this repo uses "holding" as an ordinary word:
slot-holding,paint-holding,and a shipped page description about "the 1,765 lines of statement holding up
OpenAI's ten proofs". A blind substitution would have edited the wire.
Four shapes slipped past the rules, each found by the audit rather than by a
failing check:
/holding\/p-dict/.`holding${p}.html`, ingen-md-twins.mjs.import()specifiers, which the static-import rule missed.join(ROOT, "lwe-pipeline", "specs"), a path assembled from segments.Number 2 was live. It silently dropped the Markdown twins for
/access,/garage,/lweand/pixel-peeper, and pushed the other 32 onto thestaged-tree fallback that the file's own comment says never to read. The build
still passed. That is the argument for the byte-diff below over a green suite.
contract-tests.mjsresolved 57 paths againstimport.meta.url, so it carriesa named
ROOTanchor now. That is what those calls always meant, and it costsone line the next time the file moves.
Verification
The whole 1430-file build was diffed against a build of the merge base. Every
difference is accounted for:
www/icons.svgcarries a provenance commentnaming the generator script that moved, so the sprite re-hashed, and every
page referencing
/a/icons.<hash>.svgchanged with it.Two properties hold across the served trees.
git diffoverwww/,cal/andserendipity/reports zero changed lines that do not mention holding or www.And 1062 of the 1067 moved files are byte-identical to the base with the
path substitution applied; the 5 remaining are bare
"holding"stringarguments, each checked by hand.
That second check earned its place: it caught a
pnpm run search:indexI ran asa sanity check, which regenerated
www/search-index.jsonand pulled unrelatedcontent drift (new
/accessand/lwe/leanrecords) into a structural change.Reverted to base-plus-path-rewrite. The committed search index is stale on
mainand worth regenerating on its own.All checks pass at parity with the base: 239 contract tests, 59 cal tests, 130
routes through the in-process Worker, typecheck, page contracts,
infra:check --offline, and the wrangler dry-run.Two things to know before this ships
p-dictsnapshots are stale untilpnpm run pages:rollruns after thisdeploys. Normal post-release step, just triggered for every page at once.
perf-diffwill fail once. It builds the merge base using this branch'sworkflow, and the base has no
scripts/build.mjs. Same shape as the pnpmmigration. It gates nothing and self-resolves once this is on
main.