Skip to content

refactor: reorganize the repo so the layout explains itself - #340

Merged
oddharsh merged 1 commit into
mainfrom
repo-reorg
Aug 12, 2026
Merged

oddharsh merged 1 commit into
mainfrom
repo-reorg

Conversation

@oddharsh

Copy link
Copy Markdown
Owner

The root held 38 entries and the one holding every served byte was called
holding/. Four dev scripts sat at the root beside a scripts/ directory
holding 22 more, and three *-pipeline/ directories sat beside both. None of
that told a newcomer, human or agent, where anything lives.

moved to
holding/ (1063 files) www/
build.mjs, contract-tests.mjs, perf-budget.mjs, verify-routes.mjs scripts/
infra.json, site-manifest.json, tsconfig.json config/
MAINTENANCE.md, PHOTO-PIPELINE.md, DEPENDENCIES.md, UNDERSTANDING-REVIEW.md docs/
{content,garage,lwe}-pipeline/ pipelines/{content,garage,lwe}/

Root is 21 entries now. CLAUDE.md opens with a table naming every one of them
and the four groups that stay at the root because their tooling demands it.
README.md gets a shorter version in its own voice.

Both wrangler configs deliberately stay put. Workers Builds runs from the
repo root, wrangler resolves main and assets.directory relative to its own
config file, and the deploy command is mirrored in config/infra.json, so
moving them would mean editing the Cloudflare dashboard first. That is the one
surface CLAUDE.md calls silently undoable, and a legibility win is not worth
spending it.

How the rewrite was done

Path-shaped rules, then a hand audit of every remaining occurrence, because
this repo uses "holding" as an ordinary word: slot-holding, paint-holding,
and a shipped page description about "the 1,765 lines of statement holding up
OpenAI's ten proofs". A blind substitution would have edited the wire.

Four shapes slipped past the rules, each found by the audit rather than by a
failing check:

  1. Regex literals with escaped slashes, /holding\/p-dict/.
  2. A template literal, `holding${p}.html`, in gen-md-twins.mjs.
  3. Dynamic import() specifiers, which the static-import rule missed.
  4. join(ROOT, "lwe-pipeline", "specs"), a path assembled from segments.

Number 2 was live. It silently dropped the Markdown twins for /access,
/garage, /lwe and /pixel-peeper, and pushed the other 32 onto the
staged-tree fallback that the file's own comment says never to read. The build
still passed. That is the argument for the byte-diff below over a green suite.

contract-tests.mjs resolved 57 paths against import.meta.url, so it carries
a named ROOT anchor now. That is what those calls always meant, and it costs
one line the next time the file moves.

Verification

The whole 1430-file build was diffed against a build of the merge base. Every
difference is accounted for:

  • 51 files are one cascade. www/icons.svg carries a provenance comment
    naming the generator script that moved, so the sprite re-hashed, and every
    page referencing /a/icons.<hash>.svg changed with it.
  • 35 files are files whose source text genuinely changed.

Two properties hold across the served trees. git diff over www/, cal/ and
serendipity/ reports zero changed lines that do not mention holding or www.
And 1062 of the 1067 moved files are byte-identical to the base with the
path substitution applied; the 5 remaining are bare "holding" string
arguments, each checked by hand.

That second check earned its place: it caught a pnpm run search:index I ran as
a sanity check, which regenerated www/search-index.json and pulled unrelated
content drift (new /access and /lwe/lean records) into a structural change.
Reverted to base-plus-path-rewrite. The committed search index is stale on
main
and worth regenerating on its own.

All checks pass at parity with the base: 239 contract tests, 59 cal tests, 130
routes through the in-process Worker, typecheck, page contracts, infra:check --offline, and the wrangler dry-run.

Two things to know before this ships

  1. Every page's bytes changed, via the icons re-hash, so all 46 committed
    p-dict snapshots are stale until pnpm run pages:roll runs after this
    deploys. Normal post-release step, just triggered for every page at once.
  2. perf-diff will fail once. It builds the merge base using this branch's
    workflow, and the base has no scripts/build.mjs. Same shape as the pnpm
    migration. It gates nothing and self-resolves once this is on main.

The root held 38 entries and the name of the one holding every served byte
was `holding/`. Four dev scripts sat at the root beside a `scripts/` directory
holding 22 more, and three `*-pipeline/` directories sat beside both. None of
that told a newcomer, human or agent, where anything lives.

  holding/                     -> www/
  build.mjs, contract-tests.mjs,
  perf-budget.mjs,
  verify-routes.mjs            -> scripts/
  infra.json, site-manifest.json,
  tsconfig.json                -> config/
  MAINTENANCE.md, PHOTO-PIPELINE.md,
  DEPENDENCIES.md,
  UNDERSTANDING-REVIEW.md      -> docs/
  {content,garage,lwe}-pipeline -> pipelines/{content,garage,lwe}

Root is 21 entries now, and CLAUDE.md opens with a table naming every one of
them plus the four things that stay at the root because their tooling demands
it. Both wrangler configs are in that last group: Workers Builds runs from the
repo root, wrangler resolves `main` and `assets.directory` relative to its own
config file, and the deploy command is mirrored in config/infra.json, so moving
them would mean editing the Cloudflare dashboard first.

Rewriting the references was the risk, since this repo's prose uses "holding"
in its ordinary sense ("slot-holding", "paint-holding", and a page description
about "the 1,765 lines of statement holding up OpenAI's ten proofs" that ships
to the wire). So the rewrite ran path-shaped rules and then audited every
remaining occurrence by hand. Four blind spots turned up that way, each one a
shape no rule matched:

  1. Regex literals with escaped slashes, `/holding\/p-dict/`.
  2. A template literal, `` `holding${p}.html` ``, in gen-md-twins.mjs. This one
     was live: it silently dropped the twins for /access, /garage, /lwe and
     /pixel-peeper and made the other 32 fall back to the STAGED tree, which is
     exactly the tree that file's own comment says not to read.
  3. Dynamic `import()` specifiers, which the static-import rule missed.
  4. `join(ROOT, "lwe-pipeline", "specs")`, a path assembled from segments.

contract-tests.mjs resolved 57 paths against `import.meta.url`, so it now
carries a named ROOT anchor instead. That is what those calls always meant, and
it costs one line the next time the file moves.

Verified by diffing the whole 1430-file build against a pre-change baseline.
Every difference is accounted for: 51 files are the cascade from icons.svg,
whose provenance comment names the generator script that moved, and the rest
are files whose source text changed. `git diff` over www/, cal/ and serendipity/
reports zero changed lines that do not involve holding or www.

238 contract tests, 59 cal tests, 130 routes, typecheck, page contracts, infra
offline and the wrangler dry-run all pass, matching the baseline exactly.

One consequence worth knowing: the icons.svg re-hash changes every page's bytes,
so every committed p-dict snapshot is stale until `pnpm run pages:roll` runs
after this deploys. That is the normal post-release step, just triggered for all
46 pages at once rather than the handful a content change usually touches.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
aadhar-sh cf7923b Aug 12 2026, 01:45 AM

@github-actions

Copy link
Copy Markdown
Contributor

Wire-size diff vs merge base

0554148 → c872038

Brotli q11, the quality the edge serves. Every number here is deterministic:
an unchanged file produces no row. Advisory — this check fails on nothing.

Client assets

Total Brotli: 0.00 KiB → 67.49 KiB (+67.49 KiB, n/a) across 16 files, 16 changed.

file base head Δ
lens.js — 21.05 KiB new
luna.css — 9.05 KiB new
nav.js — 6.90 KiB new
nav-run.js — 4.99 KiB new
lens-browser.js — 3.70 KiB new
quiz.js — 2.95 KiB new
lens-wire.js — 2.86 KiB new
icons.svg — 2.68 KiB new
tooltip.js — 2.57 KiB new
lens-reader.js — 2.49 KiB new
notepad.js — 2.08 KiB new
infotip.js — 1.83 KiB new
nav-tray.js — 1.55 KiB new
lwe-base.css — 1.31 KiB new
hoist.js — 1.08 KiB new
sw.js — 0.40 KiB new

Pages

Total Brotli: 0.00 KiB → 484.87 KiB (+484.87 KiB, n/a) across 46 files, 46 changed.

file base head Δ
garage/horizon.html — 63.90 KiB new
access/index.html — 36.63 KiB new
lwe/vigenere.html — 18.92 KiB new
lwe/lean.html — 16.97 KiB new
lwe/dac.html — 15.79 KiB new
garage/wire.html — 14.02 KiB new
garage/encoding.html — 13.97 KiB new
garage/tooltips.html — 13.40 KiB new
lens.html — 12.84 KiB new
pixel-peeper/index.html — 11.90 KiB new
garage/blueprint.html — 11.73 KiB new
garage/teardown.html — 10.31 KiB new
garage/pqc.html — 10.31 KiB new
garage/pretext.html — 10.30 KiB new
garage/index.html — 10.20 KiB new
lwe/encoding.html — 10.01 KiB new
garage/cloudflare.html — 9.96 KiB new
garage/chunks.html — 9.79 KiB new
lwe/fhe.html — 9.75 KiB new
garage/compression.html — 9.68 KiB new
… 26 more

Worker bundle

276.19 KiB → 276.18 KiB gzip (-0.01 KiB, -0.0%)

Largest module deltas (raw bytes in bundle):

file base head Δ
.build/holding/_worker.js/lens.js 164.17 KiB — removed
.build/www/_worker.js/lens.js — 164.17 KiB new
.build/holding/_worker.js/terminal.js 46.09 KiB — removed
.build/www/_worker.js/terminal.js — 46.08 KiB new
.build/holding/_worker.js/whoareyou.js 26.57 KiB — removed
.build/www/_worker.js/whoareyou.js — 26.57 KiB new
.build/holding/_worker.js/index.js 24.16 KiB — removed
.build/www/_worker.js/index.js — 24.16 KiB new
.build/holding/_worker.js/rn.js 23.51 KiB — removed
.build/www/_worker.js/rn.js — 23.51 KiB new
… 126 more

Dictionary deltas (pd/)

0 → 138 deltas, 0.00 KiB → 529.88 KiB (+529.88 KiB).

@oddharsh
oddharsh merged commit 6e22d6c into main Aug 12, 2026
7 of 8 checks passed
@oddharsh
oddharsh deleted the repo-reorg branch August 12, 2026 01:58
@oddharsh

Copy link
Copy Markdown
Owner Author

Two bot results on this PR need reading with the rename in mind. Neither is a
code problem and neither changes the branch.

github-advanced-security (red, gates nothing). Copilot's review agent
crashed before it read the diff:

Error creating PR review request: SessionModelError: Execution failed:
CAPIError: 400 The requested model is not supported.

That is the outage CLAUDE.md gotcha 27 documents, now its seventh occurrence,
and the check run carries the matching signature (title: null,
summary: null). CodeQL passed on this same commit, validate is the only
required context, so there is nothing here to fix and no push that would turn
it green.

wire size vs merge base (+484.87 KiB, and the number is empty rather than
alarming).
The job measures BOTH sides with HEAD's copy of perf-snapshot.mjs,
on purpose, so that a change to the measurement never reports as a change in
wire size. HEAD's copy reads const BUILD = ".build/www", and the merge base
still builds .build/holding, so the base recorded zero files. Every asset then
reads as new, and the totals are the whole site appearing from nothing.

The comparison the job was reaching for is already in the PR body and is
stronger than what it could have produced: the full 1430-file build was diffed
against a build of the merge base, and 1062 of the 1067 moved files are
byte-identical to the base with the path substitution applied.

I left perf-snapshot.mjs alone rather than teaching it to fall back to
.build/holding. That branch would be correct for exactly one PR and dead the
moment this merges, and a rotting constant in the deploy-adjacent tooling is
the thing this repo keeps talking itself out of. The next PR compares
www against www and reports normally.

oddharsh added a commit that referenced this pull request Aug 12, 2026
* fix: close the tag-stripping gaps CodeQL keeps finding

All 15 open code-scanning alerts are the same 15 that were triaged and
dismissed months ago. They came back because #340 renamed holding/ to www/,
lwe-pipeline/ to pipelines/lwe/, and moved build.mjs and contract-tests.mjs
into scripts/. CodeQL tracks an alert by file path, so every dismissal whose
file moved reset to open. The alerts on files that stayed put, in
scripts/lib/html-to-md.mjs and check-page-contracts.mjs, are still dismissed,
which is what confirms the cause.

So this fixes the 11 where fixing is cheaper than re-dismissing, and leaves
4 that no code change should touch.

The security half of every old dismissal still holds: lensMarkdown returns
JSON that reaches the DOM through esc(), an excerpt is stored escaped, and
contentOf feeds link extraction rather than rendering. What those dismissals
conceded was a FIDELITY cost, and that part is real and now fixed. `</script >`
is a legal end tag and `--!>` legally closes a comment, so a stripper spelled
`</script>` or `-->` hands the body through: script source lands in a markdown
render, and minified JS lands in a webmention excerpt a human reads.

  www/_worker.js/lens.js            \s* on the closer, --!> on the comment, and
                                    \b so <head> stops swallowing a page that
                                    opens with <header> and runs to the next
                                    literal </head>
  www/_worker.js/webmention.js      same closers in excerptAround
  www/_worker.js/webmention-send.js same closers in contentOf
  pipelines/lwe/generate.mjs        replaceAll on the accent's #, and --!> on
                                    the console.log marker strip
  scripts/build.mjs                 escape the backslash and the rest of the
                                    metacharacters before turning * into .*

lensStripTags loops, and the comment at it says plainly that one pass of
/<[^>]+>/g is already a fixpoint: the match runs from a < to the next >, so a
surviving < has no > after it and a second pass cannot match. The loop is for
CodeQL, which cannot see that argument, and for whoever narrows the pattern
later, where the proof stops holding. Better to write that down than to claim
a fix the code does not make.

The contract test fails if either closer regex goes back, verified by
reverting one.

Left open deliberately, because the right resolution is a dismissal:
  #51 www/garage/horizon.html    the unsafe parse IS the demo, setHTMLUnsafe
                                 beside setHTML on the visitor's own textarea
  #48 #49 #50 contract-tests     two of these are Array.prototype.includes on
                                 a list of extracted citations, so they are not
                                 string operations at all, let alone URL checks

Suite 240 pass, build, pages:check, typecheck, and 130 routes all green.

* fix: end tags carry attributes, and one strip pass can re-form what it removes

CodeQL re-scanned the changed lines and found two things the first pass got
wrong. Both are real, and the second is the interesting one.

An end tag may carry attributes. `</script bar>` closes a script element as
surely as `</script>` does, so `<\/script\s*>` was still handing the body
through, just for a narrower set of inputs than before. Every closer in
lensMarkdown, excerptAround and contentOf is `<\/tag\b[^>]*>` now.

Removing a span can RE-FORM the thing being removed out of what is left on
either side. `<!-` + `<!--x-->` + `-` collapses to `<!--` the moment the middle
goes, so one pass leaves a live comment opener behind. The contract test
asserts that collapse directly before asserting the fix, so it demonstrates
the bug rather than asserting around it.

What explains the split in the alerts is the REPLACEMENT, not the pattern.
The webmention strippers replace with a space, which keeps the two sides from
touching and makes them immune for free; that is why CodeQL never flagged them
for this and flagged lens.js twice. lensMarkdown replaces with "" to preserve
markdown spacing, so it pays for that with a fixpoint loop instead. The old
lensStripTags is now a general untilStable(s, re, repl) covering all five
strips, and the comment at it says which of the two problems each caller has.

pipelines/lwe/generate.mjs builds its console.log label by allowlisting the
wordy part of the marker rather than denylisting delimiters it has to remember.
Output is byte-identical on all four call sites, checked against the old form.

Suite 242 pass, build, pages:check, typecheck, 130 routes all green.

* fix: keep the end-tag fix, drop the fixpoint loops that fixed nothing

The loops in the previous commit were wrong and the measurement is why. Keeping
the reasoning in the file so it is not re-argued every time this moves.

CodeQL wants a fixpoint loop on each strip in lensMarkdown, because removing a
span can re-form what was removed out of what is left either side. That really
happens here: one pass of the comment strip over `<p>a<!-<!--x-->-</p>` leaves
`<p>a<!--</p>`, since the match starts at the INNER opener.

It is unreachable anyway, twice over. The re-formed opener is eaten by the
generic `<[^>]+>` strip further down, because `<!--</p>` matches whole. And that
strip is itself already a fixpoint: its match runs from a `<` to the next `>`,
so any `<` surviving a pass has no `>` after it and a second pass cannot match.

The bill for looping all three, measured on one 256KB fixture, node, same V8:
6.29ms against 5.94ms median. Roughly 10% of the CPU-bound half of a scan, on
the exact path LENS_PARSE_CAP exists to keep under the 10ms Workers ceiling.
Real money for a bug nothing can reach, so the loops are out and the four
lens.js multi-character alerts want a dismissal instead.

The contract test I wrote for the loops is out with them. It passed with the
loops reverted, which made it decoration, and this repo has the scar tissue to
know better. The two end-tag tests stay and both fail when their fix is
reverted, verified.

What survives from that commit is the part that was real. An end tag may carry
attributes, so `</script bar>` closes a script element as surely as `</script>`
does, and `<\/script\s*>` was still handing the body through. Every closer in
lensMarkdown, excerptAround and contentOf is `<\/tag\b[^>]*>` now.

Suite 241 pass, build, pages:check, typecheck, 130 routes all green.

---------

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
oddharsh added a commit that referenced this pull request Aug 24, 2026
…#561)

The tour still opened on `www/` and `scripts/`, neither of which has existed
since the 2026-08-18 split: served bytes are `public/`, everything authored is
`src/`, and the dev tools merged into `tools/`. The one command a newcomer runs
also still started with `pnpm install`, which now fails outright, since
`packageManager` is bun@1.4.0, the lockfile is `bun.lock`, and there is no
`pnpm-lock.yaml` or `pnpm-workspace.yaml` left in the tree.

All three drifted the same way. This section was written in #340 and the only
edits since were #483 swapping `pnpm run` for `bun run` and #538 adding the
cf-garage note, so neither the layout move nor the package-manager move came
back through here. Nothing asserts on this file, which is why it could go stale
without failing anything.

Co-authored-by: Aadharsh Pannirselvam <19518661+oddharsh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant