Conversation
…y config value once (#62) The two mechanisms under Recurring operations were exercised rather than read. The backup was tested with --dry-run and the timer was read rather than triggered, so the journal still holds one entry and 2026-08-09 is still its first scheduled run. The 404 review ran end to end and found nothing owed to golden-urls.txt. What the run found became the rest of the branch: - ops/ carries the pull, both units, an installer and a README. It had lived only on the backup host, so the mechanism making the off-host copy was the one thing with no copy anywhere. - ENVIRONMENT.md describes all 28 configuration values once, and checks/check-env-docs.py fails in CI both directions. The two .example files collapse into one example.env, which needed a .gitignore negation since *.env matched it. - OPERATIONS.md gains 'Working With the VPS' and a section on reading a 404 list, including two traps that produce plausible wrong answers: scanners defeat the referer heuristic by self-referencing, and jq select piped to wc -l counts lines rather than records. - ServiceName separates an edge 404 from a site 404 mechanically. Two on record came from the proxy rather than the site, and were raised with the host side. Eleven Copilot findings, nine of them low confidence, all eleven real. The two with teeth: chmod 700 could reach / because preflight only checked non-empty, and a failing du aborted a completed pull silently. Copilot read 15 of 16 changed files and names no file list, so one file in this change has no review. Merged knowing that.
…#63) The referer rule as committed did not work. It said to compare against scheme://RequestHost + RequestPath, and read literally that matches nothing, because a scanner reaching an HTTPS site routinely sends an http:// referer for the same address. On the 2026-08-08 data the naive form keeps all 36 false positives where the normalised form keeps none. The implementation that produced the original count tested both schemes explicitly, so the answer was right and the prose anyone would rebuild from was wrong. RequestHost is null on a request that sends no Host header, which router exploits do, so a bare startswith aborts the run part way through a file that has already printed real output. String functions reject a null where equality and concatenation tolerate it, and the section now states that rather than attaching a guard where it does nothing. ServiceName's absence says the edge answered; entryPointName with RequestScheme say why. The two routerless 404s on the site host were cleartext HTTP to port 443, where the websecure router requires TLS and traefik answers its own 404. Correct behavior, not the reconfiguration window this repo guessed at, and the guess is deleted rather than qualified. The documented filter is a runnable command now rather than a description, and it returns zero on the site host as claimed.
The build gate proves the media set against files on disk. Nothing proved those files reached the server or that the server can read them: the live check requested pages and redirects and never one image, so a media tree lost between a passing build and the server was caught by neither gate. This unblocks git-restore-mtime, which was held behind it. checks/golden-media-live.txt is a handful rather than exhaustive, because the set is already proven and this is a delivery check. Its entries cover both trees, which arrive by different routes, plus the legacy /wp-content/uploads/ form that nothing else exercised against a running server. Three assertions, each for a different loss. A missing file answers 404, a file whose mode went wrong answers 403, a file truncated to nothing still answers 200 so the byte count is checked, and a server answering an error page answers 200 as text/html so the type is checked too. The 403 case is why this is not theoretical: a hard-linked file carries its inode's mode into every later release, present and unreadable, which a build-time is_file() cannot see. check_media follows one hop by hand rather than passing -L to curl, because -L would carry the auth-gate credential to wherever the rule points. Review found a pre-existing hole this change surfaced: the truncation guard could be skipped by its own failure. An unreadable list makes grep -c yield nothing, and the numeric test then errors and evaluates false, so the guard protecting every assertion below it did not fire and the run exited 0 having checked nothing. Readability and the count are both validated now. Verified against production at 1253 URLs, with every failure shape reproduced rather than assumed.
The deploy uploads with --link-dest against the previous release, and a file links only when size and mtime both match. Git stores no mtimes, so a checkout stamps every file with the moment it was written and nothing ever linked: every release has been a full copy. Measured with two independent clones each way: without the restore 0 of 1791 files linked with the restore 1052 of 1791 files linked 1052 corroborates from three directions: it is what Hugo reports as static files, what links on a locally built release, and the file count in static/. Those files are 566 MB of the 586 MB a release occupies. Three things checked rather than assumed, any of which would have made this a no-op or a breakage. Hugo preserves a static file's mtime into public/, verified by touching a source and rebuilding. The restore is deterministic, with two clones producing byte-identical mtimes across all 1052 files, because static/ has stable last-commit times. And the Debian package installs into git's exec-path rather than onto PATH, so the subcommand form resolves and the bare binary name does not. static/ only. Generated pages are written fresh by every build and can never match. ORDERING: this follows the live media check merged in #64, deliberately. While every file arrived as a fresh inode the upload re-asserted the mode contract on every deploy. Now that a third of the tree arrives as hard links, a link carries the mode its inode chain began with, so a media file that acquires a bad one would stay present, correctly named and unreadable, through every later release. The live check is what notices that, by requesting images and failing on the 403. Not exercised here: an actual pipeline deploy. The first deploy after this merges is what proves the link count against the real host, where the host side has been measuring zero shared inodes.
The migration's process lived in a 2.9 GB directory outside the repo, a blog post, and five committed docs restating each other. The provenance-script paragraph existed six times, the Blogger truncation fact four times, and OPERATIONS.md asserted the migration was documented once as a post, which was already false. capture/ is to the migration what ops/ is to the backup pull: the durable scripts plus a README that is the authoritative procedure. The fourteen capture scripts split by whether they can ever run again rather than by whether they look reusable. A re-export re-runs the conversion chain, so those five are carried. build-golden.py and classify.py are carried as the record behind an append-only contract and say so, since they cannot run once the old hosting ends. The crawl pair and the three fetchers are named rather than carried, because their outputs are the durable artifact. build-redirects.py moves from checks/ to capture/. GOVERNANCE.md defines checks/ as the contract and the gates that enforce it, and this generates rather than gates, which is why six documents had to say so in prose. The move also closes the three-copies problem, with both stale copies now named by path and marked stale. Nothing in capture/ names a machine. Four CAPTURE_* values carry the address, the paths and the account slug, documented in ENVIRONMENT.md and gated by check-env-docs.py, which was watched failing on each before the rows were added. The doc hierarchy inverts as a class: the READMEs are procedure and fact, the post is the casual account, and a post may cite a README where a README never cites the post. Review raised 21 findings and every one was real, 19 of them arriving marked low confidence. The three with teeth were all in the guard that exists to refuse a body that is not an image: a bare RIFF entry made a WAV sniff as WebP and made the correct check unreachable, wrapper detection required a literal <html> so a doctype was never followed, and two scripts printed failures and exited zero. Verified throughout: deploy/maps/ is byte-identical after every commit, which is what proves the move changed no behavior, and every script run from an unrelated directory resolves its paths into the capture rather than into the repo.
The State table said the periodic log review "has not run at all", while Recurring operations records twelve lines down that the outward pass ran end to end on 2026-08-08 traffic and found nothing to add to the URL contract. The row now names the outward pass as run once by hand, the inward pass as not run, and neither as on its cadence yet. The row's other claim is unchanged because it was re-measured and still holds: the backup timer's LAST is `-`, its NEXT is 2026-08-09 09:10:55 UTC, and the service journal holds one entry from the hand-started run, so the schedule is still unproven. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Promotes develop to main, carrying in the operational tooling and CI gates added across PRs #62–#67 (backup pull + documentation, capture/provenance tooling, live media delivery check, mtime restore for hard-linking, and the environment-value documentation gate).
Changes:
- Add
ops/backup+log pull tooling (script + systemd units + installer) and document the operational workflow. - Introduce
capture/for durable migration/provenance scripts and update docs to make directory READMEs authoritative. - Strengthen CI/verification: environment-docs gate, live media checks in
check-live-urls.sh, and restore file mtimes in deploy workflow for effective--link-dest.
Reviewed changes
Copilot reviewed 31 out of 32 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| TODO.md | Updates operational state tracking and references capture/ as the authoritative migration procedure. |
| README.md | Documents new capture/, ops/, and ENVIRONMENT.md entry points and updates env-template references. |
| ops/vps-backup-pull.timer | Adds systemd timer for scheduled VPS pull. |
| ops/vps-backup-pull.service.d-local.conf.example | Provides host-local drop-in template for User/Group/RequiresMountsFor. |
| ops/vps-backup-pull.service | Adds systemd oneshot service definition for the pull script. |
| ops/vps-backup-pull | Adds the backup+logs pull script with preflight, verification, and reporting. |
| ops/README.md | Documents purpose, install, and operational expectations for the pull tooling. |
| ops/install.sh | Adds installer that derives host-specific settings from secrets/*.env. |
| OPERATIONS.md | Reorients migration docs to READMEs, documents VPS tooling and log-review mechanics. |
| example.env | Consolidates env templates into a single root template for all configuration values. |
| ENVIRONMENT.md | Adds single-source documentation for all configuration variables, enforced by CI. |
| deploy/README.md | Updates generator references and documents additional tooling dependencies. |
| deploy/env.example | Removes the old deploy env template (superseded by example.env). |
| content/posts/2026/08/01/moving-this-blog-from-wordpress-to-hugo.md | Links the post to capture/README.md as the maintained procedure. |
| checks/README.md | Documents golden-media-live.txt and updates regeneration guidance to capture/. |
| checks/golden-media-live.txt | Adds curated media URLs for live delivery verification. |
| checks/check-live-urls.sh | Adds live media fetching/assertions and list floor checks. |
| checks/check-env-docs.py | Adds CI gate ensuring all config values are documented exactly once. |
| capture/run-wp2hugo.sh | Adds capture conversion script that shares export-selection logic with generator. |
| capture/restructure-content.py | Adds conversion reshaping script to match URL structure. |
| capture/README.md | Adds authoritative procedure/facts for provenance capture and conversion pipeline. |
| capture/localize-external.py | Adds script to localize hotlinked media and rewrite content references. |
| capture/enumerate-media.py | Adds script to enumerate media references from the legacy platform API. |
| capture/clean-content.py | Adds script to drop comments and reduce front matter to needed keys. |
| capture/classify.py | Adds script splitting verified URLs into render vs redirect sets. |
| capture/build-redirects.py | Moves/updates redirect-map generator into capture/ and improves robustness. |
| capture/build-golden.py | Adds capture-time tooling to derive/verify the legacy URL surface area. |
| .gitignore | Updates ignore rules for env templates and includes capture/ Python byproducts note. |
| .github/workflows/validate-task.yml | Extends shell linting and adds the environment-docs check step. |
| .github/workflows/deploy-site-task.yml | Installs git-restore-mtime and restores static/ mtimes before deploy. |
| .gitattributes | Updates LF pinning rules for moved/added executables and scripts. |
| .editorconfig | Updates map-file generator reference and keeps formatting rules aligned with generators. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…69) * Pin the one unpinned shebang script, and gate the pin list both ways `ops/vps-backup-pull` is extensionless, so no `*.sh` or `*.py` rule reached it, and `* -text` keeps git passive by default. A CRLF checkout would hand systemd a broken interpreter line for a script it runs unattended. The reason it was missed is the more interesting half. `.gitattributes` already pinned `deploy/blog-deploy-shell` and `deploy/authorized_keys`, and this repository has never carried either file, in any commit: they live on the server, described in OPERATIONS.md "Server Hardening". A pin for a file that does not exist binds nothing, and the comment above those two claimed the extensionless shebang case was handled. That is what made the real one invisible. Both pins are dropped, with the reason recorded where they were. `deploy/bootstrap.Caddyfile` gets a pin it never had. It is read by Caddy from the container's config directory and is the only Caddy file outside the release bundle, so it is the same daemon-parsed class as `deploy/Caddyfile` beside it, which was pinned. `checks/check-eol-pins.py` reads the list back, in both directions, because a hand-maintained list that nothing verifies is how this got here: unpinned a tracked file opening with `#!` whose resolved eol is not lf. The shebang is the test rather than the mode bit, because the two move independently and it is the interpreter line a CRLF breaks. dead a pattern matching no tracked file, which reads as coverage while binding nothing. It asks `git check-attr` for the resolved attribute instead of re-implementing the match rules, so it cannot disagree with what git actually applies on checkout, and it fails rather than passing if it finds no shebang files at all. Verified by reintroducing each defect against the gate: dropping the `ops/vps-backup-pull` line reports it unpinned, and re-adding `deploy/blog-deploy-shell` reports it dead. It also caught its own file before that was staged, which is the third direction working. `git add --renormalize .` after the change stages nothing beyond these three files, so the pins bind future checkouts and rewrite no bytes now. systemd units are deliberately not pinned. Measured rather than assumed: `systemd-analyze verify` on a CRLF copy of `vps-backup-pull.service` reports the same clean result as the LF original, and the same tool does flag an unreachable `ExecStart`, so the negative means something. Found by Copilot review on #68. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Correct the tense in two places that outlived the change making them false The workflow comment read "a hand-maintained list that nothing read back", which is both ungrammatical and, as of the step it introduces, no longer true. The script's own docstring had the same problem in the present tense: it asserted that nothing reads the list back, which its own existence disproves. Found by Copilot review on #69, which caught the workflow half. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
) * Match .gitattributes semantics when asking which files a pin covers The dead-pattern direction asked `git ls-files -- <pattern>`, and pathspec does not share gitattributes glob semantics. It is wrong in both directions: under a slash-free pattern matches its basename at any depth in gitattributes, where bare pathspec reads it as a root-relative path. `Dockerfile` covers `Docker/Dockerfile`; bare pathspec returns nothing, so a live pin reports dead and reds CI. over `*` does not cross a `/` in gitattributes and does in bare pathspec, so `pkg/*.py` wrongly picks up `pkg/sub/nested.py`. Harmless here, since it can only hide a dead pin. The first is the one with teeth: a false failure on a correct pin. `:(glob)` gives `*` and `**` their gitattributes meaning, and a `**/` prefix supplies the any-depth match for a slash-free pattern. Verified against `git check-attr`, which is what git actually applies, rather than against the documentation. On a tree holding `Docker/Dockerfile`, `pkg/mod.py` and `pkg/sub/nested.py`, check-attr resolves eol=lf for exactly the first two, and the conversion selects exactly those two where the bare form selects the wrong set both times. Differentially: on that tree the old matcher reports `Dockerfile` dead and the new one does not. No pattern in this repository changes result, bare or converted, so this fixes a latent defect rather than a live one. Nothing here is slash-free-and-literal today. Found by Copilot review on #68. Its remedy was right and its evidence was not: it predicted `*.sh` would be flagged dead here for want of a root-level `.sh` file, and `git ls-files -- '*.sh'` returns all five nested ones, because bare pathspec lets `*` cross a `/`. That is the over-match above, and the reason CI was green rather than failing as the comment predicted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Anchor a leading-slash pattern, and check the matcher against git itself `/Dockerfile` is root-anchored and covers only the root file, where a bare `Dockerfile` matches at any depth. `pathspec_for` stripped the leading slash and then tested the remainder for one, which destroys the evidence the decision needs, so it built `:(glob)**/Dockerfile` and matched `Docker/Dockerfile` too. Verified against check-attr, which resolves eol=lf for the root file alone. Anchoring is now decided before the strip. That is the second conversion defect in this function, one under-matching and one over-matching, and both were found by review rather than by the gate. Fixing each shape as it surfaces leaves the next one to be found the same way, so the matcher is now checked against `git check-attr`, which is the thing it exists to predict: every file resolving to eol=lf is matched by some converted pattern, and every file a converted pattern matches resolves to eol=lf. A missing file means the conversion under-matched, which is what produces a false dead report. An extra file means it over-matched, which hides a real one. Both are reported as matcher findings rather than as pin findings, because they are defects in this script and not in the pins. Verified by reintroducing each defect against a tree holding `Docker/Dockerfile`, a root `Dockerfile`, `pkg/mod.py` and `pkg/sub/nested.py`. The bare-pathspec version is caught in both directions at once, the leading-slash version as an over-match, and with the matcher correct the third direction reports nothing on the same tree. This holds because no pattern here clears `eol` once another has set it, which the comment records, since a future pattern that unsets it would need the comparison to account for precedence. Found by Copilot review on #70. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes
developtomain. Six commits, each already reviewed and squash-merged todevelopas its own pull request.What this carries
ops/holds the off-host backup pull, its twosystemdunits, anEnvironmentFiletemplate and an installer that derives every path fromsecrets/local.production.env.ENVIRONMENT.mdis the single description of all 31 configuration values, gated in CI bychecks/check-env-docs.py, which fails in both directions. The two.examplefiles collapse into oneexample.envat the root.checks/check-live-urls.shnow requests eight media URLs and asserts the response is an image, which is the only gate that sees a file mode ride a hard link into every later release.--link-destmatched nothing: every release has been a full copy. With the restore, 1052 of 1791 files link, which is 566 MB of the 586 MB a release occupies.capture/holds the migration's durable scripts, and the directory READMEs are the authority. The scripts split by whether they can ever run again rather than by whether they look reusable.build-redirects.pymoves fromchecks/tocapture/, because it generates rather than gates, anddeploy/maps/is byte-identical after the move. The doc hierarchy inverts as a class: a README is procedure and fact, the post is the casual account, and a post may cite a README where a README never cites the post.The ordering in #64 and #65 is deliberate
The live media check landed before the mtime restore. Making
--link-desteffective stops the per-deploy re-assertion of file modes, so the first release after #65 is also the first where a bad mode can ride a link into every release after it. The check that watches for exactly that is in place first.Verification
The
content/andchecks/changes put this promotion inside theOPERATIONS.md"Local Verification Before a Pull Request" gate, so the pair ran against the local staging mirror at392de22before this pull request was opened:The build gate passed in the same run: 328/328 golden URLs, 778/778 legacy image URLs, 1049/1049 local asset references, 98 orphans, 0 stray gallery nodes,
robots.txtbuilt.Merge
Plain
--merge, never--delete-branch: on adevelop -> mainpromotion that flag deletesdevelop.Nothing publishes on this merge.
publish-release.ymlisworkflow_dispatchonly, and a human merge never auto-publishes.