From fe9ee784dbe91f19e2f65e65e8422905db7c0506 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Mon, 24 Aug 2026 06:41:16 -0700 Subject: [PATCH 1/3] Adopt the Fleet's .secrets/ Convention and Rename SITE_ Env Keys (#107) * Adopt the fleet's .secrets/ convention and rename SITE_ env keys Restructures local secrets handling to match the fleet convention: the tracked directory is .secrets/, holding only example.env and a README catalog, gitignored otherwise. Real values live in the host-shared ~/.secrets/, named Blog...env to disambiguate from other repos' files there. Renames the env-file key names to the same generic SITE_BASE_URL, SITE_AUTH_TOKEN_ID, and SITE_AUTH_TOKEN the GitHub Environment and CI hook already use, dropping the local HUGO_BASEURL and PANGOLIN_ACCESS_TOKEN_* names everywhere except where Hugo itself requires its own name. deploy/make-release.sh bridges SITE_BASE_URL to HUGO_BASEURL internally, letting the CI hook drop its own bridge for both that value and the auth token pair. * Keep the retired secrets/ directory ignored as a migration backstop A checkout that predates this migration and still carries a pre-existing secrets/ directory would lose its ignore coverage entirely, turning any real per-environment file left there into something a routine git add -A could commit. * Fix stale HUGO_BASEURL leak and clarify the CI-only command form deploy/make-release.sh now unsets HUGO_BASEURL when SITE_BASE_URL is unset, so an inherited value from an earlier shell export can no longer silently win over hugo.yaml. The two-line build-and-check snippets in README.md and deploy/README.md now source the environment file first, since make-release.sh cannot export SITE_BASE_URL back to the caller's shell. The command-prefix SITE_BASE_URL= form in deploy/README.md and OPERATIONS.md is now marked CI-only, since a local run's sourced environment file overwrites it. TODO.md's installer description now matches what ops/install.sh actually derives from the environment file versus from the invoking environment itself. * Drop a duplicated 'with' in the SITE_BASE_URL table row --- .github/actions/deploy/action.yml | 12 ++---- .gitignore | 21 +++++----- .secrets/README.md | 24 ++++++++++++ example.env => .secrets/example.env | 32 +++++++-------- ENVIRONMENT.md | 28 +++++++------- OPERATIONS.md | 60 +++++++++++++++-------------- README.md | 7 ++-- TODO.md | 4 +- capture/README.md | 8 ++-- capture/build-golden.py | 2 +- capture/build-redirects.py | 2 +- capture/classify.py | 2 +- capture/clean-content.py | 2 +- capture/enumerate-media.py | 2 +- capture/localize-external.py | 2 +- capture/restructure-content.py | 2 +- capture/run-wp2hugo.sh | 2 +- checks/check-env-docs.py | 4 +- checks/check-live-urls.sh | 12 +++--- deploy/README.md | 43 +++++++++++---------- deploy/make-release.sh | 23 +++++++---- ops/README.md | 8 ++-- ops/install.sh | 20 ++++------ 23 files changed, 177 insertions(+), 145 deletions(-) create mode 100644 .secrets/README.md rename example.env => .secrets/example.env (81%) diff --git a/.github/actions/deploy/action.yml b/.github/actions/deploy/action.yml index 04ba9d4..6c8e9a8 100644 --- a/.github/actions/deploy/action.yml +++ b/.github/actions/deploy/action.yml @@ -1,8 +1,8 @@ # Blog's own deploy hook, run by the hub's deploy-site-task.yml in build, prune, and verify modes. # The hub task forwards SITE_BASE_URL, SITE_AUTH_TOKEN_ID, and SITE_AUTH_TOKEN as plain process env vars, not through this composite action's own expression context. # This hook therefore reads them as $SITE_BASE_URL and so on in its run: steps, never as ${{ env.* }}. -# Hugo's own HUGO_BASEURL convention is bridged from SITE_BASE_URL inside build mode only. -# Every other script in this repo keeps reading HUGO_BASEURL exactly as OPERATIONS.md documents. +# Both deploy/make-release.sh and checks/check-live-urls.sh read these same names directly. +# The bridge to Hugo's own HUGO_BASEURL convention happens inside make-release.sh, since only Hugo requires that name. name: Deploy hook (Blog) description: The build, prune, and verify modes deploy-site-task.yml calls. @@ -65,9 +65,6 @@ runs: if: inputs.mode == 'build' uses: ./.github/actions/install-hugo - # HUGO_BASEURL is set here, once, from the hub's SITE_BASE_URL. - # Everything downstream, Hugo itself and the verify step below, reads whichever name it already expects. - # No other script in this repo touches SITE_BASE_URL directly. - name: Assemble release bundle step if: inputs.mode == 'build' shell: bash @@ -76,7 +73,7 @@ runs: MTIME_RESTORED: '1' run: | set -Eeuo pipefail - HUGO_BASEURL="$SITE_BASE_URL" deploy/make-release.sh "${{ inputs.bundle-path }}" "${{ inputs.release-id }}" + deploy/make-release.sh "${{ inputs.bundle-path }}" "${{ inputs.release-id }}" # Deliberately a no-op. # OPERATIONS.md "Who Owns What" states it directly: the host's blog-prune-releases.timer owns retention, and nothing in this repo does. @@ -97,5 +94,4 @@ runs: EXPECT_RELEASE: ${{ inputs.release-id }} run: | set -Eeuo pipefail - PANGOLIN_ACCESS_TOKEN_ID="${SITE_AUTH_TOKEN_ID:-}" PANGOLIN_ACCESS_TOKEN="${SITE_AUTH_TOKEN:-}" \ - checks/check-live-urls.sh "$SITE_BASE_URL" + checks/check-live-urls.sh "$SITE_BASE_URL" diff --git a/.gitignore b/.gitignore index 0678a5b..0a16fe0 100644 --- a/.gitignore +++ b/.gitignore @@ -29,17 +29,16 @@ __pycache__/ # Host-specific values: deploy roots, base URLs, container names, and uids. # Each names one particular machine rather than the project. -# The whole directory is ignored so a value added later lands ignored by default. -# `example.env` is the committed template and sits at the repository root. -# The `*.env` pattern is the backstop for a real environment file written outside the -# directory, matching the `..env` shape those files are named for -# rather than a single literal name. It also matches the template, so the template is -# negated on the line after it, anchored so it only exempts the one at the root. Order -# matters: a negation placed before its pattern does nothing. -secrets/ -**/secrets -*.env -!/example.env +# Real values live on the host in ~/.secrets/, never in this checkout, so nothing under this directory is ever a real value except the two exemptions below. +# Order matters: a negation placed before its pattern does nothing. +.secrets/* +!.secrets/example.env +!.secrets/README.md + +# The retired convention this replaced kept real per-environment files in secrets/ at the repository root, ignored by a rule this migration otherwise removes. +# A checkout that predates the migration and still carries that directory would have those real files turn up as untracked instead, one git add -A away from being committed. +# Kept here as a permanent backstop against exactly that, even though the convention itself is gone and no current checkout carries the directory. +/secrets/ # The working copies of the host channel described in OPERATIONS.md. # They carry server internals, and the host's own backup is what makes them durable. diff --git a/.secrets/README.md b/.secrets/README.md new file mode 100644 index 0000000..642bbd5 --- /dev/null +++ b/.secrets/README.md @@ -0,0 +1,24 @@ +# .secrets + +Nothing under this directory holds a real value except [`example.env`](./example.env), the +tracked template, and this catalog. `.gitignore` un-ignores exactly those two paths and ignores +everything else here, so a fresh checkout documents its own required shape without ever +exposing one. + +## Real values live on the host, never in the checkout + +Every real value this repo's scripts read comes from `~/.secrets/`, not from this directory. +`ENV_FILE= deploy/make-release.sh` and `ops/install.sh` both resolve a relative `ENV_FILE` +against `$HOME/.secrets`, refuse a traversing one, and default to +`~/.secrets/Blog.local.production.env`. `~/.secrets/` is shared across every repo on the host, +so each of this repo's files carries the `Blog.` prefix: + +| File | Selects | +| --- | --- | +| `~/.secrets/Blog.local.production.env` | The default, read when `ENV_FILE` is unset. | +| `~/.secrets/Blog.local.staging.env` | The local staging mirror. | +| `~/.secrets/Blog.vps.production.env` | The VPS production environment. | +| `~/.secrets/Blog.vps.staging.env` | The VPS staging environment. | + +Every value each file holds, and what reads it, is described once in +[`ENVIRONMENT.md`](../ENVIRONMENT.md). Start a new one from [`example.env`](./example.env). diff --git a/example.env b/.secrets/example.env similarity index 81% rename from example.env rename to .secrets/example.env index 6f7dee9..0d81837 100644 --- a/example.env +++ b/.secrets/example.env @@ -7,36 +7,38 @@ # It fills two destinations, marked below, because a value belongs to whichever machine # holds it. Copy the section you need rather than the whole file. # -# secrets/..env on a workstation, one file per environment -# /etc/vps-backup-pull.env on the backup host, or let ops/install.sh write it +# ~/.secrets/Blog...env on a workstation, one file per environment +# /etc/vps-backup-pull.env on the backup host, or let ops/install.sh write it # # A value appearing in both sections is spelled the same way in both, deliberately. One # name per thing means the side that writes and the side that reads cannot disagree, which # is also why ops/install.sh copies values across rather than translating them. # # Naming convention: the prefix names whatever owns the value, not whatever reads it. -# HUGO_ is fixed by Hugo, DEPLOY_ is the release tooling, CADDY_ is the container, -# PANGOLIN_ is the proxy, VPS_ is the server, and a *_ROOT is a directory on this host. +# SITE_ is the generic deploy contract, DEPLOY_ is the release tooling, CADDY_ is the +# container, VPS_ is the server, and a *_ROOT is a directory on this host. deploy/make-release.sh +# and checks/check-live-urls.sh bridge SITE_ names to a tool's own native ones internally, +# HUGO_BASEURL for Hugo, only where that tool requires its own name. # ============================================================================= -# secrets/..env +# ~/.secrets/Blog...env # ============================================================================= # One file per environment, named for the server it describes and the environment on it, # with both words spelled out, and selected by ENV_FILE: -# secrets/local.production.env the default, read when ENV_FILE is unset -# secrets/local.staging.env ENV_FILE=secrets/local.staging.env deploy/make-release.sh -# secrets/vps.production.env ENV_FILE=secrets/vps.production.env deploy/make-release.sh -# secrets/vps.staging.env ENV_FILE=secrets/vps.staging.env deploy/make-release.sh +# Blog.local.production.env the default, read when ENV_FILE is unset +# Blog.local.staging.env ENV_FILE=Blog.local.staging.env deploy/make-release.sh +# Blog.vps.production.env ENV_FILE=Blog.vps.production.env deploy/make-release.sh +# Blog.vps.staging.env ENV_FILE=Blog.vps.staging.env deploy/make-release.sh # # Sourced with `set -a`, which overwrites a variable the caller exported first. A named -# file that does not exist is a hard failure rather than a fall-through. The whole -# secrets/ directory is gitignored, so no value naming a machine reaches this history. +# file that does not exist is a hard failure rather than a fall-through. Real files live +# in ~/.secrets/, never in this checkout, so no value naming a machine reaches this history. # Where a release is written. The first argument to make-release.sh wins over it. DEPLOY_ROOT=/path/to/deploy/root # The site base URL. Must be set for anything that is not production. -HUGO_BASEURL=https://blog.example.com/ +SITE_BASE_URL=https://blog.example.com/ # The container's persistent state root, outside DEPLOY_ROOT. Nothing reads it. CADDY_APPDATA=/path/to/container/appdata @@ -48,8 +50,8 @@ CADDY_CONTAINER=blog-production EXPECT_SITE_ENV=production # Resource access token for an environment behind the auth gate. Set both or neither. -PANGOLIN_ACCESS_TOKEN_ID= -PANGOLIN_ACCESS_TOKEN= +SITE_AUTH_TOKEN_ID= +SITE_AUTH_TOKEN= # Read by the deploy workflow, which resolves them from the GitHub Environment rather than # a file. Named here so the local file and the environment describe the same shape. @@ -66,7 +68,7 @@ PANGOLIN_ACCESS_TOKEN= CAPTURE_ROOT=/path/to/blog-capture # The old platform's base URL, the site the crawl and the URL verification ran against. -# NOT HUGO_BASEURL, which is where this site is served now. See ENVIRONMENT.md. +# NOT SITE_BASE_URL, which is where this site is served now. See ENVIRONMENT.md. CAPTURE_SOURCE_URL=https://blog.example.com # The old platform's REST API for that site, carrying its numeric site id. diff --git a/ENVIRONMENT.md b/ENVIRONMENT.md index c64cfcd..b5b126b 100644 --- a/ENVIRONMENT.md +++ b/ENVIRONMENT.md @@ -8,7 +8,7 @@ Every configuration value this repository reads or writes, described once. [`OPE ## The mechanism -A local run reads one file. `secrets/..env` is sourced with `set -a`, selected by `ENV_FILE`, and defaults to `secrets/local.production.env`. The whole `secrets/` directory is gitignored, so a value naming a machine never reaches the published history, and [`example.env`](./example.env) is the tracked template that documents the shape. +A local run reads one file. `~/.secrets/Blog...env` is sourced with `set -a`, selected by `ENV_FILE`, and defaults to `~/.secrets/Blog.local.production.env`. The real files live on the host, never in this checkout, and [`.secrets/example.env`](./.secrets/example.env) is the tracked template that documents the shape. Two consequences of `set -a` are worth stating because both have surprised someone. Sourcing overwrites a variable the caller exported first, so exporting `DEPLOY_ROOT` by hand does not switch environments and only `ENV_FILE` does. And a named file that does not exist is a hard failure rather than a fall-through, because on a host serving two sites the ambient value is the other site's root. @@ -16,19 +16,19 @@ CI reads no file. The deploy workflow resolves the same values from the GitHub E ## Repository environment files -Held in `secrets/..env`, one file per environment. Template: [`example.env`](./example.env). +Held in `~/.secrets/Blog...env`, one file per environment, on the host rather than in this checkout. Template: [`.secrets/example.env`](./.secrets/example.env). | Value | Names | Notes | | --- | --- | --- | | `DEPLOY_ROOT` | where a release is written, and what the container mounts read-only at `/srv/blog` | The first argument to `make-release.sh` wins over it. | -| `HUGO_BASEURL` | the site base URL | Baked into the canonical tag, the feed links, and every absolute permalink. Must be set for anything that is not production, or a mirror serves pages pointing at production and every gate still passes. | +| `SITE_BASE_URL` | the site base URL | Baked into the canonical tag, the feed links, and every absolute permalink. Must be set for anything that is not production, or a mirror serves pages pointing at production and every gate still passes. `make-release.sh` bridges it to Hugo's own `HUGO_BASEURL`, the only name Hugo itself reads. | | `CADDY_APPDATA` | the container's persistent state root, deliberately outside `DEPLOY_ROOT` | Holds `config/` with the bootstrap Caddyfile and `data/` with Caddy state. A release writes neither. Nothing reads this value, so it is recorded to keep a rebuild from depending on memory. | | `CADDY_CONTAINER` | the container serving this environment | A release needs no restart, because Caddy reloads in process. Restarting is the remedy when the watcher dies, which it does silently after one failed load. | | `EXPECT_SITE_ENV` | the environment that must answer, compared against the `X-Blog-Env` header the bundle stamps | A proxy rule aimed at the wrong container returns a healthy 200 under the right hostname, so the check refuses to start rather than proving nothing. | -| `PANGOLIN_ACCESS_TOKEN_ID` | the resource access token's id, for an environment behind the auth gate | Set both or neither. Leave both unset for a site that is public. | -| `PANGOLIN_ACCESS_TOKEN` | the token itself | Read by `check-live-urls.sh`. Staging keeps its gate on because it serves a byte-identical copy of the public site. | +| `SITE_AUTH_TOKEN_ID` | the resource access token's id, for an environment behind the auth gate | Set both or neither. Leave both unset for a site that is public. | +| `SITE_AUTH_TOKEN` | the token itself | Read by `check-live-urls.sh`, sent as Pangolin's own `P-Access-Token` header. Staging keeps its gate on because it serves a byte-identical copy of the public site. | | `CAPTURE_ROOT` | the provenance capture, holding the WordPress exports, the crawl of the old platform, and the inventories derived from it | Every script under [`capture/`](./capture/) reads beneath it, and all but one write there too. The exception is [`capture/build-redirects.py`](./capture/build-redirects.py), which writes the committed maps under `deploy/maps/` in this repository, and which also accepts the capture as a first argument that wins over this value. Environment-independent, so it belongs in the default file only. | -| `CAPTURE_SOURCE_URL` | the old platform's base URL, the site the crawl and the URL verification ran against | **Not `HUGO_BASEURL`.** The two hold the same string after the cutover and mean different things, so merging them points a verification run at the new site while every check still passes. Environment-independent. | +| `CAPTURE_SOURCE_URL` | the old platform's base URL, the site the crawl and the URL verification ran against | **Not `SITE_BASE_URL`.** The two hold the same string after the cutover and mean different things, so merging them points a verification run at the new site while every check still passes. Environment-independent. | | `CAPTURE_SOURCE_API` | the old platform's REST API for that site, carrying its numeric site id | Read for the post and page bodies in **rendered** form, which is what expands shortcodes so a media reference is seen the way a reader's browser sees it. Environment-independent. | | `CAPTURE_AUTHOR_SLUG` | the old platform's author slug, used to backfill the author archive and its pagination | Optional, and an account name rather than a site value, which is why it is a variable at all. Unset, [`capture/classify.py`](./capture/classify.py) skips the backfill and says so, rather than emitting a list that is silently short by the author URLs. Environment-independent. | | `VPS_SSH_HOST` | the VPS administrative login | Not the deploy account. See "Two credentials" below. Environment-independent. | @@ -42,7 +42,7 @@ Three more are named in the template but commented out, because CI resolves them ## The backup host -Held in `/etc/vps-backup-pull.env`, read by `vps-backup-pull` through the unit's `EnvironmentFile`. Template: [`example.env`](./example.env). [`ops/install.sh`](./ops/install.sh) generates it by copying from the repository environment file, which is why the four shared names are spelled identically in both. +Held in `/etc/vps-backup-pull.env`, read by `vps-backup-pull` through the unit's `EnvironmentFile`. Template: [`.secrets/example.env`](./.secrets/example.env). [`ops/install.sh`](./ops/install.sh) generates it by copying from the repository environment file, which is why the four shared names are spelled identically in both. | Value | Names | Notes | | --- | --- | --- | @@ -56,7 +56,7 @@ Held in `/etc/vps-backup-pull.env`, read by `vps-backup-pull` through the unit's **The three marked required carry no default on purpose.** An address and a destination belong to one host, and a wrong-but-valid destination is a backup nobody can find, so the pull names what is missing and refuses to run rather than falling back to something plausible. -**`systemd` parses this file itself rather than passing it to a shell**, so there is no expansion and no command substitution, and a `$` or a backtick is a literal character. It does strip matching quotes, verified rather than assumed, so a value containing spaces is quoted and arrives without them. That matters because [`example.env`](./example.env) is also sourced by a shell for the other destination, where an unquoted value would run everything after the first space as a command. +**`systemd` parses this file itself rather than passing it to a shell**, so there is no expansion and no command substitution, and a `$` or a backtick is a literal character. It does strip matching quotes, verified rather than assumed, so a value containing spaces is quoted and arrives without them. That matters because [`.secrets/example.env`](./.secrets/example.env) is also sourced by a shell for the other destination, where an unquoted value would run everything after the first space as a command. ## The GitHub Environments @@ -64,15 +64,15 @@ Held on the `production` and `staging` environments. The deploy workflow reads n | Value | Kind | Names | | --- | --- | --- | -| `SITE_BASE_URL` | variable | the base URL, used twice: `.github/actions/deploy/action.yml` builds the site with it (as `HUGO_BASEURL`) and points `check-live-urls.sh` at it | +| `SITE_BASE_URL` | variable | the base URL, used twice: `.github/actions/deploy/action.yml` builds the site with it and points `check-live-urls.sh` at it | | `DEPLOY_SSH_HOST` | variable | the deploy endpoint | | `DEPLOY_SSH_USER` | variable | the confined deploy account | | `DEPLOY_SSH_KNOWN_HOSTS` | variable | the pinned host key. A variable rather than a secret, deliberately, since it is public by nature | | `DEPLOY_SSH_PRIVATE_KEY` | secret | the deploy key, held behind an `rrsync` forced command | -| `SITE_AUTH_TOKEN_ID` | secret | as above, for an environment behind the gate. `.github/actions/deploy/action.yml` reads it as `PANGOLIN_ACCESS_TOKEN_ID` for `check-live-urls.sh` | -| `SITE_AUTH_TOKEN` | secret | as above, bridged to `PANGOLIN_ACCESS_TOKEN` the same way | +| `SITE_AUTH_TOKEN_ID` | secret | as above, for an environment behind the gate. Forwarded to `checks/check-live-urls.sh`, which reads this name directly | +| `SITE_AUTH_TOKEN` | secret | as above, forwarded the same way | -**`SITE_BASE_URL` being read twice is the trap worth knowing.** A wrong value bakes the wrong address into every canonical tag and then runs the full URL contract against that same wrong address, so the deploy verifies itself and passes. Its generic name is the hub's own `deploy-site-task.yml` interface, since that task is not Hugo-specific. Blog's own scripts and `OPERATIONS.md` keep reading `HUGO_BASEURL`, which the deploy hook bridges from `SITE_BASE_URL` in one place. +**`SITE_BASE_URL` being read twice is the trap worth knowing.** A wrong value bakes the wrong address into every canonical tag and then runs the full URL contract against that same wrong address, so the deploy verifies itself and passes. Its generic name is the hub's own `deploy-site-task.yml` interface, and it is also the one this repository's own scripts and `~/.secrets/Blog.*.env` files read: `make-release.sh` bridges it to Hugo's own `HUGO_BASEURL` in one place, since only Hugo requires that name. **A host rebuild regenerates the SSH host keys and the pinned value stops matching**, which fails every deploy closed and blocks the rollback path at the same moment a rebuild makes both matter. Replace `DEPLOY_SSH_KNOWN_HOSTS` on **both** environments before the first deploy after a rebuild. @@ -84,7 +84,7 @@ Set on the command line for one run rather than stored anywhere. | Value | Effect | | --- | --- | -| `ENV_FILE` | which environment file to source. Defaults to `secrets/local.production.env` | +| `ENV_FILE` | which environment file to source. Defaults to `~/.secrets/Blog.local.production.env` | | `REQUIRE_BROTLI=1` | fail rather than shipping gzip-only. CI sets it | | `NO_LINK_DEST=1` | full copy instead of hard-linking from the previous release | | `KEEP_RELEASES` | how many releases `make-release.sh` leaves behind | @@ -103,6 +103,6 @@ The two channel transfers under [`OPERATIONS.md`](./OPERATIONS.md) "The Channel ## Rules - **One name per thing.** A value that appears on two sides is spelled identically on both, so neither side needs translating into the other. -- **No value naming a machine reaches git.** Not in a script default, not in a unit, not in a template. The `.example` files carry placeholders, and the real values live in `secrets/` or on the host. +- **No value naming a machine reaches git.** Not in a script default, not in a unit, not in a template. The `.example` files carry placeholders, and the real values live in `~/.secrets/` on the host, never in this checkout. - **A description belongs here and a reference belongs everywhere else.** A `.example` file says what the format is, and this file says what the value means. - **Nothing sources some of these, and that is recorded rather than hidden.** A value kept only so a rebuild does not depend on memory is still worth holding, but a reader should not have to discover that no code reads it. diff --git a/OPERATIONS.md b/OPERATIONS.md index fe3d680..9379f3a 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -8,16 +8,16 @@ Four environments, in two pairs. Each pair is one publish site and one staging s | Environment | Address | Fronted by | Purpose | | --- | --- | --- | --- | -| Local publish mirror | a private hostname, set in `secrets/local.production.env` | Traefik, on the maintainer's own network | Proves the artifact. The redirect rules, the maps, and the release mechanics. | -| Local staging mirror | a second private hostname, set in `secrets/local.staging.env` | Traefik | Proves that two environments on one host stay independent, before that matters on a server. | -| Staging | `blog.vps.insanegenius.net`, behind the auth gate, set in `secrets/vps.staging.env` | Pangolin | Proves the infrastructure. Routing, TLS, and the deploy path. | -| Production | `blog.insanegenius.com`, set in `secrets/vps.production.env` | Pangolin | The public site. | +| Local publish mirror | a private hostname, set in `~/.secrets/Blog.local.production.env` | Traefik, on the maintainer's own network | Proves the artifact. The redirect rules, the maps, and the release mechanics. | +| Local staging mirror | a second private hostname, set in `~/.secrets/Blog.local.staging.env` | Traefik | Proves that two environments on one host stay independent, before that matters on a server. | +| Staging | `blog.vps.insanegenius.net`, behind the auth gate, set in `~/.secrets/Blog.vps.staging.env` | Pangolin | Proves the infrastructure. Routing, TLS, and the deploy path. | +| Production | `blog.insanegenius.com`, set in `~/.secrets/Blog.vps.production.env` | Pangolin | The public site. | The local mirrors are not staging. They run the same bundle against the same web server, so they catch a broken redirect or a bad permission for free, but they exercise none of the routing, authentication, or certificate machinery that only exists on the VPS. Passing locally says the artifact is right. It says nothing about whether the server in front of it is. **The two words are `production` and `staging`, spelled out, in every position.** No `prod`, no `stage`. The same two name the container, the deploy root, the environment file, the `X-Blog-Env` value, and the GitHub Environment. This is not tidiness: the environment name is a value that gets **compared**, by `EXPECT_SITE_ENV` and by the deploy, so a spelling that differs in one position fails a deploy for a reason that reads like an outage. The local mirrors prefix the same words, `mirror-production` and `mirror-staging`, so a header names exactly one of the four environments in the fleet. -Each environment is one file under `secrets/`, named `..env`, selected with `ENV_FILE`, and holding the deploy root, the base URL, and the container name. The name carries both halves because the two pairs differ in server as well as environment, so a file says which machine it describes rather than leaving that to the value inside it, and the four in the table above are the four files. `secrets/local.production.env` is the one read when `ENV_FILE` is unset. Selecting the file is how an environment is chosen: the file is sourced with `set -a`, so it overwrites a `DEPLOY_ROOT` the caller exported and setting that variable by hand does not switch anything. A named file that does not exist is a hard failure rather than a fall-through, because on a host serving two sites the ambient value is the other site's root. +Each environment is one file under `~/.secrets/`, named `Blog...env`, selected with `ENV_FILE`, and holding the deploy root, the base URL, and the container name. The name carries both halves because the two pairs differ in server as well as environment, so a file says which machine it describes rather than leaving that to the value inside it, and the four in the table above are the four files. `~/.secrets/Blog.local.production.env` is the one read when `ENV_FILE` is unset. Selecting the file is how an environment is chosen: the file is sourced with `set -a`, so it overwrites a `DEPLOY_ROOT` the caller exported and setting that variable by hand does not switch anything. A named file that does not exist is a hard failure rather than a fall-through, because on a host serving two sites the ambient value is the other site's root. **The staging FQDN sits under the VPS wildcard deliberately.** `blog.vps.insanegenius.net` needs no new certificate and no new DNS record, and it keeps the staging name off the production domain. @@ -76,17 +76,17 @@ So release to the local mirror and run the live check **before** opening a pull | `hugo.yaml`, `layouts/` | Permalink and taxonomy changes move URLs underneath the redirects that point at them. | ```sh -set -a; . secrets/local.production.env; set +a +set -a; . ~/.secrets/Blog.local.production.env; set +a deploy/make-release.sh -checks/check-live-urls.sh "$HUGO_BASEURL" +checks/check-live-urls.sh "$SITE_BASE_URL" ``` Against the staging mirror, name its file in both places, since the sourced values and the ones `make-release.sh` reads must describe the same environment: ```sh -set -a; . secrets/local.staging.env; set +a -ENV_FILE=secrets/local.staging.env deploy/make-release.sh -checks/check-live-urls.sh "$HUGO_BASEURL" +set -a; . ~/.secrets/Blog.local.staging.env; set +a +ENV_FILE=~/.secrets/Blog.local.staging.env deploy/make-release.sh +checks/check-live-urls.sh "$SITE_BASE_URL" ``` **There is no restart step, and that depends on one flag.** The container runs `caddy run --watch`, which re-adapts the config on a timer and reloads it in process. Re-adapting re-executes every `import`, so a new release's `Caddyfile` and `maps/*.map` are picked up through the unchanged `/config/Caddyfile` that the watcher actually names. Measured on this host: content is live the instant the symlink moves, and the rules follow within about a quarter of a second. @@ -102,14 +102,14 @@ That is why the check verifies the config rather than trusting it. **When the re So the bundle stamps its own version as `X-Blog-Release`, and `check-live-urls.sh` compares it against `EXPECT_RELEASE` before checking a single URL. It **waits** for a match rather than sampling once, because the reload is asynchronous and a check that starts immediately after a deploy will otherwise race it. The timeout is what still catches a container that is not watching at all, since that one never converges: ```sh -EXPECT_RELEASE= checks/check-live-urls.sh "$HUGO_BASEURL" +EXPECT_RELEASE= checks/check-live-urls.sh "$SITE_BASE_URL" ``` Sourcing the environment file first puts the deploy root and the base URL in the environment, so no literal value is typed. `make-release.sh` then needs no arguments, because its deploy root falls back to `$DEPLOY_ROOT` and its version falls back to a timestamp. It still accepts both, and [Deploying](#deploying) below passes them explicitly, which is what CI does so a pipeline run names the commit it built rather than the clock. Either form works locally, and the argument wins over the environment. -`ENV_FILE` is set as well as sourced, and the redundancy is deliberate. The script sources its own file regardless, so leaving `ENV_FILE` off would build and install against `secrets/local.production.env` while the shell's `$HUGO_BASEURL` still named staging, and the run would check the staging site after publishing to the production root. The script prints the file it read, on every build, for that reason. +`ENV_FILE` is set as well as sourced, and the redundancy is deliberate. The script sources its own file regardless, so leaving `ENV_FILE` off would build and install against `~/.secrets/Blog.local.production.env` while the shell's `$SITE_BASE_URL` still named staging, and the run would check the staging site after publishing to the production root. The script prints the file it read, on every build, for that reason. -It refuses to install a release that fails the build gate. `check-live-urls.sh` does take a base URL, which is where the sourced `$HUGO_BASEURL` goes. It follows every URL in the contract against the running mirror, checking each redirect's destination rather than trusting its status code. +It refuses to install a release that fails the build gate. `check-live-urls.sh` does take a base URL, which is where the sourced `$SITE_BASE_URL` goes. It follows every URL in the contract against the running mirror, checking each redirect's destination rather than trusting its status code. Expect a `PASS` naming the number of URLs honored, which is the two lists' combined length and grows as they do. Anything less is a finding, and the output names each URL that failed and what it answered. @@ -120,16 +120,16 @@ A documentation-only or workflow-only change does not need this. A change to the Staging keeps Pangolin's authentication on, so an unauthenticated request never reaches the site. `check-live-urls.sh` presents a Pangolin resource access token when both halves of the pair are set, and sends nothing when neither is: ```sh -set -a; . secrets/vps.staging.env; set +a -checks/check-live-urls.sh "$HUGO_BASEURL" +set -a; . ~/.secrets/Blog.vps.staging.env; set +a +checks/check-live-urls.sh "$SITE_BASE_URL" ``` -The gate is the VPS staging environment's, so this is `secrets/vps.staging.env`. The local staging mirror sits behind Traefik on the maintainer's own network and carries neither half of the pair. +The gate is the VPS staging environment's, so this is `~/.secrets/Blog.vps.staging.env`. The local staging mirror sits behind Traefik on the maintainer's own network and carries neither half of the pair. | Variable | Header | | --- | --- | -| `PANGOLIN_ACCESS_TOKEN_ID` | `P-Access-Token-Id` | -| `PANGOLIN_ACCESS_TOKEN` | `P-Access-Token` | +| `SITE_AUTH_TOKEN_ID` | `P-Access-Token-Id` | +| `SITE_AUTH_TOKEN` | `P-Access-Token` | Set both or neither. Half a pair is a typo rather than a choice, and it is rejected as one rather than presented as a failing site. @@ -142,19 +142,21 @@ Three properties of how the credential is handled, each there for a reason worth ## Deploying ```sh -HUGO_BASEURL= deploy/make-release.sh "$(git rev-parse --short HEAD)" +SITE_BASE_URL= deploy/make-release.sh "$(git rev-parse --short HEAD)" checks/check-live-urls.sh ``` -The deploy root and the base URL are the only host-specific values. A local run reads them from an untracked file under `secrets/`, one per environment, copied from [`example.env`](./example.env), and CI passes both explicitly. The whole `secrets/` directory is gitignored, so no address, path, or container name belonging to one machine reaches the published history. +The deploy root and the base URL are the only host-specific values. A local run reads them from a file under `~/.secrets/`, one per environment, copied from [`.secrets/example.env`](./.secrets/example.env), and CI passes both explicitly. The real files live on the host, never in this checkout. -**Always set `HUGO_BASEURL` for anything that is not production.** The base URL is baked into the canonical tag, the feed links, and every absolute permalink, so a mirror built without it serves pages that all point back at the production address. Nothing downstream catches this, because the pages render at the right paths and the build gate passes. The effective value is printed on every build for that reason. +**The command-prefix form above is CI-only.** A local run whose default environment file exists sources it after the command-prefix assignment and overwrites it, since `set -a` overwrites a value the caller exported first. Locally, select the environment through `ENV_FILE` instead, as the two examples earlier in this section do. + +**Always set `SITE_BASE_URL` for anything that is not production.** The base URL is baked into the canonical tag, the feed links, and every absolute permalink, so a mirror built without it serves pages that all point back at the production address. Nothing downstream catches this, because the pages render at the right paths and the build gate passes. `make-release.sh` bridges it to Hugo's own `HUGO_BASEURL` internally, and the effective value is printed on every build for that reason. | Variable | Effect | | --- | --- | -| `ENV_FILE` | Which environment file to source. Defaults to `secrets/local.production.env`. | +| `ENV_FILE` | Which environment file to source. Defaults to `~/.secrets/Blog.local.production.env`. | | `DEPLOY_ROOT` | Fallback deploy root. The first argument wins. | -| `HUGO_BASEURL` | Overrides the site base URL. | +| `SITE_BASE_URL` | Overrides the site base URL. | | `REQUIRE_BROTLI=1` | Fails rather than shipping gzip-only. CI sets this. | | `NO_LINK_DEST=1` | Full copy instead of hard-linking from the previous release. | @@ -191,10 +193,10 @@ The script asserts both halves of that rather than assuming them. It fails when ## Working With the VPS -**Every path and hostname on this page is a value in `secrets/`, never a literal to be remembered or asked for.** The convention is the one "Environments" describes and `CAPTURE_ROOT` already follows: a value naming a machine rather than the project lives in the environment file, is sourced with `set -a`, and is read from there rather than searched for. The VPS values are environment-independent, because there is one such host rather than one per environment, so they sit in the default file alongside `CAPTURE_ROOT`. +**Every path and hostname on this page is a value in `~/.secrets/`, never a literal to be remembered or asked for.** The convention is the one "Environments" describes and `CAPTURE_ROOT` already follows: a value naming a machine rather than the project lives in the environment file, is sourced with `set -a`, and is read from there rather than searched for. The VPS values are environment-independent, because there is one such host rather than one per environment, so they sit in the default file alongside `CAPTURE_ROOT`. ```sh -set -a; . secrets/local.production.env; set +a +set -a; . ~/.secrets/Blog.local.production.env; set +a ssh "$VPS_SSH_HOST" true && echo reachable ``` @@ -213,10 +215,10 @@ ssh "$VPS_SSH_HOST" true && echo reachable **It is a pull rather than a push, and nothing on the VPS knows it happens.** That direction is the security property rather than an implementation detail: the backup host holds a key the VPS trusts, and the VPS holds no credential reaching any other system, so a compromise of the web server cannot walk into the backups that exist to survive it. -**Both sides use one set of names, so there is nothing to reconcile.** The pull writes `BACKUP_ARCHIVE_ROOT` and `LOG_ARCHIVE_ROOT` and the log review reads the same two, spelled the same way, and [`ops/install.sh`](./ops/install.sh) generates the pull's `EnvironmentFile` from this repository's `secrets/` file by copying rather than translating. Every value is described once, in [`ENVIRONMENT.md`](./ENVIRONMENT.md), and [`checks/check-env-docs.py`](./checks/check-env-docs.py) fails if one is declared without a description or described without existing. +**Both sides use one set of names, so there is nothing to reconcile.** The pull writes `BACKUP_ARCHIVE_ROOT` and `LOG_ARCHIVE_ROOT` and the log review reads the same two, spelled the same way, and [`ops/install.sh`](./ops/install.sh) generates the pull's `EnvironmentFile` from this repository's `~/.secrets/` file by copying rather than translating. Every value is described once, in [`ENVIRONMENT.md`](./ENVIRONMENT.md), and [`checks/check-env-docs.py`](./checks/check-env-docs.py) fails if one is declared without a description or described without existing. ```sh -set -a; . secrets/local.production.env; set +a +set -a; . ~/.secrets/Blog.local.production.env; set +a ls -d "$LOG_ARCHIVE_ROOT" "$BACKUP_ARCHIVE_ROOT" ``` @@ -345,7 +347,7 @@ rsync -a root@:/srv/agent-comms/vps-agent.md comms/vps-agent.md rsync -a --no-o --no-g --chmod=F644 comms/blog-agent.md root@:/srv/agent-comms/blog-agent.md ``` -**Spell both commands out rather than reading the host and directory from `secrets/`**, which is the opposite of the rule "Working With the VPS" sets for every other path, and is deliberate. These two are allowlisted in `.claude/settings.local.json`, and an allow rule matches the text of the command rather than the value it expands to, so replacing the literals with `$VPS_SSH_HOST` and `$VPS_COMMS_DIR` turns an allowed transfer into one that prompts. The same rule is why neither may be chained behind `cd` or `&&`: an allow rule matches a standalone command only. +**Spell both commands out rather than reading the host and directory from `~/.secrets/`**, which is the opposite of the rule "Working With the VPS" sets for every other path, and is deliberate. These two are allowlisted in `.claude/settings.local.json`, and an allow rule matches the text of the command rather than the value it expands to, so replacing the literals with `$VPS_SSH_HOST` and `$VPS_COMMS_DIR` turns an allowed transfer into one that prompts. The same rule is why neither may be chained behind `cd` or `&&`: an allow rule matches a standalone command only. **The push suppresses owner and group deliberately.** `-a` implies `-o` and `-g`, and the transfer connects as root, so a plain `rsync -a` carries this workstation's numeric uid onto a host that has no such user and leaves the file owned by a number. @@ -391,7 +393,7 @@ The container reads three host paths, and only one of them a release ever writes Because it sits outside the bundle, no release updates it. Install or refresh it explicitly, once per environment, which is the same command against a different sourced file: ```sh -set -a; . secrets/local.production.env; set +a # or any other secrets/..env +set -a; . ~/.secrets/Blog.local.production.env; set +a # or any other ~/.secrets/Blog...env install -m 644 deploy/bootstrap.Caddyfile "$CADDY_APPDATA/config/Caddyfile" docker restart "$CADDY_CONTAINER" # only this file needs one, see below ``` diff --git a/README.md b/README.md index 3d89568..3a03b33 100644 --- a/README.md +++ b/README.md @@ -178,11 +178,12 @@ checks/check-url-parity.py public Build a release and verify it against a running server: ```sh +set -a; . ~/.secrets/Blog.local.production.env; set +a deploy/make-release.sh -checks/check-live-urls.sh "$HUGO_BASEURL" +checks/check-live-urls.sh "$SITE_BASE_URL" ``` -The deploy root and the base URL come from an untracked file per environment under `secrets/`, named `..env`, copied from [example.env][env-example] and selected with `ENV_FILE`. `secrets/local.production.env` is the one read when `ENV_FILE` is unset. The whole `secrets/` directory is gitignored, so host-specific values stay out of the published history. +The deploy root and the base URL come from a file per environment in `~/.secrets/`, named `Blog...env`, copied from [example.env][env-example] and selected with `ENV_FILE`. `~/.secrets/Blog.local.production.env` is the one read when `ENV_FILE` is unset. The real files live on the host, never in this checkout. ## 3rd Party Tools @@ -221,7 +222,7 @@ Licensed under the [MIT License][license]\ [checks]: ./checks/ [deploy]: ./deploy/ [deploy-readme]: ./deploy/README.md -[env-example]: ./example.env +[env-example]: ./.secrets/example.env [environment]: ./ENVIRONMENT.md [history]: ./HISTORY.md [hugo-config]: ./hugo.yaml diff --git a/TODO.md b/TODO.md index 1a5d53b..7c4607e 100644 --- a/TODO.md +++ b/TODO.md @@ -93,7 +93,7 @@ The reference leaf the hub now ships carries one step this repo's deploy does no ## Open decisions -- **Resolved for the backup pull, 2026-08-08: it is in this repository at [`ops/`](./ops/).** The script, both `systemd` units, an `EnvironmentFile` template naming every path it uses, and a README covering what it does and how to check it. [`OPERATIONS.md`](./OPERATIONS.md) "Working With the VPS" names it and states which of its variables pair with which of this repo's. The reasoning below stands as the record of why, and the same question is still open for everything under it. **Installed 2026-08-08 with `ops/install.sh`**, which derives the address, both destinations, the account, the group and the mount from `secrets/local.production.env`, so nothing is typed twice. Verified after the fact rather than from the installer's own output: `systemd` resolves `User=pieter`, `Group=users` and `RequiresMountsFor=/data/backup` from the drop-in, and the environment file is `600 root:root`. The running script is byte-identical to the committed one. Re-running the installer after the shell-gate reformat also exercised its idempotent path, which reported both config files already correct and replaced only the script, so a changed value is applied by running it again rather than by editing anything on the host. The root guard was exercised and refused. Installing did not spend the evidence for the first scheduled run, which then arrived: **the timer fired unattended on 2026-08-09 at 09:11:01 UTC**, with a second journal entry and `LAST` off `-`. Separately, [#53][issue-53] reconciles the VPS's older copy in both directions rather than by overwriting either. +- **Resolved for the backup pull, 2026-08-08: it is in this repository at [`ops/`](./ops/).** The script, both `systemd` units, an `EnvironmentFile` template naming every path it uses, and a README covering what it does and how to check it. [`OPERATIONS.md`](./OPERATIONS.md) "Working With the VPS" names it and states which of its variables pair with which of this repo's. The reasoning below stands as the record of why, and the same question is still open for everything under it. **Installed 2026-08-08 with `ops/install.sh`**, which derives the address and both destinations from `~/.secrets/Blog.local.production.env`, and the account, the group and the mount from the invoking environment itself, so nothing is typed twice. Verified after the fact rather than from the installer's own output: `systemd` resolves `User=pieter`, `Group=users` and `RequiresMountsFor=/data/backup` from the drop-in, and the environment file is `600 root:root`. The running script is byte-identical to the committed one. Re-running the installer after the shell-gate reformat also exercised its idempotent path, which reported both config files already correct and replaced only the script, so a changed value is applied by running it again rather than by editing anything on the host. The root guard was exercised and refused. Installing did not spend the evidence for the first scheduled run, which then arrived: **the timer fired unattended on 2026-08-09 at 09:11:01 UTC**, with a second journal entry and `LAST` off `-`. Separately, [#53][issue-53] reconciles the VPS's older copy in both directions rather than by overwriting either. - **Where the rest of the operational tooling lives, given that today it lives nowhere.** `vps-backup-pull`, its `systemd` units, and the environment variables naming both ends of the copy were an operational asset built from another agent's instructions, and they existed only on the Proxmox host. That host is the machine the backup runs *from*, so losing it loses both the copies and the means of making them, and the instructions that produced them are in a channel file this repository deliberately does not carry. Two candidate homes, and the choice is open: **here**, beside the deploy tooling the same host runs, or **the home-automation config repository**, with the rest of that host's configuration. The argument for the second is that nothing about the pull is specific to this site; the argument for the first is that [`OPERATIONS.md`](./OPERATIONS.md) "Log Review" is the thing that stops working without it. - **The pull itself is resolved and the reasoning is kept because it applies to everything still listed here.** What made it urgent was measured: the copy protected everywhere was the VPS's older one, while the copy that actually ran, carrying the log leg the review depends on, was in no snapshot and no repository. Committing it is what closed that, not the backup host's own off-site copy, which never reached the script. - **The directory holding it is named as though it were disposable.** `~/vps-backup-pull-patch` reads as a patch staged against a source, and there is no source: it is the most complete copy of the script in existence. A directory named for a temporary artifact is the one a cleanup deletes, and nothing here would notice until a restore produced the wrong script. @@ -200,7 +200,7 @@ Secrets and variables, per environment. The App-token pair is repository-scoped `DEPLOY_SSH_PRIVATE_KEY` holds the same key in both environments, per the decision above. The environment split still carries the base URL, the SSH endpoint, and the staging-only token pair, so it is not decorative. -The deploy root is deliberately absent from this table. The rsync destination is anchored at the deploy key's confinement root, so the workflow names an environment rather than a host path, and a declared-but-unread name is drift no audit can tell from a missing one. The local `DEPLOY_ROOT` in `secrets/..env` is a different value and is still read. +The deploy root is deliberately absent from this table. The rsync destination is anchored at the deploy key's confinement root, so the workflow names an environment rather than a host path, and a declared-but-unread name is drift no audit can tell from a missing one. The local `DEPLOY_ROOT` in `~/.secrets/Blog...env` is a different value and is still read. diff --git a/capture/README.md b/capture/README.md index 76e5e8a..d9b6624 100644 --- a/capture/README.md +++ b/capture/README.md @@ -2,7 +2,7 @@ Everything in this repository is derived. What it was derived *from* is a capture directory that lives outside it, and this directory holds the scripts that read that capture. -The capture's path is `CAPTURE_ROOT`, recorded alongside the other values that name a machine rather than the project. It is environment-independent, so unlike a deploy root it belongs in `secrets/local.production.env` alone rather than in a copy per environment: there is one capture, and four copies of its path is four chances for three of them to be wrong. It is not a git repository, so it has no history to revert to, and it is read-only in normal use. Nothing here writes into it except the steps below that say they do. +The capture's path is `CAPTURE_ROOT`, recorded alongside the other values that name a machine rather than the project. It is environment-independent, so unlike a deploy root it belongs in `~/.secrets/Blog.local.production.env` alone rather than in a copy per environment: there is one capture, and four copies of its path is four chances for three of them to be wrong. It is not a git repository, so it has no history to revert to, and it is read-only in normal use. Nothing here writes into it except the steps below that say they do. **None of this runs in CI, and none of it runs on a schedule.** These are provenance tools, run by hand, and their outputs are committed. That is the whole difference between this directory and [`checks/`](../checks/), which holds gates that run on every change. @@ -46,7 +46,7 @@ In order. Each Python step is a dry run by default and takes `--apply` to write. Everything here is standard library except `clean-content.py`, which needs PyYAML to read front matter. It names the package if it is missing rather than raising an import error. ```sh -set -a; . secrets/local.production.env; set +a +set -a; . ~/.secrets/Blog.local.production.env; set +a capture/run-wp2hugo.sh # convert, into $CAPTURE_ROOT/converted/ capture/clean-content.py --apply # drop comments, reduce the front matter @@ -111,11 +111,11 @@ Seven scripts stayed behind, and each is named here so nobody goes looking for s ## Variables -Every path and address is a variable, so nothing here names a machine. [`example.env`](../example.env) lists them and [`ENVIRONMENT.md`](../ENVIRONMENT.md) describes them. +Every path and address is a variable, so nothing here names a machine. [`example.env`](../.secrets/example.env) lists them and [`ENVIRONMENT.md`](../ENVIRONMENT.md) describes them. A required value that is unset stops the script and names what is missing, rather than falling back to something plausible. A wrong-but-valid capture directory produces empty maps that are indistinguishable from working ones until the redirects are live, which is the failure this rule exists for. -`CAPTURE_SOURCE_URL` is the **old** platform. It holds the same string as `HUGO_BASEURL` after the cutover and means something different, so merging the two would point a verification run at the new site while every check still passed. +`CAPTURE_SOURCE_URL` is the **old** platform. It holds the same string as `SITE_BASE_URL` after the cutover and means something different, so merging the two would point a verification run at the new site while every check still passed. ## This directory is the source diff --git a/capture/build-golden.py b/capture/build-golden.py index 8cd0045..2f2b5cb 100755 --- a/capture/build-golden.py +++ b/capture/build-golden.py @@ -37,7 +37,7 @@ def env(name: str) -> str: """ v = os.environ.get(name, "") if not v: - sys.exit(f"{name} is not set -- see example.env and ENVIRONMENT.md") + sys.exit(f"{name} is not set -- see .secrets/example.env and ENVIRONMENT.md") return v diff --git a/capture/build-redirects.py b/capture/build-redirects.py index fa7eb25..a7d76d9 100755 --- a/capture/build-redirects.py +++ b/capture/build-redirects.py @@ -91,7 +91,7 @@ def main(argv): if not root: print(f"usage: {argv[0]} [--print-export] [capture-dir]", file=sys.stderr) print("CAPTURE_ROOT is not set and no capture directory was given", file=sys.stderr) - print("see example.env and ENVIRONMENT.md", file=sys.stderr) + print("see .secrets/example.env and ENVIRONMENT.md", file=sys.stderr) return 2 capture = pathlib.Path(root) out = REPO / "deploy" / "maps" diff --git a/capture/classify.py b/capture/classify.py index 0c52eba..c9b22cf 100755 --- a/capture/classify.py +++ b/capture/classify.py @@ -26,7 +26,7 @@ def env(name: str) -> str: """ v = os.environ.get(name, "") if not v: - sys.exit(f"{name} is not set -- see example.env and ENVIRONMENT.md") + sys.exit(f"{name} is not set -- see .secrets/example.env and ENVIRONMENT.md") return v diff --git a/capture/clean-content.py b/capture/clean-content.py index a20205d..29337a0 100755 --- a/capture/clean-content.py +++ b/capture/clean-content.py @@ -55,7 +55,7 @@ def capture_root() -> pathlib.Path: """ root = os.environ.get("CAPTURE_ROOT", "") if not root: - sys.exit("CAPTURE_ROOT is not set -- see example.env and ENVIRONMENT.md") + sys.exit("CAPTURE_ROOT is not set -- see .secrets/example.env and ENVIRONMENT.md") return pathlib.Path(root) diff --git a/capture/enumerate-media.py b/capture/enumerate-media.py index f62adbe..3551f3d 100755 --- a/capture/enumerate-media.py +++ b/capture/enumerate-media.py @@ -32,7 +32,7 @@ def env(name: str) -> str: """ v = os.environ.get(name, "") if not v: - sys.exit(f"{name} is not set -- see example.env and ENVIRONMENT.md") + sys.exit(f"{name} is not set -- see .secrets/example.env and ENVIRONMENT.md") return v diff --git a/capture/localize-external.py b/capture/localize-external.py index bfc8f12..e1db034 100755 --- a/capture/localize-external.py +++ b/capture/localize-external.py @@ -55,7 +55,7 @@ def capture_root() -> pathlib.Path: """ root = os.environ.get("CAPTURE_ROOT", "") if not root: - sys.exit("CAPTURE_ROOT is not set -- see example.env and ENVIRONMENT.md") + sys.exit("CAPTURE_ROOT is not set -- see .secrets/example.env and ENVIRONMENT.md") return pathlib.Path(root) diff --git a/capture/restructure-content.py b/capture/restructure-content.py index 0a44c27..51d6d72 100755 --- a/capture/restructure-content.py +++ b/capture/restructure-content.py @@ -39,7 +39,7 @@ def capture_root() -> pathlib.Path: """ root = os.environ.get("CAPTURE_ROOT", "") if not root: - sys.exit("CAPTURE_ROOT is not set -- see example.env and ENVIRONMENT.md") + sys.exit("CAPTURE_ROOT is not set -- see .secrets/example.env and ENVIRONMENT.md") return pathlib.Path(root) diff --git a/capture/run-wp2hugo.sh b/capture/run-wp2hugo.sh index 175c4b6..436b7f4 100755 --- a/capture/run-wp2hugo.sh +++ b/capture/run-wp2hugo.sh @@ -15,7 +15,7 @@ set -Eeuo pipefail # A `go install`ed wp2hugo lands here. $HOME rather than a literal path. export PATH="$HOME/.local/bin:$PATH" -: "${CAPTURE_ROOT:?CAPTURE_ROOT is not set -- see example.env and ENVIRONMENT.md}" +: "${CAPTURE_ROOT:?CAPTURE_ROOT is not set -- see .secrets/example.env and ENVIRONMENT.md}" [ -d "$CAPTURE_ROOT" ] || { echo "CAPTURE_ROOT is not a directory: $CAPTURE_ROOT" >&2 exit 1 diff --git a/checks/check-env-docs.py b/checks/check-env-docs.py index 4fae408..4bdd9b3 100755 --- a/checks/check-env-docs.py +++ b/checks/check-env-docs.py @@ -2,7 +2,7 @@ """Fail if a configuration value is declared without a description in ENVIRONMENT.md, or described there and declared nowhere. -The declared surface is three things, and nothing else: the keys in example.env, the +The declared surface is three things, and nothing else: the keys in .secrets/example.env, the vars.X and secrets.X a workflow references, and the KNOBS list below. A variable a script merely reads is NOT in scope, because a script-local name and a configuration value are the same shape and no pattern separates them, so widening this would report the difference as @@ -31,7 +31,7 @@ DOC = REPO / "ENVIRONMENT.md" # The one template, whose keys are the declared configuration surface. -TEMPLATES = [REPO / "example.env"] +TEMPLATES = [REPO / ".secrets" / "example.env"] # Workflow references. `vars.X` and `secrets.X` are the GitHub Environment surface, and a # value added there is exactly as undocumented as one added to a template. diff --git a/checks/check-live-urls.sh b/checks/check-live-urls.sh index feb66b3..925cf99 100755 --- a/checks/check-live-urls.sh +++ b/checks/check-live-urls.sh @@ -115,7 +115,7 @@ echo "==> tagging requests X-Blog-Check: $CHECK_TAG" # A resource access token opens the proxy's auth gate. # It goes into a curl config file because bash cannot export an array to the parallel checks. # A command line is also world-readable in ps output, and every request would carry it. -if [ -n "${PANGOLIN_ACCESS_TOKEN_ID:-}" ] && [ -n "${PANGOLIN_ACCESS_TOKEN:-}" ]; then +if [ -n "${SITE_AUTH_TOKEN_ID:-}" ] && [ -n "${SITE_AUTH_TOKEN:-}" ]; then # Same hazard as CHECK_TAG above and the same reason, but a narrower rule, because the # grammar of a credential is the issuer's to define and not this script's. Only the # characters that break out of a quoted config line are refused, and none is legal in an @@ -125,7 +125,7 @@ if [ -n "${PANGOLIN_ACCESS_TOKEN_ID:-}" ] && [ -n "${PANGOLIN_ACCESS_TOKEN:-}" ] # Carriage return counts as a line ending here as much as newline does. Header injection # is classically CRLF, and a lone CR is enough on its own, so refusing LF while allowing # CR would leave the shape this guard exists for. - for name in PANGOLIN_ACCESS_TOKEN_ID PANGOLIN_ACCESS_TOKEN; do + for name in SITE_AUTH_TOKEN_ID SITE_AUTH_TOKEN; do case "${!name}" in *'"'* | *$'\n'* | *$'\r'*) echo "FAIL $name contains a quote, a newline, or a carriage return, none of which can appear in an HTTP header value" >&2 @@ -136,11 +136,11 @@ if [ -n "${PANGOLIN_ACCESS_TOKEN_ID:-}" ] && [ -n "${PANGOLIN_ACCESS_TOKEN:-}" ] CURLRC="$(mktemp)" chmod 600 "$CURLRC" printf 'header = "P-Access-Token-Id: %s"\nheader = "P-Access-Token: %s"\n' \ - "$PANGOLIN_ACCESS_TOKEN_ID" "$PANGOLIN_ACCESS_TOKEN" >"$CURLRC" + "$SITE_AUTH_TOKEN_ID" "$SITE_AUTH_TOKEN" >"$CURLRC" echo "==> sending a Pangolin access token" -elif [ -n "${PANGOLIN_ACCESS_TOKEN_ID:-}" ] || [ -n "${PANGOLIN_ACCESS_TOKEN:-}" ]; then +elif [ -n "${SITE_AUTH_TOKEN_ID:-}" ] || [ -n "${SITE_AUTH_TOKEN:-}" ]; then # Half a credential is a typo rather than a choice, and it would otherwise fail as an outage. - echo "FAIL set both PANGOLIN_ACCESS_TOKEN_ID and PANGOLIN_ACCESS_TOKEN, or neither" >&2 + echo "FAIL set both SITE_AUTH_TOKEN_ID and SITE_AUTH_TOKEN, or neither" >&2 exit 2 fi @@ -295,7 +295,7 @@ if [ "$preflight" != "200" ]; then echo " a token was sent, so check the pair is valid for this resource" >&2 else echo " no token was sent. If this site is behind the auth gate, set" >&2 - echo " PANGOLIN_ACCESS_TOKEN_ID and PANGOLIN_ACCESS_TOKEN" >&2 + echo " SITE_AUTH_TOKEN_ID and SITE_AUTH_TOKEN" >&2 fi exit 1 fi diff --git a/deploy/README.md b/deploy/README.md index 89efabe..610439e 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -53,39 +53,41 @@ neither. Seven facts are the whole contract: ## Building a release ```sh +set -a; . ~/.secrets/Blog.local.production.env; set +a deploy/make-release.sh -checks/check-live-urls.sh "$HUGO_BASEURL" +checks/check-live-urls.sh "$SITE_BASE_URL" ``` The deploy root and the base URL are the only host-specific values, and they pair per -environment. Copy [`example.env`](../example.env) to `secrets/local.production.env`, which is the -file read when `ENV_FILE` is unset, and add `secrets/..env` for each further -environment. A single environment therefore needs `secrets/local.production.env` and nothing -else, since a differently named file is read only when `ENV_FILE` names it. `secrets/` is -gitignored as a whole directory, so a value naming one machine cannot reach a public repo by -being added to a file nobody remembered to ignore. CI passes them explicitly instead, which +environment. Copy [`example.env`](../.secrets/example.env) to `~/.secrets/Blog.local.production.env`, which is the +file read when `ENV_FILE` is unset, and add `~/.secrets/Blog...env` for each further +environment. A single environment therefore needs `~/.secrets/Blog.local.production.env` and nothing +else, since a differently named file is read only when `ENV_FILE` names it. The real files +live on the host, in `~/.secrets/`, never in this checkout. CI passes them explicitly instead, which keeps a pipeline run self-describing: ```sh -HUGO_BASEURL= deploy/make-release.sh "$(git rev-parse --short HEAD)" +SITE_BASE_URL= deploy/make-release.sh "$(git rev-parse --short HEAD)" ``` -**One file per environment, named `secrets/..env`, selected by `ENV_FILE`.** +**That command-prefix form is CI-only.** A local run whose default environment file exists sources it after the command-prefix assignment and overwrites it, since `set -a` overwrites a value the caller exported first. Locally, select the environment through `ENV_FILE` instead, per the table below. + +**One file per environment, named `~/.secrets/Blog...env`, selected by `ENV_FILE`.** Both halves are spelled out, so a name says which machine it describes as well as which -environment on it, and the four in the fleet read as one set. `secrets/local.production.env` is +environment on it, and the four in the fleet read as one set. `~/.secrets/Blog.local.production.env` is the default and is read when `ENV_FILE` is unset, so a single-environment host needs nothing else: ```sh -deploy/make-release.sh # secrets/local.production.env -ENV_FILE=secrets/local.staging.env deploy/make-release.sh # the staging site on the same host +deploy/make-release.sh # ~/.secrets/Blog.local.production.env +ENV_FILE=~/.secrets/Blog.local.staging.env deploy/make-release.sh # the staging site on the same host ``` A file whose `DEPLOY_SSH_HOST` is set describes a root on another machine, so this script refuses to create that path here and asks for a local one to assemble a bundle into: ```sh -ENV_FILE=secrets/vps.staging.env deploy/make-release.sh /path/to/bundle +ENV_FILE=~/.secrets/Blog.vps.staging.env deploy/make-release.sh /path/to/bundle ``` Selecting the file is the only way to switch environments. The file is sourced with `set -a`, @@ -95,19 +97,20 @@ still wins, because it is read after the file. A named file that does not exist failure rather than a fall-through to the ambient environment, since on a host running two sites the ambient value is the other site's root. -**Always set `HUGO_BASEURL` for anything that is not production.** The base URL is baked into +**Always set `SITE_BASE_URL` for anything that is not production.** The base URL is baked into the canonical tag, the feed links, and every absolute permalink, so a mirror built without it serves pages that all point back at production. Nothing downstream catches this, because the -pages render at the right paths and the parity gate passes. The effective value is printed on -every build for that reason. +pages render at the right paths and the parity gate passes. `make-release.sh` bridges it to +Hugo's own `HUGO_BASEURL` internally, and the effective value is printed on every build for +that reason. ### Environment variables | Variable | Effect | | --- | --- | -| `ENV_FILE` | Which environment file to source. Defaults to `secrets/local.production.env`. | +| `ENV_FILE` | Which environment file to source. Defaults to `~/.secrets/Blog.local.production.env`. | | `DEPLOY_ROOT` | Fallback deploy root. The first argument wins. | -| `HUGO_BASEURL` | Overrides the site base URL. Hugo maps `HUGO_` onto config natively. | +| `SITE_BASE_URL` | Overrides the site base URL. Bridged internally to `HUGO_BASEURL`, the name Hugo maps `HUGO_` onto config natively. | | `REQUIRE_BROTLI=1` | Fails rather than shipping gzip-only. CI sets this. | | `NO_LINK_DEST=1` | Full copy instead of hard-linking from the previous release. | @@ -116,8 +119,8 @@ open the auth gate: | Variable | Effect | | --- | --- | -| `PANGOLIN_ACCESS_TOKEN_ID` | Resource access token id, sent as the `P-Access-Token-Id` header. | -| `PANGOLIN_ACCESS_TOKEN` | The token itself, sent as `P-Access-Token`. | +| `SITE_AUTH_TOKEN_ID` | Resource access token id, sent as the `P-Access-Token-Id` header. | +| `SITE_AUTH_TOKEN` | The token itself, sent as `P-Access-Token`. | Set both or neither; half a pair is rejected as the typo it is. They go to curl through a mode-`600` config file rather than as `-H` arguments, which keeps the credential out of the diff --git a/deploy/make-release.sh b/deploy/make-release.sh index 45f6b8a..f8ffc60 100755 --- a/deploy/make-release.sh +++ b/deploy/make-release.sh @@ -12,7 +12,7 @@ KEEP_RELEASES=10 usage() { echo "usage: $0 [deploy-root] [version]" >&2 echo " deploy-root defaults to DEPLOY_ROOT, from the environment or \$ENV_FILE" >&2 - echo " ENV_FILE defaults to secrets/local.production.env, and a relative path resolves against the repo" >&2 + echo " ENV_FILE defaults to ~/.secrets/Blog.local.production.env, and a relative path resolves against ~/.secrets" >&2 echo " deploy-root is required when the environment file sets DEPLOY_SSH_HOST, since that root is on another host" >&2 exit 2 } @@ -26,19 +26,19 @@ MTIME_MIN=2025.08 # The deploy root and the base URL are the only host-specific values, and they pair per environment. # ENV_FILE selects the environment, because `set -a` overwrites a value the caller exported. # The first argument overrides the root, being read after this. -# Files are named secrets/..env, both words spelled out, so the default names -# the environment it actually selects rather than being the one file whose name says nothing. -DEFAULT_ENV_FILE="$REPO/secrets/local.production.env" +# Files are named ~/.secrets/Blog...env, both words spelled out, so the default names the environment it actually selects rather than being the one file whose name says nothing. +# The leading Blog segment disambiguates this repo's files in ~/.secrets, since other repos on the same host also write there. +DEFAULT_ENV_FILE="$HOME/.secrets/Blog.local.production.env" ENV_FILE="${ENV_FILE:-$DEFAULT_ENV_FILE}" -# A relative name resolves against the repo, so it means the same from any working directory. -# Traversal is refused rather than resolved, since a relative name is meant to reach secrets/. +# A relative name resolves against ~/.secrets, so it means the same from any working directory. +# Traversal is refused rather than resolved, since a relative name is meant to reach ~/.secrets. case "$ENV_FILE" in /*) ;; *..*) echo "ENV_FILE must not traverse: $ENV_FILE" >&2 exit 1 ;; -*) ENV_FILE="$REPO/$ENV_FILE" ;; +*) ENV_FILE="$HOME/.secrets/$ENV_FILE" ;; esac if [ -f "$ENV_FILE" ]; then echo "==> environment: $ENV_FILE" @@ -53,6 +53,15 @@ elif [ "$ENV_FILE" != "$DEFAULT_ENV_FILE" ]; then exit 1 fi +# Hugo maps HUGO_ onto config natively, and only that name, so this is the one place SITE_BASE_URL becomes the name Hugo actually reads. +# CI reads no env file here, it already has SITE_BASE_URL in the process environment, so this bridge covers both callers alike. +# The unset branch matters too: an inherited HUGO_BASEURL left over from an older shell export would otherwise survive an unset SITE_BASE_URL and quietly win, since Hugo gives the environment variable precedence over hugo.yaml. +if [ -n "${SITE_BASE_URL:-}" ]; then + export HUGO_BASEURL="$SITE_BASE_URL" +else + unset HUGO_BASEURL +fi + # This script installs to a local path, so a remote environment's DEPLOY_ROOT would be built here. # The guard is on the fallback rather than the variable. # An explicit first argument names a local path and is always honoured, which is what CI passes. diff --git a/ops/README.md b/ops/README.md index c6d2914..f192e23 100644 --- a/ops/README.md +++ b/ops/README.md @@ -9,7 +9,7 @@ The pull that copies the VPS's backup set and its access logs off the VPS. It ru | `vps-backup-pull.service` | `/etc/systemd/system/` | | `vps-backup-pull.timer` | `/etc/systemd/system/` | | `vps-backup-pull.service.d-local.conf.example` | `/etc/systemd/system/vps-backup-pull.service.d/local.conf` | -| [`example.env`](../example.env) | `/etc/vps-backup-pull.env` | +| [`example.env`](../.secrets/example.env) | `/etc/vps-backup-pull.env` | **The last two are required, not optional, and `install.sh` generates both.** Nothing in this directory names a machine, so the address, the destination paths, and the account are supplied at install time from values this repository already holds. A missing value stops the pull with the name of what is missing rather than falling back to something plausible, since a wrong-but-valid destination is a backup nobody can find. The two `.example` files document the format and are not the install path. @@ -42,7 +42,7 @@ ops/install.sh --check # derive, validate, print, write nothing ops/install.sh # the same, then install ``` -**Nothing is typed twice.** The address, both destinations, and the account are already known to this checkout, so `install.sh` copies them rather than asking: `VPS_SSH_HOST`, `BACKUP_ARCHIVE_ROOT` and `LOG_ARCHIVE_ROOT` come straight from `secrets/..env`, the account is whoever runs the script, the group is read from the destination, and the mount is resolved with `findmnt`. +**Nothing is typed twice.** The address, both destinations, and the account are already known to this checkout, so `install.sh` copies them rather than asking: `VPS_SSH_HOST`, `BACKUP_ARCHIVE_ROOT` and `LOG_ARCHIVE_ROOT` come straight from `~/.secrets/Blog...env`, the account is whoever runs the script, the group is read from the destination, and the mount is resolved with `findmnt`. **Two derivations are worth knowing, because the obvious answer is wrong for both.** The group comes from the destination rather than from `id -gn`, since `Group=` sets the process's primary group and the account's own group is usually not the one owning the backup tree. And `RequiresMountsFor=` needs the mount point rather than the destination path below it. @@ -65,9 +65,9 @@ systemctl list-timers vps-backup-pull.timer --all ## Variables -Every path is a variable, so a host states its own layout rather than editing a file git owns. [`example.env`](../example.env) lists them and [`ENVIRONMENT.md`](../ENVIRONMENT.md) describes them. The three `systemd` settings that cannot come from an environment file are in [`vps-backup-pull.service.d-local.conf.example`](./vps-backup-pull.service.d-local.conf.example). +Every path is a variable, so a host states its own layout rather than editing a file git owns. [`example.env`](../.secrets/example.env) lists them and [`ENVIRONMENT.md`](../ENVIRONMENT.md) describes them. The three `systemd` settings that cannot come from an environment file are in [`vps-backup-pull.service.d-local.conf.example`](./vps-backup-pull.service.d-local.conf.example). -**They are the same names Blog's own `secrets/` file uses, which is the point.** `VPS_*` is something on the VPS and `*_ROOT` is something on this host, and the pull writes the two roots that the log review reads. One name per directory means the writing side and the reading side cannot disagree, and it is why `install.sh` copies rather than translates. Every value is described in [`ENVIRONMENT.md`](../ENVIRONMENT.md). +**They are the same names Blog's own `~/.secrets/` file uses, which is the point.** `VPS_*` is something on the VPS and `*_ROOT` is something on this host, and the pull writes the two roots that the log review reads. One name per directory means the writing side and the reading side cannot disagree, and it is why `install.sh` copies rather than translates. Every value is described in [`ENVIRONMENT.md`](../ENVIRONMENT.md). ## This directory is the source diff --git a/ops/install.sh b/ops/install.sh index f05f365..22baf0e 100755 --- a/ops/install.sh +++ b/ops/install.sh @@ -2,10 +2,8 @@ # Install the backup pull on this host, deriving every host-specific value from the # environment file this repository already keeps. # -# The pull needs four things this repository deliberately does not carry: an address, two -# destination paths, and the account to run as. The first three are already in -# secrets/..env under the same names the pull itself uses, so this -# copies them rather than translating them, and the account is whoever runs this. +# The pull needs four things this repository deliberately does not carry: an address, two destination paths, and the account to run as. +# The first three are already in ~/.secrets/Blog...env under the same names the pull itself uses, so this copies them rather than translating them, and the account is whoever runs this. # # There is no name mapping here, because both sides spell every shared value the same way. # Keep it that way: a translation table is a thing to get wrong every time one side changes. @@ -49,15 +47,13 @@ die() { note() { printf ' %s\n' "$*"; } REPO=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) -ENV_FILE=${ENV_FILE:-$REPO/secrets/local.production.env} -# Same rule deploy/make-release.sh applies, because the two read the same files and a name -# that means different things depending on where you stood is worse here: this one installs. -# A relative name resolves against the repo, so it means the same from any directory, and -# traversal is refused rather than resolved, since a relative name is meant to reach secrets/. +ENV_FILE=${ENV_FILE:-$HOME/.secrets/Blog.local.production.env} +# The same rule deploy/make-release.sh applies, because the two read the same files and a name that means different things depending on where you stood is worse here: this one installs. +# A relative name resolves against ~/.secrets, so it means the same from any directory, and traversal is refused rather than resolved, since a relative name is meant to reach ~/.secrets. case "$ENV_FILE" in /*) ;; *..*) die "ENV_FILE must not traverse: $ENV_FILE" ;; -*) ENV_FILE="$REPO/$ENV_FILE" ;; +*) ENV_FILE="$HOME/.secrets/$ENV_FILE" ;; esac [[ $EUID -ne 0 ]] || die "do not run this under sudo -- run it as the account that will own the backup; it calls sudo for the steps that need it" @@ -95,7 +91,7 @@ MOUNT=$(findmnt -no TARGET --target "$ANCESTOR" 2>/dev/null) || VPS_TRAEFIK_ARCHIVE=${VPS_TRAEFIK_LOG_ARCHIVE:-/var/log/traefik/archive} -printf '=== derived from %s\n' "${ENV_FILE#"$REPO"/}" +printf '=== derived from %s\n' "${ENV_FILE/#"$HOME"/\~}" note "VPS_SSH_HOST $VPS_SSH_HOST" note "BACKUP_ARCHIVE_ROOT $BACKUP_ARCHIVE_ROOT" note "LOG_ARCHIVE_ROOT $LOG_ARCHIVE_ROOT" @@ -109,7 +105,7 @@ DROPIN_DEST=$DROPIN_DIR/local.conf ENV_BODY=$( cat < Date: Mon, 24 Aug 2026 08:32:34 -0700 Subject: [PATCH 2/3] Address Review Findings from the Develop-to-Main Promotion PR (#109) * Address review findings from the develop-to-main promotion PR .secrets/README.md now scopes the 'every real value comes from ~/.secrets/' claim to local invocations, since CI supplies the same values directly from the GitHub Environment. OPERATIONS.md, README.md, and deploy/README.md now name ENV_FILE explicitly alongside sourcing it, since make-release.sh sources it independently and a value already exported earlier in the same shell session would otherwise win silently. OPERATIONS.md's Deploying section now reuses one release id across both the release and the live check, so EXPECT_RELEASE actually verifies the release that command just built rather than skipping the release-stamp guard entirely. ops/install.sh's HOME-to-~ abbreviation now requires a directory boundary after the prefix, so a sibling directory sharing the same prefix (/home/alice2 under HOME=/home/alice) is no longer misread as a subdirectory of HOME. * Fix a scope claim and a bash-version-sensitive display escape .secrets/README.md now notes that an absolute ENV_FILE is honored as given, an escape hatch outside the documented ~/.secrets/ convention, rather than claiming every local invocation is confined to it. ops/install.sh no longer relies on the \~ parameter-expansion escape for its HOME-to-~ display, since bash versions differ on whether that yields a literal backslash. A case statement builds the same display value without it. --- .secrets/README.md | 9 ++++++--- OPERATIONS.md | 9 +++++---- README.md | 2 +- deploy/README.md | 2 +- ops/install.sh | 13 +++++++++++-- 5 files changed, 24 insertions(+), 11 deletions(-) diff --git a/.secrets/README.md b/.secrets/README.md index 642bbd5..e97323e 100644 --- a/.secrets/README.md +++ b/.secrets/README.md @@ -7,11 +7,14 @@ exposing one. ## Real values live on the host, never in the checkout -Every real value this repo's scripts read comes from `~/.secrets/`, not from this directory. +The documented local convention is `~/.secrets/`, not this directory. CI supplies the same +values directly from the GitHub Environment instead, reading no file here at all. `ENV_FILE= deploy/make-release.sh` and `ops/install.sh` both resolve a relative `ENV_FILE` against `$HOME/.secrets`, refuse a traversing one, and default to -`~/.secrets/Blog.local.production.env`. `~/.secrets/` is shared across every repo on the host, -so each of this repo's files carries the `Blog.` prefix: +`~/.secrets/Blog.local.production.env`. An absolute `ENV_FILE` is honored as given rather than +resolved against `~/.secrets/`, an escape hatch rather than the documented shape. +`~/.secrets/` is shared across every repo on the host, so each of this repo's files carries the +`Blog.` prefix: | File | Selects | | --- | --- | diff --git a/OPERATIONS.md b/OPERATIONS.md index 9379f3a..0a839aa 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -77,11 +77,11 @@ So release to the local mirror and run the live check **before** opening a pull ```sh set -a; . ~/.secrets/Blog.local.production.env; set +a -deploy/make-release.sh +ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh checks/check-live-urls.sh "$SITE_BASE_URL" ``` -Against the staging mirror, name its file in both places, since the sourced values and the ones `make-release.sh` reads must describe the same environment: +Name the file in both places, even when it is the default, since `make-release.sh` sources `ENV_FILE` independently of the shell above and a value already exported earlier in the same session would otherwise win silently over the sourced one: ```sh set -a; . ~/.secrets/Blog.local.staging.env; set +a @@ -142,8 +142,9 @@ Three properties of how the credential is handled, each there for a reason worth ## Deploying ```sh -SITE_BASE_URL= deploy/make-release.sh "$(git rev-parse --short HEAD)" -checks/check-live-urls.sh +RELEASE="$(git rev-parse --short HEAD)" +SITE_BASE_URL= deploy/make-release.sh "$RELEASE" +EXPECT_RELEASE="$RELEASE" checks/check-live-urls.sh ``` The deploy root and the base URL are the only host-specific values. A local run reads them from a file under `~/.secrets/`, one per environment, copied from [`.secrets/example.env`](./.secrets/example.env), and CI passes both explicitly. The real files live on the host, never in this checkout. diff --git a/README.md b/README.md index 3a03b33..6996220 100644 --- a/README.md +++ b/README.md @@ -179,7 +179,7 @@ Build a release and verify it against a running server: ```sh set -a; . ~/.secrets/Blog.local.production.env; set +a -deploy/make-release.sh +ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh checks/check-live-urls.sh "$SITE_BASE_URL" ``` diff --git a/deploy/README.md b/deploy/README.md index 610439e..6c36bad 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -54,7 +54,7 @@ neither. Seven facts are the whole contract: ```sh set -a; . ~/.secrets/Blog.local.production.env; set +a -deploy/make-release.sh +ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh checks/check-live-urls.sh "$SITE_BASE_URL" ``` diff --git a/ops/install.sh b/ops/install.sh index 22baf0e..f5fb64a 100755 --- a/ops/install.sh +++ b/ops/install.sh @@ -91,7 +91,16 @@ MOUNT=$(findmnt -no TARGET --target "$ANCESTOR" 2>/dev/null) || VPS_TRAEFIK_ARCHIVE=${VPS_TRAEFIK_LOG_ARCHIVE:-/var/log/traefik/archive} -printf '=== derived from %s\n' "${ENV_FILE/#"$HOME"/\~}" +# A display value only, built without the \~ parameter-expansion escape, since bash versions differ on whether that yields a literal backslash, and this is what an operator reads to trust the source file. +ENV_FILE_DISPLAY=$ENV_FILE +case "$ENV_FILE_DISPLAY" in +"$HOME"/*) + # shellcheck disable=SC2088 # Literal display text, deliberately not expanded. + ENV_FILE_DISPLAY="~/${ENV_FILE_DISPLAY#"$HOME"/}" + ;; +esac + +printf '=== derived from %s\n' "$ENV_FILE_DISPLAY" note "VPS_SSH_HOST $VPS_SSH_HOST" note "BACKUP_ARCHIVE_ROOT $BACKUP_ARCHIVE_ROOT" note "LOG_ARCHIVE_ROOT $LOG_ARCHIVE_ROOT" @@ -105,7 +114,7 @@ DROPIN_DEST=$DROPIN_DIR/local.conf ENV_BODY=$( cat < Date: Mon, 24 Aug 2026 09:09:06 -0700 Subject: [PATCH 3/3] Reuse One Release ID in Every Local Verification Example (#110) * Reuse one release id in every local verification example Each of the three build-and-verify snippets in OPERATIONS.md, README.md, and deploy/README.md omitted EXPECT_RELEASE, so the live check never verified the running Caddy rules belonged to the release the command just built. Each now captures one RELEASE value and passes it to both commands. * Fail closed in every release snippet's git rev-parse A failed git rev-parse left RELEASE empty in every snippet, which the builder covers with its own timestamp fallback while check-live-urls.sh silently skips its release-stamp verification on an empty EXPECT_RELEASE. set -e now stops each snippet at that failure instead. --- OPERATIONS.md | 15 ++++++++++----- README.md | 6 ++++-- deploy/README.md | 6 ++++-- 3 files changed, 18 insertions(+), 9 deletions(-) diff --git a/OPERATIONS.md b/OPERATIONS.md index 0a839aa..f59fa7e 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -76,17 +76,21 @@ So release to the local mirror and run the live check **before** opening a pull | `hugo.yaml`, `layouts/` | Permalink and taxonomy changes move URLs underneath the redirects that point at them. | ```sh +set -e set -a; . ~/.secrets/Blog.local.production.env; set +a -ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh -checks/check-live-urls.sh "$SITE_BASE_URL" +RELEASE="$(git rev-parse --short HEAD)" +ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh "" "$RELEASE" +EXPECT_RELEASE="$RELEASE" checks/check-live-urls.sh "$SITE_BASE_URL" ``` -Name the file in both places, even when it is the default, since `make-release.sh` sources `ENV_FILE` independently of the shell above and a value already exported earlier in the same session would otherwise win silently over the sourced one: +Name the file in both places, even when it is the default, since `make-release.sh` sources `ENV_FILE` independently of the shell above and a value already exported earlier in the same session would otherwise win silently over the sourced one. The empty first argument leaves the deploy root at the sourced `DEPLOY_ROOT`, and `RELEASE` is reused so `EXPECT_RELEASE` verifies the release the command just built rather than skipping the release-stamp guard. `set -e` matters here too: a failed `git rev-parse` would otherwise leave `RELEASE` empty, which silently skips the check's own release-stamp verification instead of failing loud: ```sh +set -e set -a; . ~/.secrets/Blog.local.staging.env; set +a -ENV_FILE=~/.secrets/Blog.local.staging.env deploy/make-release.sh -checks/check-live-urls.sh "$SITE_BASE_URL" +RELEASE="$(git rev-parse --short HEAD)" +ENV_FILE=~/.secrets/Blog.local.staging.env deploy/make-release.sh "" "$RELEASE" +EXPECT_RELEASE="$RELEASE" checks/check-live-urls.sh "$SITE_BASE_URL" ``` **There is no restart step, and that depends on one flag.** The container runs `caddy run --watch`, which re-adapts the config on a timer and reloads it in process. Re-adapting re-executes every `import`, so a new release's `Caddyfile` and `maps/*.map` are picked up through the unchanged `/config/Caddyfile` that the watcher actually names. Measured on this host: content is live the instant the symlink moves, and the rules follow within about a quarter of a second. @@ -142,6 +146,7 @@ Three properties of how the credential is handled, each there for a reason worth ## Deploying ```sh +set -e RELEASE="$(git rev-parse --short HEAD)" SITE_BASE_URL= deploy/make-release.sh "$RELEASE" EXPECT_RELEASE="$RELEASE" checks/check-live-urls.sh diff --git a/README.md b/README.md index 6996220..b36127e 100644 --- a/README.md +++ b/README.md @@ -178,9 +178,11 @@ checks/check-url-parity.py public Build a release and verify it against a running server: ```sh +set -e set -a; . ~/.secrets/Blog.local.production.env; set +a -ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh -checks/check-live-urls.sh "$SITE_BASE_URL" +RELEASE="$(git rev-parse --short HEAD)" +ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh "" "$RELEASE" +EXPECT_RELEASE="$RELEASE" checks/check-live-urls.sh "$SITE_BASE_URL" ``` The deploy root and the base URL come from a file per environment in `~/.secrets/`, named `Blog...env`, copied from [example.env][env-example] and selected with `ENV_FILE`. `~/.secrets/Blog.local.production.env` is the one read when `ENV_FILE` is unset. The real files live on the host, never in this checkout. diff --git a/deploy/README.md b/deploy/README.md index 6c36bad..63411ac 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -53,9 +53,11 @@ neither. Seven facts are the whole contract: ## Building a release ```sh +set -e set -a; . ~/.secrets/Blog.local.production.env; set +a -ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh -checks/check-live-urls.sh "$SITE_BASE_URL" +RELEASE="$(git rev-parse --short HEAD)" +ENV_FILE=~/.secrets/Blog.local.production.env deploy/make-release.sh "" "$RELEASE" +EXPECT_RELEASE="$RELEASE" checks/check-live-urls.sh "$SITE_BASE_URL" ``` The deploy root and the base URL are the only host-specific values, and they pair per