diff --git a/.gitignore b/.gitignore index 6872d64..8c5d34f 100644 --- a/.gitignore +++ b/.gitignore @@ -31,9 +31,12 @@ __pycache__/ # Each names one particular machine rather than the project. # The whole directory is ignored so a value added later lands ignored by default. # `deploy/env.example` is the committed template and sits outside the directory. +# The last pattern is the backstop for one written outside the directory, matching the +# `..env` shape those files are named for rather than a single literal name. +# `deploy/env.example` does not end in `.env`, so it is unaffected. secrets/ **/secrets -.env +*.env # The working copies of the host channel described in OPERATIONS.md. # They carry server internals, and the host's own backup is what makes them durable. diff --git a/OPERATIONS.md b/OPERATIONS.md index 5ac3142..225a56e 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -8,16 +8,16 @@ Four environments, in two pairs. Each pair is one publish site and one staging s | Environment | Address | Fronted by | Purpose | | --- | --- | --- | --- | -| Local publish mirror | a private hostname, set in `secrets/.env` | Traefik, on the maintainer's own network | Proves the artifact. The redirect rules, the maps, and the release mechanics. | -| Local staging mirror | a second private hostname, set in `secrets/staging.env` | Traefik | Proves that two environments on one host stay independent, before that matters on a server. | -| Staging | `blog.vps.insanegenius.net`, behind the auth gate | Pangolin | Proves the infrastructure. Routing, TLS, and the deploy path. | -| Production | `blog.insanegenius.com` | Pangolin | The public site. | +| Local publish mirror | a private hostname, set in `secrets/local.production.env` | Traefik, on the maintainer's own network | Proves the artifact. The redirect rules, the maps, and the release mechanics. | +| Local staging mirror | a second private hostname, set in `secrets/local.staging.env` | Traefik | Proves that two environments on one host stay independent, before that matters on a server. | +| Staging | `blog.vps.insanegenius.net`, behind the auth gate, set in `secrets/vps.staging.env` | Pangolin | Proves the infrastructure. Routing, TLS, and the deploy path. | +| Production | `blog.insanegenius.com`, set in `secrets/vps.production.env` | Pangolin | The public site. | The local mirrors are not staging. They run the same bundle against the same web server, so they catch a broken redirect or a bad permission for free, but they exercise none of the routing, authentication, or certificate machinery that only exists on the VPS. Passing locally says the artifact is right. It says nothing about whether the server in front of it is. **The two words are `production` and `staging`, spelled out, in every position.** No `prod`, no `stage`. The same two name the container, the deploy root, the environment file, the `X-Blog-Env` value, and the GitHub Environment. This is not tidiness: the environment name is a value that gets **compared**, by `EXPECT_SITE_ENV` and by the deploy, so a spelling that differs in one position fails a deploy for a reason that reads like an outage. The local mirrors prefix the same words, `mirror-production` and `mirror-staging`, so a header names exactly one of the four environments in the fleet. -Each environment is one file under `secrets/`, selected with `ENV_FILE`, holding the deploy root, the base URL, and the container name. Selecting the file is how an environment is chosen: the file is sourced with `set -a`, so it overwrites a `DEPLOY_ROOT` the caller exported and setting that variable by hand does not switch anything. A named file that does not exist is a hard failure rather than a fall-through, because on a host serving two sites the ambient value is the other site's root. +Each environment is one file under `secrets/`, named `..env`, selected with `ENV_FILE`, and holding the deploy root, the base URL, and the container name. The name carries both halves because the two pairs differ in server as well as environment, so a file says which machine it describes rather than leaving that to the value inside it, and the four in the table above are the four files. `secrets/local.production.env` is the one read when `ENV_FILE` is unset. Selecting the file is how an environment is chosen: the file is sourced with `set -a`, so it overwrites a `DEPLOY_ROOT` the caller exported and setting that variable by hand does not switch anything. A named file that does not exist is a hard failure rather than a fall-through, because on a host serving two sites the ambient value is the other site's root. **The staging FQDN sits under the VPS wildcard deliberately.** `blog.vps.insanegenius.net` needs no new certificate and no new DNS record, and it keeps the staging name off the production domain. @@ -56,7 +56,7 @@ This site has served the same domain across earlier platforms, so its whole oper ### Rebuilding from the Exports -Everything derived is in this repository. Everything it was derived *from* is in a capture directory outside it, which is where a rebuild starts. **The capture path is `CAPTURE_ROOT` in `secrets/.env`**, recorded alongside the other values that name a machine rather than the project, so it is read from there rather than searched for. The capture is not a git repository, so it has no history to revert to, and it is read-only in normal use. +Everything derived is in this repository. Everything it was derived *from* is in a capture directory outside it, which is where a rebuild starts. **The capture path is `CAPTURE_ROOT` in `secrets/local.production.env`**, recorded alongside the other values that name a machine rather than the project, so it is read from there rather than searched for. The capture is not a git repository, so it has no history to revert to, and it is read-only in normal use. | Under the capture | Holds | Recoverable | | --- | --- | --- | @@ -83,7 +83,7 @@ So release to the local mirror and run the live check **before** opening a pull | `hugo.yaml`, `layouts/` | Permalink and taxonomy changes move URLs underneath the redirects that point at them. | ```sh -set -a; . secrets/.env; set +a +set -a; . secrets/local.production.env; set +a deploy/make-release.sh checks/check-live-urls.sh "$HUGO_BASEURL" ``` @@ -91,8 +91,8 @@ checks/check-live-urls.sh "$HUGO_BASEURL" Against the staging mirror, name its file in both places, since the sourced values and the ones `make-release.sh` reads must describe the same environment: ```sh -set -a; . secrets/staging.env; set +a -ENV_FILE=secrets/staging.env deploy/make-release.sh +set -a; . secrets/local.staging.env; set +a +ENV_FILE=secrets/local.staging.env deploy/make-release.sh checks/check-live-urls.sh "$HUGO_BASEURL" ``` @@ -114,7 +114,7 @@ EXPECT_RELEASE= checks/check-live-urls.sh "$HUGO_BASEURL" Sourcing the environment file first puts the deploy root and the base URL in the environment, so no literal value is typed. `make-release.sh` then needs no arguments, because its deploy root falls back to `$DEPLOY_ROOT` and its version falls back to a timestamp. It still accepts both, and [Deploying](#deploying) below passes them explicitly, which is what CI does so a pipeline run names the commit it built rather than the clock. Either form works locally, and the argument wins over the environment. -`ENV_FILE` is set as well as sourced, and the redundancy is deliberate. The script sources its own file regardless, so leaving `ENV_FILE` off would build and install against `secrets/.env` while the shell's `$HUGO_BASEURL` still named staging, and the run would check the staging site after publishing to the production root. The script prints the file it read, on every build, for that reason. +`ENV_FILE` is set as well as sourced, and the redundancy is deliberate. The script sources its own file regardless, so leaving `ENV_FILE` off would build and install against `secrets/local.production.env` while the shell's `$HUGO_BASEURL` still named staging, and the run would check the staging site after publishing to the production root. The script prints the file it read, on every build, for that reason. It refuses to install a release that fails the build gate. `check-live-urls.sh` does take a base URL, which is where the sourced `$HUGO_BASEURL` goes. It follows every URL in the contract against the running mirror, checking each redirect's destination rather than trusting its status code. @@ -127,10 +127,12 @@ A documentation-only or workflow-only change does not need this. A change to the Staging keeps Pangolin's authentication on, so an unauthenticated request never reaches the site. `check-live-urls.sh` presents a Pangolin resource access token when both halves of the pair are set, and sends nothing when neither is: ```sh -set -a; . secrets/staging.env; set +a +set -a; . secrets/vps.staging.env; set +a checks/check-live-urls.sh "$HUGO_BASEURL" ``` +The gate is the VPS staging environment's, so this is `secrets/vps.staging.env`. The local staging mirror sits behind Traefik on the maintainer's own network and carries neither half of the pair. + | Variable | Header | | --- | --- | | `PANGOLIN_ACCESS_TOKEN_ID` | `P-Access-Token-Id` | @@ -157,7 +159,7 @@ The deploy root and the base URL are the only host-specific values. A local run | Variable | Effect | | --- | --- | -| `ENV_FILE` | Which environment file to source. Defaults to `secrets/.env`. | +| `ENV_FILE` | Which environment file to source. Defaults to `secrets/local.production.env`. | | `DEPLOY_ROOT` | Fallback deploy root. The first argument wins. | | `HUGO_BASEURL` | Overrides the site base URL. | | `REQUIRE_BROTLI=1` | Fails rather than shipping gzip-only. CI sets this. | @@ -311,7 +313,7 @@ The container reads three host paths, and only one of them a release ever writes Because it sits outside the bundle, no release updates it. Install or refresh it explicitly, once per environment, which is the same command against a different sourced file: ```sh -set -a; . secrets/.env; set +a # or secrets/staging.env +set -a; . secrets/local.production.env; set +a # or any other secrets/..env install -m 644 deploy/bootstrap.Caddyfile "$CADDY_APPDATA/config/Caddyfile" docker restart "$CADDY_CONTAINER" # only this file needs one, see below ``` diff --git a/README.md b/README.md index a81aa2e..10cb4dd 100644 --- a/README.md +++ b/README.md @@ -179,7 +179,7 @@ deploy/make-release.sh checks/check-live-urls.sh "$HUGO_BASEURL" ``` -The deploy root and the base URL come from an untracked file per environment under `secrets/`, copied from [deploy/env.example][env-example] and selected with `ENV_FILE`. The whole `secrets/` directory is gitignored, so host-specific values stay out of the published history. +The deploy root and the base URL come from an untracked file per environment under `secrets/`, named `..env`, copied from [deploy/env.example][env-example] and selected with `ENV_FILE`. `secrets/local.production.env` is the one read when `ENV_FILE` is unset. The whole `secrets/` directory is gitignored, so host-specific values stay out of the published history. ## 3rd Party Tools diff --git a/TODO.md b/TODO.md index 5959b55..e419543 100644 --- a/TODO.md +++ b/TODO.md @@ -143,7 +143,7 @@ Secrets and variables, per environment. The App-token pair is repository-scoped `DEPLOY_SSH_PRIVATE_KEY` holds the same key in both environments, per the decision above. The environment split still carries the base URL, the SSH endpoint, and the staging-only token pair, so it is not decorative. -The deploy root is deliberately absent from this table. The rsync destination is anchored at the deploy key's confinement root, so the workflow names an environment rather than a host path, and a declared-but-unread name is drift no audit can tell from a missing one. The local `DEPLOY_ROOT` in `secrets/.env` is a different value and is still read. +The deploy root is deliberately absent from this table. The rsync destination is anchored at the deploy key's confinement root, so the workflow names an environment rather than a host path, and a declared-but-unread name is drift no audit can tell from a missing one. The local `DEPLOY_ROOT` in `secrets/..env` is a different value and is still read. diff --git a/checks/README.md b/checks/README.md index 38da17f..435916f 100644 --- a/checks/README.md +++ b/checks/README.md @@ -64,7 +64,7 @@ Two properties of the maps are non-obvious and easy to break when regenerating t **Adding a URL.** Real traffic finds what the lists missed. When a server log shows a 404 for an address that should work, append it to the appropriate list and add a redirect rule or map entry to cover it. The lists are append-only, per Directionality below. -**Regenerating the maps.** `build-redirects.py` rebuilds everything under `deploy/maps/` from the source export, which lives in a capture directory outside this repo and is passed as an argument. **That directory's path is `CAPTURE_ROOT` in `secrets/.env`**, and `OPERATIONS.md` "Rebuilding from the Exports" records what it holds and which parts of it a person can fetch again. It is a provenance tool rather than a CI step, and it selects the export **by content**, failing unless exactly one candidate contains published posts. The capture holds both a full export and a media-only one with zero posts, and taking the wrong one yields empty maps that are indistinguishable from working ones until the redirects are live. +**Regenerating the maps.** `build-redirects.py` rebuilds everything under `deploy/maps/` from the source export, which lives in a capture directory outside this repo and is passed as an argument. **That directory's path is `CAPTURE_ROOT` in `secrets/local.production.env`**, and `OPERATIONS.md` "Rebuilding from the Exports" records what it holds and which parts of it a person can fetch again. It is a provenance tool rather than a CI step, and it selects the export **by content**, failing unless exactly one candidate contains published posts. The capture holds both a full export and a media-only one with zero posts, and taking the wrong one yields empty maps that are indistinguishable from working ones until the redirects are live. **Checking a count.** Every count above is derivable from the files, so check rather than trust: diff --git a/deploy/README.md b/deploy/README.md index 0c34f69..43124fb 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -57,23 +57,34 @@ checks/check-live-urls.sh "$HUGO_BASEURL" ``` The deploy root and the base URL are the only host-specific values, and they pair per -environment. Copy [`env.example`](./env.example) to `secrets/.env`, which is the file read when -`ENV_FILE` is unset, and add `secrets/.env` for each further environment. A single -environment therefore needs `secrets/.env` and nothing else, since a differently named file is -read only when `ENV_FILE` names it. `secrets/` is gitignored as a whole directory, so a value -naming one machine cannot reach a public repo by being added to a file nobody remembered to -ignore. CI passes them explicitly instead, which keeps a pipeline run self-describing: +environment. Copy [`env.example`](./env.example) to `secrets/local.production.env`, which is the +file read when `ENV_FILE` is unset, and add `secrets/..env` for each further +environment. A single environment therefore needs `secrets/local.production.env` and nothing +else, since a differently named file is read only when `ENV_FILE` names it. `secrets/` is +gitignored as a whole directory, so a value naming one machine cannot reach a public repo by +being added to a file nobody remembered to ignore. CI passes them explicitly instead, which +keeps a pipeline run self-describing: ```sh HUGO_BASEURL= deploy/make-release.sh "$(git rev-parse --short HEAD)" ``` -**One file per environment, selected by `ENV_FILE`.** `secrets/.env` is the default and is read -when `ENV_FILE` is unset, so a single-environment host needs nothing else: +**One file per environment, named `secrets/..env`, selected by `ENV_FILE`.** +Both halves are spelled out, so a name says which machine it describes as well as which +environment on it, and the four in the fleet read as one set. `secrets/local.production.env` is +the default and is read when `ENV_FILE` is unset, so a single-environment host needs nothing +else: ```sh -deploy/make-release.sh # secrets/.env -ENV_FILE=secrets/staging.env deploy/make-release.sh # the staging site on the same host +deploy/make-release.sh # secrets/local.production.env +ENV_FILE=secrets/local.staging.env deploy/make-release.sh # the staging site on the same host +``` + +A file whose `DEPLOY_SSH_HOST` is set describes a root on another machine, so this script refuses +to create that path here and asks for a local one to assemble a bundle into: + +```sh +ENV_FILE=secrets/vps.staging.env deploy/make-release.sh /path/to/bundle ``` Selecting the file is the only way to switch environments. The file is sourced with `set -a`, @@ -93,7 +104,7 @@ every build for that reason. | Variable | Effect | | --- | --- | -| `ENV_FILE` | Which environment file to source. Defaults to `secrets/.env`. | +| `ENV_FILE` | Which environment file to source. Defaults to `secrets/local.production.env`. | | `DEPLOY_ROOT` | Fallback deploy root. The first argument wins. | | `HUGO_BASEURL` | Overrides the site base URL. Hugo maps `HUGO_` onto config natively. | | `REQUIRE_BROTLI=1` | Fails rather than shipping gzip-only. CI sets this. | diff --git a/deploy/env.example b/deploy/env.example index 0201846..7978bdd 100644 --- a/deploy/env.example +++ b/deploy/env.example @@ -1,10 +1,13 @@ -# Copy to secrets/.env and set for this host. +# Copy to secrets/..env and set for this host. # Every value here names a machine rather than the project, so secrets/ is gitignored whole. # CI sets the deploy values from environment secrets and reads no file. # -# One file per environment, selected by ENV_FILE: -# secrets/.env the default, read when ENV_FILE is unset -# secrets/staging.env ENV_FILE=secrets/staging.env deploy/make-release.sh +# One file per environment, named for the server it describes and the environment on it, +# with both words spelled out, and selected by ENV_FILE: +# secrets/local.production.env the default, read when ENV_FILE is unset +# secrets/local.staging.env ENV_FILE=secrets/local.staging.env deploy/make-release.sh +# secrets/vps.production.env ENV_FILE=secrets/vps.production.env deploy/make-release.sh +# secrets/vps.staging.env ENV_FILE=secrets/vps.staging.env deploy/make-release.sh # # The file is sourced with `set -a`, which overwrites a variable the caller exported first. # Selecting the file is therefore how an environment is chosen. @@ -62,6 +65,6 @@ EXPECT_SITE_ENV=production # The provenance capture, holding the WordPress exports, the crawl of the old platform, and the inventories derived from it. # checks/build-redirects.py takes this directory as its one argument and rebuilds deploy/maps/ from it. -# Environment-independent, unlike every value above, so keep it in secrets/.env and drop it from any per-environment copy of this template. +# Environment-independent, unlike every value above, so keep it in the default file, secrets/local.production.env, and drop it from any per-environment copy of this template. # Nothing sources this value, so it is recorded to keep a rebuild from depending on memory. CAPTURE_ROOT=/path/to/blog-capture diff --git a/deploy/make-release.sh b/deploy/make-release.sh index d1e86d9..89c5eec 100755 --- a/deploy/make-release.sh +++ b/deploy/make-release.sh @@ -12,7 +12,8 @@ KEEP_RELEASES=10 usage() { echo "usage: $0 [deploy-root] [version]" >&2 echo " deploy-root defaults to DEPLOY_ROOT, from the environment or \$ENV_FILE" >&2 - echo " ENV_FILE defaults to secrets/.env, and a relative path resolves against the repo" >&2 + echo " ENV_FILE defaults to secrets/local.production.env, and a relative path resolves against the repo" >&2 + echo " deploy-root is required when the environment file sets DEPLOY_SSH_HOST, since that root is on another host" >&2 exit 2 } @@ -21,7 +22,9 @@ REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # The deploy root and the base URL are the only host-specific values, and they pair per environment. # ENV_FILE selects the environment, because `set -a` overwrites a value the caller exported. # The first argument overrides the root, being read after this. -DEFAULT_ENV_FILE="$REPO/secrets/.env" +# Files are named secrets/..env, both words spelled out, so the default names +# the environment it actually selects rather than being the one file whose name says nothing. +DEFAULT_ENV_FILE="$REPO/secrets/local.production.env" ENV_FILE="${ENV_FILE:-$DEFAULT_ENV_FILE}" # A relative name resolves against the repo, so it means the same from any working directory. # Traversal is refused rather than resolved, since a relative name is meant to reach secrets/.