Skip to content

Promote develop to main: the log filter, and the negative-result rule - #83

Merged
ptr727 merged 4 commits into
mainfrom
develop
Aug 9, 2026
Merged

Promote develop to main: the log filter, and the negative-result rule#83
ptr727 merged 4 commits into
mainfrom
develop

Conversation

@ptr727

@ptr727 ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Promotes develop to main. Four commits, each already reviewed and squash-merged to develop.

Documentation and comments only. Three files: OPERATIONS.md, TODO.md, and two comments in checks/check-live-urls.sh. No behaviour changes, so nothing here alters a build or a deploy.

What this carries

[#81] The outward log pass filters by X-Blog-Check rather than by user agent. The user-agent key was a coincidence: the CI runner's curl and the VPS host's curl are byte-identical, so only the client address separated them and the CI half rotates every run. The old method stays, scoped to the days logged before the field existed, since it is the only thing that can read them.

It also records the gap the host side measured: check-live-urls.sh sends the header on every request and a bare curl sends nothing, so an interactive probe passes -H "X-Blog-Check: proxmox/media-dev" or it lands in the visitor set.

[#82] The negative-result trap is stated as a demand for evidence. It said a query matching nothing reads as a clean result, which is true and names no action:

Before believing a negative, establish that the check could have produced a positive.

Four costumes are listed under it, each one this repository actually hit: a query that cannot see its target, a rule naming a target that does not exist, a filter whose precondition is unstated, and a tool reporting success having done nothing.

[#84] X-Blog-Check is added to the access log's keep-list, which the same file described without it. The two statements contradicted each other, and chasing it found a worse problem in the same sentence, predating today: "request headers are dropped" reads as Traefik stripping them from the request, which would mean the Pangolin token never reaches the site. It now names the three JSON keys the log carries and says the rest are omitted from the record rather than from the request.

[#85] Two counts are removed rather than corrected. The HUGO_BASEURL counterfactual said a wrong base URL would have run 1,245 requests, and check-live-urls.sh said in two comments that it runs 1,245 of them. It runs 1,253.

The distinction that came out of #85, and is worth carrying

1,245 is the URL contract, 328 render plus 917 redirect. 1,253 is the live check, which requests the contract plus eight media URLs. Both numbers are real and they mean different things, which is why three other occurrences in the repository are correct and were deliberately left alone: the migration post contrasting the contract with the sitemap's 111, the audit report quoting a 2026-08-04 run, and the parity note about a build baked with the wrong host.

Where a count was removed it was replaced by a statement that cannot go stale, rather than by a corrected number.

Two State rows re-measured

  • Production carries the hard-linking result: 1,052 of 3,275 files shared with the previous release, the linked set being exactly the restored generation, all 1,048 linked media files at mode 644, and 11.4 MB incremental against 585 MB for a full copy. Staging reproduced the same 1,052 independently, against a different predecessor.
  • The backup timer fired unattended on 2026-08-09 at 09:11:01 UTC, proven by both halves of the standard rather than either alone: a second journal entry and LAST moving off -.

The serving release id is no longer repeated outside the State table, since it moves with every deploy.

Merge

Plain --merge, never --delete-branch: on a develop -> main promotion that flag deletes develop.

Nothing publishes on this merge.

ptr727 and others added 2 commits August 9, 2026 10:13
* Filter the log by the header, and say a hand probe has to carry it

The outward pass told a reader to separate our traffic from real visitors
by user agent, which is the coincidence X-Blog-Check was added to replace:
the CI runner's curl and the host's own curl are byte-identical, so only
the client address separates them and the CI half rotates every run.

The header is the mechanism now, and its value carries provenance, so a
run is identifiable rather than merely excludable.

The gap worth recording is that check-live-urls.sh sends it and a bare
curl does not, so an interactive probe lands in the visitor set unless
whoever runs it passes -H. Two untagged probes turned up against 3,100
tagged ones in the 2026-08-09 deploy window, measured on the host side.

The user-agent method stays, scoped to what it can still read: the days
logged before the field existed.

Kept as sub-bullets under the first filter rather than as four more bold
paragraphs, because the section opens by saying it is four filters and a
reader has to be able to count them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Show real tag values, and say the field depends on the edge logging it

The examples were angle-bracket placeholders, and the tag is validated:
exactly one slash, letters, digits, dot, underscore, hyphen. Pasting one
of those literally is refused, so the section now shows values that work
and says the shape is enforced rather than conventional.

The larger omission is that the filter depends on something this
repository does not own. `request_X-Blog-Check` appears only because the
edge is configured to log that header, so an absent field means either an
untagged request or a capture that stopped, and the log alone does not
distinguish them. Reading a day of absence as a day of real traffic is the
wrong answer that shape produces.

Both found by review, which is also the general form the host side named:
before believing a negative, establish that the check could have produced
a positive.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Put a pasteable value in the pasteable command

The prose examples were corrected and the curl one-liner was not, which
is the one a reader actually copies. It now carries proxmox/media-dev.

ENVIRONMENT.md keeps its angle brackets deliberately. There they describe
the enforced grammar and the value the script derives under Actions,
neither of which anyone types, and the settable example beside them is
already a real one.

Found by Copilot review on #81, as a suppressed comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Claim only what the tagging actually covers

The lead said every synthetic request announces itself, and the bullet
below it said a hand curl sends nothing unless someone adds the header.
The section contradicted itself in the direction that flatters the filter,
which is the direction that produces a wrong answer nobody checks.

It now says the scripted checks send it on every request they make, and
that the filter is only as complete as the tagging is.

Found by Copilot review on #81, as a suppressed comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Say which half of the tag carries the purpose

"Naming the purpose rather than the host" implied the whole value is a
purpose label, when proxmox is the source half and only the id half is
free. A reader following that would have written something like
media-dev/probe, inverting the grammar the section just described.

Found by Copilot review on #81, as a suppressed comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* State the negative-result trap as a demand for evidence

The trap said a query matching nothing reads as a clean result, which is
true and names no action. The host side put the general form better:
before believing a negative, establish that the check could have produced
a positive. That is the same rule with a step in it, and the step is what
was missing every time this cost something.

Four costumes are now listed under it, because seeing them as one failure
is the whole value:

  a query that cannot see its target, including a journal grep run from an
  account not in adm or systemd-journal, where the lines exist and are
  simply not shown to it

  a rule naming a target that does not exist, which is the .gitattributes
  pin for two files this repository has never carried

  a filter whose precondition is unstated, which is reading an absent
  X-Blog-Check as a visitor when the field exists only because the edge is
  configured to log it

  a tool reporting success having done nothing, which is git restore-mtime
  v2022.12 printing a count and processing none of it

No count is claimed for them. An earlier draft said six false passes and
the list does not add to six, some were near-misses caught in time, and
one has not happened at all.

Two State rows are re-measured while here, since both had gone false
today. Production serves a new release and the hard-linking result now
exists, 1,052 of 3,275 with every linked media file at 644. The backup
timer fired unattended at 09:11:01 UTC, proven by both halves of the
standard rather than either alone.

The serving release id is no longer repeated outside the State table. A
value that moves with every deploy should exist in one place, and that
duplicate had already gone stale twice in a day.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Match the file on a code literal and a tool name

The file mode was plain text beside literals the same table wraps in
code, and the tool was named by its subcommand invocation where the two
nearby mentions use the binary name. Both now read the same way
throughout.

The distinction is worth keeping straight rather than merely consistent:
git-restore-mtime is the tool, and git restore-mtime is one of the two
ways it resolves. This sentence is about the tool.

Found by Copilot review on #82.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 9, 2026 17:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Promotes the already-reviewed develop documentation updates to main, refining operational guidance around edge-log filtering (X-Blog-Check) and codifying the “negative-result trap” as an evidence requirement, plus updating measured state entries.

Changes:

  • Update the outward log-review filter guidance to use X-Blog-Check provenance (and document hand-probe tagging expectations).
  • Re-state the “negative-result trap” as an explicit requirement to prove a check could have produced a positive.
  • Refresh TODO “State” measurements (hard-linking counts and unattended backup-timer proof) and remove duplicated moving release IDs outside the State table.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
TODO.md Updates State measurements and rewrites the negative-result rule section to require evidence for empty/negative checks.
OPERATIONS.md Revises outward-pass log filtering guidance to key off X-Blog-Check rather than user-agent, with additional operational cautions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread OPERATIONS.md
* Add X-Blog-Check to the keep-list this document already describes

The access-log section said request headers are dropped except Referer
and User-Agent. The host added X-Blog-Check to that allowlist when the
header was introduced, which is the only reason request_X-Blog-Check
appears at all, so the outward pass and the log description contradicted
each other in the same file.

The keep-list is named as the precondition rather than left as trivia. A
header absent from it does not appear in the log, which reads exactly like
a request that never sent one, and that is the negative-result trap this
file now states as a rule.

The reason the list is an allowlist is unchanged and still the point: it
is what keeps the Pangolin resource access token out of a file that is
retained and copied off the host.

Found by Copilot review on #83.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Say the log omits headers rather than that headers are dropped

"Request headers are dropped except" reads as Traefik stripping them
from the request. That would mean the Pangolin access token never
reaches the site, and the staging gate works, so a reader who took it
that way would be left with two facts that cannot both hold.

It now says the log records three and omits the rest from the record
rather than from the request, which still arrives intact.

Found by Copilot review on #84.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Name the log keys, since that is what a reader greps for

The sentence named the headers where the rest of the section names the
JSON keys, so it described the policy without giving the thing anyone
parsing the file actually needs. The three now appear as
request_Referer, request_User-Agent and request_X-Blog-Check, which is
the form already used in the jq examples above and below it.

Found by Copilot review on #84, as a suppressed comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 18:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

TODO.md:34

  • This bullet still says the live check runs 1,245 requests, but the State table above (and the URL-contract row) now documents the live check as 328 + 917 + 8 = 1,253. Keeping 1,245 here makes the doc internally inconsistent and could mislead someone validating a deploy.
  - **`HUGO_BASEURL` on the `production` environment is set to `https://blog.insanegenius.net/`**, done 2026-08-07. It held `https://blog.insanegenius.com/`, the live WordPress address, which is what the workflow both builds with and points the live check at, so a deploy would have baked the old platform's address into every canonical tag, feed link and `sitemap.xml` and then run 1,245 requests at the live site to verify it. **Setting it back to `.com` at M7b is the other half and is not done.**

@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Answering the suppressed finding, which carries no thread. Fixed in #85.

This bullet still says the live check runs 1,245 requests, but the State table above now documents the live check as 328 + 917 + 8 = 1,253.

I declined this once already, earlier today, on the grounds that the sentence is a counterfactual about 2026-08-07 and the check genuinely was 1,245 that day. That reasoning is correct and it was not sufficient.

Being accurate-as-history is not the same as being useful. A reader validating a deploy meets 1,245 and 1,253 in one file with nothing connecting them, and no way to know one is a vintage. I even wrote at the time that changing it "would be the reverse error" — which is true of editing the number and not of removing it.

The number was never load-bearing. The point is that the check would have been aimed at the old platform, which the sentence makes without a count:

and then run the whole live check at the live site to verify it

1,253 now appears only where it describes the live check as it is; 1,245 appears nowhere. Same reasoning as dropping the duplicated release id in #82a value that has to be dated to stay true is better removed than dated.

* Drop a count that only stayed true by being dated

The HUGO_BASEURL item said a wrong base URL would have run 1,245 requests
at the live site. That was the live check size on 2026-08-07 and the
sentence is a counterfactual about that day, so it was defensible, and
leaving it meant a reader validating a deploy met 1,245 and 1,253 in one
file with nothing connecting them.

The number was never load-bearing there. The point is that the check
would have been aimed at the old platform, which the sentence makes
without it.

Same reasoning as removing the duplicated release id: a value that has to
be dated to stay true is better removed than dated, because the next
reader has no way to know which vintage they are holding.

Found by Copilot review on #83, as a suppressed comment, having been
declined once on the grounds that the counterfactual was accurate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Stop the live check miscounting itself in its own comments

Two comments said the script runs 1,245 requests. It runs 1,253: the
1,245-URL contract plus the eight media probes added when the live media
check landed. The file describing what it does was the last thing to know
what it does.

Neither states a number now. One says it runs one request per URL checked
and the other says one request before the rest, which is what each
comment was actually about, and neither goes stale when a list grows.

The three other occurrences in the repository are correct and are left
alone. 1,245 is the URL contract, 328 render plus 917 redirect, and 1,253
is what the live check requests because it adds eight media URLs. The
blog post contrasting 1,245 with the sitemap 111, the audit report
quoting a 2026-08-04 run, and the parity note about a build baked with
the wrong host all mean the contract and all say so correctly.

Found by Copilot review on #85, which caught the PR description claiming
1,245 appeared nowhere when it had only been grepped out of two files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Say every request carries it, since a URL is not one request

The comment claimed one invocation per URL checked. A render check is
one curl, a redirect check is three, and the media check reaches six,
so the count was wrong in the direction that understates the exposure
the comment exists to explain.

It now says every request would carry the token, which is the actual
reason the credential goes in a config file rather than on a command
line, and which no list or code path can make stale.

Found by Copilot review on #85, as a suppressed comment, on wording I
had introduced one commit earlier to remove a different stale count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 18:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

checks/check-live-urls.sh:118

  • PR description says “OPERATIONS.md and TODO.md are the only files touched”, but this PR also changes checks/check-live-urls.sh (e.g., updated comments at lines 117 and 272). Please either update the PR description to include this file, or drop these edits if the intent is to keep the promotion strictly to those two docs.
# A resource access token opens the proxy's auth gate.
# It goes into a curl config file because bash cannot export an array to the parallel checks.
# A command line is also world-readable in ps output, and every request would carry it.
if [ -n "${PANGOLIN_ACCESS_TOKEN_ID:-}" ] && [ -n "${PANGOLIN_ACCESS_TOKEN:-}" ]; then

@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Answering the two suppressed findings, neither of which carries a thread.

The description was stale

PR description says "OPERATIONS.md and TODO.md are the only files touched", but this PR also changes checks/check-live-urls.sh.

Correct. That was true when I wrote it and stopped being true when #85 merged into develop beneath it. A promotion's description describes a moving branch, which is a way for it to go wrong that a normal PR body cannot, and I had not accounted for it.

Rewritten against the actual diff:

OPERATIONS.md             | 12 ++++++++++--
TODO.md                   | 16 ++++++++++------
checks/check-live-urls.sh |  4 ++--

Still documentation and comments only — the check-live-urls.sh change is two comments, no behaviour.

The 1,245 finding, from an earlier round

This bullet still says the live check runs 1,245 requests.

Fixed in #85, which is now in this promotion, and it produced the distinction worth carrying: 1,245 is the URL contract (328 render + 917 redirect) and 1,253 is the live check (contract + 8 media). Both are real.

So two occurrences were genuinely stale and three were correct and deliberately left: the migration post contrasting the contract with the sitemap's 111, the audit report quoting a 2026-08-04 run, and the parity note about a build baked with the wrong host. Plus false positives worth not re-greping — the Adaptec 51245 controller in four 2009 posts, and a PaperMod SVG path.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@ptr727
ptr727 merged commit b54e7a1 into main Aug 9, 2026
6 checks passed
@ptr727
ptr727 deployed to staging August 9, 2026 18:58 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants