Skip to content

Promote develop to main: repair the fleet-wide false ruleset DEFECT - #555

Merged
ptr727 merged 4 commits into
mainfrom
develop
Aug 3, 2026
Merged

Promote develop to main: repair the fleet-wide false ruleset DEFECT#555
ptr727 merged 4 commits into
mainfrom
develop

Conversation

@ptr727

@ptr727 ptr727 commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Three commits, and the first two are repairs to defects this hub shipped earlier today. Merged as a merge commit, never squashed.

Commit PR What
71b7691 #547 Catalogs Blog and records its first audit
82e3658 #551 Stops the audit asserting a bypass list the config no longer manages
fe01060 #553 Defines walk in the filter rather than calling it, for jq 1.5

Why this one should not wait

main is currently wrong about every repository in the fleet. The previous promotion took bypass_actors out of the ruleset payloads, but spec/audit.py compares live-against-payload over a subset that still included that field. A payload deliberately declaring no bypass, against a live ruleset that has one, reads as a normalized diff, so every repo audits as two ruleset DEFECTs:

== Utilities (csharp, nuget; release) @ main@e179288 ==
  DEFECT ruleset: develop diverges from repo-config/develop.json (normalized diff)
  DEFECT ruleset: main diverges from repo-config/main.json (normalized diff)

Anyone auditing against main today gets a false failure on every repository. #551 removes the field from the compared subset, because the defect was two tools comparing one field under opposite policies, not the field's value: after the bypass change, apply writes the live list back unchanged and check reports it without asserting, so the audit asserting it contradicted the config directly.

Verified on develop: Utilities and PlexCleaner went from two ruleset DEFECTs each to none, and Utilities' remaining two findings are LETTER results for absent GOVERNANCE.md and OPERATIONS.md, which is the real propagation gap.

The second repair

#553 closes #549. The payload-driven comparison was built on walk/1, which arrived in jq 1.6. On jq 1.5 the filter does not degrade, it fails to compile, so check_ruleset would report drift on every parameterized rule it never compared. Reproduced on a genuine jq-1.5-1-a5b5cbe and confirmed fixed, with output byte-identical to jq 1.7.

Both defects were found by review rather than by a gate: #551 while reviewing the Blog registration, #553 by a Copilot review on a downstream re-vendor of the same canonical.

Blog

#547 catalogs Blog, taking the registry to 22 repositories. Its committed report claims zero defects, which was true when written, briefly false while the audit regression stood, and is true again on this branch. Confirmed rather than assumed: spec/audit.py Blog reports 0 defect/letter/error here.

Verification

spec/validate.py OK at 22 cataloged, diff-scoped prose_lint clean, editorconfig clean, live read-only configure.sh check against this repo passes, and the audit reports the corrected verdicts above.

ptr727 and others added 3 commits August 3, 2026 15:55
Blog was stood up on 2026-08-01 and cut release `1.0.11` the same day,
but it was never added to `registry/repos.json`. Every hub tool is
registry-driven, so `spec/audit.py` and `spec/fidelity_honesty.py` had
never measured it, and `reports/divergences.md` was under-reporting the
fleet by exactly one repo while reading as complete. This catalogs it
and lands its first audit.

## The entry records what ran, not what was predicted

`reports/conformance-matrix.md` already named Blog as the reference repo
for the `source-only` + `release` shape, and predicted `releaseTrigger:
none` with an empty `publish[]` until a deploy ran. What actually
shipped is a dispatch-only publisher that cut a tag and a source
archive, so the entry is `dispatch-only` with the GitHub release
declared, and the matrix row is corrected to match. The VPS deploy still
has not run, so the static-site type stays deferred to #456 and
`publish[]` gets revisited when it does.

## What the audit found

Nine findings, no defects, verdict **operational**. Every one is the hub
advancing after the carry rather than the repo regressing:

- **Eight verbatim units behind the canonical.** Six are stale copies.
Two never arrived at all, `AGENTS.md > Fleet Bootstrap` and
`GOVERNANCE.md > Representative Data in Agent-Authored Text`, and that
distinction matters because a stale copy still states the rule in an
older form while an absent one states nothing.
- **One undeclared section**, `AGENTS.md > Project Conventions`, whose
four rules are genuinely repo-specific and duplicate no verbatim
section.

The convergence PRs against Blog follow separately, one per drift class
per AUDIT.md section 10.

## Measured against `main`, with `develop` recorded separately

The run reads hub `main` (`3a7cc64`) per AUDIT.md section 1. A re-run
from a tree at `develop` (`362aec8`) reports two more re-vendors and two
DEFECTs, all four from #545 taking `bypass_actors` out of the ruleset
payloads an hour earlier. Those are recorded in the report rather than
counted, because measuring against un-promoted hub content reports work
in flight as a conformance failure. The same four will appear against
every fleet repo when that promotes, and the live-ruleset half is a
settings change on a protected branch, so it is yours to apply rather
than an agent's.

## Two spec questions, raised rather than resolved

1. **The hub contradicts itself on whether a repo may carry its own
`AGENTS.md` section.** `AGENTS.md` says a project's own conventions live
in that project's `AGENTS.md`. `spec/section-model.md` says an
undeclared section is drift to reconcile and lists four destinations,
none of them `AGENTS.md`. A repo following the one is flagged for
violating the other, and Blog is exactly that case.
2. **The character-set tiers do not classify the COPYRIGHT SIGN**, which
any repo generating a feed or a rendered document will hit.

A third, smaller one: the report directory here is `reports/blog/`,
lowercase, matching every existing report directory rather than the
repo's CamelCase name. AUDIT.md section 8 writes it as
`reports/<repo>/audit.md`, which reads as exact case, and the two have
quietly disagreed since the first report. Worth settling in the wording,
and worth settling before someone clones on a case-insensitive
filesystem.

## Verification

- `python3 spec/validate.py`: 22 cataloged, 0 backlog, classifying
cleanly.
- `python3 spec/audit.py Blog` from a tree at hub `main`: 9 drift, 0
defect/letter/error.
- `python3 spec/fidelity_honesty.py --report`: regenerated, and the
whole diff is Blog appearing in the rows it belongs in.
- `python3 scripts/prose_lint.py reports/blog/audit.md`: clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…551)

Regression fix. #545 removed `bypass_actors` from the ruleset payloads,
and **every repo in the fleet has been reporting two ruleset DEFECTs
since it promoted**.

## What broke

`spec/audit.py` compares the live ruleset against the payload over a
fixed subset, and that subset included `bypass_actors`. A payload that
deliberately declares no bypass list, against a live ruleset that still
has one, is a normalized diff — so the audit called it divergence:

```text
== Utilities (csharp, nuget; release) @ main@e179288 ==
  DEFECT ruleset: develop diverges from repo-config/develop.json (normalized diff)
  DEFECT ruleset: main diverges from repo-config/main.json (normalized diff)
```

Reproduced on Utilities, PlexCleaner and Blog. It is every repo, because
every live ruleset carries the entry the payloads stopped declaring.

## The fix, and why it is this way round

The field is dropped from the compared subset rather than restored to
the payloads.

Who may bypass a ruleset is a per-repository human decision taken in the
UI, and `configure.sh` already treats it that way after #545: `apply`
writes the live list back unchanged, `check` reports it without
asserting. The audit asserting the same field contradicted that
directly.

**The defect was two tools comparing one field under opposite policies,
not the field's value.** The comment says so and records what happened
when it was not, because the next person to look at that list will
otherwise put it back.

## How it was found

While reviewing the Blog registration in #547. That report claims nine
drift and no defects, which was true when it was written against hub
`main` at `3a7cc64`, and stopped being true an hour later when #545
promoted. The PR body had even predicted it: "The same four will appear
against every fleet repo when that promotes."

It did. This is the same staleness the audit's own run-stamp rule exists
to expose, arriving from the hub's side rather than the repo's.

## Verification

- Utilities: 2 ruleset DEFECTs before, **0** after. Its remaining two
findings are LETTER results for absent `GOVERNANCE.md` and
`OPERATIONS.md`, which are the real propagation gap rather than this
regression.
- PlexCleaner: same, 2 before and 0 after.
- `spec/validate.py` OK, diff-scoped `prose_lint` clean, editorconfig
clean, syntax checked.

## Follow-on

#547's `reports/blog/audit.md` should be regenerated against this
corrected tool before it merges, so the committed report matches what
the audit now reports rather than what it reported an hour before the
payload change landed.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes #549, raised by the Blog agent from a Copilot review on a
downstream re-vendor and declined there because `configure.sh` is
carried `verbatim`.

## The defect

The payload-driven ruleset comparison added in #540 built its normalizer
on `walk/1`, which arrived in **jq 1.6**. On a host carrying jq 1.5 the
filter does not degrade — it **fails to compile**, so `check_ruleset`
reports drift on every parameterized rule it never actually compared.

That is the inverse of the false clean #540 was written to close, and
arguably worse: a false *failure* teaches an operator to distrust the
tool, where a false pass merely fails to warn them.

## Verified on a real jq 1.5, not argued

```console
jq version: jq-1.5-1-a5b5cbe

--- OLD filter (calls walk):
def n: walk(if type=="array" then sort else . end); n
jq: 1 compile error

--- NEW filter (defines its own):
{"allowed_merge_methods":["merge","squash"],"nested":{"deep":[1,2,3]},"required_status_checks":[{"context":"a"},{"context":"b"}]}
```

The reported failure reproduces exactly, and the fix compiles and
returns the sorted document. Output is byte-identical to jq 1.7 on the
same input, including a deliberately nested array to confirm the local
definition still recurses.

Finding a genuine jq 1.5 took three attempts — `imega/jq:1.5` does not
exist, Debian buster's archives are gone, and Alpine 3.8 ships a master
build that already has `walk`. Ubuntu 18.04 has it. Worth recording,
because "I could not reproduce it" would have been the wrong conclusion
from the first two.

## The choice

`walk` is defined inside the filter, so the script calls nothing jq 1.5
lacks and keeps one code path across versions.

The alternative in the issue — assert a jq version up front and fail
with a message — was considered and declined for the reason the issue
itself gives: `check` is the read-only mode, and refusing to run at all
is a worse outcome than running. The issue stated that as a preference
rather than a decision, and I agree with it.

## Verification

`bash -n` clean, diff-scoped `prose_lint` clean, and a live `check`
against this repo still passes on all three parameterized rules of both
rulesets.

## Downstream

`configure.sh` is carried `verbatim` with `appliesTo: "*"`, so this
joins the re-vendor the ledger already lists. It is the same file that
generated a re-vendor three times today, which is precisely the evidence
behind the vendored-tooling entry added in #546.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 3, 2026 23:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Promotes develop to main while repairing two fleet-wide auditing/comparison regressions (ruleset bypass list handling in spec/audit.py, and jq 1.5 compatibility in repo-config/configure.sh), and catalogs the Blog repository with its first recorded audit/report updates.

Changes:

  • Stop spec/audit.py from asserting bypass_actors in ruleset comparisons, aligning it with repo-config/configure.sh behavior.
  • Make repo-config/configure.sh ruleset normalization portable to jq 1.5 by defining a local walk-equivalent (w/1) instead of calling walk/1.
  • Catalog Blog in registry/repos.json and update generated reports/matrix to include the new repo and its audit status.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
spec/audit.py Removes bypass_actors from the ruleset comparison subset and drops related normalization logic.
repo-config/configure.sh Replaces walk/1 usage with a locally-defined recursive walker to support jq 1.5 in check mode.
registry/repos.json Adds the Blog repository entry to the fleet registry.
reports/blog/audit.md Adds Blog’s first audit report.
reports/divergences.md Updates the generated divergence report to include Blog in relevant rows.
reports/conformance-matrix.md Updates the shape reference row for Blog with the audit date and corrected notes.

Comment thread spec/audit.py Outdated
Comment thread reports/blog/audit.md Outdated
Both findings from the review on promotion #555. A promotion's head is
`develop`, so the fix lands here and #555 picks it up.

## The report predicted, and the prediction was wrong

`reports/blog/audit.md` closed a paragraph with:

> ...and the same four will appear against every fleet repo the moment
that change promotes:

Two things wrong with that, and the second is why it matters.

**A report records what a run observed.** A prediction cannot be
verified by re-running the audit, which is the one check the run-stamp
discipline offers a reader.

**And it turned out to be false.** The change promoted. The four
findings were then traced to `spec/audit.py` asserting a `bypass_actors`
list that `configure.sh` had deliberately stopped managing, and #551
removed that assertion. So they do not appear against every fleet repo.
Verified: `spec/audit.py Blog` reports **0 defect/letter/error** on this
branch.

A prediction goes stale the moment reality diverges and nothing
re-checks it. That is the same failure the registry's `driftNote` rules
already name, arriving in a report instead — and it is the second
instance today of a document asserting a future that stopped being true.

The sentence now ends at the reason those findings were not counted, and
hands off to the enumeration that follows it.

## The comment attached its clause to the wrong noun

```diff
-# Phrases in a registry driftNote that assert work is still outstanding.
+# Phrases that assert work is still outstanding, matched against a registry driftNote.
```

The subject is the plural *phrases*, so `assert` was correct, but
`driftNote that assert` sits adjacent and reads as a disagreement.
Reordered so the clause sits with what it describes. This is the second
pass over these two lines today, the first having added a missing verb.

## Verification

Diff-scoped `prose_lint` clean, editorconfig clean, CRLF preserved on
the report, `spec/audit.py` parses, and the audit still reports Blog at
zero findings.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 3, 2026 23:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

repo-config/configure.sh:247

  • The jq normalizer is intended to work on jq 1.5, but it uses keys_unsorted, which is not available until jq 1.6. On jq 1.5 this can still fail to compile (similar to the original walk/1 issue), preventing check_ruleset from comparing parameterized rule parameters.
            | if type == "object" then reduce keys_unsorted[] as $k ({}; . + { ($k): ($in[$k] | w(f)) }) | f

@ptr727

ptr727 commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

Answering the suppressed finding on repo-config/configure.sh:247, which carries no thread:

The jq normalizer is intended to work on jq 1.5, but it uses keys_unsorted, which is not available until jq 1.6. On jq 1.5 this can still fail to compile (similar to the original walk/1 issue), preventing check_ruleset from comparing parameterized rule parameters.

Declined: the premise is wrong, and it was checked rather than reasoned about. keys_unsorted predates jq 1.5. Run against the same build that reproduced the original walk/1 failure:

jq version: jq-1.5-1-a5b5cbe

$ echo '{"b":1,"a":2}' | jq -c 'keys_unsorted'
["b","a"]

$ echo '{"p":{"m":["squash","merge"],"c":[{"context":"b"},{"context":"a"}]}}' | jq -S -c '<the shipped filter>'
{"p":{"c":[{"context":"a"},{"context":"b"}],"m":["merge","squash"]}}

The builtin compiles, and the full normalizer runs on that version and returns the sorted document, including the nested array that proves the local walk definition still recurses. That is the same jq that gives jq: 1 compile error on the walk/1 form this change replaced, so the two builtins are not in the same position.

The finding is a good shape of question, and the analogy to walk/1 is exactly the right instinct. It is simply wrong on this builtin's history, which is why #553 was verified against a real jq 1.5 rather than argued from a version table.

Worth noting for the record that this class is anticipated in the review runbook: Copilot is sometimes factually wrong about a tool's capabilities, so a finding is verified before it is fixed and before it is dismissed. This one was verified, and it does not hold.

No change to this promotion.

@ptr727
ptr727 merged commit 3b802b9 into main Aug 3, 2026
7 checks passed
ptr727 added a commit that referenced this pull request Aug 6, 2026
…574)

Promotes the nine commits `develop` has carried since the previous
promotion (#555) to `main`. Merge commit only, no squash, and `develop`
is not deleted.

## What lands

- **#560** Standardize the static-site-deploy repo type and its
destination, including the deploy-verification guarantee and a retention
rule that records which side owns the prune.
- **#561** Ask for a blocked decision instead of reporting it, and scope
the clickable-link rule to a surface that renders markdown.
- **#564** Read a suppressed section nested inside the review-details
wrapper, so `scripts/pr_review.py` stops reporting a clean round over
findings that reach no thread.
- **#566** Settle Markdown capitalization in `CODESTYLE.md`, and fix the
two `TODO.md` prose defects the digest defect had hidden.
- **#568** Check a `driftNote` on every run rather than only on an
otherwise clean audit, and state what the audit does not evaluate.
- **#569** Name who trips the production ref gate, and state that a
harness refusal is a different thing from the maintainer's permission
and is not lifted by it.
- **#571** Re-audit `ptr727/Blog` against the hugo type and bump the
conformance matrix.
- **#573** Scope the semicolon exemption to the sentence its list lives
in, with the 44 newly reported occurrences recorded on #519 rather than
swept here.
- **#572** Exempt a verbatim section from the coordination-reference
scan.

## Issues closed

Fixes #562. Closes #565.

The closing keywords sit on this promotion rather than on the feature
pull requests, because GitHub fires them only on a merge into the
default branch. #456, #558, #563, #567, and #570 are already closed
against their merged fixes.

## Verification

`git merge-tree` reports no conflict between `origin/main` and
`origin/develop`, so this promotion needs no throwaway resolution
branch. Every constituent pull request merged green with its review loop
closed.
ptr727 added a commit that referenced this pull request Aug 7, 2026
Clears the `A Home for a Disproved Finding` cluster from `TODO.md`. Its
`Checked` anchor was re-verified against `develop` at `756a53e` before
anything was written: the file still says the reviewer is sometimes
factually wrong, still requires a decline to carry evidence, and still
keeps its list of known non-working request paths, with nowhere to put
the proof itself.

## The gap

A decline that carries proof proves something about this tree, and
[GOVERNANCE.md "Every Finding Ends in an
Action"](https://github.com/ptr727/ProjectTemplate/blob/develop/GOVERNANCE.md#every-finding-ends-in-an-action)
is right that the thread is where it belongs while the pull request is
open. Afterwards the thread is the wrong place. The pull request merges,
the next round starts with no memory of the last, and the second
occurrence of the same claim reaches a maintainer with no way to tell it
from a first.

`Disproved Claims` is a new `###` under the runbook, so the three
declared `##` sections in `spec/files.json` are unchanged. An entry
names the claim, what was run or read to disprove it, the revision it
was proved against, and what ends it. **It is deliberately not a list to
append to**: an entry outliving the code it was proved against becomes a
reason not to check, which is strictly worse than proving the claim
again, so an entry whose subject moves is deleted by the change that
moves it rather than edited to look current. Two guards ride with it.
The record answers a repeated claim and never dismisses a new one, so a
finding is judged on its merits first and matched second, and a reply
carries the proof re-read rather than a pointer to a file the reviewer
cannot open. And the entries are this repository's own, so a repository
holding a copy carries the shape and deletes an entry whose subject it
does not carry.

## The three worked examples

**`keys_unsorted` requires jq 1.6.** A suppressed finding on #555
against the normalizer in `repo-config/configure.sh`, reasoning by
analogy from the `walk/1` failure #553 fixed. Re-run in this session
rather than quoted from the thread, on `jq-1.5-1-a5b5cbe` in
`ubuntu:18.04`, the build that reproduces `walk/1`:

```console
jq-1.5-1-a5b5cbe
$ echo '{"b":1,"a":2}' | jq -c 'keys_unsorted'
["b","a"]
$ echo '{"b":1}' | jq -c 'walk(.)'
jq: error: walk/1 is not defined at <top-level>, line 1:
jq: 1 compile error
```

**The write-guard's fallback parse, from #601.** Declined on the ground
that the arm cannot execute, since `punctuation_chars` arrived in Python
3.6, the module uses f-strings throughout, and `install.py` refuses
below 3.7. That is exactly the kind of disproof that expires, which is
why the entry names the floor as what ends it. It also records that the
finding earned a test case rather than a change, since only `ValueError`
from unbalanced quoting reaches that path in practice and nothing
covered it.

**The bare-SHA design, from #602.** This one came from this repository's
own backlog rather than from a reviewer, and it is here because a
rejected method costs the same to re-propose as a declined finding costs
to re-derive, while a backlog has a place for a claim the tree
contradicts and none for a method a measurement rejects. Over the 25
most recent merged pull requests the bare-SHA arm raised four references
and all four were correct prose, and a path arm flagged 54 of 215
backticked candidates.

Both #601 and #602 are folded under disposition **Amends "Record what
was tested and against which revision, and delete an entry whose subject
changes."** They land as the second and third worked examples in the
shipped section rather than as `Settled` bullets, because the entry
ships in the same change.

## Verification

- `python3 scripts/prose_lint.py . --diff develop` clean,
`markdownlint-cli2` clean on both files.
- `python3 -m unittest discover -s scripts` reports 372 tests OK and
`python3 spec/audit.py --selftest` passes, neither touched by this
change but both run because the file is carried.
- Every factual claim in the new section was read against the tree at
`756a53e`: the `keys_unsorted` call in `repo-config/configure.sh`,
`_git_subcommand_arglists` and `_PUNCTUATION_CHARS` in
`gh-write-guard.py`, the `(3, 7)` floor in `install.py`, and the verb
alternation in `pr_review.py`.

## TODO.md

The cluster is deleted per the file's own step 9. What this change does
not carry becomes a new `ready` cluster: `GOVERNANCE.md` outcome 2 ends
at the thread and nothing agent-agnostic points at the record, so an
agent that never opens the provider runbook posts a decline and has
nowhere to put the proof. That is a byte-locked carried section and a
fleet re-vendor, which is why it is a separate change rather than a
second surface in this one.

## Downstream

`.github/copilot-instructions.md` is carried `whole` at `intent` with
`appliesTo: "*"`, so this joins the re-vendor the ledger already lists.
A repository taking it carries the section and its rules, and starts its
own entries empty.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

configure.sh check mode needs jq 1.6+, and fails to compile rather than degrade on jq 1.5

2 participants