Skip to content

Encode the README Structure, Links, and Tools as Auditable Rules - #632

Merged
ptr727 merged 15 commits into
developfrom
readme-structure-rework
Aug 9, 2026
Merged

Encode the README Structure, Links, and Tools as Auditable Rules#632
ptr727 merged 15 commits into
developfrom
readme-structure-rework

Conversation

@ptr727

@ptr727 ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner

The README spec was advisory: spec/project-types.json declared readme.sections at verdict intent, so nothing mechanically checked section presence, order, shields, or link naming. Two wanted changes had also left this repo's own README deliberately ahead of the spec that governs it.

The two changes

The License section closes the file and carries the license shield, which 15 of the 16 repos with a License section already did. The first line under the H1 is the tagline, and it alone carries the 100-character link-free rule and the mirrors, so a README may now say more about itself above the fold.

That second one was nearly free. The audit already took the first line for the About and Docker Hub mirrors (intro.split("\n")[0]), and only the HISTORY.md comparison read the whole intro region, so the change is one line plus the prose. TODO.md's claim that this was blocked on the registry field, "since taking this first means writing an extraction rule the registry change deletes", overstated the coupling, and that note is corrected here.

Settled alongside them, from a survey of all 22 non-archived non-fork repos: a Table of Contents in every README with no size threshold, Use Cases renamed to Overview with no accepted aliases, 3rd Party Tools required, and Questions or Issues required in a public repo.

Made auditable

spec/readme-sections.json declares the section order and per-section requirement, the additive shield classes, the five link groups, the reference-name suffixes, and the canonical names for the destinations every repo has. spec/third-party-tools.json declares 40 shared tools, so a tool two repos use is linked and described identically in both. spec/audit.py reads both, and readme.sections moves from intent to letter.

Two design points keep this from over-reaching:

  • A heading the model does not name is dropped before the order comparison, so the roughly 60 genuinely repo-specific sections across the fleet are never findings.
  • The tool catalog is a standard set rather than a complete one, so a tool only one repo uses is unaudited. Of the 36 tools the fleet lists today, 24 are used by exactly one repo.

What the corpus run changed

Running the checks against all 22 READMEs before shipping them corrected the model three times, which is the point of measuring first:

  • Keying shields on alt text flagged three repos for caption wording. The fleet spells one badge Release Status, Releases Build, Build Status, Workflow Status and Lint Build against a single endpoint. Matching moved to the shields.io endpoint, with requireQuery/forbidQuery separating the two shields that share one.
  • A Coverage shield class would have told 7 repos to add a badge nobody chose: 10 repos declare CODECOV_TOKEN and 3 display a badge. Dropped.
  • Requiring both Docker version shields is wrong for NxWitness, which carries 40 across six images and four channels that are not develop. The class now asserts a version shield exists; the latest-plus-develop pair stays in the spec as the single-image form.

It also caught a plain bug: extract_section matches heading text exactly, so ## License <!-- omit from toc --> was not the License section, and four repos suffix every heading that way. That alone was 12 findings for sub-sections that were present.

A later pass found two more, both fixed: keying "shield" on img.shields.io alone asked three repos to rename a byob.yarr.is badge to -link, a rename away from the convention; and matching Docker Hub by host alone told ESPHome-Config to rename its link to upstream's image.

Verification

  • python3 spec/audit.py --selftest passes, with 50 new cases covering sections, shields, links, and the tool catalog.
  • python3 spec/validate.py passes, and now shape-checks the tool catalog: a duplicate name would silently shadow, since the audit keys on it.
  • Corpus run over all 22 READMEs, read in both directions, with every model correction above driven by it.
  • This repo's own README reports zero findings across all four checks.
  • prose_lint.py, editorconfig-checker, markdownlint-cli2 (44 files), and cspell all clean. CRLF preserved; spec/audit.py and spec/validate.py stay LF.

Fleet consequences, not carried here

No downstream repo is edited by this change. TODO.md records the backlog each repo works off on its next visit: 73 structure findings, 56 link-naming letters, 28 grouping drifts, and 55 tool-catalog findings.

Two items need a maintainer's eye:

  1. GOVERNANCE.md "Repository Details" is a verbatim carried section, so this edit puts all 21 downstream repos into verbatim drift until re-vendored. That is the normal propagation path for a rule change, but it is fleet-wide rather than hub-local.
  2. The byob.yarr.is last-build badge is retired. Three repos render it and two already serve HTTP 404, and PlexCleaner lists it as a tool besides. deprecatedShields reports it so it does not depend on anyone remembering.

The README spec was advisory: `spec/project-types.json` declared
`readme.sections` at verdict `intent`, so nothing mechanically checked
section presence, order, shields, or link naming. Two wanted changes had
also left the hub's own README deliberately ahead of the spec that
governs it.

## The two changes

**The License section closes the file** and carries the license shield,
which 15 of the 16 repos with a License section already did. **The first
line under the H1 is the tagline**, and it alone carries the
100-character link-free rule and the mirrors, so a README may now say
more about itself above the fold. That second one was nearly free: the
audit already took the first line for the About and Docker Hub mirrors,
and only the `HISTORY.md` comparison read the whole intro region.

Also settled, from a survey of all 22 non-archived non-fork repos: a
Table of Contents in every README with no size threshold, `Use Cases`
renamed to `Overview` with no aliases, `3rd Party Tools` required, and
`Questions or Issues` required in a public repo.

## Made auditable

`spec/readme-sections.json` declares the section order, the requirement
per section, the additive shield classes, the five link groups, the
reference-name suffixes, and the canonical names for destinations every
repo has. `spec/third-party-tools.json` declares 40 shared tools so a
tool used by two repos is linked and described identically in both.
`spec/audit.py` reads both, and `readme.sections` moves from `intent` to
`letter`.

A heading the model does not name is dropped before the order
comparison, so the ~60 genuinely repo-specific sections across the fleet
are never findings. The tool catalog is a standard set rather than a
complete one, so a tool only one repo uses is unaudited.

## What the corpus run changed

Running the checks against all 22 READMEs before shipping them corrected
the model three times. Keying shields on alt text flagged three repos for
caption wording, since the fleet spells one badge five ways against one
endpoint; matching moved to the shields.io endpoint. A `Coverage` class
would have told 7 repos to add a badge nobody chose, since 10 declare
`CODECOV_TOKEN` and 3 display one; it was dropped. Requiring both Docker
version shields is wrong for NxWitness, which carries 40 across six
images and four channels that are not `develop`.

It also caught a plain bug: `extract_section` matches heading text
exactly, so `## License <!-- omit from toc -->` was not the License
section, and four repos suffix every heading that way.

Remaining fleet backlog, recorded in `TODO.md`: 73 structure findings,
56 link-naming findings, 28 grouping drifts, and 55 tool-catalog
findings. No downstream repo is edited here. The hub reports zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 9, 2026 00:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request turns the README structure rules from advisory prose into declared, auditable requirements, and extends the audit tooling to mechanically verify README sections/order, shields, reference-link naming/grouping, and shared 3rd-party tool catalog consistency across the fleet.

Changes:

  • Add declared models for README sections/shields/links (spec/readme-sections.json) and shared tool catalog (spec/third-party-tools.json) plus JSON schemas.
  • Extend spec/audit.py to audit README sections/order, shield placement, reference-link rules, tool catalog intersection checks, and tagline-only mirroring.
  • Update project policy docs and this repo’s README.md to conform to the new auditable rules (tagline semantics, License placement/shield, link grouping, 3rd Party Tools section).

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
TODO.md Updates backlog notes to reflect shipped README tagline/spec work and new remaining items.
spec/validate.py Adds validation for spec/third-party-tools.json shape, uniqueness, sentence style, and sorting.
spec/third-party-tools.schema.json Introduces JSON Schema for the shared third-party tool catalog.
spec/third-party-tools.json Adds the shared third-party tool catalog (names/links/descriptions).
spec/readme-structure.md Updates the prose spec to match the new declared/audited README model (tagline semantics, required sections, link groups, shield rules).
spec/readme-sections.schema.json Introduces JSON Schema for the declared README section/shield/link model.
spec/readme-sections.json Adds the declared README model consumed by the audit (sections, shield classes, link groups/naming, canonical links).
spec/project-types.json Moves README checks from intent to letter and splits into specific checks (sections/shields/links/tools/tagline).
spec/audit.py Implements mechanical README auditing: tagline extraction, section order/requirements, shield endpoint matching and placement, link definition naming/grouping, tool catalog intersection checks, plus selftests.
README.md Updates this repo README to conform (canonical github-link, adds 3rd Party Tools section, adds Distribution link group).
GOVERNANCE.md Updates repository description policy to reference the README “tagline” line and its mirror chain.
CODESTYLE.md Updates HISTORY.md mirroring rule to mirror only the tagline rather than the whole intro paragraph.
AUDIT.md Documents that README structure is now mechanically checked against the declared model.
Suppressed comments (1)

spec/readme-sections.json:27

  • The linkNaming note for kind "shield" says "A badge under img.shields.io", but the rule in spec/readme-structure.md defines a shield by how the reference is used (rendered as an image), not by its host. Updating the note avoids reintroducing the earlier host-based interpretation.
        { "kind": "shield", "suffix": "-shield", "note": "A badge under img.shields.io." },

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread spec/audit.py Outdated
Comment thread spec/readme-sections.json Outdated
Copilot review of #632 found the audit contradicting the rule it had
just written down. `spec/readme-structure.md` defines a shield as a
reference the document renders as an image, "judged by how the document
uses the reference, not by which host serves it", while `link_kind`
short-circuited on an `img.shields.io` host before ever consulting that
usage, and two `note` fields in `spec/readme-sections.json` still
described the Shields group as holding badges "under img.shields.io".

The short-circuit is removed. Measured across the fleet, all 119
img.shields.io definitions are rendered as images, so a host test
classifies nothing usage does not and can only ever contradict the rule
beside it. An A/B over all 22 READMEs with and without the short-circuit
returns 82 link findings either way, in identical sets, so this changes
what the code means without changing what any repo is told.

A selftest case pins the direction that was previously unreachable: an
`img.shields.io` definition the document never renders is an ordinary
URI, so naming it `-shield` and filing it under Shields is now reported
rather than accepted. The conformant fixture had itself been relying on
the host short-circuit, defining a license shield it never rendered, and
now renders it as a real README does.

Also corrects four counts in `TODO.md` that had gone stale as this
branch fixed the hub's own README: the structure backlog is 73 rather
than 75, link naming 55 letters and 27 drifts rather than 56 and 28, and
the tool catalog 54 across four repos rather than 55 across five. The
tool split is 48 absent descriptions, 3 that disagree, and 3 links that
disagree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 00:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

TODO.md:506

  • The <!-- Repo --> reference definitions are no longer alphabetized by reference name (e.g., third-party-tools is placed between readme-sections and readme-structure). GOVERNANCE.md requires each link-definition group to be alphabetized by reference name (GOVERNANCE.md:166).
[prose-gate]: ./.github/actions/prose-gate/action.yml
[readme-sections]: ./spec/readme-sections.json
[third-party-tools]: ./spec/third-party-tools.json
[readme-structure]: ./spec/readme-structure.md

spec/validate.py:79

  • load("spec/third-party-tools.json") is assumed to return an object; if the JSON is malformed (e.g., top-level array), .get("tools") will raise AttributeError and crash validation instead of reporting a clear shape error.
    tools = load("spec/third-party-tools.json").get("tools")

Both fixes come from the suppressed half of the round-two Copilot
review, which is worth reading rather than trusting the suppression:
neither was noise.

`TODO.md` gained a `[third-party-tools]` reference inserted next to the
`[readme-sections]` line it relates to rather than at its sorted
position, which put it between `readme-sections` and `readme-structure`
and broke the alphabetical order GOVERNANCE.md requires of a link group.
It now sits after `[standup]`. Sweeping all 44 tracked Markdown files
for the same defect finds no other group out of order, so this was
isolated rather than a pattern.

`spec/validate.py` read `load("spec/third-party-tools.json").get(...)`,
which raises AttributeError on a malformed top level and crashes the
run. That is the opposite of what shape-checking there is for, since the
block exists so a bad file reports a clear message instead of taking
down the cross-reference loops below it. A bare-array top level now
reports two errors and exits 1, verified by substituting one and
restoring the file byte-identical afterwards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 01:06
@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Round two generated no new threads, but its two suppressed comments were both real, and both are fixed in f0f9da8.

TODO.md:506 - reference group no longer alphabetized. Correct. I added [third-party-tools] next to the [readme-sections] line it relates to rather than at its sorted position, which put it between readme-sections and readme-structure. It now sits after [standup]. I swept all 44 tracked Markdown files for the same defect rather than fixing only the flagged line, and no other reference group is out of order, so this was isolated rather than a pattern.

spec/validate.py:79 - .get() on an assumed object. Correct, and it contradicts that block's own purpose: the shape check exists so a malformed file reports a clear message instead of crashing the cross-reference loops below it, and load(...).get("tools") would have raised AttributeError on a bare-array top level instead. Now guarded. Verified by substituting a [] top level, which reports two errors and exits 1, then restoring the file byte-identical.

Both are the kind of finding worth reading past the suppression for, so thanks for surfacing them.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

spec/audit.py:606

  • readme_link_findings() (and the rendered-image detection it uses for shield classification) does not skip fenced code blocks. A Markdown code sample containing <!-- ... -->, [ref]: url, or ![alt][ref] will be misread as real link-group headers/definitions or rendered shields, producing false DRIFT/LETTER findings.
    # Walk the definitions in order, tracking which group header each one falls under.
    seen_headers, current, in_group = [], None, {}
    for ln in normalize(text).split("\n"):
        s = ln.strip()
        h = re.fullmatch(r"<!--\s*(.*?)\s*-->", s)
        if h and "omit from toc" not in h.group(1):
            current = h.group(1)
            seen_headers.append(current)
            in_group.setdefault(current, [])
            continue
        d = _LINK_DEF.match(s)
        if d:
            in_group.setdefault(current, []).append((d.group(1), d.group(2)))

    all_defs = [p for v in in_group.values() for p in v]
    # A reference the document renders as an image is a shield whatever host serves it, so resolve those first.
    by_ref = dict(all_defs)
    rendered = {by_ref[m.group(1)] for m in _MD_IMAGE_REF.finditer(norm) if m.group(1) in by_ref}

spec/audit.py:519

  • link_kind() currently classifies NuGet and PyPI URLs as "distribution" only when the package name is derived from the GitHub owner (nuget: .../packages/<owner>., pypi: .../project/<owner>-). The new README link rules (spec/readme-structure.md / readme-sections.json canonicalLinks) describe NuGet/PyPI distribution links by host/path pattern, not by owner-derived naming, so this can misclassify a repo's own NuGet/PyPI links as external and prevent canonical naming / grouping enforcement for them.
    if url.startswith("#"):
        return "anchor"
    if not url.startswith(("http://", "https://")):
        return "local"
    owner = slug.split("/")[0].lower()
    own = (f"https://github.com/{slug}".lower(), f"https://hub.docker.com/r/{owner}/",
           f"https://www.nuget.org/packages/{owner}.", f"https://nuget.org/packages/{owner}.",
           f"https://pypi.org/project/{owner}-")
    if url.lower().startswith(own):
        return "distribution"
    return "shield" if url in rendered else "external"

Both fixes come from the suppressed half of the round-three Copilot
review, which generated no new threads. Neither was noise.

`readme_link_findings` read the whole document for group headers, link
definitions, and rendered images, while `ordered_headings` and
`extract_section` had skipped fenced blocks all along. A README
documenting badge markup carries `[ref]: url`, `<!-- Shields -->` and
`![alt][ref]` as samples, and reading those invents a group, a
definition, and a rendered shield the document does not have. The walk
now skips fences and the rendered-image scan reads the unfenced text, so
one document is parsed one way. No fleet README trips this today,
measured across all 22, so this is a latent defect rather than a live
one.

`link_kind` also carried its own list of URL prefixes deciding what
counts as this project's own, while `canonicalLinks` matched NuGet and
PyPI by host and path alone. Ownership stated twice is two things that
can disagree, and the code's owner-scoping was the half doing the real
work: without it a dependency link to Serilog's package would be renamed
`nuget-link`. The prefixes move into the model as `distribution`, with
`{slug}` and `{owner}` placeholders, and `link_kind` reads them from
there.

Verified: the declared prefixes reproduce the removed hardcoded set
exactly for all 22 repos, and the corpus returns the same 82 link
findings as before. Four selftest cases pin the directions that were
unreachable, covering a fenced sample, a third-party package that keeps
its name, and an owner's own package that does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 01:15
@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Round three generated no new threads, and again both suppressed comments were real. Both fixed in 021fe19.

spec/audit.py:606 - fenced code blocks not skipped. Correct, and it was an inconsistency inside my own code: ordered_headings and extract_section had skipped fences all along, while readme_link_findings read the whole document. One document was being parsed two ways. The walk now skips fences and the rendered-image scan reads the unfenced text. I measured all 22 READMEs for fenced [ref]:, <!-- ... --> and ![alt][ref] constructs and found none, so this was latent rather than live, which is a reason to fix it rather than to leave it.

spec/audit.py:519 - NuGet/PyPI ownership defined in two places. Correct. link_kind carried its own prefix list while canonicalLinks matched by host and path alone, and the code half was the one doing the real work: without owner-scoping, a dependency link to Serilog's package would be renamed nuget-link. Rather than reconcile two lists, ownership moves into the model as a distribution.urlPrefixes declaration with {slug} and {owner} placeholders, and link_kind reads it from there. Same class of defect as round one, where the code and the declared rule disagreed.

Verified rather than asserted: the declared prefixes reproduce the removed hardcoded set exactly for all 22 repos, and the corpus returns the same 82 link findings as before, so this changes what the code means without changing what any repo is told. Four new selftest cases pin the previously unreachable directions, including a third-party package that keeps its name and an owner-published one that does not.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

spec/audit.py:588

  • norm is computed but never used in readme_link_findings(). This is dead code and makes it harder to tell which normalized form the function actually operates on.
    findings = []
    norm = _HTML_COMMENT.sub(lambda m: "\x00" + m.group(0) + "\x00", normalize(text))
    suffixes = {n["kind"]: n["suffix"] for n in model["linkNaming"]}

spec/audit.py:626

  • In readme_link_findings(), rendered is built as a set of URLs and link_kind() treats any definition whose URL is in that set as a shield. If two different references point at the same URL, a non-rendered reference can be misclassified as a shield just because another reference to the same URL is rendered. The spec language here is per-reference ("a badge ... the document renders as an image"), so classification should be based on whether that reference name is rendered as an image, not whether the URL appears elsewhere.
    by_ref = dict(all_defs)
    rendered = {by_ref[m.group(1)] for m in _MD_IMAGE_REF.finditer("\n".join(unfenced)) if m.group(1) in by_ref}
    described = {"shield": "a badge", "anchor": "an in-page anchor", "local": "a path in this repo"}
    for ref, url in all_defs:
        kind = link_kind(url, slug, rendered, prefixes)
        want = suffixes.get(kind, "")

Both fixes come from the suppressed half of the round-four Copilot
review, which generated no new threads.

`readme_link_findings` still computed `norm`, which the previous commit
orphaned when the rendered-image scan moved to the unfenced text. It is
removed. A normalized form nothing reads is worse than absent, because
the next reader has to work out which of the two the function is
actually operating on.

`rendered` was a set of URLs, so `link_kind` called any definition whose
URL appeared in it a shield. Two reference names may point at one URL,
and only one of them may be rendered, in which case the plain link was
classified a shield because its twin is a badge. The spec is worded per
reference, "a badge, judged by the document rendering the reference as
an image", so `rendered` now holds reference names and `link_kind` takes
whether this one reference is rendered.

No fleet README trips this today: of the 22, the only duplicated URL is
PlexCleaner's Docker Hub page under two names, and neither is rendered.
The new selftest case discriminates rather than passing vacuously, since
the old URL keying returns two findings on it where the shipped code
returns none. The corpus returns the same 82 link findings as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 01:21
@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Round four generated no new threads, and both suppressed comments were real again. Both fixed in 7abab7a. That is six for six across rounds two through four, which is a fair argument for reading past the suppression every time.

spec/audit.py:588 - norm is dead code. Correct, and self-inflicted: the previous commit orphaned it when the rendered-image scan moved to the unfenced text. Removed. A normalized form nothing reads is worse than absent, since the next reader has to work out which of the two the function actually operates on.

spec/audit.py:626 - rendered keyed by URL rather than by reference. Correct, and the sharper of the two. Two reference names may point at one URL with only one rendered, in which case the plain link was classified a shield because its twin is a badge. Your reading of the spec wording is the right one, so rendered now holds reference names and link_kind takes whether this reference is rendered.

Measured before fixing: no fleet README trips it today. Of the 22, the only duplicated URL is PlexCleaner's Docker Hub page under two names, and neither is rendered. So this was latent.

I also checked the new selftest case discriminates rather than passing vacuously, by simulating the old URL keying against it: two findings under the old behaviour, none under the shipped one. The corpus returns the same 82 link findings as before, so this changes what the code means without changing what any repo is told.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (1)

spec/readme-sections.schema.json:6

  • distribution is optional in the schema, but the README link-audit logic relies on distribution.urlPrefixes to classify a repo's own GitHub/Docker/NuGet/PyPI URLs as distribution (which then drives canonical link naming and group validation). If distribution is omitted, the audit silently stops enforcing canonical naming for those URLs and can misclassify the Distribution group contents. Making distribution required here prevents a broken model from slipping through schema validation.
    "required": ["sections", "shieldClasses", "linkGroups", "linkNaming", "canonicalLinks"],

From the suppressed half of the round-five Copilot review, which
generated no new threads. The finding is a fail-open, which is the kind
worth fixing even with no live case.

`distribution` was optional in the schema while `link_kind` depends on
`distribution.urlPrefixes` to tell a repo's own GitHub, Docker Hub,
NuGet and PyPI URLs from a third party's. Omit the key and
`distribution_prefixes` returns empty, every own URL classifies as
external, canonical naming and Distribution-group validation quietly
stop being enforced, and the audit still reports green. A check that
stops checking without saying so is worse than one that was never
written, because the green is read as a result.

The schema now marks `distribution` required. That alone would not have
caught it, since CI runs no JSON-schema validation, so the guard that
actually runs is a shape check in `spec/validate.py`, beside the one for
the tool catalog. It covers two failure modes: absent or empty prefixes,
and prefixes carrying neither `{slug}` nor `{owner}`, which would not be
repo-scoped and would match another owner's URLs.

Verified by removing the key and by flattening the prefixes to a bare
host, each reporting a named error and exiting 1, then restoring the
file byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 01:26
@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Round five generated no new threads, and its single suppressed comment was real. Fixed in 68bcde3. That is seven for seven across rounds two through five.

spec/readme-sections.schema.json:6 - distribution optional while the audit depends on it. Correct, and it is a fail-open, which makes it the most worthwhile of the seven. Omit the key and distribution_prefixes returns empty, every own URL classifies as external, canonical naming and Distribution-group validation quietly stop being enforced, and the audit still reports green. A check that stops checking without saying so is worse than one that was never written, because the green gets read as a result.

The schema now marks it required. That alone would not have caught it, since CI runs no JSON-schema validation, so the guard that actually runs is a shape check in spec/validate.py, beside the one for the tool catalog. It covers two failure modes rather than one: absent or empty prefixes, and prefixes carrying neither {slug} nor {owner}, which would not be repo-scoped and would match another owner's URLs.

Verified by inducing both: removing the key, and flattening the prefixes to a bare https://github.com/. Each reports a named error and exits 1, and the file was restored byte-identical afterwards.

Worth naming the pattern across this loop, since five of the seven were the same defect shape: the implementation quietly disagreeing with the declared rule beside it, whether that was host versus usage, ownership defined twice, URL versus reference, or now a required input the schema called optional. Moving rules into declared data does not by itself make the code obey them.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (1)

spec/audit.py:526

  • link_kind() treats any non-http(s) URL as a repo-local path. That misclassifies other valid URI schemes (e.g. mailto:, ssh:, ftp:) as "local", which would make the audit enforce bare reference names and the Repo link group for those URIs. Treat scheme-based non-http(s) URLs as external URIs instead, while keeping relative paths as local.
    if url.startswith("#"):
        return "anchor"
    if not url.startswith(("http://", "https://")):
        return "local"

From the suppressed half of the round-six Copilot review, which
generated no new threads.

`link_kind` treated every non-http target as a repo-local path, so a
`mailto:`, `ssh:` or `ftp:` reference would have been told to drop its
`-link` suffix and move to the Repo group, which holds files. A contact
address in Questions or Issues is the obvious case. No fleet README
carries a scheme-bearing reference today, measured across all 22, so
this is a shape the rule has to get right before one arrives rather than
a defect being repaired.

A reference is local only when it carries no URI scheme. The scheme test
requires two or more characters so a `C:` drive letter is not read as
one, and every scheme a README actually carries is longer than that.
Verified across the boundary: `./LICENSE`, `LICENSE`, `docs/x.md`,
`../a/b.md` and `C:/win/path` stay local, `#anchor` stays an anchor,
`mailto:` and `ftp://` become external, and this repo's own URL still
classifies as distribution.

The corpus returns the same 82 link findings as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 01:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Comment thread spec/validate.py
From the round-eleven Copilot review, generalizing an earlier finding
rather than repeating it.

`spec/validate.py` shape-checked `distribution` and `canonicalLinks`
while `spec/audit.py` indexes `sections`, `shieldClasses`, `linkGroups`
and `linkNaming` directly too. Removing any of those four left
validation exiting 0 and the audit raising a KeyError mid-run. The
schema marks each required, but CI runs no JSON-schema validation, so
the schema was never the guard that ran.

The four unguarded keys were unguarded for no better reason than that
the earlier reviews had named the other two, so the keys are now checked
as a set rather than one at a time, which is what stops the next added
key from being the next gap.

Verified by removing each of the six in turn: every one reports a named
error and exits 1, where before four of them exited 0. Removing
`linkNaming` and calling the audit directly raises `KeyError:
'linkNaming'`, which is the failure this converts into a report. The
model was restored byte-identical after each.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 02:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (3)

spec/audit.py:420

  • shield_endpoints() scans the raw region text for image markup, so an ![alt][ref] (or inline image) inside a fenced code block would be counted as a rendered shield. That can let a README satisfy required badges via a code sample, or trip exclusive checks, even though fenced blocks are explicitly treated as non-semantic elsewhere (e.g., ordered_headings()/unfenced_text()).
def shield_endpoints(region, defs):
    """The image URLs a Markdown region renders, from both its `![alt][ref]` and its `![alt](url)` uses.

    Keyed on the endpoint rather than the alt text or the reference name, because those are captions. The
    fleet writes Release Status, Releases Build, Build Status, Workflow Status and Lint Build for one badge,
    and names its reference `last-commit-shield` and `lastcommit-shield` in the same breath, while the
    endpoint under img.shields.io is identical in every repo.

    Inline uses are resolved as well as reference ones. Reading references alone made an inline shield
    invisible rather than wrong, so a repo writing every badge inline, which the reference-link rule forbids
    for a separate reason, would have passed the shield check by carrying nothing the check could see.
    A `[ref]: url` definition line is not an image, so it is never counted as a use, which is what keeps a
    shield's own definition from reading as a second placement of it.
    """
    region = region or ""
    urls = [defs[m.group(1)] for m in _MD_IMAGE_REF.finditer(region) if m.group(1) in defs]
    return urls + [m.group(1) for m in _MD_IMAGE_INLINE.finditer(region)]

spec/audit.py:750

  • readme_shield_findings() builds defs from normalize(text), which includes fenced blocks. That means a [ref]: url inside a code sample can be treated as a real badge definition and (for deprecatedShields) flagged even if it is only shown as an example. Other README parsing in this file explicitly uses unfenced_text() to avoid this.
    findings = []
    defs = {m.group(1): m.group(2) for m in _LINK_DEF.finditer(normalize(text))}
    # A retired badge service is scanned across the whole document rather than per section, since a dead badge is wrong wherever it sits.
    # It renders broken rather than absent, which a visitor reads as a failing build rather than as a stale badge.
    for dep in model.get("deprecatedShields", []):
        for ref, url in sorted(defs.items()):
            if dep["match"] in url:
                findings.append(("LETTER", f"readme: `[{ref}]` renders {dep['label']}, which is retired - {dep['reason']} (spec/readme-structure.md)"))

spec/audit.py:708

  • third_party_tool_findings() iterates over the raw extracted section body, so a fenced code sample inside ## 3rd Party Tools that happens to contain a tool table/bullet could be interpreted as real entries and produce false findings. Since the link/definition parsing is already fence-aware via unfenced_text(), the body scan should be too.
    body = readme_region(text, "3rd Party Tools")
    if body is None:
        return []  # the absent section is already one finding from readme_section_findings
    declared = {t["name"].lower(): t for t in catalog["tools"]}

From the suppressed half of the round-twelve Copilot review, three
findings of one class, and the same generalization failure as round
eleven: fence-awareness was added where a review had pointed, and left
everywhere else.

`readme_region` did not strip fences, so every region built from it
inherited that. A badge shown as a code sample could satisfy a required
shield, a tool table shown as a sample could be read as entries, and a
`[ref]: url` sample could supply a definition. `readme_shield_findings`
read its definition map from the whole document for the same reason, so
`deprecatedShields` would flag a retired badge that a README was only
demonstrating.

Fixed at the root rather than at the three call sites. `readme_region`
strips fences, so every region the README checks read is fence-free by
construction, and the two whole-document scans, the shield definition
map and the exclusive-placement scan, read `unfenced_text`. The
exclusive scan was not among the three reported and is the same defect.

`ordered_headings` carried its own fence loop, a second definition of
the idea that let the two drift apart in the first place, and now shares
the one helper.

Verified: the corpus is unchanged at 71 section, 5 shield, 55 link
letter, 27 link drift and 55 tool findings, and this repo still reports
zero. Three selftest cases were each checked against the reverted
behaviour rather than assumed to discriminate, and the first attempt at
that check was wrong, reverting only `readme_region` and leaving the
scan it did not cover, which made a real case look inert.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 02:06
@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Round twelve passed CI with no threads, and all three suppressed comments were real. Fixed in 95978bf. Nineteen for nineteen.

All three are one class, and it is the same generalization failure round eleven found: fence-awareness was added where a review had pointed and left everywhere else. readme_region did not strip fences, so every region built from it inherited that, which is why a badge, a tool table and a definition shown as code samples could each be read as real.

Fixed at the root rather than at the three reported call sites:

  • readme_region strips fences, so every region the README checks read is fence-free by construction.
  • The two whole-document scans, the shield definition map and the exclusive-placement scan, read unfenced_text.
  • ordered_headings carried its own fence loop, a second definition of the idea that let these drift apart in the first place, and now shares the one helper.

The exclusive-placement scan was not among the three you reported and is the same defect, so it is fixed here too. Finding it required listing every text-parsing site in the file rather than fixing the ones named, which is the lesson I should have taken from round eleven and did not.

Verified: the corpus is unchanged at 71 section, 5 shield, 55 link letter, 27 link drift and 55 tool findings, and this repo still reports zero. Each of the three new selftest cases was checked against the reverted behaviour rather than assumed to discriminate, and my first attempt at that check was wrong: it reverted only readme_region and left the scan it did not cover, which made a genuinely discriminating case look inert. Reverting all the fence stripping at once showed all three discriminate.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

spec/audit.py:728

  • third_party_tool_findings() only validates cataloged tools' link/description, but spec/readme-structure.md states the 3rd Party Tools list is alphabetized. As written, a README can be non-alphabetized and still pass the readme.tools check. Consider checking that the tool entries in the section are in case-insensitive sorted order (across all tools, not just those present in the shared catalog).
    findings = []
    for ln in body.split("\n"):
        m = _TOOL_ROW.match(ln.strip()) or _TOOL_BULLET.match(ln.strip())
        if not m:
            continue
        name, ref, desc = m.group(1), m.group(2), m.group(3).strip(" -:")

spec/audit.py:642

  • readme_link_findings() intends to avoid treating directive comments like <!-- markdownlint-disable ... --> as link-group headers (see the comment on the preceding lines), but the current logic treats any HTML comment line (except omit from toc) as a header. If a README places a markdownlint directive inside the reference-definition block (before some [ref]: url lines), those definitions will be mis-grouped under the directive and reported as an unknown link group.
        h = re.fullmatch(r"<!--\s*(.*?)\s*-->", s)
        if h and "omit from toc" not in h.group(1):
            current = h.group(1)
            continue

From the suppressed half of the round-thirteen Copilot review, which
generated no new threads and passed CI.

`spec/readme-structure.md` says the 3rd Party Tools list is alphabetized
and nothing checked it, so a rule was stated and unenforced, which is
the gap this whole change exists to close. The check covers every tool
listed rather than only the cataloged ones, since a reader scans the
whole list. It reports one repo: Blog, which orders Hugo, PaperMod,
Caddy.

The directive-comment fix in 6e81742 only held where no definition
followed. A `<!-- markdownlint-disable -->` placed inside the reference
block does have definitions under it, so those would be grouped under
the directive and reported as an unknown link group, and the code
comment claimed otherwise. Directives are now skipped outright, from a
`directiveComments` list in the model rather than inferred from the
shape of the text, since which tools a repo instructs is a fact about
the fleet rather than something to guess at.

No fleet README carries a directive comment today, measured across all
22, so that half is a shape the rule has to get right before one arrives.

Verified: both new selftest cases were checked against the reverted
behaviour, the directive case by emptying the declared list and the
ordering case against its own absence. The corpus moves only by the one
Blog finding, from 55 tool findings to 56, and this repo still reports
zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 9, 2026 02:13
The alphabetization check added in a64c0d0 reports one repo, so the
backlog entry reads 56 rather than 55 and names what the extra finding
is: Blog listing Hugo, PaperMod, Caddy out of order. A committed count
that no longer matches what the tool reports is the claim this file
exists to make checkable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ptr727

ptr727 commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Round thirteen passed CI with no threads, and both suppressed comments were real. Fixed in a64c0d0, with the backlog count carried in c702d85. Twenty-one for twenty-one.

spec/audit.py:728 - the tool list order was unenforced. Correct, and the most pointed finding of the loop: spec/readme-structure.md says the list is alphabetized and nothing checked it. A rule stated and unenforced is exactly the gap this change exists to close, so having shipped one is worth naming. Per your note it covers every tool listed rather than only the cataloged ones, since a reader scans the whole list. It reports one repo, Blog, which orders Hugo, PaperMod, Caddy.

spec/audit.py:642 - the directive fix only held where no definition followed. Correct. A <!-- markdownlint-disable --> inside the reference block does have definitions under it, so they would be grouped under the directive and reported as an unknown group, and my code comment claimed otherwise. Directives are now skipped outright, from a directiveComments list in the model rather than inferred from the shape of the text, since which tools a repo instructs is a fact about the fleet rather than something to guess at. No fleet README carries one today, measured across all 22, so that half is a shape to get right before one arrives.

Verified: both cases were checked against the reverted behaviour, the directive one by emptying the declared list and the ordering one against its own absence. The corpus moves only by the single Blog finding, 55 tool findings to 56, and this repo still reports zero.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

spec/audit.py:716

  • third_party_tool_findings() does not enforce the spec/readme-structure.md rule that the "3rd Party Tools" table carries no License column. Because _TOOL_ROW only parses the first two columns, a 3-column table (e.g., "| Tool | Role | License |") would currently pass this check silently.
_TOOL_ROW = re.compile(r"^\|\s*\[([^\]]+)\]\[([^\]]+)\]\s*\|\s*([^|]*?)\s*\|")
_TOOL_BULLET = re.compile(r"^[-*]\s*\[([^\]]+)\]\[([^\]]+)\]\s*(.*)$")


def third_party_tool_findings(text, catalog):

spec/validate.py:90

  • validate.py now checks that spec/readme-sections.json top-level keys exist and have the right container type, but it still doesn't validate the per-item shapes that spec/audit.py indexes directly (e.g., linkGroups items need name/holds, shieldClasses items need trigger/shields, etc.). Since CI does not run JSON schema validation, a malformed item would still crash spec/audit.py with KeyError/TypeError rather than being reported by validate.py.
        for key, want in (("sections", list), ("shieldClasses", list), ("linkGroups", list),
                          ("linkNaming", list), ("canonicalLinks", list), ("distribution", dict)):
            value = readme_model.get(key)
            if not isinstance(value, want) or not value:
                errors.append(f"readme-sections.json: '{key}' must be a non-empty {'array' if want is list else 'object'}, and spec/audit.py indexes it directly")

Copilot AI review requested due to automatic review settings August 9, 2026 02:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

@ptr727
ptr727 merged commit c2ce145 into develop Aug 9, 2026
7 checks passed
@ptr727
ptr727 deleted the readme-structure-rework branch August 9, 2026 02:34
ptr727 added a commit that referenced this pull request Aug 9, 2026
…ate (#635)

Promotion of develop at 2082547, six squashes since the last one. Closes #633.

#631 routes the README by reader and documents the GH_WRITE_GUARD_ALLOW grant where a denied cross-owner write puts the reader. #632 moves readme.sections from intent to letter with four checks beside it, backed by spec/readme-sections.json and spec/third-party-tools.json, and settles the tagline rule. #634 adds repo_gate.py --check eol-coverage, reading the line-ending pins against the tree rather than only against .editorconfig.

#636 and #637 repair two readers Copilot found on this pull request, both defects develop already carried: a tool row required both outer table pipes that GitHub's Markdown makes optional, and a retired badge written as an inline image was invisible to a scan that read reference definitions alone. #638 turns the host contract's presence check into a version gate, and retires two gh workarounds that were artifacts of a stale distribution package, re-tested on an upgraded host rather than inferred.

Four carried files owe a downstream re-vendor and none is recorded in the TODO.md entry yet. GOVERNANCE.md Repository Details is verbatim, so the audit reports it, and it propagates a rule: the About description is the tagline alone, and Docker Hub receives it from the About panel rather than from the README. CODESTYLE.md item 4 and .gitattributes are intent, so nothing reports them, and the second couples to the new gate through the forward-declared mark. .github/copilot-instructions.md is intent and propagates a correction rather than a refresh, so a repo left on the old copy is wrong rather than merely stale.
ptr727 added a commit that referenced this pull request Aug 9, 2026
Backlog bookkeeping for a question that is settled, and for a number
that was doing two jobs.

#577 asked that the tagline be stated as a distinct thing from the
intro, so a README may carry a second paragraph without the
100-character cap or the mirrors reaching it. That is what shipped in
`c2ce145` (#632): `spec/readme-structure.md` item 1 states that further
paragraphs below the tagline are free prose read by no mirror,
`CODESTYLE.md` item 4 narrows the `HISTORY.md` mirror to the tagline
alone, and `spec/audit.py` extracts the first line of the intro region
under a self-test that names the two-paragraph case. #577 is closed
against that evidence.

Grepping the number before closing it found it in a second place. "The
Declared Repository Description" had given #577 to its README-to-About
hop entry, which is work the issue's body never covers, so closing the
issue on its own correct evidence would have left that entry pointing at
a closed number. The hop now carries #639 of its own and the entry cites
it.

The "Recorded for the Maintainer" bullet that held #577 behind the
declared description is deleted rather than rewritten, because the
action it recorded is taken. Its reasoning did not survive the check
either: the extraction rule it feared writing already existed for the
About and Docker Hub mirrors, so narrowing the changelog mirror to match
it was one line rather than a new parser, which is recorded on the
closing comment so it is not re-derived.

## Verification

`scripts/prose_lint.py`, `scripts/repo_gate.py` (`eol`, `eol-coverage`,
`sha-pin`), `editorconfig-checker` and `markdownlint-cli2` all pass on
the working tree. The file is CRLF and stayed CRLF, which
`editorconfig-checker` is the gate that would have caught.

Closes #577

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ptr727 added a commit that referenced this pull request Aug 9, 2026
… Gates (#643)

Three commits, all backlog bookkeeping and local tooling. No rule
changes meaning, and nothing here alters what CI gates.

**#640, the tagline question.** #577 asked that the tagline be a
distinct thing from the intro so a README may carry a second paragraph
without the cap or the mirrors reaching it. That shipped in `c2ce145`
(#632), so the issue is closed against it. Grepping the number before
closing found `TODO.md` had also given #577 to a different entry, the
README-to-About hop, which the issue's body never covers. That hop now
carries #639 of its own.

**#641, the divergence ledger.** `reports/divergences.md` is a live pass
over each repo's ground-truth branch, so a committed copy is only as
current as its last run. Regenerating moved three rows: `AGENTS.md`
"Fleet Bootstrap" is now divergent at Blog and HomeAutomation-Config,
and `GOVERNANCE.md` "Verification Discipline" and "Workflow YAML
Conventions" widened from one repo to four. The reason `TODO.md` gave
for regenerating did not survive the run and is corrected. Two spent
backlog claims went with it.

**#642, the pre-commit hook.** The chore was to reconsider whether the
hook should run the doc gates now that they are diff-scoped. Measuring
first found this repository had no hook at all, while `GOVERNANCE.md`
and `scripts/README.md` both described what one runs. There is now a
committed `.husky/pre-commit` running the prose gate diff-scoped and the
line-ending check repo-wide, with the `.gitattributes` and
`.editorconfig` pins that `GOVERNANCE.md` "Line Endings" makes mandatory
for an extensionless hook.

The formatting half the fleet convention names is deliberately absent,
and that is the finding worth carrying: this repository declares
`[tool.ruff]` in `pyproject.toml` and `spec/project-types.json` declares
`python.ruff.config`, yet no workflow runs ruff and the tree does not
pass it, at 13 of 57 files unformatted and 106 lint errors. A ruff step
would have blocked every commit from the moment it landed. The
`GOVERNANCE.md` bullet now states the condition rather than a tool list,
that a repo adds each half once its tree passes that half, and the
measurement is recorded as a `TODO.md` chore.

## Carried content in this promotion

`GOVERNANCE.md` "Running the Linters Locally" changed, and that section
is carried `verbatim`, so every downstream copy is byte-mismatched until
the next fleet visit. It rides the re-vendor entry already tracked under
`TODO.md` "Fleet Sweeps" rather than owing a sweep of its own.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants