Encode the README Structure, Links, and Tools as Auditable Rules - #632
Conversation
The README spec was advisory: `spec/project-types.json` declared `readme.sections` at verdict `intent`, so nothing mechanically checked section presence, order, shields, or link naming. Two wanted changes had also left the hub's own README deliberately ahead of the spec that governs it. ## The two changes **The License section closes the file** and carries the license shield, which 15 of the 16 repos with a License section already did. **The first line under the H1 is the tagline**, and it alone carries the 100-character link-free rule and the mirrors, so a README may now say more about itself above the fold. That second one was nearly free: the audit already took the first line for the About and Docker Hub mirrors, and only the `HISTORY.md` comparison read the whole intro region. Also settled, from a survey of all 22 non-archived non-fork repos: a Table of Contents in every README with no size threshold, `Use Cases` renamed to `Overview` with no aliases, `3rd Party Tools` required, and `Questions or Issues` required in a public repo. ## Made auditable `spec/readme-sections.json` declares the section order, the requirement per section, the additive shield classes, the five link groups, the reference-name suffixes, and the canonical names for destinations every repo has. `spec/third-party-tools.json` declares 40 shared tools so a tool used by two repos is linked and described identically in both. `spec/audit.py` reads both, and `readme.sections` moves from `intent` to `letter`. A heading the model does not name is dropped before the order comparison, so the ~60 genuinely repo-specific sections across the fleet are never findings. The tool catalog is a standard set rather than a complete one, so a tool only one repo uses is unaudited. ## What the corpus run changed Running the checks against all 22 READMEs before shipping them corrected the model three times. Keying shields on alt text flagged three repos for caption wording, since the fleet spells one badge five ways against one endpoint; matching moved to the shields.io endpoint. A `Coverage` class would have told 7 repos to add a badge nobody chose, since 10 declare `CODECOV_TOKEN` and 3 display one; it was dropped. Requiring both Docker version shields is wrong for NxWitness, which carries 40 across six images and four channels that are not `develop`. It also caught a plain bug: `extract_section` matches heading text exactly, so `## License <!-- omit from toc -->` was not the License section, and four repos suffix every heading that way. Remaining fleet backlog, recorded in `TODO.md`: 73 structure findings, 56 link-naming findings, 28 grouping drifts, and 55 tool-catalog findings. No downstream repo is edited here. The hub reports zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This pull request turns the README structure rules from advisory prose into declared, auditable requirements, and extends the audit tooling to mechanically verify README sections/order, shields, reference-link naming/grouping, and shared 3rd-party tool catalog consistency across the fleet.
Changes:
- Add declared models for README sections/shields/links (
spec/readme-sections.json) and shared tool catalog (spec/third-party-tools.json) plus JSON schemas. - Extend
spec/audit.pyto audit README sections/order, shield placement, reference-link rules, tool catalog intersection checks, and tagline-only mirroring. - Update project policy docs and this repo’s
README.mdto conform to the new auditable rules (tagline semantics, License placement/shield, link grouping, 3rd Party Tools section).
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| TODO.md | Updates backlog notes to reflect shipped README tagline/spec work and new remaining items. |
| spec/validate.py | Adds validation for spec/third-party-tools.json shape, uniqueness, sentence style, and sorting. |
| spec/third-party-tools.schema.json | Introduces JSON Schema for the shared third-party tool catalog. |
| spec/third-party-tools.json | Adds the shared third-party tool catalog (names/links/descriptions). |
| spec/readme-structure.md | Updates the prose spec to match the new declared/audited README model (tagline semantics, required sections, link groups, shield rules). |
| spec/readme-sections.schema.json | Introduces JSON Schema for the declared README section/shield/link model. |
| spec/readme-sections.json | Adds the declared README model consumed by the audit (sections, shield classes, link groups/naming, canonical links). |
| spec/project-types.json | Moves README checks from intent to letter and splits into specific checks (sections/shields/links/tools/tagline). |
| spec/audit.py | Implements mechanical README auditing: tagline extraction, section order/requirements, shield endpoint matching and placement, link definition naming/grouping, tool catalog intersection checks, plus selftests. |
| README.md | Updates this repo README to conform (canonical github-link, adds 3rd Party Tools section, adds Distribution link group). |
| GOVERNANCE.md | Updates repository description policy to reference the README “tagline” line and its mirror chain. |
| CODESTYLE.md | Updates HISTORY.md mirroring rule to mirror only the tagline rather than the whole intro paragraph. |
| AUDIT.md | Documents that README structure is now mechanically checked against the declared model. |
Suppressed comments (1)
spec/readme-sections.json:27
- The linkNaming note for kind "shield" says "A badge under img.shields.io", but the rule in spec/readme-structure.md defines a shield by how the reference is used (rendered as an image), not by its host. Updating the note avoids reintroducing the earlier host-based interpretation.
{ "kind": "shield", "suffix": "-shield", "note": "A badge under img.shields.io." },
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Copilot review of #632 found the audit contradicting the rule it had just written down. `spec/readme-structure.md` defines a shield as a reference the document renders as an image, "judged by how the document uses the reference, not by which host serves it", while `link_kind` short-circuited on an `img.shields.io` host before ever consulting that usage, and two `note` fields in `spec/readme-sections.json` still described the Shields group as holding badges "under img.shields.io". The short-circuit is removed. Measured across the fleet, all 119 img.shields.io definitions are rendered as images, so a host test classifies nothing usage does not and can only ever contradict the rule beside it. An A/B over all 22 READMEs with and without the short-circuit returns 82 link findings either way, in identical sets, so this changes what the code means without changing what any repo is told. A selftest case pins the direction that was previously unreachable: an `img.shields.io` definition the document never renders is an ordinary URI, so naming it `-shield` and filing it under Shields is now reported rather than accepted. The conformant fixture had itself been relying on the host short-circuit, defining a license shield it never rendered, and now renders it as a real README does. Also corrects four counts in `TODO.md` that had gone stale as this branch fixed the hub's own README: the structure backlog is 73 rather than 75, link naming 55 letters and 27 drifts rather than 56 and 28, and the tool catalog 54 across four repos rather than 55 across five. The tool split is 48 absent descriptions, 3 that disagree, and 3 links that disagree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (2)
TODO.md:506
- The
<!-- Repo -->reference definitions are no longer alphabetized by reference name (e.g.,third-party-toolsis placed betweenreadme-sectionsandreadme-structure). GOVERNANCE.md requires each link-definition group to be alphabetized by reference name (GOVERNANCE.md:166).
[prose-gate]: ./.github/actions/prose-gate/action.yml
[readme-sections]: ./spec/readme-sections.json
[third-party-tools]: ./spec/third-party-tools.json
[readme-structure]: ./spec/readme-structure.md
spec/validate.py:79
load("spec/third-party-tools.json")is assumed to return an object; if the JSON is malformed (e.g., top-level array),.get("tools")will raiseAttributeErrorand crash validation instead of reporting a clear shape error.
tools = load("spec/third-party-tools.json").get("tools")
Both fixes come from the suppressed half of the round-two Copilot
review, which is worth reading rather than trusting the suppression:
neither was noise.
`TODO.md` gained a `[third-party-tools]` reference inserted next to the
`[readme-sections]` line it relates to rather than at its sorted
position, which put it between `readme-sections` and `readme-structure`
and broke the alphabetical order GOVERNANCE.md requires of a link group.
It now sits after `[standup]`. Sweeping all 44 tracked Markdown files
for the same defect finds no other group out of order, so this was
isolated rather than a pattern.
`spec/validate.py` read `load("spec/third-party-tools.json").get(...)`,
which raises AttributeError on a malformed top level and crashes the
run. That is the opposite of what shape-checking there is for, since the
block exists so a bad file reports a clear message instead of taking
down the cross-reference loops below it. A bare-array top level now
reports two errors and exits 1, verified by substituting one and
restoring the file byte-identical afterwards.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round two generated no new threads, but its two suppressed comments were both real, and both are fixed in f0f9da8.
Both are the kind of finding worth reading past the suppression for, so thanks for surfacing them. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (2)
spec/audit.py:606
- readme_link_findings() (and the rendered-image detection it uses for shield classification) does not skip fenced code blocks. A Markdown code sample containing
<!-- ... -->,[ref]: url, or![alt][ref]will be misread as real link-group headers/definitions or rendered shields, producing false DRIFT/LETTER findings.
# Walk the definitions in order, tracking which group header each one falls under.
seen_headers, current, in_group = [], None, {}
for ln in normalize(text).split("\n"):
s = ln.strip()
h = re.fullmatch(r"<!--\s*(.*?)\s*-->", s)
if h and "omit from toc" not in h.group(1):
current = h.group(1)
seen_headers.append(current)
in_group.setdefault(current, [])
continue
d = _LINK_DEF.match(s)
if d:
in_group.setdefault(current, []).append((d.group(1), d.group(2)))
all_defs = [p for v in in_group.values() for p in v]
# A reference the document renders as an image is a shield whatever host serves it, so resolve those first.
by_ref = dict(all_defs)
rendered = {by_ref[m.group(1)] for m in _MD_IMAGE_REF.finditer(norm) if m.group(1) in by_ref}
spec/audit.py:519
- link_kind() currently classifies NuGet and PyPI URLs as "distribution" only when the package name is derived from the GitHub owner (nuget:
.../packages/<owner>., pypi:.../project/<owner>-). The new README link rules (spec/readme-structure.md / readme-sections.json canonicalLinks) describe NuGet/PyPI distribution links by host/path pattern, not by owner-derived naming, so this can misclassify a repo's own NuGet/PyPI links as external and prevent canonical naming / grouping enforcement for them.
if url.startswith("#"):
return "anchor"
if not url.startswith(("http://", "https://")):
return "local"
owner = slug.split("/")[0].lower()
own = (f"https://github.com/{slug}".lower(), f"https://hub.docker.com/r/{owner}/",
f"https://www.nuget.org/packages/{owner}.", f"https://nuget.org/packages/{owner}.",
f"https://pypi.org/project/{owner}-")
if url.lower().startswith(own):
return "distribution"
return "shield" if url in rendered else "external"
Both fixes come from the suppressed half of the round-three Copilot
review, which generated no new threads. Neither was noise.
`readme_link_findings` read the whole document for group headers, link
definitions, and rendered images, while `ordered_headings` and
`extract_section` had skipped fenced blocks all along. A README
documenting badge markup carries `[ref]: url`, `<!-- Shields -->` and
`![alt][ref]` as samples, and reading those invents a group, a
definition, and a rendered shield the document does not have. The walk
now skips fences and the rendered-image scan reads the unfenced text, so
one document is parsed one way. No fleet README trips this today,
measured across all 22, so this is a latent defect rather than a live
one.
`link_kind` also carried its own list of URL prefixes deciding what
counts as this project's own, while `canonicalLinks` matched NuGet and
PyPI by host and path alone. Ownership stated twice is two things that
can disagree, and the code's owner-scoping was the half doing the real
work: without it a dependency link to Serilog's package would be renamed
`nuget-link`. The prefixes move into the model as `distribution`, with
`{slug}` and `{owner}` placeholders, and `link_kind` reads them from
there.
Verified: the declared prefixes reproduce the removed hardcoded set
exactly for all 22 repos, and the corpus returns the same 82 link
findings as before. Four selftest cases pin the directions that were
unreachable, covering a fenced sample, a third-party package that keeps
its name, and an owner's own package that does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round three generated no new threads, and again both suppressed comments were real. Both fixed in 021fe19.
Verified rather than asserted: the declared prefixes reproduce the removed hardcoded set exactly for all 22 repos, and the corpus returns the same 82 link findings as before, so this changes what the code means without changing what any repo is told. Four new selftest cases pin the previously unreachable directions, including a third-party package that keeps its name and an owner-published one that does not. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (2)
spec/audit.py:588
normis computed but never used in readme_link_findings(). This is dead code and makes it harder to tell which normalized form the function actually operates on.
findings = []
norm = _HTML_COMMENT.sub(lambda m: "\x00" + m.group(0) + "\x00", normalize(text))
suffixes = {n["kind"]: n["suffix"] for n in model["linkNaming"]}
spec/audit.py:626
- In readme_link_findings(),
renderedis built as a set of URLs and link_kind() treats any definition whose URL is in that set as a shield. If two different references point at the same URL, a non-rendered reference can be misclassified as a shield just because another reference to the same URL is rendered. The spec language here is per-reference ("a badge ... the document renders as an image"), so classification should be based on whether that reference name is rendered as an image, not whether the URL appears elsewhere.
by_ref = dict(all_defs)
rendered = {by_ref[m.group(1)] for m in _MD_IMAGE_REF.finditer("\n".join(unfenced)) if m.group(1) in by_ref}
described = {"shield": "a badge", "anchor": "an in-page anchor", "local": "a path in this repo"}
for ref, url in all_defs:
kind = link_kind(url, slug, rendered, prefixes)
want = suffixes.get(kind, "")
Both fixes come from the suppressed half of the round-four Copilot review, which generated no new threads. `readme_link_findings` still computed `norm`, which the previous commit orphaned when the rendered-image scan moved to the unfenced text. It is removed. A normalized form nothing reads is worse than absent, because the next reader has to work out which of the two the function is actually operating on. `rendered` was a set of URLs, so `link_kind` called any definition whose URL appeared in it a shield. Two reference names may point at one URL, and only one of them may be rendered, in which case the plain link was classified a shield because its twin is a badge. The spec is worded per reference, "a badge, judged by the document rendering the reference as an image", so `rendered` now holds reference names and `link_kind` takes whether this one reference is rendered. No fleet README trips this today: of the 22, the only duplicated URL is PlexCleaner's Docker Hub page under two names, and neither is rendered. The new selftest case discriminates rather than passing vacuously, since the old URL keying returns two findings on it where the shipped code returns none. The corpus returns the same 82 link findings as before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round four generated no new threads, and both suppressed comments were real again. Both fixed in 7abab7a. That is six for six across rounds two through four, which is a fair argument for reading past the suppression every time.
Measured before fixing: no fleet README trips it today. Of the 22, the only duplicated URL is PlexCleaner's Docker Hub page under two names, and neither is rendered. So this was latent. I also checked the new selftest case discriminates rather than passing vacuously, by simulating the old URL keying against it: two findings under the old behaviour, none under the shipped one. The corpus returns the same 82 link findings as before, so this changes what the code means without changing what any repo is told. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (1)
spec/readme-sections.schema.json:6
distributionis optional in the schema, but the README link-audit logic relies ondistribution.urlPrefixesto classify a repo's own GitHub/Docker/NuGet/PyPI URLs asdistribution(which then drives canonical link naming and group validation). Ifdistributionis omitted, the audit silently stops enforcing canonical naming for those URLs and can misclassify the Distribution group contents. Makingdistributionrequired here prevents a broken model from slipping through schema validation.
"required": ["sections", "shieldClasses", "linkGroups", "linkNaming", "canonicalLinks"],
From the suppressed half of the round-five Copilot review, which
generated no new threads. The finding is a fail-open, which is the kind
worth fixing even with no live case.
`distribution` was optional in the schema while `link_kind` depends on
`distribution.urlPrefixes` to tell a repo's own GitHub, Docker Hub,
NuGet and PyPI URLs from a third party's. Omit the key and
`distribution_prefixes` returns empty, every own URL classifies as
external, canonical naming and Distribution-group validation quietly
stop being enforced, and the audit still reports green. A check that
stops checking without saying so is worse than one that was never
written, because the green is read as a result.
The schema now marks `distribution` required. That alone would not have
caught it, since CI runs no JSON-schema validation, so the guard that
actually runs is a shape check in `spec/validate.py`, beside the one for
the tool catalog. It covers two failure modes: absent or empty prefixes,
and prefixes carrying neither `{slug}` nor `{owner}`, which would not be
repo-scoped and would match another owner's URLs.
Verified by removing the key and by flattening the prefixes to a bare
host, each reporting a named error and exiting 1, then restoring the
file byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round five generated no new threads, and its single suppressed comment was real. Fixed in 68bcde3. That is seven for seven across rounds two through five.
The schema now marks it required. That alone would not have caught it, since CI runs no JSON-schema validation, so the guard that actually runs is a shape check in Verified by inducing both: removing the key, and flattening the prefixes to a bare Worth naming the pattern across this loop, since five of the seven were the same defect shape: the implementation quietly disagreeing with the declared rule beside it, whether that was host versus usage, ownership defined twice, URL versus reference, or now a required input the schema called optional. Moving rules into declared data does not by itself make the code obey them. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (1)
spec/audit.py:526
- link_kind() treats any non-http(s) URL as a repo-local path. That misclassifies other valid URI schemes (e.g. mailto:, ssh:, ftp:) as "local", which would make the audit enforce bare reference names and the Repo link group for those URIs. Treat scheme-based non-http(s) URLs as external URIs instead, while keeping relative paths as local.
if url.startswith("#"):
return "anchor"
if not url.startswith(("http://", "https://")):
return "local"
From the suppressed half of the round-six Copilot review, which generated no new threads. `link_kind` treated every non-http target as a repo-local path, so a `mailto:`, `ssh:` or `ftp:` reference would have been told to drop its `-link` suffix and move to the Repo group, which holds files. A contact address in Questions or Issues is the obvious case. No fleet README carries a scheme-bearing reference today, measured across all 22, so this is a shape the rule has to get right before one arrives rather than a defect being repaired. A reference is local only when it carries no URI scheme. The scheme test requires two or more characters so a `C:` drive letter is not read as one, and every scheme a README actually carries is longer than that. Verified across the boundary: `./LICENSE`, `LICENSE`, `docs/x.md`, `../a/b.md` and `C:/win/path` stay local, `#anchor` stays an anchor, `mailto:` and `ftp://` become external, and this repo's own URL still classifies as distribution. The corpus returns the same 82 link findings as before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
From the round-eleven Copilot review, generalizing an earlier finding rather than repeating it. `spec/validate.py` shape-checked `distribution` and `canonicalLinks` while `spec/audit.py` indexes `sections`, `shieldClasses`, `linkGroups` and `linkNaming` directly too. Removing any of those four left validation exiting 0 and the audit raising a KeyError mid-run. The schema marks each required, but CI runs no JSON-schema validation, so the schema was never the guard that ran. The four unguarded keys were unguarded for no better reason than that the earlier reviews had named the other two, so the keys are now checked as a set rather than one at a time, which is what stops the next added key from being the next gap. Verified by removing each of the six in turn: every one reports a named error and exits 1, where before four of them exited 0. Removing `linkNaming` and calling the audit directly raises `KeyError: 'linkNaming'`, which is the failure this converts into a report. The model was restored byte-identical after each. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (3)
spec/audit.py:420
- shield_endpoints() scans the raw region text for image markup, so an
![alt][ref](or inline image) inside a fenced code block would be counted as a rendered shield. That can let a README satisfy required badges via a code sample, or tripexclusivechecks, even though fenced blocks are explicitly treated as non-semantic elsewhere (e.g., ordered_headings()/unfenced_text()).
def shield_endpoints(region, defs):
"""The image URLs a Markdown region renders, from both its `![alt][ref]` and its `` uses.
Keyed on the endpoint rather than the alt text or the reference name, because those are captions. The
fleet writes Release Status, Releases Build, Build Status, Workflow Status and Lint Build for one badge,
and names its reference `last-commit-shield` and `lastcommit-shield` in the same breath, while the
endpoint under img.shields.io is identical in every repo.
Inline uses are resolved as well as reference ones. Reading references alone made an inline shield
invisible rather than wrong, so a repo writing every badge inline, which the reference-link rule forbids
for a separate reason, would have passed the shield check by carrying nothing the check could see.
A `[ref]: url` definition line is not an image, so it is never counted as a use, which is what keeps a
shield's own definition from reading as a second placement of it.
"""
region = region or ""
urls = [defs[m.group(1)] for m in _MD_IMAGE_REF.finditer(region) if m.group(1) in defs]
return urls + [m.group(1) for m in _MD_IMAGE_INLINE.finditer(region)]
spec/audit.py:750
- readme_shield_findings() builds
defsfrom normalize(text), which includes fenced blocks. That means a[ref]: urlinside a code sample can be treated as a real badge definition and (for deprecatedShields) flagged even if it is only shown as an example. Other README parsing in this file explicitly uses unfenced_text() to avoid this.
findings = []
defs = {m.group(1): m.group(2) for m in _LINK_DEF.finditer(normalize(text))}
# A retired badge service is scanned across the whole document rather than per section, since a dead badge is wrong wherever it sits.
# It renders broken rather than absent, which a visitor reads as a failing build rather than as a stale badge.
for dep in model.get("deprecatedShields", []):
for ref, url in sorted(defs.items()):
if dep["match"] in url:
findings.append(("LETTER", f"readme: `[{ref}]` renders {dep['label']}, which is retired - {dep['reason']} (spec/readme-structure.md)"))
spec/audit.py:708
- third_party_tool_findings() iterates over the raw extracted section body, so a fenced code sample inside
## 3rd Party Toolsthat happens to contain a tool table/bullet could be interpreted as real entries and produce false findings. Since the link/definition parsing is already fence-aware via unfenced_text(), the body scan should be too.
body = readme_region(text, "3rd Party Tools")
if body is None:
return [] # the absent section is already one finding from readme_section_findings
declared = {t["name"].lower(): t for t in catalog["tools"]}
From the suppressed half of the round-twelve Copilot review, three findings of one class, and the same generalization failure as round eleven: fence-awareness was added where a review had pointed, and left everywhere else. `readme_region` did not strip fences, so every region built from it inherited that. A badge shown as a code sample could satisfy a required shield, a tool table shown as a sample could be read as entries, and a `[ref]: url` sample could supply a definition. `readme_shield_findings` read its definition map from the whole document for the same reason, so `deprecatedShields` would flag a retired badge that a README was only demonstrating. Fixed at the root rather than at the three call sites. `readme_region` strips fences, so every region the README checks read is fence-free by construction, and the two whole-document scans, the shield definition map and the exclusive-placement scan, read `unfenced_text`. The exclusive scan was not among the three reported and is the same defect. `ordered_headings` carried its own fence loop, a second definition of the idea that let the two drift apart in the first place, and now shares the one helper. Verified: the corpus is unchanged at 71 section, 5 shield, 55 link letter, 27 link drift and 55 tool findings, and this repo still reports zero. Three selftest cases were each checked against the reverted behaviour rather than assumed to discriminate, and the first attempt at that check was wrong, reverting only `readme_region` and leaving the scan it did not cover, which made a real case look inert. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round twelve passed CI with no threads, and all three suppressed comments were real. Fixed in 95978bf. Nineteen for nineteen. All three are one class, and it is the same generalization failure round eleven found: fence-awareness was added where a review had pointed and left everywhere else. Fixed at the root rather than at the three reported call sites:
The exclusive-placement scan was not among the three you reported and is the same defect, so it is fixed here too. Finding it required listing every text-parsing site in the file rather than fixing the ones named, which is the lesson I should have taken from round eleven and did not. Verified: the corpus is unchanged at 71 section, 5 shield, 55 link letter, 27 link drift and 55 tool findings, and this repo still reports zero. Each of the three new selftest cases was checked against the reverted behaviour rather than assumed to discriminate, and my first attempt at that check was wrong: it reverted only |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (2)
spec/audit.py:728
- third_party_tool_findings() only validates cataloged tools' link/description, but spec/readme-structure.md states the 3rd Party Tools list is alphabetized. As written, a README can be non-alphabetized and still pass the readme.tools check. Consider checking that the tool entries in the section are in case-insensitive sorted order (across all tools, not just those present in the shared catalog).
findings = []
for ln in body.split("\n"):
m = _TOOL_ROW.match(ln.strip()) or _TOOL_BULLET.match(ln.strip())
if not m:
continue
name, ref, desc = m.group(1), m.group(2), m.group(3).strip(" -:")
spec/audit.py:642
- readme_link_findings() intends to avoid treating directive comments like
<!-- markdownlint-disable ... -->as link-group headers (see the comment on the preceding lines), but the current logic treats any HTML comment line (exceptomit from toc) as a header. If a README places a markdownlint directive inside the reference-definition block (before some[ref]: urllines), those definitions will be mis-grouped under the directive and reported as an unknown link group.
h = re.fullmatch(r"<!--\s*(.*?)\s*-->", s)
if h and "omit from toc" not in h.group(1):
current = h.group(1)
continue
From the suppressed half of the round-thirteen Copilot review, which generated no new threads and passed CI. `spec/readme-structure.md` says the 3rd Party Tools list is alphabetized and nothing checked it, so a rule was stated and unenforced, which is the gap this whole change exists to close. The check covers every tool listed rather than only the cataloged ones, since a reader scans the whole list. It reports one repo: Blog, which orders Hugo, PaperMod, Caddy. The directive-comment fix in 6e81742 only held where no definition followed. A `<!-- markdownlint-disable -->` placed inside the reference block does have definitions under it, so those would be grouped under the directive and reported as an unknown link group, and the code comment claimed otherwise. Directives are now skipped outright, from a `directiveComments` list in the model rather than inferred from the shape of the text, since which tools a repo instructs is a fact about the fleet rather than something to guess at. No fleet README carries a directive comment today, measured across all 22, so that half is a shape the rule has to get right before one arrives. Verified: both new selftest cases were checked against the reverted behaviour, the directive case by emptying the declared list and the ordering case against its own absence. The corpus moves only by the one Blog finding, from 55 tool findings to 56, and this repo still reports zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The alphabetization check added in a64c0d0 reports one repo, so the backlog entry reads 56 rather than 55 and names what the extra finding is: Blog listing Hugo, PaperMod, Caddy out of order. A committed count that no longer matches what the tool reports is the claim this file exists to make checkable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Round thirteen passed CI with no threads, and both suppressed comments were real. Fixed in a64c0d0, with the backlog count carried in c702d85. Twenty-one for twenty-one.
Verified: both cases were checked against the reverted behaviour, the directive one by emptying the declared list and the ordering one against its own absence. The corpus moves only by the single Blog finding, 55 tool findings to 56, and this repo still reports zero. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Suppressed comments (2)
spec/audit.py:716
- third_party_tool_findings() does not enforce the spec/readme-structure.md rule that the "3rd Party Tools" table carries no License column. Because _TOOL_ROW only parses the first two columns, a 3-column table (e.g., "| Tool | Role | License |") would currently pass this check silently.
_TOOL_ROW = re.compile(r"^\|\s*\[([^\]]+)\]\[([^\]]+)\]\s*\|\s*([^|]*?)\s*\|")
_TOOL_BULLET = re.compile(r"^[-*]\s*\[([^\]]+)\]\[([^\]]+)\]\s*(.*)$")
def third_party_tool_findings(text, catalog):
spec/validate.py:90
- validate.py now checks that spec/readme-sections.json top-level keys exist and have the right container type, but it still doesn't validate the per-item shapes that spec/audit.py indexes directly (e.g., linkGroups items need name/holds, shieldClasses items need trigger/shields, etc.). Since CI does not run JSON schema validation, a malformed item would still crash spec/audit.py with KeyError/TypeError rather than being reported by validate.py.
for key, want in (("sections", list), ("shieldClasses", list), ("linkGroups", list),
("linkNaming", list), ("canonicalLinks", list), ("distribution", dict)):
value = readme_model.get(key)
if not isinstance(value, want) or not value:
errors.append(f"readme-sections.json: '{key}' must be a non-empty {'array' if want is list else 'object'}, and spec/audit.py indexes it directly")
…ate (#635) Promotion of develop at 2082547, six squashes since the last one. Closes #633. #631 routes the README by reader and documents the GH_WRITE_GUARD_ALLOW grant where a denied cross-owner write puts the reader. #632 moves readme.sections from intent to letter with four checks beside it, backed by spec/readme-sections.json and spec/third-party-tools.json, and settles the tagline rule. #634 adds repo_gate.py --check eol-coverage, reading the line-ending pins against the tree rather than only against .editorconfig. #636 and #637 repair two readers Copilot found on this pull request, both defects develop already carried: a tool row required both outer table pipes that GitHub's Markdown makes optional, and a retired badge written as an inline image was invisible to a scan that read reference definitions alone. #638 turns the host contract's presence check into a version gate, and retires two gh workarounds that were artifacts of a stale distribution package, re-tested on an upgraded host rather than inferred. Four carried files owe a downstream re-vendor and none is recorded in the TODO.md entry yet. GOVERNANCE.md Repository Details is verbatim, so the audit reports it, and it propagates a rule: the About description is the tagline alone, and Docker Hub receives it from the About panel rather than from the README. CODESTYLE.md item 4 and .gitattributes are intent, so nothing reports them, and the second couples to the new gate through the forward-declared mark. .github/copilot-instructions.md is intent and propagates a correction rather than a refresh, so a repo left on the old copy is wrong rather than merely stale.
Backlog bookkeeping for a question that is settled, and for a number that was doing two jobs. #577 asked that the tagline be stated as a distinct thing from the intro, so a README may carry a second paragraph without the 100-character cap or the mirrors reaching it. That is what shipped in `c2ce145` (#632): `spec/readme-structure.md` item 1 states that further paragraphs below the tagline are free prose read by no mirror, `CODESTYLE.md` item 4 narrows the `HISTORY.md` mirror to the tagline alone, and `spec/audit.py` extracts the first line of the intro region under a self-test that names the two-paragraph case. #577 is closed against that evidence. Grepping the number before closing it found it in a second place. "The Declared Repository Description" had given #577 to its README-to-About hop entry, which is work the issue's body never covers, so closing the issue on its own correct evidence would have left that entry pointing at a closed number. The hop now carries #639 of its own and the entry cites it. The "Recorded for the Maintainer" bullet that held #577 behind the declared description is deleted rather than rewritten, because the action it recorded is taken. Its reasoning did not survive the check either: the extraction rule it feared writing already existed for the About and Docker Hub mirrors, so narrowing the changelog mirror to match it was one line rather than a new parser, which is recorded on the closing comment so it is not re-derived. ## Verification `scripts/prose_lint.py`, `scripts/repo_gate.py` (`eol`, `eol-coverage`, `sha-pin`), `editorconfig-checker` and `markdownlint-cli2` all pass on the working tree. The file is CRLF and stayed CRLF, which `editorconfig-checker` is the gate that would have caught. Closes #577 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… Gates (#643) Three commits, all backlog bookkeeping and local tooling. No rule changes meaning, and nothing here alters what CI gates. **#640, the tagline question.** #577 asked that the tagline be a distinct thing from the intro so a README may carry a second paragraph without the cap or the mirrors reaching it. That shipped in `c2ce145` (#632), so the issue is closed against it. Grepping the number before closing found `TODO.md` had also given #577 to a different entry, the README-to-About hop, which the issue's body never covers. That hop now carries #639 of its own. **#641, the divergence ledger.** `reports/divergences.md` is a live pass over each repo's ground-truth branch, so a committed copy is only as current as its last run. Regenerating moved three rows: `AGENTS.md` "Fleet Bootstrap" is now divergent at Blog and HomeAutomation-Config, and `GOVERNANCE.md` "Verification Discipline" and "Workflow YAML Conventions" widened from one repo to four. The reason `TODO.md` gave for regenerating did not survive the run and is corrected. Two spent backlog claims went with it. **#642, the pre-commit hook.** The chore was to reconsider whether the hook should run the doc gates now that they are diff-scoped. Measuring first found this repository had no hook at all, while `GOVERNANCE.md` and `scripts/README.md` both described what one runs. There is now a committed `.husky/pre-commit` running the prose gate diff-scoped and the line-ending check repo-wide, with the `.gitattributes` and `.editorconfig` pins that `GOVERNANCE.md` "Line Endings" makes mandatory for an extensionless hook. The formatting half the fleet convention names is deliberately absent, and that is the finding worth carrying: this repository declares `[tool.ruff]` in `pyproject.toml` and `spec/project-types.json` declares `python.ruff.config`, yet no workflow runs ruff and the tree does not pass it, at 13 of 57 files unformatted and 106 lint errors. A ruff step would have blocked every commit from the moment it landed. The `GOVERNANCE.md` bullet now states the condition rather than a tool list, that a repo adds each half once its tree passes that half, and the measurement is recorded as a `TODO.md` chore. ## Carried content in this promotion `GOVERNANCE.md` "Running the Linters Locally" changed, and that section is carried `verbatim`, so every downstream copy is byte-mismatched until the next fleet visit. It rides the re-vendor entry already tracked under `TODO.md` "Fleet Sweeps" rather than owing a sweep of its own. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
The README spec was advisory:
spec/project-types.jsondeclaredreadme.sectionsat verdictintent, so nothing mechanically checked section presence, order, shields, or link naming. Two wanted changes had also left this repo's own README deliberately ahead of the spec that governs it.The two changes
The License section closes the file and carries the license shield, which 15 of the 16 repos with a License section already did. The first line under the H1 is the tagline, and it alone carries the 100-character link-free rule and the mirrors, so a README may now say more about itself above the fold.
That second one was nearly free. The audit already took the first line for the About and Docker Hub mirrors (
intro.split("\n")[0]), and only theHISTORY.mdcomparison read the whole intro region, so the change is one line plus the prose.TODO.md's claim that this was blocked on the registry field, "since taking this first means writing an extraction rule the registry change deletes", overstated the coupling, and that note is corrected here.Settled alongside them, from a survey of all 22 non-archived non-fork repos: a Table of Contents in every README with no size threshold,
Use Casesrenamed toOverviewwith no accepted aliases,3rd Party Toolsrequired, andQuestions or Issuesrequired in a public repo.Made auditable
spec/readme-sections.jsondeclares the section order and per-section requirement, the additive shield classes, the five link groups, the reference-name suffixes, and the canonical names for the destinations every repo has.spec/third-party-tools.jsondeclares 40 shared tools, so a tool two repos use is linked and described identically in both.spec/audit.pyreads both, andreadme.sectionsmoves fromintenttoletter.Two design points keep this from over-reaching:
What the corpus run changed
Running the checks against all 22 READMEs before shipping them corrected the model three times, which is the point of measuring first:
Release Status,Releases Build,Build Status,Workflow StatusandLint Buildagainst a single endpoint. Matching moved to the shields.io endpoint, withrequireQuery/forbidQueryseparating the two shields that share one.Coverageshield class would have told 7 repos to add a badge nobody chose: 10 repos declareCODECOV_TOKENand 3 display a badge. Dropped.develop. The class now asserts a version shield exists; the latest-plus-develop pair stays in the spec as the single-image form.It also caught a plain bug:
extract_sectionmatches heading text exactly, so## License <!-- omit from toc -->was not the License section, and four repos suffix every heading that way. That alone was 12 findings for sub-sections that were present.A later pass found two more, both fixed: keying "shield" on
img.shields.ioalone asked three repos to rename abyob.yarr.isbadge to-link, a rename away from the convention; and matching Docker Hub by host alone told ESPHome-Config to rename its link to upstream's image.Verification
python3 spec/audit.py --selftestpasses, with 50 new cases covering sections, shields, links, and the tool catalog.python3 spec/validate.pypasses, and now shape-checks the tool catalog: a duplicate name would silently shadow, since the audit keys on it.prose_lint.py,editorconfig-checker,markdownlint-cli2(44 files), andcspellall clean. CRLF preserved;spec/audit.pyandspec/validate.pystay LF.Fleet consequences, not carried here
No downstream repo is edited by this change.
TODO.mdrecords the backlog each repo works off on its next visit: 73 structure findings, 56 link-naming letters, 28 grouping drifts, and 55 tool-catalog findings.Two items need a maintainer's eye:
GOVERNANCE.md"Repository Details" is averbatimcarried section, so this edit puts all 21 downstream repos into verbatim drift until re-vendored. That is the normal propagation path for a rule change, but it is fleet-wide rather than hub-local.byob.yarr.islast-build badge is retired. Three repos render it and two already serve HTTP 404, and PlexCleaner lists it as a tool besides.deprecatedShieldsreports it so it does not depend on anyone remembering.