Split AGENTS.md into a router, and prove the scripts/ gates by ptr727 · Pull Request #455 · ptr727/ProjectTemplate

ptr727 · 2026-07-30T00:47:37Z

Five commits. The first three were held back for isolated testing; the last two are the gate work.

What this carries

AGENTS.md becomes a thin router (87,457 -> 7,789 bytes) with the rule text moving to GOVERNANCE.md. docs/token-cost.md records the measurement behind it: re-reading the pre-split file cost $273 over 216 reads, and only 36.9% of those reads used a range.

The scripts/ gates get tests, because they landed with none and running them turned up five defects that all report success while doing nothing:

Defect	Evidence
`SUGGEST` omitted U+21D2, U+2264, U+2265, three characters the charset rule names	`--check ascii` exited 0 on all three
`DUP_ALLOW` carried `the the` while the comment beside it claimed it was flagged	exited 0 on `the the thing`
Discovery reached 337 files, 246 of them `.mypy_cache` and `.artifacts`	106 tracked files match
`lstrip("./")` took a character set, not a prefix, so every dot-path normalized to a key no git diff emits	`--diff` was blind to all of `.github/`
Writing the charset table put 14 literal non-ASCII characters into the file the rule scans	caught, escaped, now asserted

The first and third are the failure mode Verification Discipline already names, live in this repo own gate. The third also violates the line-endings rule, which requires git ls-files over a raw walk and names those two directories.

The gates were watched failing

Removing U+2264 from the table fails two independent cases: the coupling case with [] != ["U+2264"], naming the codepoint rather than a count, and the floor at 13 not >= 14. Breaking the workflow glob makes repo_gate --check sha-pin print 0 issue(s) - a clean pass - while three cases object, including 0 not >= 4.

The charset table is driven by the codepoints parsed out of GOVERNANCE.md rather than a copy of them, because bait built from a table cannot notice the table is incomplete.

Scope and posture

Scope is now every text file git tracks, binaries skipped by a NUL-byte check. An extension allowlist covers what its author thought of and stops covering whatever is added next, the same argument that governs directory lists. That reaches .editorconfig, the .jsonc configs, and the husky snippet for the first time.

CI gates the self-tests, both repo_gate checks, and the charset and duplicate-word rules, all clean tree-wide. The semicolon rule stays warn-only: going green needs the sweep the prose rule itself forbids.

Notes for review

reports/homeassistant-purpleair/audit.md classified U+2264 and U+2265 as the scientific carve-out, contradicting the charset rule naming both must-replace. The classification is corrected and all eight characters are named by codepoint.
The 26 semicolon splices this branch introduced are recast in their own commit. They are prose written this week, not the legacy backlog, so the fix-as-edited carve-out does not cover them.
Sections declared verbatim are edited here. Doing it pre-push costs no downstream staleness, since nothing has vendored them yet.
Still to come, deliberately not here: the British-to-US spelling rule, the tiered charset rule, and tokenize-based comment extraction.

🤖 Generated with Claude Code

AGENTS.md was 87 KB and read whole 216 times across two months of agent sessions. Because a session re-reads its whole context on every request, each of those reads was re-billed for the rest of the session: measured at $273 of a $3,465 cache-read bill for this one file, with a median of 199 requests still to run after the read. The rule text cannot simply be trimmed - 16 of its 18 sections are declared fidelity verbatim and carried byte-identically to the fleet - so the file is split instead. AGENTS.md keeps only what every task needs and a map from a task to the section that governs it, and GOVERNANCE.md carries the topical rule text. All 18 carried sections move byte-identically, so no verbatim drift wave follows. Repository Layout is the one edited section, and it is fidelity intent. The split stays agent-agnostic deliberately. A Claude-specific pointer file would have solved the same problem for one harness, while Codex and Copilot read AGENTS.md too. Adds the Context and Delegation Discipline section: session scope, reading by range, bounding command output, and delegation including a brief shape that keeps a subagent from reading a governance file to find its own rules. Also makes an existing documented guarantee real. section-model.md claimed validate.py proves every declared section resolves in the hub file; it never opened the file. It does now, so a renamed or mistyped section name fails loud instead of silently ceasing to be checked anywhere. Held on a branch: the manifest and audit changes re-vendor across the fleet, so this is for isolated testing before any promotion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

markdownlint, cspell, actionlint, and editorconfig-checker all pass on prose that breaks a documented Documentation Style Conventions rule, so nothing enforced those rules and a reviewer kept finding them by hand. An audit of 1,047 Copilot findings on this repo put roughly half in categories a deterministic check can catch before a push. prose_lint.py covers the prose rules: substitutable typographic Unicode, a semicolon joining two independent clauses, and a duplicated word. It scopes to changed lines by default, matching the standing rule that existing prose is corrected as each file is next edited rather than swept. Replayed against the exact strings Copilot quoted in #437, #413, and #423 it catches all of them, and it does not fire on a list semicolon or a C-style for loop. The two documented non-ASCII exceptions are deliberately not flagged. repo_gate.py covers two more: every workflow `uses:` is a 40-hex SHA, and every .gitattributes LF pin has the matching .editorconfig override the line-ending rule requires. A third check, a stale-backticked-path detector, was written and rejected - a template repo legitimately cites paths that live in downstream repos, so it produced 34 false positives on a clean tree. pr_review.py collapses a Copilot poll cycle into one invocation. Polling was 4,513 measured `gh` calls at exactly one call per agent turn, and since each turn re-bills the whole session context, the round-trips rather than the bytes are the cost. It is read-only: mutations stay explicit `gh` calls so they remain visible to the gh-write-guard hook. Hub-only, not declared in spec/files.json, on the same footing as spec/audit.py. Promoting one to fleet-carried is a separate deliberate act. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The Context and Delegation Discipline rules were derived from a measurement, not from preference, and the measurement lived only in a local scratch folder. Without it those rules read as unmotivated and a later agent deletes them. Records what was measured over 220 transcripts and 383 pull requests: the cost equation, the 7.05 billion prompt tokens behind it, the distribution that put 55% of spend on requests carrying over 600k of context, and the counterfactual that made session scope a rule. Also records the two findings that inverted an obvious fix - output verbosity is 0.30% of prompt volume so terser prose is not the lever, and `gh` output already averages 574 bytes so the round-trips rather than the bytes are the cost. States plainly what is not settled: model tiering has obvious headroom but no measured saving, fresh-context self-review was tried and underperformed the external reviewer, and the prose gate is warn-first until its hits are compared against a real batch of findings. Adds `docs/` to Repository Layout, which never listed it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…catch The three scripts landed with no tests, and running them turned up five defects that all report success while doing nothing, which is the failure mode Verification Discipline names. `SUGGEST` omitted U+21D2, U+2264, and U+2265, three characters the charset rule explicitly names, so `--check ascii` exited 0 on all of them. `DUP_ALLOW` carried `the the` while the comment beside it claimed the phrase was still flagged. Discovery reached 337 files of which 246 were `.mypy_cache` and `.artifacts`, against a line-endings rule that already requires `git ls-files` over a raw walk and names those two directories. `lstrip('./')` took a character set rather than a prefix, so every dot-path normalized to a key no git diff emits and `--diff` was blind to all of `.github/`. Writing the charset table the first time put 14 literal non-ASCII characters into the file the rule scans. Each fix carries the case that would have caught it. The charset table is driven by the codepoints parsed out of GOVERNANCE.md rather than a copy of them, because a bait built from the table cannot notice the table is incomplete. Both modules assert a floor on what a healthy run reaches: with the workflow glob broken, `repo_gate --check sha-pin` prints `0 issue(s)` and reads exactly like a clean tree. Scope is now every text file git tracks, binaries skipped by a NUL-byte check. An extension allowlist covers what its author thought of and stops covering whatever is added next, which is the same argument that governs directory lists. That reaches `.editorconfig`, the `.jsonc` configs, and the husky snippet for the first time. A double-quoted span in markdown is read as a quotation, so a rule that states its own counter-example no longer reports the document that documents it. `reports/homeassistant-purpleair/audit.md` classified U+2264 and U+2265 as the scientific carve-out, which contradicts the charset rule naming both must-replace. The classification is corrected and all eight characters are named by codepoint, the form the rule itself uses. CI gates the self-tests, both repo_gate checks, and the charset and duplicate-word rules, all clean tree-wide. The semicolon rule stays warn-only, since going green needs the sweep the prose rule itself forbids. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The split authored GOVERNANCE.md fresh, so the 26 splices in it are not the legacy backlog the "corrected as each file is next edited" carve-out covers - they are prose written this week, against a rule stated a few lines away from most of them. Each one becomes two sentences, or a comma where the clauses are short and closely linked, which is what the rule itself prescribes. The sections involved are declared verbatim, so downstream copies would go stale on an edit; doing it before the branch is pushed costs nothing, because nothing has vendored them yet. The bait strings in the test module are assembled from two literals so the file does not hold the constructions it feeds the gate. A test full of rejected input otherwise reports itself, which is the same self-flagging problem the quoted-span exemption solves for prose. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR restructures the repo’s governance documentation to reduce agent context load by splitting AGENTS.md into a router and moving the full rule text into a new GOVERNANCE.md, and it hardens the scripts/ “gates” by adding deterministic self-tests and wiring them into CI.

Changes:

Introduce GOVERNANCE.md as the authoritative rules document; convert AGENTS.md into a thin entry-point/router and update cross-references across docs/spec/audit.
Add new repo gate scripts (prose_lint.py, repo_gate.py, pr_review.py) plus unit tests that prove each gate fails when it should.
Enforce the new script gates in CI and extend Python type-check scope to include scripts/.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
WORKFLOW.md	Update governance references from AGENTS to GOVERNANCE
spec/validate.py	Enforce that declared sections resolve to real H2 headings
spec/section-model.md	Update section model for split AGENTS/GOVERNANCE layout
spec/readme-structure.md	Point “Repository Details” authority to GOVERNANCE
spec/project-types.json	Update spec intentRefs to GOVERNANCE where applicable
spec/files.json	Split baseline section declarations across AGENTS + GOVERNANCE
spec/fidelity-model.md	Update workflow interface authority reference to GOVERNANCE
spec/divergences.json	Update divergence ledger examples to GOVERNANCE section keys
spec/audit.py	Extend carried-doc scans and references to include GOVERNANCE
scripts/test_repo_gate.py	Add unit tests proving repo_gate checks fail correctly
scripts/test_prose_lint.py	Add unit tests proving prose_lint rules fail correctly
scripts/repo_gate.py	New deterministic repo gate checks (sha-pin, eol)
scripts/README.md	Document hub-only scripts and how to run them
scripts/prose_lint.py	New prose rules gate (ascii/semicolon/dupword/sentence-split)
scripts/pr_review.py	New compact PR review-status polling helper
reports/homeassistant-purpleair/audit.md	Correct charset carve-out classification notes
repo-config/README.md	Update Branching Model references to GOVERNANCE
README.md	Update links to governance sections and anchors
pyproject.toml	Include `scripts/` in mypy/pyright scopes
host-setup/agent-safety/README.md	Update write-safety doc references to GOVERNANCE
host-setup/agent-safety/gh-write-guard.py	Update prose references to GOVERNANCE write-safety rules
host-setup/agent-safety/claude-md-safety.md	Update prose references to GOVERNANCE write-safety rules
GOVERNANCE.md	New consolidated governance rules document
docs/token-cost.md	New measurement doc motivating the governance split
docs/repo-config-carry.md	Update governance references from AGENTS to GOVERNANCE
CODESTYLE.md	Update Line Endings reference to GOVERNANCE
AUDIT.md	Update Branching/Review etiquette link targets to GOVERNANCE
AGENTS.md	Become a router: context/delegation + map to GOVERNANCE sections
.github/workflows/validate-task.yml	Run script self-tests + gates in CI
.github/copilot-instructions.md	Update canonical references to GOVERNANCE sections
.gitattributes	Pin LF for scripts and their tests
.editorconfig	Pin LF for `scripts/*.py`

Comments suppressed due to low confidence (1)

scripts/repo_gate.py:26

This comment still references AGENTS.md as the source of the sha-pin exception set, but that documentation is now in GOVERNANCE.md. Keeping this accurate matters because test_repo_gate.py couples the exception set to the doc text.

# AGENTS.md documents exactly one floating-ref exception.
SHA_EXCEPTIONS = {'dotnet/nbgv'}

Review flagged four stale citations. The class is larger: the split left AGENTS.md owning Context and Delegation Discipline and Where the Rules Live, so every reference citing it for git, branching, release, doc style, line endings, write safety, linter invocations, or review etiquette named the wrong file. 25 references across 13 files, found by sweeping for the class rather than patching the sampled instances. Two were self-contradicting. WORKFLOW.md said GOVERNANCE.md is authoritative where the two overlap and then handed the tie to AGENTS.md, and README.md used link text naming one file over an anchor pointing at the other. The README doc list had no GOVERNANCE.md entry at all, while its AGENTS.md entry described GOVERNANCE.md's contents. Both are now listed for what they hold. `spec/project-types.json`'s cspell assertion points at CODESTYLE.md, which is where the extension/CLI/CI parity rule for spelling actually lives. Removing the last `[agents]` uses orphaned four reference definitions, which markdownlint MD053 caught - the failure the documentation rule about removing a link's definition describes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 33 out of 33 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

GOVERNANCE.md:137

The documentation-style rules still treat only AGENTS.md and .github/copilot-instructions.md as the agent-instruction files, and the carried-files list omits GOVERNANCE.md. Since this PR introduces GOVERNANCE.md as a carried governance document, both lists should include it to keep the rule text accurate and self-consistent.

- **Carried files carry no coordination references.** In the files the fleet carries - `AGENTS.md`, `CODESTYLE.md`, `WORKFLOW.md`, `.github/copilot-instructions.md`, the `repo-config/` and `spec/` files, the carried `AUDIT.md` - two things are banned. **Any reference to the template repo**, in prose or in a link: it is private, so a link 404s for a downstream repo's users, and the coordination flow is machinery a consumer of that repo should never have to see. Where a carried file must express a template-level behavior - "report a rule discrepancy upstream" - state the behavior rather than the destination. The maintainer supplies the destination out of band. And **a sibling fleet repo named as an illustrative example** ("repo X does it this way", "see repo Y's adoption"), which couples the repos and rots as they diverge. To point at a current good example, name it in the onboarding or conformance issue, never in a carried doc. **A contextually relevant link to a related project is not a coordination reference, and is expected.** Where another repo is part of this repo's subject matter - the image that consumes this config, the builder that generates this hardware, a library this depends on - link it normally. The test is whether the link serves a reader of *this* repo's content, not whether the target happens to be in the fleet. This rule governs carried template content. A repo's own `README.md` and topical docs are its own content, not carried verbatim, and it does not reach them. This pairs with the present-tense rule below: state the current shape, not a history of which repo it came from.

### Markdown

- **Reference-style links in human-facing docs.** Every markdown file **except** the agent-instruction files (`AGENTS.md` and `.github/copilot-instructions.md`, which optimize for agents and keep inline links) uses reference-style links only: every URI - internal path, anchor, external URL, or shield image - is defined at the **bottom of the file**, split into groups by type under an HTML-comment header (e.g. `<!-- Shields -->`, `<!-- Workflow -->`, `<!-- Repo -->`, `<!-- External -->`) with each group's definitions alphabetized by reference name. **Reference names are contextual and encode the target and its group** - `foo-shield` for a shield image, `foo-link` for an external URL, and a bare `foo` for a local path or anchor (e.g. `[license-shield]`, `[releases-link]`, `[repo-config]`) - never numeric (`[1]`) or opaque. No inline `[text](uri)` targets in prose. **A URL inside a fenced code block stays inline** - reference links do not resolve in code blocks, so do not extract it, and exclude fenced code from any link-integrity check (bracket literals like `["a", "b"]` otherwise read as undefined references). **Removing a link also removes its reference definition** - an orphaned definition fails the no-unused-defs rule. The one exception is the Table of Contents, whose entries stay inline anchor links (see Table of Contents below).

scripts/README.md:9

This README currently says every script exits non-zero on a finding and that each script has a test_<script>.py beside it. In this PR, pr_review.py is a read-only status tool (and has no matching test_pr_review.py), so the wording should be narrowed to gate scripts to avoid a false claim.

Python only, standard library only, no third-party packages. Every script is read-only and exits non-zero on a finding.

Each script has a `test_<script>.py` beside it, driving its gates against input they must reject, because a gate nobody has watched fail is a gate nobody knows works. Where a case covers a table it reads the live table rather than restating it, and each one asserts a floor on what a healthy run reaches - a check whose scan matches nothing reports zero findings and reads exactly like a pass.

.github/copilot-instructions.md:9

This sentence still says any reviewer-binding rule must live in AGENTS.md, but this PR moves most rule text into GOVERNANCE.md with AGENTS.md as a router. The statement should allow GOVERNANCE.md as a valid authority location to avoid contradicting the new structure.

Do not duplicate language-specific rules here. **Project-specific conventions and API/behavioral contracts also belong in [GOVERNANCE.md](../GOVERNANCE.md), not here** - this file is intentionally limited to the inline commit/PR-title summary, the guidance for reviewing carried fleet content, and the GitHub Copilot Review Runbook. Non-Copilot agents (Claude Code, Codex, Cursor, ...) are not directed to this file and don't read it by default, so any rule a reviewer must honor has to live in `AGENTS.md` to be provider-independent.

spec/fidelity-model.md:22

This verbatim-fidelity bullet still refers to "the universal rule sections of AGENTS.md" and to classifying AGENTS.md sections. In this PR, the universal rule text moves to GOVERNANCE.md (with AGENTS.md as a router), and section classification now spans both files, so this description is outdated.

- **verbatim** - `.markdownlint-cli2.jsonc` (fleet-generic, no governed divergence), the `github-release` job region of the release task (the canonical orchestration a repo must not fork), and the universal rule sections of `AGENTS.md` - fleet-law with no repo-specific content (no SHAs, no `ptr727/<repo>` references), where a paraphrase or a missing rule is a defect, not an adaptation. Which `AGENTS.md` sections are verbatim and which stay intent (the few that describe one repo) is defined in [section-model.md][section-model].

…ain (#457) Promotes 3974ac2 (#455) and c4d3980 (#458). AGENTS.md becomes a thin router (87,457 -> 7,789 bytes) with the rule text in GOVERNANCE.md, and docs/token-cost.md records the measurement behind it. The scripts/ gates get tests. Five defects in them all reported success while doing nothing: the charset table omitted three characters the rule names, the duplicate-word allowlist contradicted its own comment, discovery reached 246 generated files against a rule that names those directories, lstrip ate every dotfile prefix so --diff was blind to .github/, and the table was first written with literal non-ASCII in the file the rule scans. Each fix carries the case that would have caught it. CI gates the self-tests, both repo_gate checks, and the charset and duplicate-word rules. The semicolon rule stays warn-only. Promoted now so a new repo scaffolds against the router shape rather than the pre-split monolith: groundTruthBranch cannot name develop (AUDIT.md section 1, and #340 settled it for HomeAssistant-Config), so until main carries GOVERNANCE.md a newly scaffolded repo either holds the pre-split shape or is audited against a branch lacking what it carries. Sections declared verbatim changed, so every downstream copy is stale until re-vendored. spec/fidelity_honesty.py distinguishes stale from modified by hash.

ptr727 and others added 5 commits July 26, 2026 07:56

Copilot AI review requested due to automatic review settings July 30, 2026 00:47

Copilot started reviewing on behalf of ptr727 July 30, 2026 00:48 View session

ptr727 mentioned this pull request Jul 30, 2026

Intake: Hugo static site with content deploy to a VPS on release (new target type) #456

Closed

Copilot AI reviewed Jul 30, 2026

View reviewed changes

Comment thread WORKFLOW.md Outdated

Comment thread README.md Outdated

Comment thread scripts/repo_gate.py Outdated

Comment thread spec/project-types.json

Copilot AI review requested due to automatic review settings July 30, 2026 00:56

Copilot started reviewing on behalf of ptr727 July 30, 2026 00:57 View session

Copilot AI reviewed Jul 30, 2026

View reviewed changes

ptr727 merged commit 3974ac2 into develop Jul 30, 2026
7 checks passed

ptr727 deleted the docs/agents-router-governance-split branch July 30, 2026 01:09

ptr727 mentioned this pull request Jul 30, 2026

Promote the AGENTS.md router split and the scripts/ gate harness to main #457

Merged

Navigation Menu

Split AGENTS.md into a router, and prove the scripts/ gates - #455

Split AGENTS.md into a router, and prove the scripts/ gates#455
ptr727 merged 6 commits into
developfrom
docs/agents-router-governance-split

ptr727 commented Jul 30, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ptr727 commented Jul 30, 2026

What this carries

The gates were watched failing

Scope and posture

Notes for review

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants