Skip to content

Gate the pattern-detectable half of the representative-data rule - #596

Merged
ptr727 merged 4 commits into
developfrom
feat/representative-data-path-check
Aug 7, 2026
Merged

Gate the pattern-detectable half of the representative-data rule#596
ptr727 merged 4 commits into
developfrom
feat/representative-data-path-check

Conversation

@ptr727

@ptr727 ptr727 commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Why

GOVERNANCE.md "Representative Data in Agent-Authored Text" says agent-authored text illustrates with constructed data rather than data observed in the maintainer's environment, and says in the same breath that no pattern closes it. This gates the half a pattern can reach.

It is introduced as that half deliberately. The leak that motivated the rule was name-shaped and sat in a pull request comment, which no committed-file linter reads, so a check mistaken for the answer is what stops anyone looking at the exposure the rule actually exists for.

Carries the TODO.md cluster "The Representative-Data Path Check", which #595 released from blocked once the scope floor shipped in #594.

The shapes were measured, not assumed

The backlog recorded four shapes, /home/<name>, /Users/<name>, C:\Users\<name>, and a bare drive letter. Run against this repository, a bare drive letter matched 11 files and named a path in none of them.

An escaped newline after any word ending in a letter and a colon reads as a drive letter, so a YAML fixture is full of them:

Matched text What it actually is
e:\n needs jobs:\n needs inside a test fixture string
s:\n a jobs:\n a: inside a test fixture string
C:\tmp (8 hits) a temp path in fixtures for the string-literal rule

That shape is dropped. The one kept is a drive letter followed by Users, which matches nothing in the tree today. This is narrower than the committed prose, which says "an absolute home path or a drive letter", so the narrowing is stated here rather than left for a reader to discover.

The exemption is built, not listed

A wrong exemption hands out a work list that damages correct documents, and this rule has two self-referential problems that a file-name allowlist would solve badly and briefly.

The rule's own wording quotes the shapes it forbids. Requiring a real user segment solves it by construction: the documentation placeholder is not a valid account name, so it describes the shape without matching it. No exemption names GOVERNANCE.md, and none goes stale when the text moves.

Probe Verdict
C:\Users\<name> in documentation no match
/home/<name> in documentation no match
/home/vscode/.ssh exempt, container account
a real absolute home path flagged

Container accounts are exempt by name. Every home path in this repository, all 15, is /home/vscode from the devcontainer snippets and the doc describing them. Without that exemption the rule would open with 15 findings and no true positive among them.

The gate reads its own source, so the bait paths in the tests are assembled from parts and a case asserts that neither prose_lint.py nor its test file carries a literal. That is the approach the file already takes for non-ASCII, which it writes as escapes for exactly this reason.

Operational repositories are exempt, and told so

An operational repository's runbook carries the literal path an operator types. That is the repository's own content rather than an agent quoting an environment it observed, which is the distinction the rule is about.

The model is read from what the repository carries rather than from the hub registry, which a downstream checkout cannot see: spec/files.json declares repo-config/operational/develop.json for the operational model and repo-config/develop.json for the release one. The hub carries both, being the template for each, so it reads as release rather than exempting itself from a rule it authors.

The skip is announced on stderr, because a rule that silently stops running reads as a rule that passed, which is the same failure #594 closed on the other side.

Scope note

Unlike every other prose rule, this one reads fenced blocks and configuration values rather than stripped prose. A transcript pasted from a terminal and a bind mount naming a real home are the exposure in its most consequential forms, and neither is a sentence.

Verification

  • python3 scripts/test_prose_lint.py - 188 pass, up from 173. Of the 14 new cases, 10 fail against develop and 4 pass against both, holding the must-not-fire direction.
  • python3 scripts/prose_lint.py . --check home-path - clean across the tracked tree.
  • python3 scripts/prose_lint.py . --summary - 557 violations across 45 files, the prose_lint: the hub's own docs, spec, and catalog do not pass the gate it ships #519 backlog figure unchanged, with home-path contributing none. Adding a rule to DEFAULT_RULES did not move the number.
  • python3 spec/validate.py, python3 spec/audit.py --selftest, python3 scripts/repo_gate.py - clean.
  • The blocking CI prose run - clean.

Correction: this branch carries the TODO bookkeeping too

An earlier revision of this description said the TODO.md edits were held back to avoid conflicting with #595. That was wrong. This branch was cut from #595's branch rather than from develop, so it already contained that commit, and the description claimed an absence a reader could disprove from the diff.

The review then caught the real consequence: the cluster entry still promised a bare drive letter after the implementation dropped that shape. Correcting the bullet was the smaller half of the answer, since this is the pull request that ships the cluster, so c6fca9e applies step 9 and deletes it rather than leaving a corrected description of finished work. The disproved claim is recorded here and in the commit message, alongside the measurement that disproved it.

This makes #595 redundant, since every line of it is already in this branch. Closing it as superseded is the maintainer's call rather than something this branch does.

🤖 Generated with Claude Code

ptr727 and others added 2 commits August 6, 2026 17:51
Selection step 9 applied to c8687c5. The cluster it carried filed no issue,
so nothing moves to "Verified Complete, Awaiting Close" and the deletion is
the whole of the bookkeeping for it.

"The Representative-Data Path Check" waited on exactly that floor, on the
reasoning that gating a path shape is worth having only once the gate can
prove it read something, so its state goes to ready and the dependency is
recorded as settled rather than dropped silently. Its freshness anchor is
rewritten against the tree rather than against the issue, per step 4: the
rule is stated in GOVERNANCE.md and none of the gate's nine rules reads a
path shape.

Its one open question survives untouched, whether a home path in an
operational repo's runbook is a finding, since that is a scoping decision
with fleet reach rather than something this commit settles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GOVERNANCE.md states that agent-authored text illustrates with constructed
data rather than data observed in the maintainer's environment, and says in
the same breath that no pattern closes it. This gates the half a pattern can
reach and is introduced as that half, since a check mistaken for the answer
is what stops anyone looking at the name-shaped exposure the rule exists for.

The shapes were chosen by measurement rather than from the list the backlog
recorded. A bare drive letter is not one of them: against this repository it
matched 11 files and named a path in none, because an escaped newline after
any word ending in a letter and a colon reads as a drive letter, so a YAML
fixture is full of them. The shape kept is a drive letter followed by Users.

The exemption carries the burden here, so it is built rather than listed. A
real user segment is required, which is what lets the rule's own wording
quote its shapes without matching them, so no exemption names the files that
describe the rule and none goes stale as they move. Container and runner
accounts are exempt by name because every home path in this repository, all
15, is /home/vscode from the devcontainer snippets and their doc: without
that exemption the rule would open with 15 findings and no true positive.

An operational repository is exempt entirely, since its runbook carries the
literal path an operator types, which is the repository's own content rather
than an agent quoting an environment. The model is read from what the
repository carries, spec/files.json declaring the operational payload for one
model and the release payload for the other, so nothing reaches the hub
registry to ask and the hub, carrying both, reads as release rather than
exempting itself from a rule it authors. The skip is announced, because a
rule that silently stops running reads as a rule that passed.

Unlike every other prose rule this one reads fenced blocks and config values,
since a transcript pasted from a terminal and a bind mount naming a real home
are the exposure in its most consequential forms rather than prose.

Both source files are read by the rule they implement, so the bait paths are
assembled from parts and a case asserts neither file carries a literal. That
is the approach this file already takes for non-ASCII, which it writes as
escapes for the same reason.

The rule reports nothing across the tracked tree and leaves the #519 backlog
at 557 violations across 45 files, unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 7, 2026 01:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new home-path rule to scripts/prose_lint.py to gate the pattern-detectable subset of the "Representative Data in Agent-Authored Text" policy (absolute home paths that look like real user homes), including tests and an operational-repository exemption, and updates the corresponding TODO cluster state.

Changes:

  • Add home-path detection to scripts/prose_lint.py, include it in DEFAULT_RULES, and exempt operational checkouts (with an explicit stderr note).
  • Add focused unit tests for home-path, including placeholder non-matches, service-account exemptions, fenced/config coverage, and operational exemption behavior.
  • Update TODO.md to mark the representative-data path check cluster as ready and refresh its Checked anchor.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
TODO.md Updates the work-cluster state/anchors for the representative-data path check.
scripts/prose_lint.py Implements the home-path rule, adds it to defaults, and adds an operational-checkout exemption.
scripts/test_prose_lint.py Adds unit tests covering the new home-path rule and the operational exemption behavior.
Suppressed comments (1)

TODO.md:37

  • This list of shapes still includes "a bare drive letter", but the new home-path regex intentionally excludes that form due to false positives. Consider updating the shapes here to match what the gate actually checks.
  - **Settled** - The shapes are `/home/<name>`, `/Users/<name>`, `C:\Users\<name>`, and a bare drive letter.

Comment thread TODO.md Outdated
The review found the entry still promising a bare drive letter after the
implementation dropped that shape, which is a contradiction between the
backlog and the tree. Correcting the bullet is the smaller half of the
answer: this branch is the pull request that ships the cluster, so step 9
deletes it rather than leaving a corrected description of finished work.

The claim the corpus disproved is recorded where a reader will find it, in
this branch's commit message and pull request description, which is where
the measurement that disproved it also lives. A bare drive letter matched 11
files here and named a path in none.

The orphaned prose-lint link definition goes with it, since MD053 is enabled
by default and the two clusters that cited it have both shipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 7, 2026 01:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

scripts/prose_lint.py:64

  • HOME_PATH matches only "\Users\" with a capital U. Windows paths are case-insensitive, so a pasted path like "C:\users\alice" would currently be missed (false negative). Consider allowing lowercase as well to match real-world logs and terminal output.
HOME_PATH = re.compile(r'(?:/home/|/Users/|[A-Za-z]:\\Users\\)(?P<user>[A-Za-z][A-Za-z0-9._-]*)')

scripts/prose_lint.py:160

  • home_path_findings includes the full matched absolute path in the lint output. Since the purpose of this rule is to prevent leaking real home paths, echoing the literal path into CI logs can unnecessarily amplify the exposure. The file+line is already reported, so the message can stay actionable without repeating the sensitive value.
        out.append((lineno, 'home-path',
                    f'absolute home path {m.group(0)!r} -> use a constructed path, not an '
                    'observed one'))

scripts/test_prose_lint.py:1585

  • Test name reads as a grammatical typo ("persons" vs "person"). Renaming improves readability and avoids introducing a new inconsistent identifier.
    def test_a_container_account_is_not_a_persons_home(self) -> None:

The review found a false negative: the Windows branch required a literal
Users while that filesystem is case-insensitive, so C:\users\alice and
c:\USERS\alice were both missed. A pasted terminal path is the case this rule
exists for and it carries whatever casing the shell printed.

The fix is scoped to the drive-letter branch rather than applied to the whole
pattern, which is the part worth stating. A lowercase /users/ is the
commonest REST path there is, so widening the POSIX branch the same way would
flag an API route in every document that has one. Two cases hold both halves.

The container-account test loses a possessive that read as a typo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 7, 2026 01:13
@ptr727

ptr727 commented Aug 7, 2026

Copy link
Copy Markdown
Owner Author

Answering the three suppressed comments from the round on c6fca9e, since a suppressed finding has no thread to pair an answer to. Two accepted, one declined, all fixed or answered in f8d6090.

Accepted, prose_lint.py:64 - case sensitivity on the Windows branch. Correct and a real false negative. Verified before fixing:

Probe Before After
C:\Users\<user> match match
C:\users\<user> miss match
c:\USERS\<user> miss match

The fix is scoped to the drive-letter branch rather than applied to the whole pattern, which is the part worth recording. Widening the POSIX branch the same way would match https://api.example.com/users/<user>, and a lowercase /users/ is the commonest REST path there is, so it would flag an API route in every document that has one. Two cases now hold both halves.

Accepted, test_prose_lint.py:1585 - the possessive read as a typo. Renamed to test_a_container_account_is_not_a_personal_home.

Declined, prose_lint.py:160 - echoing the matched path into CI logs. The reasoning is sound in general and I checked it rather than dismissing it, but its premise does not hold here. The value being echoed is already in a committed file in this repository, and the log line points at that file and line. The reader of the log and the reader of the file are the same audience with the same access, so the log adds no exposure that the file does not already carry. What it would cost is real: the message is what tells an author which of several paths on a line is the finding, and dropping it leaves a rule name and a line number for a rule whose whole subject is a specific string.

The case the finding is reaching for is a path that never reaches a commit, which this gate cannot see, since it reads committed files. That gap is genuine and is stated in the description rather than closed here: the leak that motivated the rule was in a pull request comment, which no committed-file linter reads.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@ptr727
ptr727 merged commit 8e10a2c into develop Aug 7, 2026
7 checks passed
@ptr727
ptr727 deleted the feat/representative-data-path-check branch August 7, 2026 01:25
ptr727 added a commit that referenced this pull request Aug 8, 2026
… that blocks on what it cannot read (#609)

Promotes 14 commits from `develop`. Merge with a **merge commit** (`gh
pr merge --merge`), never a squash, and **without `--delete-branch`**,
since this pull request's head is `develop` itself.

Closes #607 through the closing keyword already carried in `530cf71`,
which is why it is not repeated here.

## The prose backlog, cleared end to end

`#600`, `#604`, `#605`, `#606` took the tree from **557 findings across
45 files to 0 across 0**, in four batches ordered by surface: snippets,
comments, hub-only Markdown, then the carried files. Each batch measured
the checker's own exemption against the live corpus *before* sweeping,
and twice the measured answer was **"do not change the checker"**, which
is a result of that pass rather than a skipped one.

`#594` added the floor that makes those numbers trustworthy: a
diff-scoped run now asserts what it actually scanned, since a check
whose scan matches nothing reports zero findings and reads exactly like
a pass.

One finding from that work is worth carrying up: an exemption that is
too **loose** produces silence rather than false positives. #519
recorded the governance files as clean; today's checker reports 38
findings against those same files as they stood at the commit that
measured them.

## A review loop that fails closed

`#599`, `#601`, `#602`, `#603` and `#608` are one arc on
`scripts/pr_review.py`, each removing a shape in which the loop reported
a clean pass over a review it had misread:

- **`#599`** removed the shape a reply kept failing in, by taking the
thread's *words* rather than an id, so there is no argument a hand-typed
`PRRT_...` fits in.
- **`#602`** made `claims` resolve what a description points at rather
than what it looks like.
- **`#603`** gave a disproved claim a home the next round reads.
- **`#608`** reads the file-coverage line, and then generalizes: every
reader keys on a structural marker, so a marker that changes spelling is
a section the reader stops finding and reports as absent. The digest now
vets headings, `<summary>` texts, metadata labels, coverage wordings and
the reviewer login against an inventory measured from **332 review
bodies**, and **blocks on anything outside it**, exit `43`, with the
remedy stated as filing an issue on the hub. Whether to merge regardless
is the maintainer's decision.

`GOVERNANCE.md` merge gate went from four preconditions to **five**
accordingly.

## Governance and tooling

- **`#593`** states which checkout an agent works in and what the hub
is, which is the host-wide routing the repositories that most need it
cannot carry.
- **`#596`** gates the pattern-detectable half of the
representative-data rule, honest that no pattern closes the name-shaped
case.
- **`#598`** declares where a repository states what CI cannot verify.
- **`#592`** regrouped `TODO.md` by what ships rather than by what it
touches, so a `###` heading is one pull request.
- **`#601`** ended a `gh push` argument list at a newline rather than
only at `&&`, fixing a write-guard over-block.

## Verification

Run on `develop` at `530cf71` immediately before opening this:
`test_pr_review.py` (174), `test_prose_lint.py`, `test_repo_gate.py`,
`spec/audit.py --selftest`, `gh-write-guard.py --selftest`,
`spec/validate.py`, `repo_gate.py`, the prose gate in both CI
invocations, markdownlint and editorconfig-checker. All clean.

## Not carried by this promotion

- **#519 is complete and still open.** `TODO.md` holds its closing
evidence under "Verified Complete, Awaiting Close". Closing it is the
maintainer's call, so no keyword for it appears here.
- **The re-vendor debt is now nine files.** `#606` queued seven, and
`#608` changed `GOVERNANCE.md` "PR Review Etiquette" (`verbatim`) and
`.github/copilot-instructions.md` (`intent`) on top. The `intent` half
produces no hash and therefore no audit finding, which is why the Fleet
Sweeps entry names those files by hand.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants