Add the verbatim engine - content-hash fixed units against canonical (#374, PR-C)

[ptr727]

PR-C of the fidelity series (#374). Adds the second engine: verbatim content-hashing of fixed units against the hub's canonical.

What it does

• normalize (EOL-only) → content_hash (sha256) → classify_verbatim: None (matches current canonical), stale (matches a past hub revision → re-vendor), or modified (matches no revision → the repo changed fixed content). The discriminator is the content hash walked over the canonical's git history, never a version stamp (spoofable).
• Runs at DRIFT — a byte diff is a hint to review, never proof of breakage; exit code unchanged.
• Enabled on two real units, exercising both granularities:
  • Whole file — .markdownlint-cli2.jsonc.
  • Job region — the canonical github-release job of build-release-task.yml, extracted by job key from both sides (reference → catalog snippet, contract.verbatimJobs).

Normalization is EOL-only (design note)

A verbatim unit carries no per-repo placeholder. The files that declare placeholders are fidelity intent (judged by hand, never hashed). Masking could not serve a hash anyway: a downstream copy holds the substituted value (ptr727), not the token (<owner>), so masking the token in the canonical alone would guarantee a mismatch. A unit that ever needed per-repo substitution would require template-matching, not this hash — none does today.

Verification

• python3 spec/audit.py --selftest — interface (14) + verbatim: identical / CRLF / bare-CR match, body edit → violated, past-revision → stale, stale-modulo-EOL, no-revision → violated, forked region hashes differently.
• python3 spec/validate.py green (21 cataloged).
• Positive controls: .markdownlint-cli2.jsonc and the snippet github-release region self-hash clean; both deliberate forks → DRIFT.
• Live audit.py PlexCleaner exit 0 — surfaced a real advisory finding (PlexCleaner forked the canonical github-release orchestration).

Held on develop; no develop → main promotion.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds the second fidelity engine to spec/audit.py: a verbatim content-hash check (EOL-normalized) that can classify mismatches as stale (matches a past canonical revision) vs violated (matches no canonical revision), and wires it into the baseline via spec/files.json.

Changes:

• Extend spec/audit.py with EOL-only normalization + sha256 hashing and a verbatim checker that consults the hub file’s git history.
• Mark the github-release job region in build-release-task.yml as verbatim-pinned via the interface contract (verbatimJobs) and document the model update.
• Update audit procedure docs to include verbatim checks in the mechanized subset.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
spec/files.json	Adds verbatimJobs for the build-release interface contract and points canonical comparison at the catalog snippet via reference.
spec/fidelity-model.md	Updates the verbatim definition/normalization rules to EOL-only and documents stale-vs-violated semantics.
spec/audit.py	Implements verbatim hashing/classification and integrates interface+verbatim checks into the audit loop and selftests.
AUDIT.md	Documents verbatim checks as part of the mechanized audit subset (needs terminology alignment per review comment).

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

spec/audit.py:303

• check_verbatim() always walks the canonical's git history (git_file_history) even when the downstream content matches the current canonical. That forces at least one full history scan per canonical per run, even for clean repos, and also does unnecessary work before discovering an extract region is missing. Compute the current canonical match first (after any extract) and only call git_file_history() when there's an actual mismatch to classify as stale vs modified.

    try:
        canon_text = (ROOT / canonical_rel).read_text(encoding="utf-8")
    except OSError:
        return [("DRIFT", f"verbatim: {label} canonical {canonical_rel} is unreadable from the hub (spec error?)")]
    history = git_file_history(canonical_rel)
    if extract is not None:

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

spec/audit.py:250

• The normalize() docstring is longer than it needs to be. AGENTS.md comment guidance says one line is the default; this can be expressed tersely without wrapping while still pointing to the spec doc for rationale.

    """Reduce a carried unit to its comparable form: neutralize line endings, since EOL variance is governed
    separately, not a fidelity deviation. No placeholder masking - see spec/fidelity-model.md "Normalization".
    """

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

spec/audit.py:250

• normalize()'s docstring is still a multi-line design narrative. Per AGENTS.md comment guidance (one line is the default), this can be collapsed to a single-line docstring that states the non-obvious behavior (EOL normalization) and points to spec/fidelity-model.md for rationale.

def normalize(text):
    """Reduce a carried unit to its comparable form: neutralize line endings, since EOL variance is governed
    separately, not a fidelity deviation. No placeholder masking - see spec/fidelity-model.md "Normalization".
    """

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

spec/audit.py:251

• normalize()'s docstring is wrapped over multiple lines with design rationale. Per AGENTS.md comment guidance (one line default), collapse this to a single-line docstring that just states the non-obvious behavior and points to the model doc for rationale (AGENTS.md:146-148).

def normalize(text):
    """Reduce a carried unit to its comparable form: neutralize line endings, since EOL variance is governed
    separately, not a fidelity deviation. No placeholder masking - see spec/fidelity-model.md "Normalization".
    """
    return text.replace("\r\n", "\n").replace("\r", "\n")

spec/audit.py:254

• This inline comment uses a semicolon to join two independent clauses, which the repo's prose rules forbid for agent-authored text (AGENTS.md:158). Recast without a semicolon.

@functools.lru_cache(maxsize=1024)  # bounded; the keys that recur across repos are the canonical and its history