feat(provenance): documentation-provenance audit plugin (phases 1 to 5) - #3456
Conversation
Brief for the planned copied-external-content plugin: LLM-led breadcrumb-first detection with reasoning-free-only scripts, guarded fix dispositions, evidence-gated confidence tiers, hand-scored golden-set evals, capped budgets, and a staged delivery arc that reopens the upstream-drift recorded decision at sweep completion. Locked through three interview rounds, two verified discovery waves, blindspot and brainstorm passes, and a two-validator audit-answers pass; working artifacts live in the untracked .work/copied-external-content/ slice. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
Design-stage artifacts per /planning:design: capability matrix (17 rows split script vs LLM per the reasoning-free constraint), type inventory (evidence tiers, finding record, script contracts, draft crosswalk rows, config schema, golden-set case shape), plugin topology (audit+setup skills, six scripts with paired tests, versioned rubric catalog), fifteen design threads with statuses, and the drafted upstream-drift convention engagement that fires at sweep completion. Folds in both S2 amendments (quote-stripping as fingerprint preprocessing incl. inline quotes; matched-span verdicts, never whole-file containment) and the S1 sibling-breadcrumb finding. Naming pass recommends `provenance`; the user picks at the plan gate (Q19). All files ai-slop clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
The user accepted the design and picked provenance from the naming shortlist at the design acceptance gate, ahead of the plan-gate schedule the Brief allowed. Thread T1 moves DIRECTIONAL -> RESOLVED. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
Fill PLAN.md's Plan section per /planning:plan: eight phases (scaffold, fingerprint module, deterministic scripts, reference artifacts, skills and evals, golden set with improvement loop, crosswalk registration, sweep sub-topic promotion), TDD-first test strategy, execution-shape analysis with an optional parallel Wave A, and recorded Q10/Q16 resolutions. The user resolved Q16 as an accuracy-first, all-tunable posture, which adds gates and accuracy blocks to the config schema (type-inventory amended with a dated note). Draft status: fresh-context stress-tests are running; user approval gate still ahead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
Fold in all 13 plan-reviewer findings (1 CRITICAL, 5 IMPORTANT, 7 SUGGESTION), the load-bearing three verified against the repo: Phase 1 now regenerates the catalog and cheat-sheet views and gates on validate-plugins.sh; the evals-warrant sanity command takes its base-ref form; emit-findings.sh splits model-side rung resolution from reasoning-free composition per the ai-slop precedent (type-inventory amended with a dated note); the min-n arithmetic's v1 consequence (fix mode dark for every class until the golden set grows) is stated as the designed path; Phase 6 runs sidecar-only until the crosswalk rows land; prerequisites, conformance greps, runnable sanity commands, four missing standards rows, and sweep resume semantics added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
Fold in the second stress-test's 15 findings, each verified against the repo: Phase 1 adds the settings.json enablement key and gates on check-plugin-catalog-enablement.sh; Phase 2 adds the .test.sh discovery wrapper so the fingerprint suite actually runs in CI; Phase 5 adds the argued audit leaf-name registry row and orders evals.json fixture references around the orphan and unresolvable-fixture gates; Phase 6 gains the golden-harness seam (config-layer fixture exclusion per the ai-slop #3041 resolution, offline source.md short-circuit) and a named exit condition for the min-n gate; Phase 7 reshapes the crosswalk rows into the registry's exact column set; the accuracy block loses its duplicate judge_samples key, gains presence-gating for deep research and defined review-veto semantics (leave-with-reason, no tier reassignment). Design artifacts carry dated amendments; stress-test summary recorded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
The plan passed the fresh-context review and devils-advocate hardening passes; the user approved it, so execution begins at Phase 1. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
Manifest, README, and CHANGELOG for the documentation-provenance audit, plus marketplace registration, catalog regeneration, and enablement. The README carries what a consumer needs without this repository: the boundary against every adjacent owner (extract-ssot, doc-drift-detector, audit-derivability, ai-slop, point-dont-copy, audit-comment-residue), the config schema with the Q16/Q10 defaults, the fence and stamped-record marker forms, the evidence-tier table, and a prerequisites section naming the degraded branch when web search is unavailable (breadcrumb-only resolution, the rest landing on the neutral not-found disposition). It states plainly that this is prose provenance, not software supply chain, since the name admits an SLSA reading the skill descriptions must scope away. Deliberately absent: a per-instance suppression marker. Allowances stay categorical, which is what clears the upstream-drift recorded decision's "without a suppression list" bar; a per-finding keep routes to the consuming project's finding-suppression convention instead. The README's two skill references do not resolve yet by design; Phase 5 creates those SKILL.md files, and the cheat sheet regenerates then. Sanity checks all exit 0: jq manifest validity, marketplace entry present, scripts/validate-plugins.sh, scripts/check-plugin-catalog-enablement.sh (72 plugins, keys sorted). Lint clean: em-dash grep, typos, markdownlint, editorconfig-checker. Plan: docs/topics/copied-external-content/PLAN.md Phase 1. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
The plugin's one pure library: word 5-shingles, containment, Jaccard, and contiguous matched spans between a local passage and an already-fetched source, behind a thin CLI. It decides nothing. It reports lexical overlap, and the audit flow maps that evidence to a tier, which is what keeps the deterministic layer reasoning-free. Two behaviors are contract, both earned in the spike phase and both covered by named fixtures: Quotation stripping runs INSIDE the module, over the local text, before shingling, and covers inline quotation marks as well as blockquotes and code fences. A blockquote-only stripper missed inline quotes in a real fleet file and would have reported a cited excerpt as a copy; a rubric-layer carve-out would arrive after the evidence was already wrong. Curly quotes strip identically, an unpaired mark is left in place rather than swallowing the rest of the line, and a word-internal apostrophe is not a quotation mark. Verdicts are matched spans with local line offsets, not whole-file containment. On a 2,912-shingle file a genuine 27-word match diluted to 0.019; the separation rule therefore fires on containment OR span length, and the spans are the anchor a later fix edits against, which is also what keeps span localization out of LLM offset arithmetic. TDD: the suite was written first and observed red (module absent), then green at 21 of 21. Sanity checks pass: the suite exits 0 naming both amendment fixtures, the .test.sh wrapper exits 0, and the CLI's --json output satisfies `jq -e '.matched_spans'`. The wrapper exists because run-plugin-tests.sh discovers only *.test.sh; without it the module would ship with no CI coverage. Lint clean: shellcheck, shfmt, em-dash grep, typos, editorconfig-checker. Plan: docs/topics/copied-external-content/PLAN.md Phase 2. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
…t 1) The first three of Phase 3's five deterministic scripts, each written test-first and observed red before the script made it green. list-corpus.sh enumerates tracked markdown minus the categorical carve-outs. The eval-fixture exclusion arrives through the config layer rather than unconditionally in the script, so the eval harness's own config isolation lifts it and the fixtures report real findings (the #3041 resolution). Corpus root and config root resolve separately, which is what makes that isolation expressible. extract-breadcrumbs.sh inventories URLs, HTML-comment fences, stamp lines and blockquote lines, emitted per directory so sibling breadcrumbs travel together (spike S1 resolved a real cross-file breadcrumb). check-stamps.sh flags expired stamps and declines what it cannot parse. Date arithmetic is days-from-civil rather than `date -d`, so GNU and BSD agree. Two things the corpus taught, both measured over 1,347 tracked files: - mawk panics at compile time on interval expressions, and the panic is quiet enough that the scan simply returns nothing. Every regex here uses explicit repetition instead. - "read" is an ordinary English verb, so at the wide keyword window prose like "an unconfirmed read of a shipped build" became a stamp candidate, and `context-management-2025-06-27` — an API beta identifier — became an expired stamp. A narrower window for that one keyword drops all of them and keeps every real "read <date>" form. Declined candidates fell 54 to 43 and the one false expiry finding went with them. Corpus baseline at the 180-day default: 525 stamp candidates, 482 parsed, 43 declined, 0 expired. The declined count is the honest report the constraint asks for, not a defect to tune away; the oldest parsed stamp is 2026-04-08. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
…lete) The last two of Phase 3's five deterministic scripts, both test-first and observed red, plus the phase's plan and changelog entries. emit-findings.sh projects relay-eligible findings into a conforming findings file. It enforces the relay boundary rather than trusting its caller: only fingerprint-confirmed copies and the two stamp rules become rows; judgment verdicts are counted in `## Surfaces` and their tier names are deliberately kept out of the file, because a tier name in the apply relay's input invites a consumer to act on a verdict this producer withheld on purpose. A finding it cannot map to a relay rule goes to `## Unparsed` verbatim rather than being dropped. Cell escaping is the idempotent parity walk the ai-slop precedent established, so an excerpt that already contains `\|` is not double-escaped into a live delimiter. Destination resolution stays model-side per the type-inventory amendment: the script receives an already-resolved `--out`. score-golden.sh tallies case-level precision and recall. The confusion matrix lets one case be both a false negative and a false positive, because a positive case where the run missed the real span and emitted a different one has failed twice and one cell would flatter one of the metrics. A golden case the run never scored is declined and excluded from the tally, never counted as a miss: that would report a recall failure describing the harness rather than the detector. Making that refusal possible is why the actual file declares `cases_run`, and why its absence surfaces as `coverage_declared: false` instead of a silent assumption. Written against synthetic fixtures, which resolves the open question of whether it had to wait for Phase 6's golden set. Phase 3 Sanity Check: the suite loop exits 0 at 244 cases across six suites. The second command exits 3, not 0, and the plan's expectation of 0 was unreachable from the moment Phase 2 landed rather than being a regression here: affected-tests.sh reserves exit 1 for a failing suite, and exit 3 means every shell suite passed while a Node suite was also selected, whose runner it declines to guess. Both lanes verified green, 29 shell suites and 21 of 21 in fingerprint.test.mjs. PLAN.md records that reading for later phases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
The five prose surfaces the audit flow loads at the step that needs them: rubric.md (version 1), dispositions.md, source-fetch.md, nomination.md, and context/persist-findings.md. The rubric opens with its own boundary, because the four criterion names resemble fair-use factors and the resemblance misleads. The verdicts here are editorial and the remedies are maintenance remedies: a finding says a passage should point at its source instead of restating it, never that a passage is unlawful. Carve-outs are graded before any criterion, since several of them make the criteria meaningless rather than merely satisfied, and every criterion requires a quoted span with UNKNOWN available when the deciding text is not in front of the judge. Two shapes exist to keep a measurement from lying about itself. Judges are blind to the fingerprint numbers and to each other, because a judge handed the containment score turns three samples into one sample repeated. The semantic-diff guard reads before and after without the rewrite rationale, because an agent told why an edit was made reliably finds that the edit achieved it. For the same reason nomination passes union rather than intersect: intersecting two recall-biased passes converts them into a precision filter and discards the recall they were spawned to buy. persist-findings.md lands the resolution the plan flagged as an open tension. The contract now resolves through three rungs: the review plugin's bundled copy when that plugin is installed, the publisher's raw URL otherwise, and a refusal to write when neither is reachable. Rung 1 is what makes an offline run able to persist at all, and it keeps a portable plugin from pointing at one organization's URL as its only route. The scope limit is stated rather than implied: that rung yields the file shape and merge rules, while severity-vocabulary mapping stays model work either way. Phase 4 Sanity Check: all four criterion ids grep individually; the two untrusted-content conformance greps match file-for-file over source-fetch.md and nomination.md, which is the correct state at this phase since the remaining two ingest surfaces are Phase 5's; the ai-slop detector reports 0 findings over the five files; markdownlint, typos, editorconfig, skill portability, the em-dash ratchet and validate-plugins all exit 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
…external-content-k5aw77
/provenance:audit (default read-only, plus explicit fix and sweep) and /provenance:setup (check by default, apply on request), with 8 and 6 eval cases, three synthetic fixtures, a context/gotchas.md, and the leaf-name registration. The audit keeps mutation behind an explicit argument, so a bare invocation scopes, judges and reports and touches nothing. The untrusted-content spine is carried at the fetch step and the fix flow's liveness check cites that statement rather than restating it, keeping one contract in the file instead of two wordings of it. All four T8 ingest surfaces are now covered. The setup skill is human-invoked for a specific reason rather than a ceremonial one: config decides what the audit may ignore, so a model proposing its own exclusions could quiet its own findings. Its evals pin the refusals that matter -- declining to hardcode the fixture exclusion into list-corpus.sh, and correcting the premise that raising a gate shortens a report when gates bind fix eligibility only and never filter the report. Eval fixtures describe a fictional build tool. A fixture planting real copied text would make this plugin's own repository carry the defect it exists to find. gotchas.md records failure history the build actually produced, each with the symptom it presents as: mawk's silent interval-expression panic that makes a dead rule look like a clean corpus, an API beta identifier read as an expired stamp, a declined count that is the honest answer rather than a defect, and an exit 3 that is not a failure. Three corrections this phase, each caught by a gate rather than by review. The branch was 22 commits behind main, which made the skill warrant report a dropped-trigger regression in ai-slop, a plugin this branch never touched; merging main cleared it. The setup skill's action vocabulary was wrong -- the contract requires check as the leading action with apply documented, not show/init/set. And setup skills must not carry cheat-sheet metadata, since they are excluded from it as infra setup. Phase 5 Sanity Check: check-changed-skills exits 0 with both skills at 0 errors and 0 warnings; check-skill-leaf-names --check exits 0; both generators run and validate-plugins exits 0; fix appears only in the router row and under an explicit-invocation heading; ai-slop, markdownlint, typos and the eval-quality lint are clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
|
Claude finished @kyle-sexton's task in 1m 52s —— View job Code review in progress
|
|
Claude finished @kyle-sexton's task in 2m 52s —— View job Security review (
|
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
|
Last security-reviewed head: |
…lice
The contract-slice prune gate is correct and this change set was in breach:
docs/topics/<slug>/ is Contract tier, committed on a task branch only and
pruned before merge. Adding the slug to the grandfathered baseline is
explicitly not the remedy -- that list is a snapshot of debt that predates the
gate, and its own header says new work prunes before merge.
Following the topic-docs contract-slice lifecycle:
- Durable design artifacts graduate to docs/specs/ with `git mv`, so history
follows them: provenance-type-inventory.md (script contracts, finding record,
tier enum, config schema, golden-set case shape, draft crosswalk rows),
provenance-capability-matrix.md, provenance-design-threads.md,
provenance-plugin-topology.md, provenance-convention-engagement.md.
- Actionable follow-ups graduate to the work-item tracker: phases 6 to 8, with
the Brief's goal carried verbatim, the measured corpus baseline, and each
phase's sanity check.
- Every in-plugin pointer to the old docs/topics/ paths is rewritten, so a
script header names a contract that still resolves rather than a path the
prune removed.
Two review findings addressed in the same push, both real:
- Security review flagged `Bash(node:*)` in the audit skill's allowed-tools as
an unscoped wildcard while every sibling entry is path-scoped. In a skill
whose whole job is processing content its own docs treat as an active
injection surface, that is the hard permission gate rather than prose, so it
is now scoped to the one documented invocation,
`Bash(node ${CLAUDE_SKILL_DIR}/scripts/fingerprint.mjs:*)`. No functional
cost.
- shell-portability-lint read `back\slash` in two test fixtures as a GNU-only
`\s` construct. It is literal document text round-tripping through the JSON
escaper, which is exactly what those cases assert, so each site carries the
sanctioned `portability-ok:` annotation naming that reason rather than being
reworded to dodge the lint.
Verified: check-contract-slice-prune.sh and check-shell-portability.sh both
pass against origin/main; all six script suites stay green (244 cases) and
fingerprint.test.mjs exits 0 at 21 of 21; shellcheck, shfmt, markdownlint and
typos are clean over the moved and edited files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ffc0a1ada6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Claude finished @kyle-sexton's task in 57s —— View job Security review (
|
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
|
Warning Automated review did not complete — this is an infrastructure failure, not a review verdict. Treat any Claude comment on this PR (including a placeholder like "I'll analyze this and get back to you") as incomplete, not "no findings."
Re-run the job, or workflow_dispatch this workflow with the PR number, to retry the review. A new push re-triggers this lane only if the caller's |
All five were real. Each was reproduced before being fixed, and each fix is covered by a case that was observed red first. list-corpus.sh reported an EMPTY corpus for `.`. The repository root has several spellings and every one means "the whole corpus", but `.` reached the directory-prefix filter as a literal prefix and matched no tracked path. On this repository that was 0 files instead of 1,353, returned with no error, so it read as a clean repository rather than a broken invocation. Every root spelling now normalizes to the empty prefix. An explicit `"excluded_paths": []` could not clear an inherited exclusion. Treating "no elements" as "key absent" left the earlier layer's value standing, so an overlay could add exclusions but never remove one -- not the per-key override the config-cascade contract specifies. Presence, not emptiness, now decides whether a layer overrides. emit-findings.sh reported success having written nothing. With `set -e` deliberately off, an uncreatable directory or an unwritable path fell through to the "wrote" message and exit 0. That is the worst failure a persistence step can have, because nothing downstream contradicts it: the audit says its findings are relayed and the consumer never scans a file that does not exist. Both writing steps are checked now, under a new exit 5. Configured separation thresholds never reached the fingerprint module. The module reads no config by design, so a repository that tuned min_containment or min_span_words silently got the bundled 0.3 and 15 -- the constants that decide fix eligibility. The audit flow resolves them through the cascade, passes them explicitly, and reports the values it used. --show-config did not say which layer supplied a value. The setup skill promises per-value provenance and tells the operator to read it from there rather than parsing the layers by hand; listing layers and effective values separately did not deliver that. Each value is attributed now, to its layer, to the overriding flag, or to the bundled defaults. Also from the security lane, and taken as the safer fix: the audit skill's `Bash(node:*)` grant was an unscoped wildcard while every sibling entry was path-scoped, in a skill whose whole job is processing content its own docs treat as an active injection surface. Scoped to the one documented invocation. Verified: 256 cases across the six suites, all green, up from 244 as the new cases landed; fingerprint.test.mjs 21 of 21; shellcheck, shfmt, check-shell-portability.sh, check-changed-skills.sh (0 errors on both skills), ai-slop, markdownlint and typos all clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
|
All five Codex findings and the security finding are fixed in P1, P1, separation thresholds — confirmed. P1, P2, empty overlay cannot clear inherited exclusions — confirmed. P2, Security lane, Two notes on things I did not change:
Also worth flagging for whoever reads the checks: the automated code-review lane reported an infrastructure failure ( Generated by Claude Code |
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
|
Warning Automated security review did not complete — this is an infrastructure failure, not a review verdict. Treat any Claude comment on this PR (including a placeholder like "I'll analyze this and get back to you") as incomplete, not "no findings."
The check is green on purpose, and it is not evidence. It certifies that a security pass ran, and this one did not complete — but the cause is outside this PR's control, so merging is deliberately left unblocked rather than locking every merge for the length of the outage. Nothing was reviewed at this head. Where this check is required, it is satisfied without that evidence; a human should review security-sensitive changes here before merging. Re-run the job, or workflow_dispatch this workflow with the PR number, to retry the review (ci-workflows#227). A new push also retries it only if the caller's Re-running does NOT help for every class:
|
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
The changelog-parity gate is right and I missed it: 0.1.0 was published when #3456 merged, so a change set that modifies files under plugins/provenance/ cannot reuse that version number. Manifest to 0.2.0, and the changelog split at the real boundary. The three entries this PR adds -- rubric version 2, the contested c10 class, and the golden set with its measurement and quote-pairing fixes -- move under ## [0.2.0]. Everything from the merged PR stays under ## [0.1.0] untouched, so no published entry is rewritten. Verified: --check-bump, --check, --check-order and --check-preserved all exit 0, the last confirming every version heading the file carried at the merge base survives. validate-plugins, catalog enablement, both generators and markdownlint clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A
… it found (phases 6 to 8) (#3467) Closes #3458 Completes the provenance build plan. Phases 1 to 5 shipped in #3456; this lands Phase 6 (the golden set and the first measurement), Phase 7 (crosswalk registration), and Phase 8 (the sweep sub-topic promotion, now #3465) — plus two real defects the measurement exposed. - Ten synthetic golden cases, each judged offline against its own source. 8 tp / 0 fp / 0 fn / 2 tn; every class ships report-only, which is the gate's own arithmetic at n below min_n_per_class rather than a shortfall. - The T15 rotation probe records a real limit: at one substitution per four words containment goes to 0.0 against a source that was fetched and identity-checked. Constants left at the bundled 0.3 and 15. - Three rule ids registered in the detector-findings crosswalk, all IMPORTANT, each argued from the severity walk. 29 rows to 32. - Rubric version 2: C3 and C4 were phrased so their intuitive answer was exculpatory while their worked examples labelled it PASS, contradicting the "STANDS only if all four PASS" verdict rule. Found by blind adjudication, not by four rounds of review. - fingerprint.mjs stripped inline quotes per line, so a quotation spanning a hard wrap survived into the shingles. Stripping now spans line breaks within a paragraph, bounded by blank lines, fences and blockquote lines. Line numbering is preserved. 21 tests to 40.
2 participants
No linked issue
Summary
Adds
provenance, a plugin that finds prose in tracked markdown restating content an externalsource owns without a pointer or a conforming stamped record, and converts those copies into
links, quoted citations, or four-part stamped records. The harm it reduces is drift, not
plagiarism: a copied paragraph starts accurate and silently stops being accurate the next time
the upstream page changes, with nothing in the repository recording that it did.
This lands phases 1 to 5 of the approved build plan — the plugin, its six scripts, its reference
surfaces, and its two skills. Phases 6 to 8 are tracked in #3458.
Fix
fingerprint module from phase 2):
list-corpus.sh,extract-breadcrumbs.sh,check-stamps.sh,emit-findings.sh,score-golden.sh. Every script is reasoning-free; alljudgment is model-side.
rubric.md(version 1),dispositions.md,source-fetch.md,nomination.md, andcontext/persist-findings.md./provenance:audit(read-only by default, with explicitfixandsweep) and/provenance:setup(checkby default,applyon request), with 8 and 6 eval cases.Design points that are load-bearing rather than incidental:
exclusion would decline the fixtures under the eval harness's own config isolation, leaving an
eval author reading prose instead of results. Making that expressible is why
list-corpus.shresolves the corpus root and the config root separately.
identifies an unfenced copy's source.
emit-findings.shenforces the relay boundary itself rather than trusting its input: onlyfingerprint-confirmed copies and the two stamp rules become rows. Judgment verdicts are counted
in
## Surfaces, and their tier names are deliberately absent, since a tier name in the applyrelay's input invites a consumer to act on a verdict this producer withheld on purpose.
before and after without the rewrite rationale. Both exist because an agent told the answer
reliably finds it.
persist-findings.mdresolves the detector-findings contract through three rungs — thereviewplugin's bundled copy when installed, the publisher URL otherwise, refusal when neither is
reachable. The first rung is new against the ai-slop precedent: fetching a contract from one
organization's URL made every offline run report-only.
Two findings the corpus produced, both recorded in
context/gotchas.md:{0,4}), and the panic is quiet enoughthat the scan returns nothing while still exiting 0. A whole rule stopped firing and the run
looked healthy. Every regex here now uses explicit repetition.
context-management-2025-06-27— an API beta identifier — became an expired-stamp finding, andprose like "an unconfirmed read of a shipped build" became a stamp candidate. Narrowing the
window for that one keyword dropped every such case and kept the real
read <date>forms.Review findings addressed
Bash(node:*)inallowed-tools. Valid and taken: every sibling entry waspath-scoped while this one was an unscoped wildcard, in a skill whose job is processing content
its own docs treat as an active injection surface. Now scoped to the single documented
invocation,
Bash(node ${CLAUDE_SKILL_DIR}/scripts/fingerprint.mjs:*), at no functional cost.shell-portability-lint. It readback\slashin two test fixtures as a GNU-only\sconstruct. That text is literal document prose round-tripping through the JSON escaper, which is
what those cases assert, so each site carries the sanctioned
portability-ok:annotation namingthe reason rather than being reworded to dodge the lint.
Verification
Measured over this repository, 1,347 tracked markdown files after carve-outs (20 vendored paths
declined): 525 stamp candidates, 482 parsed, 43 declined, 0 expired at the 180-day default. The
oldest parsed stamp is 2026-04-08, which is why the default window fires on nothing; at a 60-day
window the same corpus yields 9 findings, and the full
list-corpus→check-stamps→emit-findingspipeline produces a conforming findings file with repo-relative locations. Thedeclined count is the honest report the design asks for: the corpus genuinely carries month-name
and bare-year stamp forms, and a parser that guessed at them would manufacture findings against
dates nobody wrote down.
Gates run locally, all green:
node .../fingerprint.test.mjsexits 0 at 21 of 21.scripts/check-changed-skills.sh origin/mainexits 0 with both new skills in scope, at 0 errorsand 0 warnings each.
check-contract-slice-prune.sh(both modes),check-shell-portability.sh,check-skill-leaf-names.sh --check,validate-plugins.sh,check-plugin-catalog-enablement.sh,check-detector-findings-crosswalk.sh --check,check-skill-portability.sh --all,check-purged-em-dashes.sh --check,check-silent-skips.sh --check, andcheck-evals-quality.shall exit 0. Both generators werere-run.
editorconfig-checker, shellcheck and shfmt are clean.
ingest surfaces.
One result worth stating plainly rather than leaving to be rediscovered:
scripts/affected-tests.sh --runexits 3, not 0, for any diff touching this plugin, and 3 is not a failure. Exit 1 is afailing suite; exit 3 means every shell suite passed and a Node suite was also selected, whose
runner that script deliberately declines to guess. This was true from the moment phase 2 landed
fingerprint.test.mjs, so the original plan's expectation of exit 0 was unreachable rather thanregressed. Both lanes were verified green.
Contract-slice graduation (topic-docs lifecycle)
The
copied-external-contentcontract slice was pruned before merge, perdocs/conventions/topic-docs/README.md"Contract-slice lifecycle (prune with pointer)". Adding theslug to
scripts/contract-slice-baseline.txtwas not an option and was not attempted: that list isa snapshot of debt predating the gate, and its own header says new work prunes before merge.
Pre-prune commit SHA:
ffc0a1ada6c8fc834d957ed1ad47e2b984aa13fe— the last commit that stillcontained the slice. Retention of that object is best-effort and carries no promised lifetime; the
graduation targets below are the load-bearing record.
Durable outcomes graduated to
docs/specs/withgit mv, so history follows them:design/type-inventory.mddocs/specs/provenance-type-inventory.mddesign/capability-matrix.mddocs/specs/provenance-capability-matrix.mddesign/design-threads.mddocs/specs/provenance-design-threads.mddesign/plugin-topology.mddocs/specs/provenance-plugin-topology.mddesign/convention-engagement.mddocs/specs/provenance-convention-engagement.mdActionable follow-ups graduated to #3458, which carries the Brief's goal verbatim, the measured
corpus baseline, and phases 6 to 8 with each phase's sanity check and its stated consequences (among
them: every finding class ships report-only at v1 by the gate's own arithmetic, and Phase 6 runs are
sidecar-only until Phase 7's crosswalk rows exist).
Every in-plugin pointer to the old
docs/topics/paths was rewritten in the same commit, so ascript header names a contract that still resolves.
Related
(phase 8), not from this PR.
it is the phase 8 convention engagement, and sweep completion is its stated trigger.
🤖 Generated with Claude Code
https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A