Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
14a3808
docs(topics): lock the copied-external-content interview contract
claude Aug 27, 2026
1a4cf73
docs(topics): design the copied-external-content plugin
claude Aug 27, 2026
e28c8cc
docs(topics): lock the provenance name (Q19 resolved at design gate)
claude Aug 27, 2026
6e3f23f
docs(topics): draft the provenance build plan (pre-approval)
claude Aug 27, 2026
0b76676
docs(topics): fix the provenance plan per the fresh-context review
claude Aug 27, 2026
b4cb6c6
docs(topics): harden the provenance plan per the devils-advocate pass
claude Aug 27, 2026
f8b0800
docs(topics): record the user's approval of the provenance build plan
claude Aug 27, 2026
d5f6eca
feat(provenance): scaffold and register the provenance plugin (Phase 1)
claude Aug 27, 2026
313f1f0
feat(provenance): the fingerprint module, test-first (Phase 2)
claude Aug 27, 2026
33dccc5
feat(provenance): corpus, breadcrumb, and stamp scripts (Phase 3, par…
claude Aug 28, 2026
d85a3d9
feat(provenance): the emitter and the golden-set scorer (Phase 3 comp…
claude Aug 28, 2026
a2ec73c
feat(provenance): the reference artifacts (Phase 4)
claude Aug 28, 2026
d4bee80
Merge remote-tracking branch 'origin/main' into claude/detect-copied-…
claude Aug 28, 2026
ffc0a1a
feat(provenance): the two skills and their evals (Phase 5)
claude Aug 28, 2026
20b1418
chore(provenance): graduate the design specs and prune the contract s…
claude Aug 28, 2026
5a0a0f7
fix(provenance): five review findings, each verified by execution first
claude Aug 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,24 @@
"realign",
"skill"
]
},
{
"name": "provenance",
"displayName": "Provenance",
"source": "./plugins/provenance",
"category": "quality",
"tags": [
"provenance",
"citation",
"attribution",
"copied-content",
"upstream-drift",
"documentation",
"detector",
"prose",
"audit",
"skill"
]
}
],
"renames": {
Expand Down
1 change: 1 addition & 0 deletions .claude/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@
"plugin-quality@melodic-software": true,
"powershell-format@melodic-software": true,
"prototype@melodic-software": true,
"provenance@melodic-software": true,
"rate-limit-guard@melodic-software": true,
"repo-fleet-hygiene@melodic-software": true,
"repo-hygiene@melodic-software": true,
Expand Down
1 change: 1 addition & 0 deletions docs/CATALOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ plugin manifests and kept in sync by CI — never hand-edit it; the category voc
- [`ai-slop`](../plugins/ai-slop) — Detects and removes AI-writing tells (slop) in checked-in markdown prose: em dashes, emoji formatting, AI vocabulary, negative parallelisms, chatbot phrases, filler, stacked hedging, citation artifacts, model-era phrases, and the rest of a catalog distilled from Wikipedia's Signs of AI writing plus a repo-owned, evidence-graded inventory of current-generation model vocabulary. Read-only audit by default with a deterministic detector plus a judgment rubric; an explicit fix action rewrites findings behind a semantic-diff guard. Findings conform to the detector-findings convention so the review fanout fix relay can consume them.
- [`overengineering`](../plugins/overengineering) — Evidence-earned-keep audit of an existing enforcement surface — agent hooks and standing instructions, repository and version-control hooks, CI lanes and gate scripts, branch protections, forge apps, declared external integrations — treating every incumbent mechanism as a retirement candidate until empirical evidence earns its keep, arguing every verdict in cost of carry, capping retirement-direction verdicts on security-class artifacts at FLAG-FOR-HUMAN, and realigning to the simplest adequate solution behind an explicit per-item human gate. The audit is read-only and emits a diffable findings artifact; realignment is a separate, explicitly invoked skill; and a third read-only lane re-runs the audit on whatever cadence the consumer wires and reports only what moved since the last run, above a configurable noise budget.
- [`improvement`](../plugins/improvement) — Evidence-first, cross-dimension improvement finder — point it at a repo, feature, concept, or process surface and it produces a ranked, evidence-cited list of improvement candidates led by value-to-effort, interviews on the pick, and hands off to the planning pipeline; runnable unattended as a tech-debt-sweep routine.
- [`provenance`](../plugins/provenance) — Finds prose in tracked markdown that restates content an external source owns (vendor docs, blogs, articles) without adequate attribution, confirms the source, and refactors the copy into a pointer, a citation, or a dated stamped record. Documentation provenance, not software supply chain. Nomination and judgment are LLM work; the scripts do only reasoning-free work (corpus scoping, breadcrumb extraction, stamp expiry, fingerprint compare of two concrete texts). Read-only audit by default; explicit fix and sweep actions apply dispositions behind a semantic-diff guard and live pointer verification. Findings conform to the detector-findings convention.

## Maintenance

Expand Down
1 change: 1 addition & 0 deletions docs/SKILL-CHEAT-SHEET.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,7 @@ owned by [docs/CATALOG-TAXONOMY.md](CATALOG-TAXONOMY.md).
| [`/playbooks:boris`](../plugins/playbooks/skills/boris/SKILL.md) | `playbooks` | Boris Cherny's Claude Code workflow tips across 115 sections |
| [`/playbooks:fable-5`](../plugins/playbooks/skills/fable-5/SKILL.md) | `playbooks` | Fable 5's operating doctrine loaded as standing session instructions |
| [`/playbooks:skill-authoring`](../plugins/playbooks/skills/skill-authoring/SKILL.md) | `playbooks` | Anthropic's internal skill-authoring playbook and patterns |
| [`/provenance:audit`](../plugins/provenance/skills/audit/SKILL.md) | `provenance` | Find prose copied from external sources and convert it into pointers |
| [`/repo-hygiene:clean`](../plugins/repo-hygiene/skills/clean/SKILL.md) | `repo-hygiene` | Clean caches, build artifacts, stale branches, and stashes per repo |
| [`/session-flow:show-options`](../plugins/session-flow/skills/show-options/SKILL.md) | `session-flow` | Lay out the skills that fit this moment as a ranked, nothing-hidden menu |
| [`/session-flow:workflow`](../plugins/session-flow/skills/workflow/SKILL.md) | `session-flow` | Navigate the staged dev workflow and suggest the next stage |
Expand Down
209 changes: 209 additions & 0 deletions docs/specs/provenance-capability-matrix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
# Capability matrix — copied-external-content plugin

Design-stage decomposition per /planning:design, written 2026-08-27 against the Brief in
`../PLAN.md`. Working plugin name: `provenance` (recommended by the naming pass; final pick is
the user's at the plan approval gate, thread T1). `<name>` below reads as that working name.

Every capability row states what it does, whether it is deterministic-script or LLM work (the
Brief's C1 split: scripts do only reasoning-free operations), and where it lives in the plugin.

| # | Capability | Kind | Home |
|---|---|---|---|
| 1 | Corpus enumeration and exclusion filtering | Script | `skills/audit/scripts/list-corpus.sh` |
| 2 | Breadcrumb inventory (links, fences, stamps) | Script | `skills/audit/scripts/extract-breadcrumbs.sh` |
| 3 | Stamp expiry check (portable baseline) | Script | `skills/audit/scripts/check-stamps.sh` |
| 4 | Trigger-less-stamp check (repo override only) | Script | same script, off-by-default flag |
| 5 | Nomination of suspect passages | LLM (fresh context) | audit flow + `reference/nomination.md` |
| 6 | Source resolution (breadcrumb-first, budgeted search) | LLM + budgets | audit flow |
| 7 | Source fetch (rung ladder, identity checks, cache) | Mechanical fetch, LLM-orchestrated | audit flow + `reference/source-fetch.md` |
| 8 | Fingerprint verify (quote-strip, shingle, matched spans) | Script | `skills/audit/scripts/fingerprint.mjs` |
| 9 | Rubric judgment (4 binary criteria, 3 blind judges) | LLM (fresh contexts) | `reference/rubric.md` + audit flow |
| 10 | Tier mapping and dispositions | Fixed mapping, applied by the flow | `reference/rubric.md` tier table |
| 11 | Human report | LLM | audit flow |
| 12 | Relay persistence (findings file) | Script | `skills/audit/scripts/emit-findings.sh` + `context/persist-findings.md` |
| 13 | Fix (three dispositions, guarded) | LLM behind explicit `fix` | audit skill `fix` action + `reference/dispositions.md` |
| 14 | Sweep (execution-contract mode) | LLM behind explicit `sweep` | audit skill `sweep` action |
| 15 | Configuration and carve-out management | LLM + config-cascade | `setup` skill, `.claude/<name>.json` |
| 16 | Evals and golden set | Fixtures + scorer script | `skills/audit/evals/` |
| 17 | Convention engagement at sweep completion | One-time repo work, not plugin machinery | `design/convention-engagement.md` |

## 1. Corpus enumeration and exclusion filtering

Enumerates tracked markdown for a target (file, directory, or repo-wide) and removes the
categorical carve-outs before anything reads a byte. Reasoning-free: path matching only.

- Built-in categorical exclusions (the Brief's carve-out constraint): vendored trees
(`**/vendor/**` and linguist-vendored path attributes), the plugin's own eval-fixture tree,
and consumer `excluded_paths` from config.
- Carve-outs that need reading (conforming stamped records, quotation contexts, owned content,
distilled-product genre) are NOT path-expressible; they are evaluated by capability 9 before
criteria, and by capability 8's preprocessing for quotations. The script never guesses at
them.
- Declined paths are counted and reported per the detector-findings declined-candidate rule,
never silently dropped.
- Invariant, amended 2026-08-27 at plan time (stress-test finding): the fixture-tree
exclusion is a CONFIG-LAYER entry (this repo's `.claude/provenance.json`
`excluded_paths`), not unconditional in the script. This is the ai-slop resolution of
#3041: an unconditional exclusion would decline fixtures under the eval harness's own
isolation, leaving the eval author trusting prose. Under the consuming repo's config every
normal run declines the tree and says so; the harness isolation lifts the config layer and
the fixtures report their real findings.

Cross-app reuse: none. This is plugin-specific plumbing.

## 2. Breadcrumb inventory

Extracts, per corpus file, the provenance signals already present: URLs with line numbers,
HTML-comment fence pairs (source URL + date), stamp lines, and blockquote/citation markers.
Reasoning-free extraction; no judgment about which breadcrumb explains which passage.

- Scope includes SIBLING files: S1 resolved a cross-file breadcrumb (a neighbor's citation
identified an unfenced copy's source), so the inventory is emitted per directory, and the
audit flow hands the nominating and resolving steps the whole directory's inventory, not just
the flagged file's.
- Output is JSON to stdout so the LLM layers consume it without re-reading files.

## 3. Stamp expiry check (portable baseline)

The one deterministic stamp check the portable baseline ships (Brief constraint, Q13): a
four-part record whose as-of date is older than the configured expiry window is flagged with
the run's own values (stamp date, window, days over). Parsing tolerates the fleet's known
stamp forms but claims only what it parses; unparsed candidate stamps are counted as declined
with the reason, because the live corpus carries stamp dates in at least four prose forms and a
guessing parser would manufacture findings.

## 4. Trigger-less-stamp check (repo override only)

The upstream-drift convention's named-not-built check (flag a dated stamp whose surface states
no recheck trigger). Ships built but OFF by default; a consuming repo that standardizes
greppable stamp forms enables it via config. It lands through the convention engagement
(capability 17), which is where its build trigger is formally answered.

## 5. Nomination

A fresh-context subagent reads a chunk of corpus files plus their breadcrumb inventories and
nominates suspect passages: file, approximate line range, suspected class (verbatim,
near-verbatim, paraphrase, summary), candidate source URLs (breadcrumbs first, sibling
breadcrumbs included), and the provenance signals that raised suspicion. Nomination is
recall-biased; precision comes from verification and judgment downstream.

- Nomination needs only file plus approximate line range. Exact spans for fix-eligible findings
come deterministically from capability 8's matched-span output, which resolves the handoff's
exact-offset open question without asking the nomination prompt to do offset arithmetic.
- The nomination prompt carries the untrusted-content spine for the file contents it reads
(repository files under exploration are an ingest surface).

## 6. Source resolution

Per nomination, in order: (a) breadcrumbs in or near the passage, (b) sibling-file breadcrumbs,
(c) budgeted WebSearch enrichment, only when no breadcrumb exists and only inside the Q10
budgets (per-candidate caps, convergence early-stop: same top source twice with no new
evidence, corpus-level fetch ceiling). Exhaustion produces the neutral disposition "source not
identified (budget exhausted; searched: ...)" naming every surface checked.

## 7. Source fetch

Fetches candidate sources for verification, under the upstream-drift fetch discipline carried
operationally in `reference/source-fetch.md` (raw-markdown channel first where one exists,
wholeness check, page-identity check before trusting a body, no absence claim from a truncated
read). Responses are cached for the run (lychee `--cache` is the in-repo model) and fetch
counts land in the budget log. Every fetch surface carries the untrusted-content framing spine
inline, byte-identical, per that convention's inline form.

## 8. Fingerprint verify

The liftable pure module from S2, rewritten for the plugin per prototype discipline: word
5-shingles, Jaccard plus containment plus longest matched span, comparing a local passage
against a fetched source text. Two spike-earned amendments are part of the module's contract,
not the rubric's:

- Quotation and fence stripping, including INLINE quotation marks and not only blockquotes, is
a PREPROCESSING step inside the module. A properly quoted excerpt never reaches shingling.
- Verdicts are reported per matched SPAN, never as whole-file containment: on real-sized files
whole-file scores dilute genuine matches to noise (a 27-word match scored 0.019 whole-file in
S2).

Output: matched spans with local line offsets, per-span word counts, and the separation-rule
inputs (containment, longest span). The working separation rule, containment >= 0.3 OR span >=
15 words after quote-stripping, ships as a named placeholder constant pair tuned at plan time
from golden-set telemetry (Q10/Q16 arbiters).

## 9. Rubric judgment

The versioned rubric catalog (`reference/rubric.md`) applied by three blind fresh-context
judges per candidate. Carve-outs are evaluated BEFORE criteria; then four binary criteria (span
correspondence to a named source, beyond common idiom, attribution adequacy, transformative
use), each graded with quoted evidence. Unanimity renders the verdict; any split routes to the
human. Judge sampling is the cost center (S5), so `judge_samples` is config with default 3 and
a floor of 3 for fix-eligible findings.

## 10. Tier mapping and dispositions

Evidence-gated tiers, fixed mapping (S4-adopted):

- fingerprint-confirmed: a matched span above the separation rule against an identity-checked
fetched source. Fix-eligible; the only tier that reaches the relay for copy findings.
- source-fetched-similar: source fetched, similarity below the deterministic rule, judges say
copy. Human flag, report-only.
- llm-suspected: no lexical evidence possible (paraphrase, summary). Report-only, permanently.
- Neutral: "source not identified (budget exhausted; searched: ...)". First-class outcome, not
a failure.

Dispositions (applied only by `fix`): convert-to-pointer, trim-to-citation,
condense-to-stamped-record. Offline-load-bearing surfaces are never bare-removed; they condense
to conforming stamped records. Read-frequency and fetch cost weigh on the disposition choice,
never as an allowance category.

## 11. Human report

All tiers, rubric grades with quoted evidence, carve-out declines with counts, budget
telemetry, and what the rubric pass did not cover. Also emitted as a machine-parseable JSON
sidecar in the memory tier (S2's design nudge), so golden-set scoring never parses prose.

## 12. Relay persistence

Script findings only, per the detector-findings convention: fingerprint-confirmed copy findings
and the deterministic stamp findings, with argued crosswalk rows, Confidence high or omitted,
rule ids leading every Finding cell. Judgment verdicts never enter the findings file (ai-slop's
V1 relay boundary, kept deliberately). The emitter fetches the producer contract at run time
and refuses to write when it is unreachable, reporting report-only as the outcome.

## 13. Fix

Explicit argument only. Applies the three dispositions to fix-eligible findings, then per file:
verify every pointer target live at edit time (fetch with identity check), run a fresh-context
semantic-diff verifier blind to the rewrite rationale (flags semantic loss, ambiguity, quote
corruption), revert flagged hunks, and close with fixed / suppressed-with-reason /
reverted-with-reason accounting. A later-dead pointer demotes back to a stamped record or an
archived-snapshot citation; the wiring to the weekly link-check lane is repo-side integration
recorded in the convention engagement, not plugin machinery.

## 14. Sweep

The Brief's execution contract as an explicit action: one tracked file at a time, apply the
verdict, verify (semantic diff plus pointer liveness), close. A file is closed when every
finding in it carries a disposition or an explicit neutral outcome. Sweep state (per-file
closure ledger) lives in the memory tier so an interrupted sweep resumes instead of
restarting. Sweep completion, never spike results, fires the convention engagement.

## 15. Configuration

`.claude/<name>.json` per the config-cascade convention, managed by the `setup` skill:
`excluded_paths`, budget constants, `stamp_expiry_days`, `trigger_less_stamp_check` (default
false), `judge_samples`, fetch-cache location. The detector scripts expose `--show-config`
naming the layer supplying each effective value (ai-slop model).

## 16. Evals and golden set

Per-skill `evals/evals.json` (house CI warrant) plus the golden set: synthetic fixtures
(shape-preserving rewrites of history cases; hard negatives including
paraphrase-styled-never-copied distractors), authored runner-agnostic so cases wrap as
case.yaml when `claude plugin eval` leaves early access. Case-level precision/recall is
hand-scored; a scorer script does the mechanical tally once verdicts exist. The fixture tree
carries the named categorical exclusion from every scan corpus.

## 17. Convention engagement

One combined engagement at sweep completion, drafted in `convention-engagement.md` beside this
file: reopen the upstream-drift recorded decision, conditional major bump, land the
trigger-less-stamp check behind the repo override, one changelog entry.
Loading
Loading