fix(provenance): four defects the first report-only sweep found, three of them in the plugin itself by kyle-sexton · Pull Request #3471 · melodic-software/claude-code-plugins

kyle-sexton · 2026-08-28T10:36:50Z

No linked issue

Summary

Ran stage 1 of the #3465 sweep — the report-only pass, which that issue states is runnable now and is what feeds the growth loop. The sweep's value turned out not to be its findings. It was that pointing the plugin at this repository, and at itself, exposed four defects: a stale baseline in the changelog, an unstated grading scope in the rubric, a judge dispatch that could not execute the rule the rubric states, and a truncation bug that silently exempted real stamps from expiry checking.

Three of the four were caught by fresh-context verifiers that disagreed with the reading they were checking, twice overturning a PASS from a verifier that had only reproduced the numbers.

Fix

1. A stale corpus baseline, and a wrong diagnosis of it

The merged 0.2.0 entry presents a Phase 6 measurement whose six figures are Phase 3 ones. All six reproduce exactly at 33dccc59 — the commit that introduces list-corpus.sh — on a clean tree with the scripts as they existed there, then were carried forward without re-measuring.

The current baseline is recorded at 619199ee with --as-of 2026-08-28 pinned, because two of its figures expire: the 180-day zero holds only until 2026-10-05 on the current oldest stamp, and the 60-day count moves daily. A reproducibility fix that shipped an expiring baseline would repeat the defect one turn later.

The delta's attribution is also corrected. It is not main moving across #3467–#3469; those contribute +1 in total. #3467 adds 20 markdown files and contributes zero — all inside the excluded fixtures tree, which is why considered rises by 20 and the fixture decline goes 3→23 while the corpus is untouched.

An earlier draft of this entry claimed the figures came from no commit at all, and that claim was false. That replay covered only history reachable from main, which the squash-merge of #3467 had made exclude the originating build branch; the reflog held it throughout. The entry now records that a history replay bounded at a squash boundary cannot answer whether a number came from a commit.

2. Rubric v3: the scope C3 is graded at

Version 2 never stated it, and the two criteria resolve in opposite directions under it. Surfaced by grading a real corpus passage (plugins/dometrain/skills/grounding/SKILL.md:50-64 at d7e391da, containment 0.589, a 142-token span against Dometrain/mcp@master fetched 2026-08-28): two readers reached the same verdict and disagreed on which scope produced it.

C3 is graded outward across the whole file; C4 on the passage. C3 tests whether the attribution's declared scope matches the derivation's — file-scope attribution discharges it when the derivation is file-wide, and does not when one lift sits inside otherwise-original material. The tempting reading, "the attribution exists and is complete", is rejected explicitly: it would let a single lift escape on a header line about something else. C4's half is only written down; its examples were already passage-scoped.

This takes the invalidation. The rubric's own rule is that a criterion change invalidates measurements pinned to the prior version, so the golden set must be re-scored before any precision figure is cited, and no class becomes fix-eligible on a superseded rubric. Stated plainly so the figures are not left under a cloud they do not deserve: no current golden case appears to turn on the scope question, so the re-score is expected to reproduce 8tp/0fp/0fn/2tn. It is still required, because the rule keys on a criterion changing rather than on a recorded case flipping.

3. The judge dispatch could not execute that rule

reference/nomination.md handed each judge the local passage, the fetched source, and the rubric — never the containing file. A C3 graded across the whole file is unanswerable from that, and both the rubric and the judge prompt instruct UNKNOWN when the text to quote is absent. A conforming judge under v3 would have graded C3 UNKNOWN on every candidate, stopping every verdict and routing every run to the human — the same class of defect as the v1 polarity inversion, reached from the other side. The motivating case proves it: the attribution that clears dometrain sits ~35 lines above the passage.

The dispatch now supplies LOCAL FILE: and says which criteria are graded against which input. Blindness here means blind to the pipeline's own suspicion — fingerprint numbers, nomination reasoning, other judges — never blind to the material a criterion is defined over. Carve-outs 1, 4 and 5 are file-level judgments too and were under-supplied before v3; the same fix closes that.

4. A conforming ISO stamp declined as a bare year, and never expiry-checked

keyword_window() sliced the window at exactly its own length, so a date that started inside but ended past it was cut and the fragment satisfied a weaker test. At docs/upstream/aihero-course.md:127, as-of 2026-08-17 was read as 2026-08-, the ISO test failed, and the bare-year fallback matched the leftover 2026. Declined lines are reported and then skipped, so the one thing the script exists to do never ran on a date that parses fine.

The window is now a distance from the keyword rather than a cut through the text, with every form required to begin at or before the window length.

Verification

Every substantive change was checked by an independent fresh-context verifier that did not produce it. Three returned FAIL and were acted on:

The baseline entry was rewritten after a verifier refuted its headline claim, having replayed 70 commits including the pre-squash build branch. A prior verifier that only re-ran the counts had passed the same claim.
The rubric change was returned FAIL for shipping a rule its execution surface could not satisfy — defect 3 above, which had gone unnoticed by its author.
The plugins/ sweep was returned PASS with a tiering correction: two findings had been tiered fingerprint-confirmed on the file-level separation flag while their per-finding spans were 14 words, below the 15 floor. Genuine short copies below the deliberate floor.

The stamp fix, red-first: 4 new assertions fail against the pre-fix script (Passed: 48 Failed: 4), pass after (Passed: 52 Failed: 0), test change purely additive. Corpus effect at --as-of 2026-08-28 over 1,352 files: candidates 535→542, parsed 491→498, declined 44→44, expired findings 0→0. That zero was confirmed by enumerating every parsed stamp before and after and diffing the sets — nothing lost, exactly 7 gained, oldest 40 days, none expired. No lapsed stamp had been hidden.

Gates green: check-stamps.test.sh (52), check-shell-portability (default awk here is mawk 1.3.4, so every run exercised it; no interval expressions introduced), all four check-changelog-parity modes, validate-plugins, check-changed-skills, check-purged-em-dashes, markdownlint. affected-tests.sh --run exits 3, which is not a failure — 2 shell suites passed and 5 PowerShell suites need a Windows lane.

Limits, stated rather than left to be discovered. The sweep met neither the two-pass nomination union nor the three-judge blind panel the design calls for, so recall is below spec and no verdict carries panel unanimity — absence of findings in a tree is weaker evidence than presence of one. One sweep leg could not obtain a verifier at all and said so rather than claiming one; the may month-name false positive is diagnosed but deliberately not fixed here, as it over-reports into the visible declined bucket and deserves its own change. Full record on #3465.

Refs provenance sweep sub-topic: repo-wide sweep, upstream-drift convention engagement, and the cleaned-state gate #3465 — the sweep sub-topic; stage 1 recorded there, stage 2 still blocked on fix-eligibility
Refs provenance plugin: phases 6 to 8 (golden set and measurement, crosswalk registration, sweep) #3458, feat(provenance): golden set, crosswalk registration, and two defects it found (phases 6 to 8) #3467 — phases 6 to 8, whose baseline this corrects
Refs feat(provenance): documentation-provenance audit plugin (phases 1 to 5) #3456 — phases 1 to 5, where the rubric and check-stamps.sh shipped
Refs chore(conventions): plugins restating upstream harness facts are not adopters of upstream-drift, so stale claims never surface #2273 — the upstream-drift decision; this run accumulated zero per-instance suppressions, the first evidence on record pointing at its trigger
Refs ai-slop: back the audit evals with real fixture files instead of prose scenarios #3041 — the config-layer fixture exclusion, confirmed working as designed

🤖 Generated with Claude Code

https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

Generated by Claude Code

…date The 0.2.0 entry presents a Phase 6 measurement whose six figures are actually Phase 3 ones: 1,347 files / 525 candidates / 482 parsed / 43 declined / 0 expired / oldest 2026-04-08 all reproduce exactly at 33dccc5, the commit that introduces list-corpus.sh, on a clean tree with the scripts as they existed there. They were carried forward without re-measuring. Records the current baseline at 619199e with --as-of 2026-08-28: 1,352 tracked markdown after carve-outs (1,395 considered, 43 declined at path level), 535 candidates, 491 parsed, 44 declined at stamp level (20 month-name, 24 bare years), 0 expired at the 180-day default, oldest parsed stamp 2026-04-08, 9 findings at a 60-day window. Two of those figures expire, so the as-of date is pinned beside the commit: 0 expired holds only until 2026-10-05 on the current oldest stamp, and the 60-day finding count moves daily. A baseline recorded without one repeats the staleness this entry corrects. Also corrects the delta's attribution. It is not main moving across #3467 to #3469 — those contribute +1 in total, one file in #3468. #3467 adds 20 markdown files and contributes zero, all of them inside the excluded fixtures tree, which is why considered rises by 20 and the fixture decline goes 3 to 23 while the corpus is untouched. An earlier draft of this entry claimed the figures came from no commit at all. That replay covered only history reachable from main, which the squash-merge of #3467 had made exclude the originating build branch; the reflog held it throughout. The entry now records that a history replay bounded at a squash boundary cannot answer whether a number came from a commit. Refs #3465. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

Version 2 never said at which scope C3 and C4 are graded, and the two resolve in opposite directions under it. Grade both at the file and a majority-adapted file clears twice; grade both at the span and a well-attributed derived file stands every time. Nothing in v2 chose. Surfaced by applying the rubric to a real corpus passage (plugins/dometrain/skills/grounding/SKILL.md:50-64, containment 0.589, 142-word span). Two readers reached the same verdict and disagreed on which scope produced it. v3 states it: C3 is graded outward across the whole file, C4 on the passage. C3 must look outside the span because attribution inside the span is the quotation carve-out, not a C3 question. What C3 tests is whether the attribution's declared scope matches the derivation's -- file-scope attribution discharges it when the derivation is file-wide, and does not when one lift sits inside otherwise-original material. The tempting reading, "the attribution exists and is complete", is rejected explicitly: it would let a single lift into an original file escape on a header line about something else. The v2 golden-set measurement does NOT carry forward. This rubric's own rule is that a criterion change invalidates measurements pinned to the prior version, and v3 adds a scope-match test that can decide a case either way. The recorded 8tp/0fp/0fn/2tn belongs to v2 and must be quoted with its version; the set must be re-scored against v3 before any precision figure is cited, and no class becomes fix-eligible on a superseded rubric. v2 took the one exception to that rule on the argument that it changed no criterion's substance; v3 cannot make that argument and does not try. Refs #3465. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

Rubric v3 grades C3 outward across the whole file, but the judge dispatch in reference/nomination.md handed each judge only the local passage, the fetched source, and the rubric. The containing file was never supplied. Both the rubric and the judge prompt instruct UNKNOWN when the text to quote is absent, and C3 requires a quoted span. So a conforming judge under v3 would have graded C3 UNKNOWN on every candidate, stopping every verdict and routing every run to the human -- the same class of defect as the v1 polarity inversion, reached from the other side. The motivating case proves it: the attribution that clears dometrain sits about 35 lines above the passage. The dispatch now supplies LOCAL FILE: and states which criteria are graded against which input. Blindness in this panel means blind to the pipeline's own suspicion (fingerprint numbers, nomination reasoning, other judges), never blind to the material a criterion is defined over. Carve-outs 1, 4 and 5 are file-level judgments too and were under-supplied before v3; the same fix closes that. The lens-diversity stance that read for "whether the attribution present already discharges the obligation" pointed judges at the reading v3 rejects, and now reads for scope match. Also corrects the v3 entry itself. The headline claimed v2 never stated the scope for both criteria; C4's examples and replacement test were already passage-scoped, so only C3's half is substantive and only it carries the invalidation. The C3 argument no longer rests on "attribution inside the span is the quotation carve-out" -- that is false as written, since carve-out 3 is gated on text presented as a quotation -- and instead points at C3's own worked examples, which already grade attribution sections away. The dometrain evidence is pinned to d7e391d with its fetch date, since recording a measurement without one is the defect 5480c16 corrected. And the entry now states that no current golden case appears to turn on the scope question, so the required re-score is expected to reproduce. Refs #3465. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

check-stamps.sh sliced the keyword window at exactly its length, so a date that started inside the window but ended past it was cut. At docs/upstream/aihero-course.md:127 "as-of 2026-08-17" was read as "2026-08-", the ISO test failed on the fragment, and the bare-year fallback matched the "2026" left behind. A declined stamp is reported and then skipped, so the expiry comparison never ran on a date that parses. The window is now a distance from the keyword rather than a cut through the text: the slice carries nine more characters, one short of the longest form matched, and every form must begin at or before the window length. Corpus at --as-of 2026-08-28 over 1,352 files: candidates 535 -> 542, parsed 491 -> 498, declined 44 -> 44, expiry findings 0 -> 0. All seven newly parsed stamps are dated 2026-07-19 or later, well inside the 180-day window, so no lapsed stamp had been hidden. The script and test changes themselves landed in 4130379. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

The 0.3.1 entry said the two new month-name declines came "within reach of the added slack". They do not. Measured on the real lines, the word `may` starts at offset 60 in `plugins/planning/skills/interview/SKILL.md:117` and 59 in `plugins/repo-hygiene/skills/clean/context/git-branch-cleanup.md:42`, both inside the 60-character window; the old slice cut the word after one character. The same truncation that hid the ISO dates was hiding these, and the `may` false positive itself reproduces identically on the previous script. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

claude · 2026-08-28T10:37:12Z

Claude finished @kyle-sexton's task in 4m 22s —— View job

Claude finished the code review

Gather context (PR diff, CLAUDE.md/AGENTS.md, changed files)
Load and follow /review:code-review instructions
Review each changed file for correctness/maintainability issues
Post inline comments for line-anchored findings
Post summary comment

Scope

This repo carries .github/workflows/claude-security-review.yml, so per REVIEW.md this lane stays out of security and covers correctness, design, and maintainability only.

What I checked

The only executable change is the keyword_window() fix in check-stamps.sh. I traced it by hand rather than trusting the PR's own narrative:

Reconstructed the pre-fix truncation bug character-by-character against the new window-edge.md fixture (plugins/provenance/skills/audit/scripts/check-stamps.test.sh:90-93): the measured keyword's 60-char window cuts 2026-02-28 after 2026-02-, the ISO regex fails on the fragment, and the bare-year fallback matches the leftover 2026. That reproduces the defect described in the changelog exactly.
Verified the fix's wlen + 9 buffer and RSTART <= wlen guard: since the longest fixed-width pattern (ISO) is 10 chars, a match starting at the last legal position (wlen) needs exactly wlen + 9 characters of window, which is what's allocated. A match that starts past wlen can't fit inside the truncated window and is correctly rejected — confirmed against the second window-edge.md line, which is deliberately built so the date starts past the window and must stay a non-candidate.
Checked which of the 5 new assertions actually flip pre-fix vs post-fix: 4 do (parsed, declined, stamp_date, days_over), 1 doesn't (candidates, since a declined line and a parsed line both still count as one candidate). That matches the changelog's claim of "4 new assertions fail against the pre-fix script" precisely — good sign the red/green claim wasn't just asserted.

No correctness issues found in that script change.

The remaining four files (rubric.md, nomination.md, CHANGELOG.md, plugin.json) are documentation/rubric/version-bump changes. I cross-checked internal consistency rather than prose style:

nomination.md's claim that "carve-outs 1, 4 and 5 are file-level judgments" matches what carve-outs 1/4/5 actually say in rubric.md (vendored trees, owned content, distilled-product architectures — all surface-level, not passage-level).
The new judge-prompt block's "Grade C1, C2 and C4 on LOCAL PASSAGE. Grade C3 against LOCAL FILE" matches the scope rule stated earlier in both files.
CHANGELOG.md's top entry version (0.3.1) matches plugin.json's bumped version.

I did not find a bug or inconsistency worth blocking on. No inline comments were posted, since there were no line-anchored findings to attach.

Not independently verified

This lane's Bash access is limited to gh pr * subcommands, so I could not execute check-stamps.test.sh, check-changelog-parity.sh, or the other gates the PR body lists as green. Those pass/fail claims are author-reported and unverified by me; my confidence above rests on the manual trace, not on re-running the suite.

claude · 2026-08-28T10:37:18Z

Claude finished @kyle-sexton's task in 1m 5s —— View job

Security review

Gather context (PR diff, changed files)
Apply skip gate
Review changed files for security-relevant issues
Post findings

Skip gate: no security-relevant surface. I read the full diff (git diff origin/main...HEAD) across all 6 changed files:

plugins/provenance/.claude-plugin/plugin.json — version bump only (0.2.0 → 0.3.1).
plugins/provenance/CHANGELOG.md — documentation additions only.
plugins/provenance/skills/audit/reference/nomination.md — prompt-template doc change adding LOCAL FILE: (local repo content, already trusted) to the judge dispatch. No new untrusted-input surface; fetched external source text was already part of the prompt before this diff.
plugins/provenance/skills/audit/reference/rubric.md — rubric documentation, no executable content.
plugins/provenance/skills/audit/scripts/check-stamps.sh — bug fix to the keyword_window() awk function so it stops truncating dates at the window boundary. Inputs are local tracked markdown files (via git ls-files or explicit args, each checked with [[ -f "$f" ]]); no shell metacharacters are ever passed through eval/sh -c, and the change only widens a text-window slice used for date parsing — no injection or path-traversal surface introduced or altered.
plugins/provenance/skills/audit/scripts/check-stamps.test.sh — test-only additions exercising the above fix.

No authentication, authorization, secrets/credential handling, injection surface, or GitHub Actions workflow changes are present in this diff. Nothing to report.

github-actions · 2026-08-28T10:38:49Z

Last security-reviewed head: 9f394d56ef7a44001363e8a5192adec307627b0c. On the next push, the relevance gate compares only the commits since this SHA; delete this comment to force a full re-review.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c750dc339

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

github-actions · 2026-08-28T10:42:04Z

Claude has reviewed this PR 4 times. The lane skips further automatic reviews after 5; deleting this comment resets the count.

…surfaces Two findings from automated review on #3471, both verified against the corpus before fixing. extract-breadcrumbs.sh:is_stamp still sliced the keyword window at exactly its length after 0.3.1 fixed the same bug in check-stamps.sh:keyword_window. The two scripts promise the same candidate definition and the audit passes the extractor's output to nomination, so they must agree. Measured, three files were short a stamp line the other counted: docs/CLOUD-SESSIONS.md (3 vs 4), docs/conventions/loop-lane/README.md (4 vs 5), and plugins/session-flow/CHANGELOG.md (7 vs 8). All three agree now. CLOUD-SESSIONS.md:320 is worse than the 0.3.1 case rather than a repeat of it: its date begins at offset 60 of 60, so the cut left a bare "2" and no form matched at all, not even the bare-year fallback that kept the earlier case visible in the declined bucket. That line did not decline, it left the inventory silently. Regression test pins the real corpus line, with a negative control at offset 64 proving the added slack does not admit a date starting outside the window. Recorded with it: is_stamp rescans from each keyword in turn, so a fixture whose line carries a second keyword beside its date passes against the unfixed script and proves nothing. Two fixtures written that way did exactly that before the offsets were measured rather than estimated. The review dispatch could not execute rubric v3 either. 0.3.0 gave the judge the containing file and left the reviewer holding only the passage, the source, and the quoted grades, while its job includes checking the C3 grade and whether a carve-out was missed. C3 is graded across the file and carve-outs 1, 4 and 5 are file-level. Review is the last stage before fix eligibility, so a reviewer waving through an unsupported C3 PASS is what puts an unsupported finding in reach of an automatic edit. Refs #3465. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

claude · 2026-08-28T10:46:36Z

Claude finished @kyle-sexton's task in 2m 32s —— View job

Code review

Gather context (PR diff, CLAUDE.md/AGENTS.md, changed files)
Load and follow /review:code-review instructions
Review each changed file for correctness/maintainability issues
Post inline comments for line-anchored findings
Post summary comment

Scope

Per REVIEW.md, this lane stays out of security (this repo carries .github/workflows/claude-security-review.yml) and covers correctness, design, and maintainability. This run is triggered by synchronize — I diffed origin/main...HEAD at the new head (fb11cf6) and focused on what changed since the prior review round (2c750dc), rather than re-litigating the parts already covered.

What changed since the last review, and what I checked

Both P2 findings from @chatgpt-codex-connector's prior round are resolved in this push, and I traced each fix rather than taking the changelog's word for it:

Review-dispatch blindness to the containing file (nomination.md). The optional review prompt (used when accuracy.review_agents > 0) now carries a LOCAL FILE: block on the same terms as the judge prompt above it, so a reviewer checking a C3 grade or a file-level carve-out (1, 4, 5) is no longer stuck with passage-only input. Confirmed the block was added to both prompt templates (judge at L122, review at L142), not just one.
extract-breadcrumbs.sh:is_stamp disagreeing with check-stamps.sh:keyword_window on the window boundary (extract-breadcrumbs.sh:190-217). This now applies the identical wlen + 9 slice / RSTART <= wlen start-boundary rule that 0.3.1 shipped for check-stamps.sh. I hand-verified the new straddle.md fixture by counting offsets character-by-character: the positive case's date starts at exactly relative position 60 of a 60-char window (matching the changelog's "offset 60 of 60" claim), and the negative control shifts it to relative position 64 — past wlen, correctly still rejected. The two scripts' pattern-check order differs (bare-year is checked 3rd in extract-breadcrumbs.sh vs. 5th in check-stamps.sh), but that's immaterial here since is_stamp only returns a boolean — order can't change which of the 5 alternatives fires.
Version bump (0.2.0 → 0.3.2 in one PR, since it never shipped between) and the new [0.3.2] changelog entry are internally consistent with plugin.json and with the two fixes above.

No new correctness or maintainability issues found. No inline comments posted — there was nothing line-anchored to flag.

Not independently verified

Bash execution in this lane is limited (script runs and python3 both require approval I don't have), so I could not execute check-stamps.test.sh, extract-breadcrumbs.test.sh, or the other gates the PR body lists as green. I instead hand-traced the awk offset arithmetic for the new fixtures against the actual fixture text (shown above) rather than trusting the changelog's numbers. The PR body's pass/fail claims for the full gate suite remain author-reported and unverified by me.

claude · 2026-08-28T10:46:44Z

Claude finished @kyle-sexton's task in 1m 12s —— View job

Security review

Gather context (diff since last-reviewed head, changed files)
Apply skip gate
Review changed files for security-relevant issues
Post findings

Scope note: the relevance gate on this lane compares only commits since the last-reviewed head (2c750dc). This push (fb11cf6) adds one new commit on top of that, addressing the two Codex review comments from the prior push (missing LOCAL FILE input to the review-agent dispatch, and the breadcrumb extractor's window truncation not matching the check-stamps.sh fix). Diff reviewed: 2c750dc339526a267b7d5137edc123e546a0319d..fb11cf6ae5f9af0cc499763144a43814e4fa39f9.

Skip gate: no security-relevant surface. Files touched by this increment:

plugins/provenance/.claude-plugin/plugin.json — version bump only (0.3.1 → 0.3.2).
plugins/provenance/CHANGELOG.md — documentation only.
plugins/provenance/skills/audit/reference/nomination.md — adds LOCAL FILE: to the optional review-agent prompt template, mirroring what the judge dispatch already receives. This is a prompt-template doc, not executable content; it widens the input an LLM reviewer is told to hold, not any code trust boundary. No new untrusted-input surface.
plugins/provenance/skills/audit/scripts/extract-breadcrumbs.sh — mirrors the check-stamps.sh keyword_window() fix from the prior commit (0.3.1) into is_stamp's awk window slice: wlen + 9 characters with a RSTART <= wlen start-boundary guard, instead of a hard cut at wlen. Same operation already reviewed clean on check-stamps.sh — pure text-window arithmetic over locally read tracked files, no eval/sh -c, no shell metacharacter interpolation, no path handling changed.
plugins/provenance/skills/audit/scripts/extract-breadcrumbs.test.sh — test-only additions exercising the above fix (straddle case + negative control).

No authentication, authorization, secrets/credential handling, injection surface, or GitHub Actions workflow changes are present in this increment. Nothing to report.

Three corrections from independent verification, all of them errors in the prose rather than the code. The 0.3.2 entry said three files were short a stamp line. It is five. The draft sampled five of the seven files the 0.3.1 fix newly parsed and reported the differences it happened to catch as the total; sweeping all seven against both versions of the extractor adds docs/topics/fresh-eyes-checkpoint-audit/design/design-resolution.md (1 to 2) and plugins/context-guard/CHANGELOG.md (4 to 5). The 0.3.1 corpus-effect figures did not reproduce from the tree they ship in. They were measured before the entry itself was written, and this changelog is inside the corpus it measures, so the paragraph moved its own totals. Restated delta-first, since the delta holds and the absolutes only hold at the commit they were taken at. Same staleness 0.2.1 was written to correct, one turn later and quieter. The 0.3.2 entry quoted a literal date as an example of stamp shape, and the corpus run then reported this changelog as carrying an expired stamp, one day over. Prose about stamp syntax is indistinguishable from a stamp to a mechanical detector, and this file is in scope. Reworded to a placeholder; expiry findings back to 0. Refs #3465. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

claude · 2026-08-28T10:51:58Z

Claude finished @kyle-sexton's task in 3m 7s —— View job

Code review

Gather context (PR diff, CLAUDE.md/AGENTS.md, changed files)
Load and follow /review:code-review instructions
Review each changed file for correctness/maintainability issues
Post inline comments for line-anchored findings
Post summary comment

Scope

Per REVIEW.md, this lane stays out of security (this repo carries .github/workflows/claude-security-review.yml) and covers correctness, design, and maintainability. Triggered by synchronize; the last-reviewed head was fb11cf6 (per the security lane's tracking comment), so I diffed fb11cf6..e6e192c rather than re-reviewing the whole PR. That range touches exactly one file: plugins/provenance/CHANGELOG.md. Both Codex P2 findings from the prior round (missing LOCAL FILE in the review-agent dispatch, extract-breadcrumbs.sh window mismatch) were already resolved in fb11cf6 and covered by the previous review round; nothing in this increment touches that code again.

What I checked

This commit corrects three counts in the 0.3.1/0.3.2 changelog entries that were themselves wrong (the PR's own stated purpose — this changelog documents a script that scans this repo's markdown for stamp-like text, and previous drafts of the entry became stamp candidates themselves, or undercounted by sampling instead of sweeping). I hand-verified the corrected numbers rather than trusting the prose:

"Five" vs. the earlier "three" file list (CHANGELOG.md:11-20): the two newly-added files are docs/topics/fresh-eyes-checkpoint-audit/design/design-resolution.md (1 against 2) and plugins/context-guard/CHANGELOG.md (4 against 5). I greped the first for the keyword+date pattern is_stamp/keyword_window match on and found exactly two candidate lines (design-resolution.md:55 and :146, both verified ... 2026-07-19) — consistent with "1 against 2" (one detected pre-fix, both post-fix).
Arithmetic in the corpus-effect paragraph (CHANGELOG.md:74-91): 538→545 candidates and 493→500 parsed both delta to +7, matching the stated "+7 candidates, +7 parsed" headline; declined is stated as a single unchanged value (45), consistent with "declined unchanged (month-name and bare-year trading 2)" — 2 bare-year declines flip to parsed while 2 new may month-name declines appear elsewhere in the same entry, netting to zero.
The pinned commit fb11cf6a (CHANGELOG.md:77) resolves unambiguously to fb11cf6ae5f9af0cc499763144a43814e4fa39f9, the actual commit these absolutes were measured at.

No correctness or maintainability issues found in this increment. No inline comments posted — nothing line-anchored to flag.

Not independently verified

This lane's Bash access is limited to gh pr * subcommands (per SKILL.md's allowed-tools), so I could not execute check-stamps.sh/extract-breadcrumbs.sh against the corpus to reproduce the exact candidate/parsed/declined totals, only hand-trace plausibility on individual files as shown above. The full-corpus counts and the gate suite the PR body lists as green remain author-reported and unverified by me.

The hygiene lane failed shellcheck on the new straddle fixtures. The markdown backticks inside the single-quoted printf read as command substitution to shellcheck, but they are literal corpus text and part of the character count that places the date at offset 60 of the window, so they cannot be dropped or reworded without destroying what the fixture tests. Annotated with the repo's existing convention for this exact case, an inline shellcheck disable carrying its reason, as used in check-skill-portability.test.sh and check-docs-only-gate.test.sh for fixture bodies that must reach the script verbatim. Reproduced the failure locally before fixing and confirmed shellcheck now exits 0 both on the changed files and across every tracked .sh, the way CI runs it. Both suites still pass (44 and 52). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tvKPLnWr7suxxAupU4p8A

claude · 2026-08-28T11:03:38Z

Claude finished @kyle-sexton's task in 2m 22s —— View job

Code review complete

Gather context (PR diff, CLAUDE.md/AGENTS.md, changed files)
Load and follow /review:code-review instructions
Review each changed file for correctness/maintainability issues
Post inline comments for line-anchored findings
Post summary comment

Scope

This repo carries .github/workflows/claude-security-review.yml, so per REVIEW.md this lane stays out of security and covers correctness, design, and maintainability only. Reviewed the full PR diff (origin/main...HEAD, head 9f394d5), with focused re-verification of what changed since the last reviewed head (e6e192c..9f394d5, one file).

What's new since the last review round

The only change since e6e192c is extract-breadcrumbs.test.sh: two # shellcheck disable=SC2016 annotations on fixture printf lines whose literal backtick-quoted commit-message text (`chore: sync standards components`) reads as command substitution to shellcheck inside single quotes. This is mechanical and matches the repo's existing convention exactly — the same per-line-disable-with-reason pattern is used throughout the codebase (claude-config, claude-ops, autonomy scripts all follow it), and I confirmed by grep that this repo has no divergent SC2016-suppression style this deviates from. The backtick text is load-bearing: it's part of the character count that places the fixture's date at the window-boundary offset the test is exercising, so rewording it would defeat the fixture's purpose. No issues.

Full-PR verification

Both prior Codex P2 findings (missing LOCAL FILE in the review-agent dispatch; extract-breadcrumbs.sh window mismatch) remain resolved as of fb11cf6, confirmed again by re-reading nomination.md (both judge and review dispatches now carry LOCAL FILE:) and extract-breadcrumbs.sh (mirrors check-stamps.sh's wlen + 9 / RSTART <= wlen boundary rule).

I independently hand-traced the keyword_window() boundary arithmetic against both new check-stamps.test.sh fixture lines (not just the one prior reviews walked through): line 3 of window-edge.md has its ISO date starting at relative offset 53 from the "measured" keyword (within wlen=60, so it parses fully under the new wlen + 9 slice); line 4 has it starting at relative offset 63 (past wlen, correctly rejected by RSTART <= wlen, and no other stamp keyword remains in the line, so it produces no candidate at all — matching the test's candidates == 1 assertion). Both check out precisely against the code at check-stamps.sh:309-339.

rubric.md, nomination.md, CHANGELOG.md, and plugin.json (version 0.3.2, matching the changelog's top entry) are internally consistent with each other and with the script changes.

No correctness or maintainability issues found in this PR. No inline comments posted — nothing line-anchored to flag.

Not independently verified

This lane's Bash access is limited (script execution requires approval I don't have), so I could not execute check-stamps.test.sh, extract-breadcrumbs.test.sh, or shellcheck itself. My confidence rests on hand-tracing the awk offset arithmetic and the shellcheck-convention grep shown above, not on re-running the suites. The gate-suite pass/fail claims in the PR body remain author-reported and unverified by me.
· Branch: claude/detect-copied-external-content-k5aw77

claude · 2026-08-28T11:03:42Z

Claude finished @kyle-sexton's task in 47s —— View job

Security review

Gather context (PR diff, changed files)
Invoke /review:security-review and follow its instructions
Review changed files for security-relevant issues
Post findings

Scope note: the relevance gate on this lane compares only commits since the last-reviewed head (fb11cf6). This push (9f394d5) adds two commits on top of that: e6e192c (docs(provenance): correct three counts this entry got wrong about itself) and 9f394d5 (fix(provenance): silence SC2016 on fixture text that must stay literal). Diff reviewed: fb11cf6ae5f9af0cc499763144a43814e4fa39f9..9f394d56ef7a44001363e8a5192adec307627b0c.

Skip gate: no security-relevant surface. Files touched by this increment:

plugins/provenance/CHANGELOG.md — corrects three counts in the already-merged 0.3.1/0.3.2 entries (three files → five files short a stamp line; delta-vs-absolute reframing of the corpus-effect paragraph; a line number added to one file reference). Documentation only, no executable content, no new claims about trust boundaries or credential handling.
plugins/provenance/skills/audit/scripts/extract-breadcrumbs.test.sh:203-211 — adds two # shellcheck disable=SC2016 comments above printf lines whose fixture text (lifted verbatim from the corpus) contains literal backticks that must not be shell-expanded. This only silences a shellcheck lint about single-quoted strings containing backticks/variables; the printf arguments themselves are unchanged single-quoted literals, so there's no new interpolation, no eval, and no injection surface introduced.

No authentication, authorization, secrets/credential handling, injection surface, or GitHub Actions workflow changes are present in this increment. Nothing to report.
— branch claude/detect-copied-external-content-k5aw77

… it found in the rubric (#3472) Rubric v3 shipped in #3471 with an obligation attached: its own rule blocks every precision figure, and with it every class's fix eligibility, until the golden set is re-scored against v3. That re-score is done here. - The re-score reproduces the recorded table exactly: 8 tp / 0 fp / 0 fn / 2 tn, thirty judges in independent processes, three per case, cases relabelled so no path reached a judge. Every panel unanimous, no verdict moved. No class becomes fix-eligible, because every class sits below min_n_per_class 10 -- the class-size block stands after the rubric-version one is lifted. - The rubric carried its own answer key. Its version-history paragraph recorded the expected tally, the panel size, and which case turns on which criterion, and the pipeline inlines the whole rubric into every judge prompt. All thirty judges read the prediction before grading, and the run withdrew its claim of a blind panel. The prediction now lives in the changelog; the rubric says a judge should be able to read all of it and still not know the answer. - A vendored-snapshot basis gains a tier rule: caps at source-fetched-similar, records source.route with each failed live fetch, never fix-eligible, because fix eligibility rests on current upstream state and a snapshot cannot establish it. - The not-found searched-surfaces listing is recorded as prose-only and unenforced, because emit-findings.sh carries no field for it. - The modal "may" is no longer read as a month name. Three sites, including the classifier a single-site fix would miss. Three reviewers then found that the "may" fix had traded over-reporting for under-reporting: a digitless "Verified this May" vanished upstream of the declined bucket, from both audit paths. Fixed by reading the capital M from the original line, which both scripts had been discarding via tolower(). The suites could not have caught it -- they asserted the two scripts agree, which passes when both are equally wrong -- so the new cases pin non-zero expected counts on each side. Every substantive change was checked by an independent fresh-context verifier. Two returned FAIL and were acted on. One, nested two tiers down and never briefed by this session, noticed the branch was 9 commits ahead rather than 1 -- which was hiding that the branch had never been reset after #3471 merged and would have deleted seven test suites added by #3459. Limits recorded rather than left to be discovered: the re-score panel was not blind, for the reason above; a second answer-key leak sits in a fixture and is deliberately unfixed because that line is inside the text the fingerprint compares, so the fix and a re-score are one atomic change; and a latent under-report from may_form()'s branch order is documented at the rule rather than chased, since the obvious one-line reorder only mirrors it.

claude added 5 commits August 28, 2026 10:12

chatgpt-codex-connector Bot reviewed Aug 28, 2026

View reviewed changes

Comment thread plugins/provenance/skills/audit/reference/nomination.md

Comment thread plugins/provenance/skills/audit/scripts/check-stamps.sh

kyle-sexton merged commit 4f11a43 into main Aug 28, 2026
59 checks passed

kyle-sexton deleted the claude/detect-copied-external-content-k5aw77 branch August 28, 2026 11:15

kyle-sexton mentioned this pull request Aug 28, 2026

fix(provenance): discharge the rubric-v3 re-score, and the answer key it found in the rubric #3472

Merged

Navigation Menu

Uh oh!

fix(provenance): four defects the first report-only sweep found, three of them in the plugin itself - #3471

fix(provenance): four defects the first report-only sweep found, three of them in the plugin itself#3471
kyle-sexton merged 8 commits into
mainfrom
claude/detect-copied-external-content-k5aw77

kyle-sexton commented Aug 28, 2026

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

claude Bot commented Aug 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kyle-sexton commented Aug 28, 2026

Summary

Fix

1. A stale corpus baseline, and a wrong diagnosis of it

2. Rubric v3: the scope C3 is graded at

3. The judge dispatch could not execute that rule

4. A conforming ISO stamp declined as a bare year, and never expiry-checked

Verification

Related

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Claude finished the code review

Scope

What I checked

Not independently verified

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Security review

Uh oh!

github-actions Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code review

Scope

What changed since the last review, and what I checked

Not independently verified

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Security review

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code review

Scope

What I checked

Not independently verified

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code review complete

Scope

What's new since the last review round

Full-PR verification

Not independently verified

Uh oh!

claude Bot commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Security review

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

claude Bot commented Aug 28, 2026 •

edited

Loading

claude Bot commented Aug 28, 2026 •

edited

Loading

github-actions Bot commented Aug 28, 2026 •

edited

Loading

github-actions Bot commented Aug 28, 2026 •

edited

Loading

claude Bot commented Aug 28, 2026 •

edited

Loading

claude Bot commented Aug 28, 2026 •

edited

Loading

claude Bot commented Aug 28, 2026 •

edited

Loading

claude Bot commented Aug 28, 2026 •

edited

Loading

claude Bot commented Aug 28, 2026 •

edited

Loading