feat(claude-config): stable audit-instructions finding identity (#4116) - #4851
Conversation
333f142 to
c46859c
Compare
Give every finding an audit-pass (check, claim, sites) identity via finding-ids.sh, and emit I30–I33 from lane findings through emit-findings.sh --from-lane with detector-findings 3.2.0 crosswalk rows. Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
c46859c to
84051dc
Compare
# Conflicts: # plugins/claude-config/.claude-plugin/plugin.json # plugins/claude-config/CHANGELOG.md
- SKILL.md back under the 500-line cap: Finding ID prose points at reference/finding-identity.md instead of restating it. - finding-ids.sh follows symlinks with plain readlink in a bounded loop instead of GNU-only readlink -f. - finding-ids.test.sh renames AB/BA to FWD/REV for typos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PT4esxbdC7eQiwQxy35Mie
Identical sentences under one heading path share a finding_id; emit the finding once and name the collision with its occurrence count in ## Surfaces, per audit-pass finding-identity. The I32 fallback target now reads only /plugin:skill or a backticked plugin:skill, so a clock time falls back to shape=. Pins the post-fence heading path in finding-ids tests and limits the adopter row's downgrade sentence to the I28 rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hx63mvV8t39y644DQ3uLnn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hx63mvV8t39y644DQ3uLnn
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Claude finished @kyle-sexton's task in 4m 13s —— View job Claude finished the security review
Reviewed the two new scripts ( IMPORTANT —
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e00f8256bb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… as typos Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 38s —— View job Security review —
|
…nces/ I33 Heading-shaped lines inside a block HTML comment outside a fence no longer enter the anchor heading path. I33 accepts references/ spokes as I31 already did. An I32 line with more than one skill-ref candidate falls back to shape= since the writer cannot tell which one is absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 25s —— View job Security review —
|
|
Claude finished @kyle-sexton's task in 5m 26s —— View job Code review (
|
|
Claude finished @kyle-sexton's task in 50s —— View job Security review —
|
|
Claude finished @kyle-sexton's task in 1m 17s —— View job Code review (
|
No related issue: partial delivery of #4115; its AC2 (five previously-retained findings still seed) cannot be verified because those five findings are not recorded anywhere, so #4115 stays open for that check. ## Summary Refs #4115. Fence the I6 pre-scan to the docs-hygiene `rule-negation-without-positive` gate set (minus its hard-guardrail carve-out), and report I33 as one finding per spoke with an opener-sentence excerpt anchor. ## Fix - I6 rows are sentences that open with a prohibition and carry neither a paired positive nor a rationale marker. Soft-wrapped lines join first; frontmatter, fences, tables, headings, and HTML comment openers are skipped. A fence closes only on the opener's character, at least the opener's length, with only whitespace after it; blockquoted fences are recognized. `--i6-counts` prints raw and surviving counts. - audit-noise's hard-guardrail carve-out is not adopted: a guardrail "never" still owes I6's fallback rationale (I7), so it stays a candidate. Stated in the scanner header and CHANGELOG. - I6 emits one row per physical line, skips Setext headings, and reads a cue wrapped in underscore emphasis. - I33 is one finding per spoke, anchored on the opener sentence with the heading path as discriminator. Phase D rolls I33 up per plugin. - `criteria.md` catalog 1.24.0. Four fixtures plus evals 27 and 28. - Version 0.51.28 with CHANGELOG. ## Verification - `instruction-scan.test.sh` 139/139 (Case 3f: fence closing; Case 3g: one row per line, Setext headings skipped, underscore-emphasized cues read), `emit-findings.test.sh` 172/172, `finding-ids.test.sh` 29/29, `lane-runs.test.sh` 52/52. - Count reproduction at `2dfaaa40` (990 files) prints `I6 raw=6608 surviving=913`: `find . -name '*.md' -not -path './.git/*' \( -path './plugins/*/skills/*' -o -path './plugins/*/agents/*' -o -path './.claude/*' -o -name CLAUDE.md -o -name AGENTS.md \) -print0 | sort -z | xargs -0 bash plugins/claude-config/skills/audit-instructions/scripts/instruction-scan.sh --i6-counts` - `scripts/validate-plugins.sh`, `scripts/check-changelog-parity.sh --check --check-order`, `scripts/check-changed-skills.sh origin/main`: pass. - Known limits: lines inside a multi-line HTML comment are still read; an enumeration after the cue (`Never commit files, logs, or caches.`) reads as a named alternative, as in audit-noise. ## Related - Refs #4115 (AC2 open). - Finding identity for I30–I33 is #4116 (PR #4851). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…e and permission lint (#5311) Refs: #3563 Refs: #3568 Refs: #4027 Refs: #4094 Refs: #4113 Refs: #4114 Refs: #4115 Refs: #4116 Refs: #4583 Refs: #4600 Refs: #4656 ## Summary Fixes the `plugins/claude-config` findings from the audit of the unattended Cursor PR run. Every change is inside `plugins/claude-config/`; `hooks/exec-bash.mjs` is untouched. - `audit-instructions`: the findings relay now follows `criteria.md` (I31 and I33 surfaces, I32 tier by arm), `finding-ids.sh` refuses non-instruction files under `$HOME`, `SKILL.md` records the subagent-window claim and the I33 dispatch rule, and `execution-and-report.md` (a second home for facts other files own) is deleted. - `unhobble`: delivers the #4094 must-fix items (product-surface class, session branch, convention oracle test, `readd` refusal) and nice-to-have items (`status` fields, ledger grouping, `decide`), and removes silence as a deletion warrant (#3563). - `audit-permission-state`: independent review of the #5154 `defaultMode` lint, which reached main under a "Do not merge" body with no review. - `audit`: the live-hook ask row carries the unattended-lane note (#4600). `conflict-criteria.md` loses its dead repo-root link (#3568). - `README.md` and `CHANGELOG.md` are corrected; version 0.52.0. ## Fix - I15 link (#3568): `conflict-criteria.md` used six `../` segments that resolved outside the repository, and an installed plugin does not carry `docs/`. It now names the path in code font. - Relay (#4116, #4656): `emit-findings.sh` admits I31 in any skill-directory file and I33 in any skill-loaded file except `SKILL.md`, names the nearest ancestor `SKILL.md` as the I33 hub, and tiers I32 CRITICAL under `plugins/` and IMPORTANT elsewhere. The persist-admission text in `criteria.md` and `persist-findings.md` names the scanner (I28, I29) and lane (I30 to I33) intakes. - `finding-ids.sh` `surface_of()` (#4116, the security-lane finding left open at merge of #4851): the user surface stays home-wide, and now admits only instruction-file shapes. Basis: Claude Code reads `CLAUDE.md`, `CLAUDE.local.md` and `AGENTS.md` from the working directory and every directory above it, and follows imports to absolute paths, so narrowing to `${CLAUDE_CONFIG_DIR:-$HOME/.claude}` would drop real user surfaces. Any markdown file, `settings.json`, `settings.local.json` and `hooks.json` inside a `.claude` tree or the resolved `CLAUDE_CONFIG_DIR`, and files beneath a `skills/` directory in those trees are admitted; `~/.ssh/config`, credentials, transcripts and dotfiles are refused as `surface-not-an-instruction-file`. The scope carries its four-part verification record in the code. This is the citation for dismissing the open PR-comment finding on #4851. `relativize_in_repo` is unchanged. - `audit-instructions` docs (#4113, #4114, #4115, #4656): lane sizing loses its ticket back-references, the subagent-window claim gets a four-part record, the read-only contract covers lane reports and run-state writes, I33 rows count as one dispatch outside per-lane verifier batches, and `SKILL.md` regains line headroom under the 500-line cap. - `audit-engine.sh` (#4600): the `HOOKS_LIVE=1` row ends with `$lane_note`, like the other ask-rule rows. - `unhobble` (#4094, #3563): the one-row watch disqualifier versus the two-row re-add grammar is this skill's own rule (`SKILL.md` Phase 4 step 1), not an upstream one. `SKILL.md` and `evals/evals.json` (evals 27 to 30 added; 18 and 22 tightened). Items 1 to 4 and 7 of #4094 were already on main (#5081). Items 5, 6, 8, 9 and 10 to 12 land here. - #5154 review (#4027): `permission-state.sh` classified a settings file `invalid-json` when the last key under `permissions` was a string, because `jq -e` reads only the last output of a per-key check, so `C2-defaultMode` and `C5-disableType` could not fire on real files. Fixed with tests that fail against the old reader. The masking claim now carries its condition, and the unsourced auto version boundary is removed. Claims were checked against pages fetched 2026-09-29 (recorded in the commit body). - #4656 equivalence proof: regenerated at the #4839 pair; every output matched byte for byte and the grep count is constant. Evidence only, no change committed. - `CHANGELOG.md` (#4027): the repeated 0.51.17 and 0.51.18 bodies became pointers, 0.51.7 records that it shipped through #5154, and the header notes the unreleased reserved versions. - Cross-group requests applied (#4027): `criteria.md` I21 and the `audit` effort-pin row state the effort defaults from the model-config resolution order fetched 2026-09-28 (Opus 5.5 and Sonnet 5.5 default to `medium`, Opus 4.7 to `xhigh`; the first-run-hold clause is gone), with Verified, Source and the recheck trigger restamped. `unhobble` `SKILL.md` and eval 15 stop restating the `block-hook-bypass` exit code and heredoc coverage and defer to the hook. In-place changelog corrections: - 0.51.18: body replaced by a pointer; it repeated the 0.51.16 Fixed entry (released by #5159) and a reworded near-duplicate of the 0.51.10 Changed entry (released by #5161). - 0.51.17: body replaced by a pointer; it repeated the 0.51.15 entry (released by #5156). - 0.51.7: states it shipped through #5154 (commit 9c2db71), not #5059 (closed unmerged). - File header: notes that 0.51.8 to 0.51.9 and 0.51.11 to 0.51.14 were reserved by parallel branches and never released. No issue is closed by this PR. #4094 stays on `Refs` because its 14-item checklist is not verified complete here, and the others are decision-held or have leftovers in other groups. ## Verification Run in `/home/kyle/worktrees/ccp-fix-claude-config` after merging `origin/main` (aebb9bb): - All 34 `plugins/claude-config/**/*.test.sh` suites: exit 0 (includes `emit-findings.test.sh`, `finding-ids.test.sh`, `finding-identity.test.sh`, `permission-plane-lint.test.sh`, `permission-state.test.sh`, `audit-engine.test.sh`, `lane-runs.test.sh`, `instruction-files.test.sh`). - `bash scripts/check-detector-findings-crosswalk.sh --check`: OK, 38 rule rows. - `bash scripts/check-changed-skills.sh origin/main`: 4 skills checked, 0 failed. - `bash scripts/check-changelog-parity.sh --check --check-order`, `--check-bump origin/main`, `--check-preserved origin/main` (190 headings compared): pass. - `bash scripts/validate-plugins.sh`: all manifests and the catalog validated. - `shellcheck` on every changed `.sh` file (the SC2016 hit on the `emit-findings.test.sh` rule fixture line is suppressed with a reason) and `check-evals-quality.sh` on the changed evals pass; `check-skill.sh` warnings that remain were present on main. #5154 review outcome (#4027): - Claims checked: the `defaultMode` masking claim, the `permissions` key shapes the lint reads, and the auto-mode version boundary, each against Claude Code docs pages fetched 2026-09-29. - Evidence: the masking claim holds only under a stated condition, now carried in the text; the auto version boundary had no source and is removed; `permission-state.sh` reproduced the `invalid-json` misclassification when the last `permissions` key was a string. - Defect fixed: the per-key `jq -e` check counted only the last output, so `C2-defaultMode` and `C5-disableType` could not fire on real files. New tests in `permission-state.test.sh` fail against the old reader and pass now. ## Related Audit report: `.work/audit/REPORT.md` findings by issue number: #3563, #3568 (row 3c), #4027 (3b), #4094 (3b), #4113 (3b), #4114, #4115, #4116, #4583, #4600, #4656 (3c). Cross-group requests (not done here): - core-docs: `docs/plugin-philosophy.md` #4583 structure and citation defects, the `docs/upstream/claude-code.md` ledger re-point from #5059 to #5154, #5159 and #5161, the row 257-089 status, the FORCE decline reconciliation, and review of the #5154 philosophy changes. - playbooks: review of the #5154 boris changes. - scripts: `check-changelog-parity.sh` should reject a CHANGELOG entry whose body is byte-identical to an earlier entry. - source-control: a body containing "Do not merge" must block the unattended merge lane. - review: closing comment on #3566. - ci: reopen #4094 and comment that this PR delivers items 5, 6, 8, 9, 10 to 12. - conventions: `docs/conventions/detector-findings/README.md` crosswalk rows for I31, I32 and I33 restate the surface set and the I32 tier as `criteria.md` defines them. Cross-group requests received and not applied here: - playbooks (docs/upstream row 257-083 rewrite): forwarded to core-docs, which owns `docs/upstream/claude-code.md` and already edits that row. - ci (#4094): items 5, 6, 8, 9 and 10 to 12 already land in this PR. - review (#3566): the review group posts that comment. - tracker (#4656): label removed and results noted on the issue. - context-guard (row 260-002 home), context-budget (Items column), skill-quality and session-flow follow-ups: added to the #4027 packet as open questions. decisions-docs (#3568): both questions added to the #3568 packet. Nothing implemented. Issue operations done: #4027 and #4115 reopened with decision packets, packets on #3568 and #4027 extended, comments on #4113, #4116, #4656, #4583 and #3563. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01EugXnFddtpHcY5gTuyEirB --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Closes #4116
Summary
Give every
audit-instructionsfinding a stableaudit-passidentity, and persist I30–I33 from the lane.Deviation from #4116: the issue says a lane-sourced finding's confidence is fixed to
highbecause "a deterministic detector still fired". No scanner produces I30–I33, so that premise does not hold; lane rows leaveConfidenceblank instead of claiming a detector confidence.Fix
(check, claim, sites):checkisclaude-config/audit-instructions/<id>, claims live inreference/finding-identity.md, anchors areanchor/v1excerpts with the heading-path discriminator, and an I15 conflict is one finding with two sites.scripts/finding-ids.shderivesfinding_id/v1andgroup/v1throughaudit-pass'sfinding-identity.sh.emit-findings.sh --from-laneemits I30–I33 (detector-findings 3.2.0 crosswalk). I32 in frontmatter and I31/I33 outside a spoke are declined with a named reason. Phase D table gains a Finding ID column.finding_idare emitted once, with anIdentity collisions: finding_id=<id> count=<n>line under## Surfaces, per the audit-pass identity contract./plugin:skillor a backtickedplugin:skill; anything else reportsshape=.Verification
finding-ids.test.sh: 28 passed (a new case proves a fenced## not a headingis not in the heading path).emit-findings.test.sh: 170 passed (new Case 16: identical lines collide to one row withcount=2;10:30yieldsshape=).scripts/check-detector-findings-crosswalk.sh,scripts/validate-plugins.sh,scripts/check-changelog-parity.sh --check --check-order,scripts/check-changed-skills.sh origin/main: pass.Related
docs/conventions/detector-findings3.2.0.plugins/review/skills/audit-enforceability/context/crosswalk.mdhas no exact rows for the four new lane rules, so they fall to theclaude-config/audit-instructions/family row ("already deterministic").🤖 Generated with Claude Code
https://claude.ai/code/session_01Hx63mvV8t39y644DQ3uLnn