Skip to content

feat(claude-config): fence I6 gate set and per-spoke I33 (#4115) - #4850

Merged
kyle-sexton merged 10 commits into
mainfrom
cursor/4115-i6-i33-fences-37e9
Sep 29, 2026
Merged

kyle-sexton merged 10 commits into
mainfrom
cursor/4115-i6-i33-fences-37e9

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

No related issue: partial delivery of #4115; its AC2 (five previously-retained findings still seed) cannot be verified because those five findings are not recorded anywhere, so #4115 stays open for that check.

Summary

Refs #4115. Fence the I6 pre-scan to the docs-hygiene rule-negation-without-positive gate set (minus its hard-guardrail carve-out), and report I33 as one finding per spoke with an opener-sentence excerpt anchor.

Fix

  • I6 rows are sentences that open with a prohibition and carry neither a paired positive nor a rationale marker. Soft-wrapped lines join first; frontmatter, fences, tables, headings, and HTML comment openers are skipped. A fence closes only on the opener's character, at least the opener's length, with only whitespace after it; blockquoted fences are recognized. --i6-counts prints raw and surviving counts.
  • audit-noise's hard-guardrail carve-out is not adopted: a guardrail "never" still owes I6's fallback rationale (I7), so it stays a candidate. Stated in the scanner header and CHANGELOG.
  • I6 emits one row per physical line, skips Setext headings, and reads a cue wrapped in underscore emphasis.
  • I33 is one finding per spoke, anchored on the opener sentence with the heading path as discriminator. Phase D rolls I33 up per plugin.
  • criteria.md catalog 1.24.0. Four fixtures plus evals 27 and 28.
  • Version 0.51.28 with CHANGELOG.

Verification

  • instruction-scan.test.sh 139/139 (Case 3f: fence closing; Case 3g: one row per line, Setext headings skipped, underscore-emphasized cues read), emit-findings.test.sh 172/172, finding-ids.test.sh 29/29, lane-runs.test.sh 52/52.
  • Count reproduction at 2dfaaa40 (990 files) prints I6 raw=6608 surviving=913:
    find . -name '*.md' -not -path './.git/*' \( -path './plugins/*/skills/*' -o -path './plugins/*/agents/*' -o -path './.claude/*' -o -name CLAUDE.md -o -name AGENTS.md \) -print0 | sort -z | xargs -0 bash plugins/claude-config/skills/audit-instructions/scripts/instruction-scan.sh --i6-counts
  • scripts/validate-plugins.sh, scripts/check-changelog-parity.sh --check --check-order, scripts/check-changed-skills.sh origin/main: pass.
  • Known limits: lines inside a multi-line HTML comment are still read; an enumeration after the cue (Never commit files, logs, or caches.) reads as a named alternative, as in audit-noise.

Related

🤖 Generated with Claude Code

@cursor
cursor Bot force-pushed the cursor/4115-i6-i33-fences-37e9 branch 3 times, most recently from 7995535 to 3970ad0 Compare September 28, 2026 12:24
…4115)

Adopt the measured I6 pre-scan gates (sentence-open prohibition, paired
positive, rationale, structural exclusions) and report I33 one finding
per spoke with excerpt anchors and a per-plugin roll-up.

Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
@cursor
cursor Bot force-pushed the cursor/4115-i6-i33-fences-37e9 branch from 3970ad0 to 1f7584e Compare September 28, 2026 12:38
kyle-sexton and others added 3 commits September 28, 2026 11:43
# Conflicts:
#	plugins/claude-config/.claude-plugin/plugin.json
#	plugins/claude-config/CHANGELOG.md
…ne cap (#4115)

Moves the I33 roll-up layout to context/execution-and-report.md and the
lane-brief fence note to the I33 catalog row; SKILL.md points at both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PT4esxbdC7eQiwQxy35Mie
kyle-sexton and others added 2 commits September 28, 2026 14:01
…o 1.24.0

A fence line carrying an info string no longer closes an open fence, and
blockquoted fences are recognized, so code after a pseudo-close is not
seeded and prose after the real closer is. The scanner header and the
CHANGELOG state why audit-noise's hard-guardrail carve-out is not adopted
and name the file set behind the 6,608 -> 913 count. criteria.md moves
to 1.24.0 for the I6 Detect and I33 Reporting edits (#4115).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hx63mvV8t39y644DQ3uLnn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review September 28, 2026 18:21
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-28T18:26:19.315556Z ed89be7 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 6s —— View job


Security review

  • Gather context (PR diff, changed files)
  • Review changed files for security-relevant issues
  • Adversarial validation of any candidate findings
  • Post final results

Reviewed all 13 changed files (git diff origin/main...HEAD) against the criteria in /review:security-review.

No security issues found.

This PR is confined to the audit-instructions skill: a bash/awk text-processing scanner
(instruction-scan.sh) plus its tests, docs (SKILL.md, criteria.md, execution-and-report.md),
fixtures, and a changelog/version bump. Checked specifically for:

  • Injection surfaces: no eval, source, xargs, or sh -c/bash -c constructs. All awk -v
    variables (I6_CUE_ALT, I6_PAIRED_ERE, stopword/transparent lists, etc.) are static strings
    defined in the script, not derived from file content or argv. File paths flow through
    "${files[@]}" correctly quoted throughout (e.g. instruction-scan.sh#L358-L362).
  • Temp file handling: i6_tmp/other_tmp use mktemp with an EXIT trap cleanup
    (instruction-scan.sh#L559-L561) — no predictable-path or race issue.
  • Trust boundary: the scanner only reads file content as text for regex/sentence matching
    (never executes it) and only reads file paths explicitly passed as arguments by the caller
    (an operator/CI audit run over repo-local markdown) — no attacker-controlled input crosses into
    a command context.
  • Actions hardening: no workflow files, triggers, or permissions: blocks are touched by this
    PR (out of scope for this lane regardless; zizmor's advisory lane owns that surface).

The doc/fixture/changelog changes carry no logic that touches secrets, auth, or external I/O.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ed89be7ecb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@claude

claude Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 5m 14s —— View job


Review complete

  • Gather context (diff vs origin/main, changed files)
  • Run /review:code-review per custom instructions
  • Post inline findings
  • Post summary

Ran the code-review lane against this PR's diff (origin/main...HEAD), scoped to the changed files in plugins/claude-config/skills/audit-instructions/. Focused on the new I6 sentence-scan logic in instruction-scan.sh, since that's where nearly all the behavioral risk in this PR lives (277 new/changed lines, dense awk).

1 finding, posted inline on instruction-scan.sh (line 430-432):

I6's new emphasis-marker stripping only works for */** (asterisk) emphasis; a cue word wrapped in _/__ (underscore) emphasis — e.g. _Never_ delete the backup file. — fails the cue's boundary check ([^[:alnum:]_] excludes _) and is silently never flagged, even though the new header comment and criteria.md both say emphasis markers around the cue are stripped. None of the new tests (Case 3b–3f) exercise underscore emphasis, only > **Do not**. Full trace and a [Fix this →] link are in the inline comment.

I did not find other functional bugs in the fence-closing, sentence-splitting, soft-wrap-join, or paired-positive/names_alternative logic after tracing them by hand against the new Case 3b–3f tests (all traced correctly). I could not execute instruction-scan.test.sh myself in this sandbox (script/awk execution was denied), so the PR's claimed 131/131 pass and the 2dfaaa40 count reproduction are author-claimed and unverified by me, not independently confirmed.

Out of scope for this lane: security findings (none observed) and the open AC2 item the PR body already flags (five previously-retained findings not recorded anywhere) — that's a process/tracking gap on issue #4115, not a code defect in this diff.

…ad underscore emphasis

A line opening two bare prohibitions now yields one I6 row, so --count
and --i6-counts no longer double it. A setext underline drops the pending
paragraph as a heading, except after a list item or blockquote, where
--- is a thematic break. An underscore after the cue now counts as a
boundary, so _Never_ and __Do not__ reach the sentence gate (#4115).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kyle-sexton added a commit that referenced this pull request Sep 29, 2026
… (#4851)

Closes #4116

## Summary

Give every `audit-instructions` finding a stable `audit-pass` identity,
and persist I30–I33 from the lane.

Deviation from #4116: the issue says a lane-sourced finding's confidence
is fixed to `high` because "a deterministic detector still fired". No
scanner produces I30–I33, so that premise does not hold; lane rows leave
`Confidence` blank instead of claiming a detector confidence.

## Fix

- Identity is `(check, claim, sites)`: `check` is
`claude-config/audit-instructions/<id>`, claims live in
`reference/finding-identity.md`, anchors are `anchor/v1` excerpts with
the heading-path discriminator, and an I15 conflict is one finding with
two sites.
- `scripts/finding-ids.sh` derives `finding_id/v1` and `group/v1`
through `audit-pass`'s `finding-identity.sh`.
- `emit-findings.sh --from-lane` emits I30–I33 (detector-findings 3.2.0
crosswalk). I32 in frontmatter and I31/I33 outside a spoke are declined
with a named reason. Phase D table gains a Finding ID column.
- Rows that share a `finding_id` are emitted once, with an `Identity
collisions: finding_id=<id> count=<n>` line under `## Surfaces`, per the
audit-pass identity contract.
- The I32 fallback target matches only `/plugin:skill` or a backticked
`plugin:skill`; anything else reports `shape=`.
- detector-findings README adopter row states the I30–I33 tier spread.
- Version 0.51.27 with CHANGELOG.

## Verification

- `finding-ids.test.sh`: 28 passed (a new case proves a fenced `## not a
heading` is not in the heading path).
- `emit-findings.test.sh`: 170 passed (new Case 16: identical lines
collide to one row with `count=2`; `10:30` yields `shape=`).
- `scripts/check-detector-findings-crosswalk.sh`,
`scripts/validate-plugins.sh`, `scripts/check-changelog-parity.sh
--check --check-order`, `scripts/check-changed-skills.sh origin/main`:
pass.

## Related

- `docs/conventions/detector-findings` 3.2.0.
- I6/I33 fences are #4115 (PR #4850).
- Follow-up outside this plugin:
`plugins/review/skills/audit-enforceability/context/crosswalk.md` has no
exact rows for the four new lane rules, so they fall to the
`claude-config/audit-instructions/` family row ("already
deterministic").

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01Hx63mvV8t39y644DQ3uLnn

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 24s —— View job


Security review

  • Gather context (PR diff, changed files, existing review comments)
  • Run /review:security-review per custom instructions
  • Post final results

Reviewed the full diff (git diff origin/main...HEAD, HEAD fb08e60) against /review:security-review's criteria. All 13 changed files are confined to the audit-instructions skill: instruction-scan.sh (the bash/awk I6 sentence-scan and I33 doc changes) plus its tests, SKILL.md/criteria.md/execution-and-report.md docs, JSON fixtures/evals, CHANGELOG.md, and a version bump.

No security issues found.

Checked specifically:

  • Injection surfaces: no eval, source, xargs, or sh -c/bash -c. Every awk -v variable (cue, anycue, paracue, paired, rationale, stopwords, transparent, the I6_*/I8_*/I23_*/etc. EREs) is a static string built from other static strings in the script, never derived from file content or argv. File paths flow through "${files[@]}"/"${files[@]:chunk_start[c]:chunk_len[c]}", correctly quoted throughout (e.g. instruction-scan.sh#L352, #L362-L489).
  • Temp file handling: i6_tmp/other_tmp via mktemp with an EXIT trap cleanup (instruction-scan.sh#L547-L549) — no predictable-path or race issue.
  • Trust boundary: the scanner only reads file content as text for regex/sentence matching (awk's getline line < p in frontmatter_end, #L266-L276, is a read-only reopen of a caller-supplied path, never executed) and only reads file paths explicitly passed as arguments by the caller (an operator/CI audit run over repo-local markdown) — no attacker-controlled input crosses into a command context.
  • New logic since the last review (b932720: I6 per-line dedup via the emitted[] array, Setext-heading exclusion, underscore-emphasis boundary) is pure detection-logic refinement over already-local trusted file content; it doesn't change how paths or content reach a shell/awk command.
  • Instruction-surface deletions: the SKILL.md/criteria.md/execution-and-report.md edits narrow and rewrite this audit tool's own detection criteria (what counts as an I6/I33 finding). None of them delete, narrow, or soften a standing behavioral guardrail (hook block list, permission rule, CLAUDE.md/AGENTS.md constraint) that governs the model's own actions, so the instruction-exception-register check doesn't apply here.
  • Actions hardening: no workflow files, triggers, or permissions: blocks are touched (out of scope for this lane regardless).

The CHANGELOG.md, plugin.json version bump, evals.json, and fixture .md files carry no logic and no secrets/auth/external-I/O surface.
· branch cursor/4115-i6-i33-fences-37e9

🤖 Generated with Claude Code

@claude

claude Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 5m 23s —— View job


Review complete

  • Gather context (diff vs origin/main, changed files)
  • Invoke /review:code-review per custom instructions
  • Post inline findings
  • Post summary

Ran the CI code-review lane (/review:code-review) against origin/main...HEAD (13 files, +579/-49), scoped per the lane's criteria to correctness/maintainability findings, with security out of scope (a separate security lane already ran clean on this PR).

Since the last review of this PR (commit ed89be7), commit b932720 fixed all three previously-reported findings from Codex and Claude:

  • I6 rows deduped per physical line (Codex)
  • Setext headings excluded (Codex)
  • Underscore-emphasized cues (_Never_, __Do not__) now reach the sentence gate (Claude)

I traced that fix commit by hand against the current instruction-scan.sh (execution of bash/awk was denied in this sandbox, so nothing below is execution-verified, only traced from source) and found 1 new finding, posted inline on instruction-scan.sh:365:

Fixing the underscore-emphasis case widened the cue regex's trailing-boundary class from [^[:alnum:]_] to [^[:alnum:]], i.e. it now treats a bare _ as a valid boundary character everywhere cue is used (judge() at line 436, names_alternative() at line 422) — not just right after a stripped opening emphasis marker. That reintroduces a false-positive path: a sentence that opens with a snake_case identifier like `never_delete_flag` controls whether files are removed. now matches the I6 cue, because _ right after "never" now satisfies the boundary, when previously _ was deliberately excluded from that class to avoid matching inside identifiers. None of the new Case 3g assertions cover a snake_case-prefixed sentence, so this isn't caught by the test suite. Full trace, repro, and a suggested narrower fix (only accept the trailing _/__/___ as a boundary right after the opening-marker strip recognizes a matching opener, rather than unconditionally) are in the inline comment, along with a [Fix this →] link.

I did not find other functional issues after tracing the fence-closing, Setext-heading, and per-line-dedup logic by hand — those three fixes look correct for the cases they target. I33's changes in this PR are documentation/process only (no scanner code), so out of scope for a code-correctness pass.
· branch cursor/4115-i6-i33-fences-37e9

Comment thread plugins/claude-config/skills/audit-instructions/scripts/instruction-scan.sh Outdated
kyle-sexton and others added 2 commits September 28, 2026 21:30
…ke_case

`_` after a cue counts as a boundary only when a non-word character or the
end follows it, so `_Never_` still reads and `never_delete_flag` does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 48s —— View job


Security review

  • Gather context (PR diff origin/main...HEAD, HEAD 593917a; prior review history)
  • Review changed files/new commits for security-relevant issues
  • Adversarial validation of any candidate findings
  • Post final results

No security issues found.

This PR remains confined to the audit-instructions skill (13 files, +582/-49): the instruction-scan.sh bash/awk text-processing scanner plus its tests, docs (SKILL.md, criteria.md, execution-and-report.md), fixtures, evals.json, and a changelog/version bump.

Since the last successful security review of this PR (fb08e60), the only new commit is ca1f076 (plus the no-op 593917a merge from origin/main):

  • ca1f076 — narrows the I6 cue-boundary regex in run_i6() from ([^[:alnum:]]|$) to _*([^[:alnum:]_]|$) (and the matching paracue pattern), at instruction-scan.sh#L365-L366. This fixes the precision regression a prior review flagged (never_delete_flag was matching the I6 cue after the earlier underscore-emphasis fix). It's a pure detection-logic tweak — the cue/paracue values are still static strings built from other static strings in the script, never derived from file content or argv, and the change touches no file I/O, path construction, or command execution. The paired instruction-scan.test.sh change just adds a fixture line and a assert_not_contains case for the snake_case identifier.

Re-confirmed the invariants from the prior passes on this PR:

  • Injection surfaces: no eval, source, xargs, or sh -c/bash -c. All awk -v variables remain static strings; file paths flow through "${files[@]}", correctly quoted throughout.
  • Temp file handling: i6_tmp/other_tmp via mktemp with an EXIT trap cleanup — no predictable-path or race issue.
  • Trust boundary: the scanner only reads file content as text for regex/sentence matching (never executes it) and only reads file paths explicitly passed as arguments by the caller — no attacker-controlled input crosses into a command context.
  • Instruction-surface deletions: the SKILL.md/criteria.md/execution-and-report.md edits narrow and rewrite this audit tool's own detection criteria (what counts as an I6/I33 finding), not a standing behavioral guardrail (hook block list, permission rule, CLAUDE.md/AGENTS.md constraint) governing the model's own actions — the instruction-exception-register check doesn't apply.
  • Actions hardening: no workflow files, triggers, or permissions: blocks are touched by this PR (out of scope for this lane regardless).
    · branch cursor/4115-i6-i33-fences-37e9

@claude

claude Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 3m 52s —— View job


Code review complete

  • Gather context (diff vs origin/main, changed files, full prior-review thread)
  • Invoke /review:code-review per custom instructions
  • Post inline findings
  • Post summary

Reviewed the current head (593917a, merge of origin/main after ca1f076) against origin/main...HEAD (13 files, +582/-49), scoped to correctness/maintainability per the CI code-review lane's criteria (security is out of scope — a separate security lane already ran clean on this PR).

No new findings. This is the fourth review pass on this PR; the prior three rounds (Codex + this lane) found four real issues, and all four are now fixed on this head:

  1. Multiple prohibitions on one physical line double-counted → fixed in b932720 (per-line emitted[] dedup), covered by Case 3g.
  2. Setext headings (===/--- underlines) read as prose → fixed in b932720, covered by Case 3g.
  3. _Never_ / __Do not__ (underscore-emphasized cues) silently never flagged → fixed in b932720.
  4. That fix's own regression — loosening cue's trailing boundary to [^[:alnum:]] made a bare _ a boundary everywhere, so `never_delete_flag` controls... would have matched I6 — fixed in ca1f076 by introducing a separate paracue (used for the paragraph/sentence-gate check, boundary _*([^[:alnum:]_]|$), i.e. "eat trailing underscores from a closed emphasis span, but don't treat a lone _ as a boundary") while restoring cue's original [^[:alnum:]_] boundary for judge()/names_alternative().

I traced fix #4 by hand against the new cue/paracue regexes at instruction-scan.sh#L365-L367:

  • _Never_ delete the ledger. → opening-marker strip removes the leading _, leaving never_ delete...; cue's _* in ^(never|...)_*([^[:alnum:]_]|$) consumes the trailing _, then the space satisfies the boundary → still flagged. ✓
  • `never_delete_flag` controls... → backticks stripped, no list/emphasis marker to strip, g = "never_delete_flag controls..."; _* can consume 0 or 1 underscores but in both cases the next character (_ or d) fails [^[:alnum:]_] → no match, correctly excluded. ✓
  • Both cases are also covered by the new instruction-scan.test.sh Case 3g assertions (lines 12/14/16 of the fixture).

I could not execute instruction-scan.test.sh, emit-findings.test.sh, or a bare awk/bash invocation myself — script execution was denied in this sandbox — so the PR's claimed 139/139, 172/172, 29/29, 52/52 pass counts and the 2dfaaa40 count reproduction remain author-claimed and unverified by me, consistent with every prior review pass on this PR.

One thing I noticed but am not flagging as a defect: the raw count (anycue, unchanged WB_L/WB_R boundaries) and the surviving count (paracue/cue, now underscore-aware) use deliberately different word-boundary definitions — a line whose only cue is _Never_ ... is not counted in raw but can still surface in surviving. This is explicitly called out in the b932720 code comment ("anycue keeps the raw count on the boundaries of the per-line rule") as an intentional scope choice, not an oversight, so it doesn't meet this lane's bar for a finding.

The I33 changes in this PR are documentation/process only (SKILL.md, criteria.md, execution-and-report.md, evals, fixtures) — no scanner code — so out of scope for a code-correctness pass, as noted in the prior review.
· branch cursor/4115-i6-i33-fences-37e9

@kyle-sexton
kyle-sexton merged commit a1733b0 into main Sep 29, 2026
19 checks passed
@kyle-sexton
kyle-sexton deleted the cursor/4115-i6-i33-fences-37e9 branch September 29, 2026 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants