[kyle-sexton]

Refs: #6598

Summary

This PR ships the #6598 concise rewrite of the /ai-slop:audit skill body. It was measured with no loss on Opus 5.5 and Sonnet 5.5 over 9 of the plugin's 10 outcome cases, and every pass that was read is VALID. The description and the setup skill are unchanged. #6598 stays open for one fix: audit-over-compression-boundary does not load (see Verification).

Fix

• plugins/ai-slop/skills/audit/SKILL.md: 347 to 336 lines, 3735 to 3629 words. These parts now take fewer words:
  • the Purpose paragraph, which drops the catalog's provenance (the catalog records it itself)
  • the rubric fan-out step
  • the step 6 relay recommendation
  • fix step 1
• These are byte-identical: the frontmatter, the gather block, the ! pre-compute line and its pointer record, the user-scope, non-repository target, digest-guard and configuration sections, and the untrusted-content spine.
• Changelog fragment .changes/ai-slop/ (patch).

Skill	Part	Decision	Basis
audit	description	unchanged	#11821 measured and shipped it on 2026-11-01. It ran a trigger pass on the committed probes/audit.json plus a blind held-out set, on Opus and Sonnet, all VALID: no lost trigger, and held-out triggering rose from 58% to 83% on Opus and from 57% to 70% on Sonnet. No trigger pass here.
audit	body	ships after text	Rules 1-3 hold on both models (table below).
setup	description, body	unchanged, unmeasured	It has no case.yaml outcome case, only a skill-creator evals.json. The runbook records that the eval sandbox denies a setup-style skill's config reads and .claude/ writes.

Verification

Protocol

• Preregistered before any paid run, mirroring Concise skills rollout: debugging (1 skills, 1 model-invoked) #6603 round 2. A version ships only if, on each model:
  • at least 8 of 10 cases are read (both versions VALID)
  • the mean with-arm score after is at least the score before minus 0.05
  • no case falls by 0.333 or more
• Suite frozen at plugins/ai-slop/evals tree b6f58d59b7edc2051d96044113461f48635b90a9, with the same digest in both run copies.
• Run setup: Claude Code 2.1.316, claude plugin eval, 3 runs per version per case, judge opus, threshold 0.8, --scaffold, --ablation none (with-arm only). Each case is granted the gated tools its allowed_tools lists. Before and after launch together per case. run-validity.py --scaffolded judges each pass at origin/main.
• /evals:validate (validate-cases.py): exit 0, warnings only.

Before the passes

• Host-safety probe: the sandbox sees no Windows drive contents or binaries, and WSL_INTEROP is unset.
• Judge calibration: all 9 calibratable llm graders scored 100% (88 judged runs). facts-kept-nothing-invented has file focus, so the builder skips it.

Meaning diff

• Two fresh docs-hygiene:compare-labeler agents, then fresh rechecks until one passed.
• The union found three losses, and each was restored before any paid run:
  • the step 3 link to context/rubric-fanout.md
  • "(V1 boundary, revisit with field history)"
  • "takes its ordinary cleanup class"
• A recheck then found that the merged DATA paragraph reworded the untrusted-content spine, which docs/contracts/untrusted-content/README.md requires byte-identical. It was restored too.
• The final recheck: PASS, with 0 semantic loss and 0 ambiguity.

Passes

• Sonnet's fix-plain-words, fix-voice-invariants and vague-attribution passes were INVALID, each on one denial of a read under the plugin copy (a known harness class). Each was rerun as a pair per the rule, and the reruns are read.

Case	Opus before	Opus after	Sonnet before	Sonnet after
audit-bare-optimism-closer	1.000	1.000	1.000	1.000
audit-challenges-closer	1.000	1.000	1.000	1.000
audit-commit-message-no-git-write	1.000	1.000	1.000	1.000
audit-curly-quotes-markdown	1.000	1.000	1.000	1.000
audit-fix-colon-crutch	0.933	0.933	1.000	1.000
audit-fix-plain-words	1.000	1.000	1.000	1.000
audit-fix-voice-invariants	0.583	0.833	0.750	0.750
audit-mechanism-free-claim	0.933	1.000	0.867	0.867
audit-vague-attribution	1.000	1.000	1.000	1.000
Mean (9 read)	0.939	0.974	0.957	0.957

• Opus: mean +0.035, and no case falls. Sonnet: mean +0.000, and no case falls. noise-report.py on each pair says "n too small to call" (one case per file) and prints no case drop line.
• Unmeasured: audit-over-compression-boundary, on both models. Its grader fixture-unchanged.md fails to load on both versions ("invalid YAML frontmatter: YAML Parse error: Unexpected EOF"), so it produced 0 runs. Its regex holds |---| table rows, which the frontmatter split reads as a delimiter. validate-cases.py passes the file. The suite stays frozen here, so the fix is follow-up work.
• Headroom: 7 of the 9 read cases score 1.0 on both versions on at least one model, so the suite has little room to show a loss there. facts-kept-nothing-invented fails on most audit-fix-voice-invariants runs on both versions and both models, and it cannot be calibrated (file focus).
• /skill-quality:check (check-skill.sh plugins/ai-slop/skills): both skills PASS before and after, with 0 errors. audit has the same ten warnings; its router estimate drops from about 6573 to 6392 tokens, still above the 5000 WARN line, and an actions/ split is left for a round with cases behind those sections. setup is unchanged.
• gitleaks git --log-opts=origin/main..HEAD --redact: no leaks.
• Eval spend: 49 passes including probe and calibration, about 39 USD (list-price estimate, sum of costUsd).

Remaining

• Concise skills rollout: ai-slop (2 skills, 1 model-invoked) #6598: fix audit-over-compression-boundary/graders/fixture-unchanged.md so the case loads, then measure that case on both versions. That is what keeps the issue open.
• Evals plugin, outside Concise skills rollout: ai-slop (2 skills, 1 model-invoked) #6598: validate-cases.py passes a grader whose frontmatter claude plugin eval cannot parse (a |---| row inside a regex).

Related

• Rollout parent: Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476. The description this PR keeps was measured in fix(ai-slop): name commit messages, PR bodies and interface copy in the audit description #11821. The suite and probes come from feat(ai-slop): audit commit messages, PR bodies and interface copy #11535.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-02T01:06:14.276728Z	c5881aa	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed c5881aa6bef94ef9a80a2021810706b3dc093b3e: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #12201, head c5881aa6

No security issues found.

I read every hunk in both changed files:

• .changes/ai-slop/feat-6598-concise-audit-body-5a9c2e71.md: a new changelog fragment containing only text.
• plugins/ai-slop/skills/audit/SKILL.md: rewrites four parts in fewer words (the Purpose paragraph, the step 3 rubric fan-out, the step 6 relay recommendation and fix step 1).

Neither file matches a trusted-path glob. The PR changes no trust control, workflow, manifest or userConfig key.

Instruction-surface check. The change shortens instruction text in a skill body, so I checked each removed sentence to see what stops working without it:

• Purpose: drops where the catalog came from (Wikipedia revision pin, the "General-prose additions" and "Model-era additions" sections). This is description, not a rule.
• Step 3: "Coverage is uniform" and the sentence on the rubric pass being independent of the detector are folded into "whether or not the detector found anything in it". "Never trade the rubric away for budget" is kept. The rubric working directory still says "with no remote fetch", and the stale-result rejection is kept.
• Step 6: "Recommend, never auto-run" is unchanged. The rule against promising the relay will apply rule-utm-params, and the condition about /ai-slop:audit being available, are both kept.
• Fix step 1: "Meaning wins over style… skip it and record why", the triad no-collapse condition, the self-audit and the voice-pass exclusions (never operative instructions or reference tables) are all kept. Only the rationale sentence "Fewer parallel items is also less to maintain" is dropped.

No rule that guards against a protected-class action (a git write, a remote fetch, untrusted-content handling, auto-running a fix) is removed or weakened. The untrusted-content DATA spine, the frontmatter and the ! pre-compute line are outside the diff, so they are unchanged. I used recognition by consequence and did not read docs/contracts/instruction-exception-register/README.md from the checkout.

Spend: none. The diff changes no turn limits, models, schedules or billing.

Prior findings: none; round 0 of 3, below the cap.

critical-findings: 0

[github-actions]

claude-review reviewed c5881aa6bef94ef9a80a2021810706b3dc093b3e: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #12201, feat(ai-slop): trim the audit skill body

No findings. I read every hunk in both changed files: plugins/ai-slop/skills/audit/SKILL.md and .changes/ai-slop/feat-6598-concise-audit-body-5a9c2e71.md.

Checks on the reworded passages

• Purpose: the new text says findings persist as "a findings file (audit step 5)". Step 5 at SKILL.md#L117 is "Persist the findings file", so the new cross-reference is correct. The PR drops the catalog's provenance and the word "conforming". Neither carries an instruction: step 5 and context/persist-findings.md still own the producer contract.

• Step 3 (rubric fan-out): these all survive the rewrite:

  • the rubric runs on every file, independent of the detector
  • never trading the rubric away for budget
  • the order of plan, extract, status and merge
  • the working-directory pointer with no remote fetch
  • the rule that rejects a stale result bound to an old scope or old content

• Step 6 (relay recommendation): these survive too:

  • the rule-utm-params exception
  • "do not promise that it will"
  • the /simplify preference and its fallback
  • that neither surface loads the rewrite guide
  • the condition that /ai-slop:audit must be available

• Fix step 1: these survive:

  • the replacement forms, including no parentheses and no en dashes
  • meaning winning over style
  • triads collapsing, with both of their exceptions
  • the scope of the voice pass
  • the self-audit before verification

  The PR drops only one sentence of rationale ("Fewer parallel items is also less to maintain"), and it gives no instruction.

• Changelog fragment: its frontmatter (bump: patch) and ### Changed section match the repo's other fragments, for example .changes/naming/feat-6611-concise-naming-3e7a1c52.md. A patch bump fits a change that only rewords the skill body.

Not in scope or not applicable

• Closing issue: none. The PR says Refs: #6598, not a closing keyword, so there are no acceptance criteria to check.
• Plugin manifests: the diff doesn't change plugin.json or marketplace.json.
• Prior findings: none. This is round 0 of a cap of 3, so the cap doesn't apply.
• Embedded instructions: the PR context contains no text that tries to direct this review lane.

Unverified author claims. The eval scores, the meaning-diff PASS, the check-skill.sh results and the gitleaks result all come from the PR author. I didn't run any of them. Spend: none from this PR. The author reports spending about 39 USD of API credit on their eval runs before opening it.