[kyle-sexton]

Closes #6600

Summary

Concise-skills rewrite of codebase-health, run as a measured before/after pass under the
rewrite protocol.
Only the parts measured with no loss on both target models ship. That is three citation-only cuts
in the audit body. The audit description, every spoke, and the setup skill are unchanged.

Fix

• plugins/codebase-health/skills/audit/SKILL.md (body only, 348 lines / 3022 words to 348 /
  2996):
  1. drops a sentence describing how the /source-control:worktree record is cited (the pointer
     record, its as-of date and recheck trigger stay);
  2. drops the duplicate Phase 1 link to context/discovery-method.md (the remaining link now
     names the scope-fencing rules);
  3. drops the docs/plugin-philosophy.md provenance aside, a marketplace-internal file a consumer
     repo does not have (the cross-vendor-advisor rule it cites stays word for word).
• Changelog fragment (patch).

Per-skill decision table:

Skill	Part	Decision	Basis
audit	description	unchanged, no decision	Kept byte-identical: it carries 4 quoted triggers, and one held-out flip in 8 is 12.5 points, past the 5-point margin. No trigger pass was run.
audit	body	ships (cuts 1-3 above)	The r2 pair is VALID and non-inferior on Opus 5.5 and Sonnet 5.5 (table below).
setup	description and body	unchanged, unmeasured	Setup is user-invoked. It reads ~/.claude/codebase-health.md outside the workspace, and apply writes under .claude/, which the eval sandbox denies.

Verification

The pre-registration was written before any paid run. Every rule was added before the run it
governs, with one exception: Opus audit-cost-gate fell outside the splice rule (its rerun was
VALID on before only), so it was resolved as unmeasured after the rerun. It cannot move the
verdict: both versions scored 0.889 in every Opus observation. Suite: the frozen plugins/codebase-health/evals (git tree f9c3125e, 5 cases, 24
deterministic graders, from #11814). Flags: --runs 3 --scaffold --allow-tools Write Edit --ablation none. Every prompt invokes the skill by slash command, and no grader is with-only, so
the no-plugin arm adds nothing. CLI: Claude Code 2.1.316. run-validity.py and noise-report.py
come from main. In each pair, before and after were launched together.

Meaning diff: two fresh docs-hygiene:compare-labeler agents compared the body as first
rewritten, with REASONS none, and both returned VERDICT: BLOCK on the same three rows. The
injected-commands pointer record was restored. The other two (citation text) were carried with
written deletion-test reasons to two more fresh labelers, and both returned VERDICT: PASS (0
SEMANTIC LOSS, 0 AMBIGUITY). This departs from the pre-registered "restore the union" wording and
is recorded as a deviation. The shipped body (after-r2) was then labeled on its own by two more
fresh labelers against a REASONS file of only its three cuts: both VERDICT: PASS, with 3 INTENDED
CUT and 0 SEMANTIC LOSS each.

First pair (body with a fourth cut, the "Adapting to your environment" section):

• Opus: VALID and non-inferior on 4 cases (+0.08, 95% interval -0.04 to +0.19).
  audit-cost-gate was unmeasured: the same user-global-config denial hit both versions in a
  paired rerun.
• Sonnet: audit-zero-findings came in at 0.917 after vs 1.000 before, which breaks the per-case
  margin. The failing run put a needs-review item into the findings table as an info row.
• Following the protocol, that cut was restored (after-r2). A paired Sonnet rerun of the case
  scored 1.000 vs 1.000, both VALID. The restore closed the gap. Causation is not established:
  before showed the same miss shape once on Opus.

Final pair, before vs after-r2 (the shipped body), full suite, fresh before control. With-arm case
means over 3 runs:

Model	run-validity	boundary	cost-gate	fix	planted-drift	zero-findings	noise report
Opus 5.5	both VALID (spliced)	1.000 / 1.000	unmeasured	1.000 / 1.000	0.800 / 0.867	0.667 / 0.750	+0.04, -0.01 to +0.08, non-inferior at 0.05
Sonnet 5.5	both VALID	1.000 / 1.000	1.000 / 1.000	1.000 / 1.000	1.000 / 1.000	1.000 / 1.000	+0.00, non-inferior at 0.05

Notes on the Opus row:

• Cells are before / after-r2.
• Opus full passes were INVALID only on sandbox denials of ~/.claude/codebase-health.md (the
  config ladder's user-global layer, outside the workspace). As pre-registered, each denied case
  was rerun as a pair and its rerun numbers spliced in.
• audit-cost-gate scored 0.889 on both versions in every Opus observation but never had a VALID
  pair, so it is unmeasured.
• The remaining Opus misses appear on both versions: a section titled other than "Verified
  non-issues", or "config-gap" phrased differently. They are grader-shape misses, recorded, not
  fixed, because the suite is frozen.

Other checks:

• /skill-quality:check plugins/codebase-health/skills: 2 passed, 0 failed. The warnings are the
  same set as before the change.
• Push hooks check-affected and the guardrails reference and content checks passed. gitleaks
  over origin/main..HEAD: no leaks.

Unmeasured: setup; audit triggering (description unchanged, so no trigger pass); and
audit-cost-gate on Opus. The blind held-out trigger set (16 queries) was written for this round,
but nothing was run on it and it is not committed, so it is still unspent.

Spend: 19.75 USD list-price estimate, CLI-reported, across 22 result files including the INVALID
passes and reruns. Eval wall time: 17:44-18:16 UTC, 2026-11-01.

Related

• Umbrella: Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476
• Frozen suite and probes: test(codebase-health): add outcome eval cases and frozen trigger probes for audit #11814
• Eval validity fixes this pass depended on: fix(evals): let trigger-measurement probe runs pass run-validity #11618 (by fix(evals): judge invocation-probe runs by their measured rate in run-validity #11639), fix(skill-quality): stop invocation-probe runs failing validity on denied searches outside the workspace #11624, test(visualization): fix the frozen suite's graders and re-measure present and visualize with VALID runs #11628

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T18:45:05.912567Z	144e0e5	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed 144e0e5513fef193c5b0ef1a1dd363a51b44d772: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Security review: no security issues found

I read the whole diff for both changed files: plugins/codebase-health/skills/audit/SKILL.md and the new changelog fragment .changes/codebase-health/feat-6600-concise-audit-body-5d1e7a20.md. Neither file matches a trusted-path glob. Neither is a plugin manifest, workflow, action or trust control.

Instruction-surface check. The diff shortens a skill body, so I checked each removed sentence for a rule it might drop:

• Cut 1, lines 26-30: removes a sentence that only describes how the worktree record is cited. The rule stays: a pre-compute block that carries git is refused and git reads go in separate Bash calls. The pointer, its as-of date and its recheck trigger stay too.
• Cut 2, lines 205-218: removes a duplicate link to context/discovery-method.md. The scope-fencing requirement is still stated in the Phase 1 sentence and named again at the one remaining link.
• Cut 3, lines 231-238: removes a provenance pointer to docs/plugin-philosophy.md. The cross-vendor-advisor rule stays word for word, including the "when one is installed and set up" condition and the same-vendor fallback. The read-only fence on each validator (its findings' files plus verification sources) also stays.

No guardrail is removed or narrowed, so no protected class is affected. I did not read docs/contracts/instruction-exception-register/README.md because nothing was removed that needed checking against it.

PR context. The description and issue comments contain no text that tries to direct this lane.

Spend. The PR reports about 19.75 USD (list-price estimate) of eval runs that already happened. The change adds no recurring spend and nothing that bills.

critical-findings: 0

[github-actions]

claude-review reviewed 144e0e5513fef193c5b0ef1a1dd363a51b44d772: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Code review: no findings

I read every hunk of both changed files: the changelog fragment and plugins/codebase-health/skills/audit/SKILL.md. The PR makes three text-only cuts, and none of them removes an instruction the skill relies on:

1. Worktree record pointer (L25-30). The removed sentence only described how the record is cited. The pointer itself stays, with its as-of date and recheck trigger: the skill name, the gather-block reference, and the quoted section title.
2. Phase 1 discovery-method link (L206-218). The first sentence now says "the scope-fencing rules of the discovery method linked below". The one remaining link at L218 now names the scope-fencing rules too, so the link still resolves and the rules stay reachable.
3. Phase 2 aside (L233-238). Only the docs/plugin-philosophy.md aside was dropped. The cross-vendor-advisor rule and its fallback match the original word for word; the rest of the change is line rewrapping.

Path-scoped rules. .claude/rules/skill-bodies-state-current-rules.md and .claude/rules/cost-claims.md match plugins/*/skills/**. Neither is broken: no history narrative was added, and no cost claim changed. The cost-gate text in Phase 1 is unchanged.

Plugin manifests. Neither plugin.json nor marketplace.json changed. The fragment's bump: patch fits a body-only rewording.

Closing issue #6600, acceptance criteria:

• /skill-quality:check reports no new FAIL: only the author says so ("2 passed, 0 failed"). I did not verify it.
• /evals:validate passes: the PR body doesn't mention this command by name. It describes eval runs and the frozen suite, but this specific check is neither shown nor verified here.
• Probe file plugins/codebase-health/probes/audit.json: it exists in the repository; it came from test(codebase-health): add outcome eval cases and frozen trigger probes for audit #11814.
• Per-skill decision table, description and body decided separately: met, in the PR body.
• Unmeasured skills and cases named with reasons: met. The body names setup, audit triggering, and Opus audit-cost-gate.
• Version or changelog entry: met, with .changes/codebase-health/feat-6600-concise-audit-body-5d1e7a20.md.

The before/after eval numbers in the PR body are the author's claims; I did not rerun them.

Spend: the PR body reports 19.75 USD of eval spend on prepaid API credit, already incurred. The change itself adds no runtime spend.

No embedded instructions aimed at the review lane appeared in the PR context.