[kyle-sexton]

Closes #13298
Refs: #6405

Summary

Plain prompts that should load /evals:plugin-eval (a WSL2 sandbox missing socat; claude plugin eval argument order) and /code-tidying:audit-dead-code ("find the dead code and delete it") did not fire either skill. #13245 and #13204 then named the skill in the cases that showed it, so no case tested the plain route. Survey: #6405 comment.

Fix

• plugin-eval description: adds "missing socat or bubblewrap" and "argument order around --allow-tools". To stay under the 1024-codepoint spec field maximum, the summary sentence drops "CLI", "runs and", "under the ceiling" and "reading"; every base trigger phrase is kept (1019/1024).
• audit-dead-code description: adds 'find the dead code and delete it' and 'remove unused code', and says a find-and-delete request starts here (this skill reports; deletion stays /code-tidying:tidy), so the request no longer routes away. 545 chars.
• New triggering cases whose prompts do not name the skill, each with only a skill-fired tool_used Skill grader (with-arm indicator) and pass/fail samples:
  • plugins/evals/evals/trigger-wsl2-missing-socat: the pre-test(evals): name plugin-eval in two cases that never fired it #13245 plain socat prompt.
  • plugins/code-tidying/evals/adc-plain-delete-request-triggers: the pre-test(code-tidying): name audit-dead-code in adc-read-only-delete-request #13204 plain prompt, same scaffold and tools.
• The argument-order phrasing has no plain-prompt case of its own; one case per skill, per the issue.
• One patch changelog fragment per plugin.

Verification

• validate-cases.py on plugins/evals and plugins/code-tidying: 0 FAIL (WARNs only; the new code-tidying case gets the usual Bash/Edit/Write grant WARNs).
• scripts/check-changed-skills.sh origin/main: both skills PASS, all base trigger phrases preserved.
• check-listing-budget.sh plugins/*/skills: advisory WARN already on main (aggregate over the default budget); this change adds 156 chars to it.
• scripts/check-affected.sh --tests on the committed head: 15 passed, 2 skipped.
• Local eval runs on a git archive copy of the head (Claude Code 2.1.318, with-without ablation, claude plugin eval <copy>/plugins/<plugin> --case <name> --runs N --allow-tools ..., --scaffold for code-tidying):
  • trigger-wsl2-missing-socat, --runs 3: skill-fired 3 of 3 with-arm, 0 of 3 without. run-validity.py: VALID (models WARN only).
  • adc-plain-delete-request-triggers, --runs 3, --allow-tools Bash Edit Write: skill-fired 3 of 3 with-arm, 0 of 3 without. run-validity.py --scaffolded: VALID (models WARN only).
  • Named-skill cases, --runs 1: sandbox-wsl2-missing-socat with 1.00 / without 0.00; target-before-json with 1.00 / without 0.00; adc-read-only-delete-request with 1.00 / without 0.50 (file kept, no Edit or Write). Each fired its skill.

Related

• feat(evals): run plugin eval suites in a CI lane and hill-climb in the background #6405 (survey), test(evals): name plugin-eval in two cases that never fired it #13245, test(code-tidying): name audit-dead-code in adc-read-only-delete-request #13204.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-04T17:34:11.801867Z	4a0ba65	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed 4a0ba65a9bf75fe116631e56dc49fea38bd302d6: read 10 of 10 changed file(s); 0 finding(s), 0 as inline comments.

Security review: no security issues found

I read every hunk of all 10 changed files. Nothing in this PR touches a trust control, a token, a merge hold, a workflow, a plugin manifest or a trusted path.

• Skill descriptions (plugins/code-tidying/skills/audit-dead-code/SKILL.md, plugins/evals/skills/plugin-eval/SKILL.md): only the description frontmatter changes, which adds trigger phrases. audit-dead-code still calls itself "Read-only" and still sends deletion to /code-tidying:tidy, so no guardrail is narrowed. The plugin-eval description drops "under the ceiling", but the skill body still enforces the cost ceiling and --max-cost-usd (plugins/evals/skills/plugin-eval/SKILL.md lines 206-220 at head). Nothing is lost under the instruction-surface lens.
• New eval cases (adc-plain-delete-request-triggers, trigger-wsl2-missing-socat): these are a prompt, a tool_used Skill grader and sample JSON. scaffold.sh calls the existing, unchanged plugins/code-tidying/evals/fixtures/adc-seed.sh with fixed fixture names that already exist in the repository. It takes no outside or consumer-supplied input. The case grants Bash, Edit, Write only inside an eval run the operator starts. Sibling ADC cases use the same pattern, so this adds no new path to a credentialed lane.
• Changelog fragments: prose only.
• Embedded text: the PR context contains no instructions aimed at the review lane.
• Spend: running the new cases (3 runs each when the operator runs them) uses prepaid API credit or subscription usage only. This is not a hold.

The instruction exception register was not consulted. No standing rule was deleted or narrowed, so the lens had no rule to match against it.

critical-findings: 0

[github-actions]

claude-review reviewed 4a0ba65a9bf75fe116631e56dc49fea38bd302d6: read 10 of 10 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13302

Result: no findings. I read every hunk of all 10 changed files.

What changed

• plugins/evals/skills/plugin-eval/SKILL.md: the description gains two trigger phrases, "missing socat or bubblewrap" and "argument order around --allow-tools". To make room, some summary words are cut. Every base trigger phrase is still in the description.
• plugins/code-tidying/skills/audit-dead-code/SKILL.md: the description gains 'find the dead code and delete it' and 'remove unused code'. It also says a find-and-delete request starts here and that deletion stays with /code-tidying:tidy. That keeps the skill read-only. tidy's own description has no matching phrases, so the two skills don't compete for these requests.
• New triggering cases: trigger-wsl2-missing-socat and adc-plain-delete-request-triggers. Each is a copy of its named-skill sibling (sandbox-wsl2-missing-socat, adc-read-only-delete-request) with the skill's name taken out of the prompt:
  • Same scaffold and fixtures. adc-seed.sh and the three adc-util-* fixtures exist.
  • Same tool grants.
  • Same skill-fired grader regex.
  • The pass and fail samples cover the qualified name, the bare name, and sibling skills that must not match.
• Changelog fragments: one bump: patch fragment per plugin. Patch fits a fix to when a skill triggers.

Closing issue #13298: done-when criteria

I couldn't run the evals in this lane, so the run counts below are the author's claims and I haven't verified them.

• Both descriptions edited: met, as shown in the diff.
• Each new triggering case fires in at least 2 of 3 runs on the head (VALID run): the author reports 3 of 3 for each, with VALID run-validity verdicts. The two new cases are the tests that could fail if this broke.
• The existing named-skill cases still pass: the author reports them passing with --runs 1. I didn't verify this.
• Listing budget (Change item 3): the author says the aggregate WARN already exists on main and this PR adds 156 characters. The 1019/1024 count for the plugin-eval description is also the author's figure. I didn't count it myself; the repository's changed-skill check is what enforces the limit.

Other checks

• Plugin manifests: no plugin.json or marketplace.json changed, so there were no manifest checks to run.
• Trusted paths: none of the changed files match the trusted-path list.
• Embedded instructions: the PR context contains no text addressed to this lane.
• Spend: the new cases add eval runs when someone runs the suite (3 runs each). That uses prepaid or subscription usage only, so it isn't a hold or a finding.