[kyle-sexton]

Closes #6582

Summary

Concise-skills rollout for github, descriptions only (scope recorded on #6476: #6476 (comment)). Freezes description-trigger probes for both model-invoked skills and ships a shorter advise description that held up in held-out trigger probes on Opus.

Fix

• plugins/github/probes/{audit,advise}.json: frozen probes, 18 queries each.
• advise description: 664 -> 528 chars, every quoted trigger phrase and the audit pointer kept.
• audit description: unchanged (unmeasured, see below).
• Skill bodies and setup unchanged.

Verification

Held-out trigger probes (16 per skill, written blind, not committed), Opus model and judge, 3 runs per case, decision rule recorded before any pass: no lost positive majority, no new false-fire majority, rates within 0.05 of before.

Skill	VALID passes	Positive fire (before -> after)	Negative quiet (before -> after)	Result
advise	8/8	23/24 -> 23/24	22/24 -> 21/24	rewrite ships
audit	6/8	n/a	n/a	unmeasured; main's description kept

• audit's before n2 pass was INVALID twice on a sandbox-root Glob denial before any Skill call; the rule recorded before the rerun treats that as the eval sandbox itself (fix(evals): stop outcome passes failing validity on a required user-scope read and on forked skills reporting 0 turns #11727), so audit is unmeasured.
• Two splits were rerun in both versions for environment reasons (a 300 s run timeout; later a full shared tmpfs, ENOSPC); the first results are kept aside, not counted.
• A run's turns field is not used as a turn-cap signal (it exceeds max_turns on normally completed runs); the cap is read from the error text or the trace's terminal reason.

Related

• Umbrella: Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476
• Eval validity gaps: fix(evals): stop outcome passes failing validity on a required user-scope read and on forked skills reporting 0 turns #11727

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T08:38:51.786473Z	0feb7ac	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed 0feb7ac74c061e3a33274a3ab9c2df84a38c4446: read 4 of 4 changed file(s); 0 finding(s), 0 as inline comments.

Security review: no security issues found

I read all 4 changed files in full in .claude-lane/pr.diff. None of them matches a trusted-path glob.

• .changes/github/feat-6582-concise-skills-7c3e1a94.md: a changelog fragment, prose only.
• plugins/github/probes/advise.json and plugins/github/probes/audit.json: new, static sets of trigger-probe queries. Neither file is a plugin manifest or a hook, and neither points to anything that runs. The query text is benign test data, and nothing in either file is written as an instruction to an automated lane.
• plugins/github/skills/advise/SKILL.md: only the description frontmatter is shorter. The new text drops two sentences: "Bare invocation performs zero mutations" and "Guidance and proposals only, never recall presented as grounded."

Removed sentences. I checked both under the instruction-surface lens and neither is a finding. The skill body still states both rules, so the change shortens the description without changing what the skill forbids:

• The zero-mutation rule is still at SKILL.md#L123-L138. That section bans GraphQL mutation documents and sends every write through --apply.
• The rule against presenting recall as grounded is still at SKILL.md#L40-L41.
• user-invocable, disable-model-invocation and argument-hint are unchanged.
• I did not open the instruction exception register (docs/contracts/instruction-exception-register/README.md). I judged by consequence: what breaks when each sentence is absent. Because the body still enforces both rules, nothing does.

Other checks:

• Manifests: no plugin.json or marketplace.json changed, so there are no manifest items to report.
• Trust controls: none of the files touches a trust control.
• PR context: the description has no embedded instructions aimed at this lane.

critical-findings: 0

[github-actions]

claude-review reviewed 0feb7ac74c061e3a33274a3ab9c2df84a38c4446: read 4 of 4 changed file(s); 0 finding(s), 0 as inline comments.

Code review: no findings

I read every hunk of all 4 changed files. None of them matches a trusted-path glob, and the PR changes no plugin manifest.

What changed

• /github:advise description: shorter. Every quoted trigger phrase is still there, and so is the pointer to the audit skill. The PR drops two sentences from the description: "Bare invocation performs zero mutations" and "never recall presented as grounded". The skill body still enforces both:

  • zero mutations: the --apply / write-capability section at plugins/github/skills/advise/SKILL.md:123
  • no recall presented as grounded: the "Refusal branch" at plugins/github/skills/advise/SKILL.md:40

  I searched the repository: no test or other file depends on the removed text.

• plugins/github/probes/{audit,advise}.json: 18 queries each, split 5 positive and 4 negative for training and the same for validation. They use the same layout as the probe files other plugins already ship (for example plugins/event-storming/probes/simulation.json). The listed competitors match the boundaries the skills draw.

• Changelog fragment: .changes/github/… with bump: patch. That level fits a description-only rewording plus added data files, and the fragment matches the sibling concise-skills rollouts.

Closing issue #6582

• "The PR merges with a per-skill decision table in its body, and every shipped description and body was measured": met. The PR body has the table. advise is the only skill whose description changed, and the PR reports it as measured. Those measurement results are the author's claim and I did not verify them; this lane cannot run evals.
• "Any skill left unmeasured, and why, is named in the PR": met. The body names audit as unmeasured, explains why (its baseline pass was invalid twice in the eval sandbox, fix(evals): stop outcome passes failing validity on a required user-scope read and on forked skills reporting 0 turns #11727), and says its description was kept unchanged. It names setup as unchanged and user-invoked.

Spend: none. The change only edits text and data files.