Skip to content

perf(skills): trim over-cap listing entries and flag uncited side-effect skills manual-only - #3677

Merged
kyle-sexton merged 2 commits into
mainfrom
perf/skill-listing-budget
Sep 2, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
perf/skill-listing-budget

Conversation

@kyle-sexton

Copy link
Copy Markdown
Contributor

No related issue: phase 6 of the hook-performance program, tracking issue #3623

Summary

Phase 6 of the hook-performance program (skill listing budget). The Claude Code skills doc states that the combined description and when_to_use text is truncated at 1,536 characters in the skill listing and that the shared listing budget scales at 1% of the model's context window. This PR rewrites the ten largest listing entries in the marketplace to spend fewer characters while keeping every single-quoted trigger phrase and every Not for / Skip when / sibling disambiguation verbatim, and records the outcome of the manual-only (disable-model-invocation: true) sweep over side-effect skills.

Baseline finding: the over-cap clause of the program ruling was already satisfied. The largest entry was 1,526 characters, so zero entries exceeded the 1,536 cap before this PR and zero exceed it after. The substance here is the aggregate reduction.

Fix

Ten SKILL.md descriptions tightened (old chars, new chars):

Skill Before After
claude-ops:audit-performance 1526 1356
claude-ops:audit-skill-visibility 1511 1391
claude-config:audit-prompting-postures 1316 1228
work-items:decompose 1266 1251
skill-quality:check 1248 1019
improvement:find 1245 1212
claude-ops:audit-native-overlap 1236 1153
code-tidying:dissolve-comments 1194 1173
overengineering:audit 1192 1177
architecture:improve 1165 1154

What was cut: mechanism narration and parenthetical enumerations (the suspect labels in audit-performance, the twenty-five-check list in check, the verdict enumeration in audit-native-overlap, the budget-drop explanation in audit-skill-visibility). What was kept: every quoted trigger phrase, every /plugin:skill reference inside the disambiguation sentences (those references are the citation edges the manual-only rule reads), and the argument names. No description grew.

Manual-only sweep: no skill gains disable-model-invocation: true. The rule applied was "a skill cited as /<plugin>:<skill> by any external file (SKILL.md, agent, plugin README, or skill spoke) is not a candidate." Every listing-eligible side-effect skill has at least one external citing file. Three side-effect skills are cited only by their own plugin README (overengineering:realign, coupling:reduce, code-tidying:dissolve-comments); they would be candidates under a stricter presence-gate reading and are listed for the program lead to re-rule.

Deferred (plugin has an open PR): source-control:babysit-loop sits in the top ten at 1,196 characters and was not touched. instruction-placement:realign is a side-effect skill in a no-touch plugin.

Eight plugins get a patch bump and a ## [<version>] CHANGELOG entry: claude-ops 0.41.8, claude-config 0.40.32, work-items 0.39.53, skill-quality 0.20.11, improvement 0.1.7, code-tidying 0.15.4, overengineering 0.3.6, architecture 0.6.8.

Verification

Listing-budget script, before (bash plugins/skill-quality/scripts/check-listing-budget.sh plugins/*/skills):

Shared listing-budget estimate over 182 listing-eligible skill(s) across 74 root(s):
  aggregate: 136209 chars
  budget:    8000 chars (documented default (SLASH_COMMAND_TOOL_CHAR_BUDGET fallback))

After:

Shared listing-budget estimate over 182 listing-eligible skill(s) across 74 root(s):
  aggregate: 135424 chars
  budget:    8000 chars (documented default (SLASH_COMMAND_TOOL_CHAR_BUDGET fallback))

Aggregate reduction: 785 characters (0.58%). Over-cap entries (combined description plus when_to_use over 1,536): 0 before, 0 after, computed per skill from frontmatter across all 243 SKILL.md files (182 listing-eligible). Phase 0 recorded 178 eligible skills at 132,552 characters; the fleet grew by four skills between phase 0 and this baseline.

check-skill.sh (with CHECK_SKILL_SKILLS_ROOT set per plugin and CHECK_SKILL_SKIP_MARKDOWNLINT=1, matching the repo gate): PASS on nine of ten, each reporting "all N base-ref trigger phrase(s) preserved" for check 3. audit-skill-visibility reports every trigger phrase preserved but FAILs on its bundled script test, scripts/audit_skill_visibility.test.sh: all 96 Python unit tests pass and the shell fixture then errors with no installed skills resolved from '<mktemp dir>/cfg'. That fixture does not read SKILL.md and the only change to that skill is the description line, so this is a Windows-host fixture-path issue, not a regression from this PR; CI runs the suite on Linux.

bash scripts/check-changelog-parity.sh --check-bump origin/main: exit 0 after re-fetching origin/main.

bash scripts/check-purged-em-dashes.sh: exit 0 (86 declared paths, 105 files scanned, no em dashes). Em-dash line counts per touched file did not grow versus HEAD; architecture:improve dropped from 2 lines to 1.

bash scripts/affected-tests.sh --explain: no suites selected. Every changed file (SKILL.md, CHANGELOG.md, plugin.json) is a recorded no-suite class covered by a non-shell CI lane.

Related

🤖 Generated with Claude Code

Phase 6 of the hook-performance program (skill listing budget, tracking
issue #3623). Claude Code truncates each skill's combined description and
when_to_use at 1,536 characters in the listing and scales the shared
listing budget at 1% of the model's context window, so the ten largest
entries were rewritten to spend fewer characters: mechanism narration and
parenthetical enumerations moved out, every single-quoted trigger phrase
and every Not for / Skip when / sibling disambiguation kept verbatim.
check-skill.sh check 3 reports every base-ref trigger phrase preserved on
all ten.

Aggregate listing estimate: 136,209 -> 135,424 chars over 182
listing-eligible skills. Over-cap entries: 0 before, 0 after (the largest
entry was already 1,526).

No skill gains disable-model-invocation: true. Every listing-eligible
side-effect skill is cited as /<plugin>:<skill> by at least one external
file (plugin READMEs included per the program ruling), so none is a
manual-only candidate. babysit-loop (source-control) sits in the top ten
but is deferred: that plugin has an open PR.

Each touched plugin gets a patch bump and a CHANGELOG entry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-02T23:06:20.822110Z 320f9c4 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 2m 39s —— View job


Code review: perf/skill-listing-budget (#3677)

  • Gather context (PR diff, CLAUDE.md/AGENTS.md, /review:code-review skill)
  • Review changed files against skip gate and criteria
  • Post findings (inline comments + summary)

Scope reviewed: the 10 SKILL.md description trims, the 8 matching plugin.json version bumps, and the 8 CHANGELOG.md entries. This is a docs-only PR (no disable-model-invocation frontmatter changed — the manual-only sweep concluded no skill qualifies, which the diff confirms).

Verified independently:

  • Every touched SKILL.md diff is a 1-line description change; I read all 10 before/after pairs. Nine of the ten preserve every quoted trigger phrase, every Skip when / Not for disambiguation clause, and every /plugin:skill reference, matching the PR's claim.
  • plugin.json version bumps line up 1:1 with the new ## [x.y.z] CHANGELOG.md heading in each of the 8 touched plugins (spot-checked architecture and work-items in full, skimmed the rest).
  • .claude-plugin/marketplace.json doesn't mirror per-plugin versions, so no cross-file sync gap there.

Finding posted inline: plugins/claude-ops/skills/audit-performance/SKILL.md:2 — the fan-out-layer sentence drops the "whether running sessions predate the settings file" diagnostic clause outright, while its four sibling suspects in the same sentence were only compressed to parenthetical labels. The underlying check still exists in scripts/audit_performance.py, so it's not a functional bug, just an inconsistent trim that costs that one capability its listing-description discoverability. Not blocking — worth a quick look.

Not independently re-verified (author-claimed, plausible, out of scope for this diff): the exact before/after character counts in the PR table, the check-listing-budget.sh aggregate numbers, and the audit-skill-visibility bundled-test failure being a pre-existing Windows-host fixture-path issue rather than a regression — I didn't have a way to run those scripts in this sandbox. CI runs the full suite on Linux per the PR's own note.

No other correctness or maintainability issues found. The description edits read as faithful, careful compressions (em dashes to parentheses, redundant phrasing dropped) rather than lossy rewrites, with the one exception noted above.

@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 55s —— View job


Claude finished the security review

  • Gather context (PR diff, changed files, custom instructions)
  • Run /review:security-review per custom instructions
  • Post findings

Skip gate hit: no security-relevant surface.

Read the full diff (gh pr diff 3677) against all 26 changed files. Every change falls into one of three categories:

  1. SKILL.md frontmatter description: strings — text-only rewording of skill-listing entries (trigger phrases, disambiguation clauses, mechanism narration removed) for the shared listing-budget program. No argument-hint, allowed-tools, disallowed-tools, or disable-model-invocation values changed in any file.
  2. plugin.json — patch-version bumps only (e.g. plugins/architecture/.claude-plugin/plugin.json, 0.6.70.6.8).
  3. CHANGELOG.md — new ## [<version>] entries documenting the description edits.

No source code, scripts, hooks, GitHub Actions workflow files, permissions, or config semantics are touched. There is no injection surface, no credential/token handling, no trust-boundary change, and no supply-chain-relevant pin change in this diff — it's a documentation-only tightening of existing SKILL.md descriptions plus routine version/changelog bookkeeping.

No security findings to report.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Last security-reviewed head: 1b95a91f82da3568e954c8caf2bee838d4454f99. On the next push, the relevance gate compares only the commits since this SHA; delete this comment to force a full re-review.

@kyle-sexton
kyle-sexton enabled auto-merge (squash) September 2, 2026 23:03
Comment thread plugins/claude-ops/skills/audit-performance/SKILL.md Outdated
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Claude has reviewed this PR 2 times. The lane skips further automatic reviews after 5; deleting this comment resets the count.

kyle-sexton added a commit that referenced this pull request Sep 2, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…udit-performance listing

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 3s —— View job


Claude finished the security review

  • Gather context (PR diff via gh pr diff, custom instructions)
  • Run /review:security-review per custom instructions
  • Post findings

Skip gate hit: no security-relevant surface.

Reviewed the full diff (gh pr diff 3677) at HEAD 1b95a91 against all 26 changed files, including the second commit (1b95a91) added since the prior security-review comment on this PR. Every change falls into one of three categories:

  1. SKILL.md frontmatter description: strings — text-only tightening of skill-listing entries (trigger phrases and disambiguation clauses preserved, mechanism narration and parenthetical enumerations trimmed). The new commit only restores one previously-dropped clause in plugins/claude-ops/skills/audit-performance/SKILL.md — still a pure description-text change. No argument-hint, allowed-tools, disallowed-tools, or disable-model-invocation values changed in any file.
  2. plugin.json — patch-version bumps only (e.g. plugins/architecture/.claude-plugin/plugin.json, 0.6.70.6.8).
  3. CHANGELOG.md — new ## [<version>] entries documenting the description edits.

No source code, scripts, hooks, GitHub Actions workflow files, permissions, or config semantics are touched. There is no injection surface, no credential/token handling, no trust-boundary change, and no supply-chain-relevant pin change in this diff — it remains a documentation-only tightening of existing SKILL.md descriptions plus routine version/changelog bookkeeping.

No security findings to report.

@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 2m 24s —— View job


Code review: perf/skill-listing-budget (#3677)

  • Gather context (PR diff, CLAUDE.md/AGENTS.md, /review:code-review skill)
  • Review changed files against skip gate and criteria
  • Post findings (inline comments + summary)

Trigger: PR synchronize. New commit since the last review: 1b95a91 ("keep the settings-predating-sessions check in the audit-performance listing").

Prior finding resolved: the earlier review flagged that plugins/claude-ops/skills/audit-performance/SKILL.md dropped the "whether running sessions predate the settings file" clause outright instead of compressing it like its sibling suspects. Commit 1b95a91 restores it as sessions that predate the settings file, consistent with how the other four suspects in that sentence are trimmed. Confirmed via git show 1b95a91 — 1-line diff, only that clause changed. No new "over budget" risk: the restored clause adds ~40 characters back, well under the 1,536-char cap the PR is targeting.

Full-diff re-check (git diff bf6321c...HEAD, 26 files, +117/-18, matches the PR's own stat):

  • All 8 plugin.json version bumps verified as patch increments matching the PR's before/after table (architecture 0.6.7→0.6.8, claude-config 0.40.31→0.40.32, claude-ops 0.41.7→0.41.8, code-tidying 0.15.3→0.15.4, improvement 0.1.6→0.1.7, overengineering 0.3.5→0.3.6, skill-quality 0.20.10→0.20.11, work-items 0.39.52→0.39.53).
  • Each bump has a matching new ## [x.y.z] CHANGELOG.md heading — checked all 8, not just the two spot-checked in the prior review pass.
  • Re-diffed all 10 SKILL.md description edits line-by-line: every quoted Use when: trigger phrase and every Not for / Skip when disambiguation clause (including the /plugin:skill references) is preserved verbatim across all ten, not just the ones previously spot-checked.
  • No new em dashes introduced by the edits (git diff ... | grep '—' shows only the pre-existing em dash inside a verbatim-preserved trigger phrase in work-items:decompose, and the architecture:improve line where em dashes were intentionally converted to parens).
  • No files outside SKILL.md / plugin.json / CHANGELOG.md are touched.

Not independently re-verified (author-claimed, same as noted in the prior pass): the exact before/after character counts, the check-listing-budget.sh aggregate numbers, and the audit-skill-visibility bundled-test failure being a pre-existing Windows-host fixture issue — this sandbox doesn't have approval to execute the repo's shell/python scripts, only git and read-only inspection. CI runs the full suite on Linux per the PR's own note.

No new findings. This addresses the one open item from the previous review pass; nothing else in the diff warrants a block.

@kyle-sexton
kyle-sexton merged commit 3d55e1f into main Sep 2, 2026
68 checks passed
@kyle-sexton
kyle-sexton deleted the perf/skill-listing-budget branch September 2, 2026 23:34
kyle-sexton added a commit that referenced this pull request Sep 2, 2026
… main

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
kyle-sexton added a commit that referenced this pull request Sep 3, 2026
No related issue: phase 8 of the hook-performance program, tracking
issue #3623

## Summary

The hook-performance program landed nine PRs today (#3621, #3662, #3666,
#3668, #3671, #3675, #3676, #3677, #3678, plus dotfiles #620). This PR
records the outcome where the convention says it lives: the hook-budget
convention gains a dated reference-figures section in spawn-equivalents
with the harness sha256, and the four plugins that lacked a hook budget
accounting row gain one under Requirements. Documentation only; no hook,
script or registration changes.

## Fix

- `docs/conventions/hook-budget/README.md`: a "Reference figures
(2026-09-02, after the hook-performance program)" section with the
harness identity, the before-and-after spawn-equivalents per surface
(before at S = 33 ms, after at S = 18 ms on `main` `5e3d749cb`), the
reference-host conversion, and a plain reading against the budget table:
per-turn rows meet the 500 ms ceiling; per-tool-call rows sit at 1.4 to
1.9 s against the 1 s typical ceiling, with the guardrails dispatcher
named as the whole of the remainder.
- `plugins/typos-format/README.md`, `plugins/eol-normalizer/README.md`,
`plugins/markdown-format/README.md`, `plugins/context-guard/README.md`:
a "Hook budget accounting" section each, carrying the measured rows
their CHANGELOG entries already state (36.3 to 26.0, 41.0 to 21.5, 41.6
to 32.0 spawn-equivalents; 11 to 2, 9 to 4 and 6 to 1 processes for
context-guard), what changed, and the residual. guardrails and
rate-limit-guard already carried one.
- Version bumps with a documentation-only CHANGELOG entry: typos-format
0.6.37, eol-normalizer 0.6.30, markdown-format 0.11.40, context-guard
0.7.36.

## Verification

- Final harness run on the installed cache at `main` `5e3d749cb`,
`--runs 3`, S = 18 ms, valid, quiet host; every measured plugin's cache
directory byte-compared against `origin/main` (0 stale files in 17);
`enabledPlugins` unchanged against the pre-program snapshot; 52
`hooks.json` entries listed, every one `type: command`, no `async` row.
The per-event block, the STATED CHECK and the reading against goal (B)
are in the program's PLAN.md and DEVIATIONS.md on
`perf/hook-performance-program`.
- Per event (ms, slowest hook): PreToolUse:Bash 2,475 before to 1,599
after; PostToolBatch 1,254 to 282; UserPromptSubmit 975 to 297; in-repo
PostToolUse:Write 13,225 to 1,949; in-repo PostToolUse:Edit 17,192 to
3,048.
- `markdownlint-cli2` 0 issues on the five files; em dash counts
unchanged; `scripts/check-changelog-parity.sh --check-bump origin/main`
exit 0; `scripts/affected-tests.sh --explain` selects no suites (every
changed file is a recorded no-suite class covered by a non-shell CI
lane).

## Related

- #3623 (tracking issue)
- Merged today: #3621, #3662, #3666, #3668, #3671, #3675, #3676, #3677,
#3678; dotfiles #620

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant