fix(skill-quality): make listing-budget report the machine's configured budget #6261

Problem

/skill-quality:check listing-budget compares the pooled description total against a fixed
8,000-character budget unless the operator sets environment overrides by hand. On a machine whose
settings raise skillListingBudgetFraction, the verdict is wrong by a large factor and the skill
gives no hint that the machine's own setting was ignored.

Observed: on a machine with skillListingBudgetFraction: 0.05 in user settings and a 1M-token model,
the pooled user root plus a marketplace checkout came to 146,574 chars and the script reported it
"~18.3x over" an 8,000-char budget. At that machine's settings, the script's own reconstruction
(tokens x chars per token x fraction) gives 1,000,000 x 4 x 0.05 = 200,000 chars (150,000 at 3
chars/token), so the same total is under budget.

On current main, plugins/skill-quality/scripts/check-listing-budget.sh:

• :188-215 resolves the budget from CHECK_SKILL_LISTING_BUDGET_CHARS, else from
  CHECK_SKILL_LISTING_CONTEXT_TOKENS x chars-per-token x fraction, else the literal BUDGET_CHARS=8000
  with the source label "documented default (SLASH_COMMAND_TOOL_CHAR_BUDGET fallback)". No settings
  file is read.
• CHECK_SKILL_LISTING_BUDGET_FRACTION is read only inside the CONTEXT_TOKENS branch (:200). Set
  on its own, it is silently ignored and the report still uses 8,000.
• The SLASH_COMMAND_TOOL_CHAR_BUDGET environment variable, which the docs say overrides the budget
  with a fixed character count, is not honored.
• The overflow summary (:489-490) says the aggregate is over budget "at the configured budget" even
  when the budget is the hard-coded default, not anything configured.
• The header (:38-46) states the design choice: the script "never asserts a live value it cannot
  observe" and reports against the documented default, deferring to /doctor. That choice is
  defensible for CI, but the SKILL.md action (skills/check/SKILL.md:163-186) runs the same bare
  command on a user's machine, where the settings value IS observable on disk and is the number the
  user is asking about.

The repo's own CI already works around this: .github/workflows/ci.yml:1844-1882 runs a second step
that reads skillListingBudgetFraction from .claude/settings.json with jq and passes it in with a
pinned 750,000-token window. Every other caller gets the 8,000 default.

Evidence

Labels follow the parent: E1 measured and attributed, E2 aggregate measured with cause inferred, E3
structural inference from code or docs, E4 suspicion.

• E1 (reported by the audit): on the audited machine, skillListingBudgetFraction was 0.05 in
  ~/.claude/settings.json; the user root alone measured 5,202/8,000 chars (OK) and the pooled user
  root plus a marketplace checkout measured 146,574 chars, reported as "18.3x over" 8,000.
• E3 (verified this pass, code at origin/main 5779559): the budget resolution, the fraction-only
  no-op, the missing SLASH_COMMAND_TOOL_CHAR_BUDGET handling, and the "configured budget" wording,
  at the lines above; the CI workaround at ci.yml:1844-1882; this repo sets
  "skillListingBudgetFraction": 0.05 at .claude/settings.json:32.
• E3 (verified this pass, docs fetched as .md with curl and quoted verbatim):
  • https://code.claude.com/docs/en/settings-reference.md, skillListingBudgetFraction: "Claude Code
    caps that listing at a share of the context window. When the listing is over the cap, Claude Code
    keeps every skill's name but drops the descriptions of the least-used skills". Scope "Any file";
    "Default: 0.01, which reserves 1% of the context window". skillListingMaxDescChars default
    1536.
  • https://code.claude.com/docs/en/skills.md, "Skill descriptions are cut short": "The budget scales
    at 1% of the model's context window"; "To raise the budget, set the skillListingBudgetFraction
    setting ... or the SLASH_COMMAND_TOOL_CHAR_BUDGET environment variable to a fixed character
    count"; "When the listing exceeds its budget, Claude Code also writes a warning to the debug log".
  • https://code.claude.com/docs/en/env-vars.md, SLASH_COMMAND_TOOL_CHAR_BUDGET: "The budget scales
    dynamically at 1% of the context window, with a fallback of 8,000 characters."
  • Not documented: the chars-per-token factor. The script's 4 and the visibility engine's 3-or-4 by
    model are inferences (the latter from binary extraction, see the sibling issue).
• Not verified: whether 146,574 chars would actually fit live. The pooled roots in the audit were a
  user root plus a marketplace checkout, not the installed fleet, and the script counts descriptions
  only (:20-24, names are "a floor this report does not estimate"). The audit separately observed
  the live listing at its cap with descriptions shed (see the sibling issue for audit-skill-visibility).

Proposed approach

Keep a default-consumer mode, and add a configured-machine mode that is the default for the skill
action.

1. In check-listing-budget.sh, add budget resolution from effective settings: read
   skillListingBudgetFraction from the settings layers in precedence order (managed, then project
   .claude/settings.local.json, project .claude/settings.json, then ~/.claude/settings.json;
   honor CLAUDE_CONFIG_DIR), and honor SLASH_COMMAND_TOOL_CHAR_BUDGET above everything, as the
   docs state. Report the source of each input (settings:<path>, env:SLASH_COMMAND_TOOL_CHAR_BUDGET,
   override, documented default).
2. Because the window is per model and not on disk, report a band rather than one number when no
   window override is given: the budget at 200k and 1M tokens, at 4 chars/token (and 3 where the
   sibling engine uses 3), with a verdict per row. Keep the single-row output when
   CHECK_SKILL_LISTING_CONTEXT_TOKENS or CHECK_SKILL_LISTING_BUDGET_CHARS pins it.
3. Add a flag (for example --default-consumer) that keeps today's behavior: ignore settings and use
   the documented 8,000 fallback. Point CI step 1 (ci.yml:1843) at it so its meaning does not change
   when the script starts reading this repo's .claude/settings.json. CI step 2 can then drop its
   jq read and rely on the script, or keep it; either way, state which.
4. Make CHECK_SKILL_LISTING_BUDGET_FRACTION on its own either take effect (through the band) or fail
   with a usage error; it must not be silently ignored.
5. Reword the overflow line so "configured" appears only when the budget came from settings or an
   override.
6. Update skills/check/SKILL.md "Action: listing-budget" step 3 to report the budget source and the
   band, and update the header contract in the script to match.

Alternatives considered:

• Leave the script static and make the SKILL.md action read settings and pass the env overrides.
  Fewer script changes, but it relies on the agent doing it every time and leaves the fraction-only
  no-op and the missing SLASH_COMMAND_TOOL_CHAR_BUDGET handling in place.
• Read settings unconditionally with no default-consumer mode. Rejected: it silently changes what CI
  step 1 measures, since this repo sets 0.05.
• Add skill names to the total. Out of scope here; the script documents names as a deliberate floor
  it does not estimate. The sibling audit-skill-visibility issue covers the name floor, and the two
  engines are documented as deliberate duplicates that must stay reconciled
  (plugins/harness-ops/skills/audit-skill-visibility/scripts/audit_skill_visibility.py:1499-1507).

Acceptance criteria

• With skillListingBudgetFraction: 0.05 in a test settings file and no overrides, the report
  names that file as the fraction source and shows budgets of 40,000 (200k x 4 x 0.05) and
  200,000 (1M x 4 x 0.05) chars, with a verdict per row.
• With SLASH_COMMAND_TOOL_CHAR_BUDGET=12345, the budget is 12,345 regardless of settings, and the
  source says so.
• CHECK_SKILL_LISTING_BUDGET_FRACTION set alone is either applied or rejected with exit 2; it is
  never silently ignored.
• --default-consumer (or the chosen name) reproduces today's output against 8,000 and ignores
  settings; CI step 1 uses it.
• The word "configured" no longer appears in the summary when the budget is the documented default.
• plugins/skill-quality/scripts/check-listing-budget.test.sh covers each case above using
  fixture settings under a temp HOME/CLAUDE_CONFIG_DIR, never the real user settings, and
  passes.
• The script still exits 0 on every successful report (advisory contract unchanged).

Constraints and gotchas

• Tests must not read the developer's real ~/.claude/settings.json; point HOME and
  CLAUDE_CONFIG_DIR at a fixture directory.
• The script is a single-process awk pass for Windows speed (:290 and perf(skill-quality): check-listing-budget.sh takes 289s over plugins/*/skills on Windows, so #2023's own cycle command cannot complete in the foreground #2216, where a per-file loop
  took 289 s). Settings reading should be one jq or awk call, not per-file work.
• Upstream constants the script encodes carry a four-part verification record (:56-66); extend it
  for SLASH_COMMAND_TOOL_CHAR_BUDGET precedence and the settings scopes, with a recheck trigger, per
  .claude/rules/skill-bodies-state-current-rules.md.
• .github/recurring-schedule.json runs the pooled call for the listing-budget watch ([Maintenance] Listing-budget watch: report the shared aggregate and drift #2023); keep its
  output comparable or note the change there.
• Bump plugins/skill-quality/.claude-plugin/plugin.json (currently 0.28.2) and add a
  plugins/skill-quality/CHANGELOG.md entry, per the repo's shared version bump and CHANGELOG rule.
• Related: claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model #4076 and PR fix(claude-ops): audit-skill-visibility reads effective settings, withholds unscored verdicts, and determines reachability #4089 made audit-skill-visibility read effective settings (the same fix in
  the sibling engine); skill-quality: check-skill.sh has no shared listing-budget check, plus three smaller gate defects #1404 and PR feat(skill-quality): add the shared listing-budget report, fix check 2's joiner #1450 introduced this report; Listing-budget decision: full-install aggregate is 86.5k chars (~10.8x the default budget) #2022 and perf(skills): skill listing is 42% over its character budget; ~50 skills lose their descriptions and become undiscoverable #3505 record listing-budget
  decisions; PR chore(config): raise skillListingBudgetFraction to 0.05 so the enabled fleet fits #3767 raised this repo's fraction to 0.05.

Context

Part of the 2026-10-04 performance audit, parent #6244.

Source: local handoff item 20261004-152632-claude-perf-audit-tooling-bugs.md, bug 2 of 6 (retired
into this issue). Found by the 2026-10-04 Windows performance audit (parent #6244). Sibling issues
from the same item: #6260 (context-budget parser) and
#6262 (audit-skill-visibility estimator, which shares
the listing arithmetic).