Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/claude-config/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "claude-config",
"version": "0.33.1",
"version": "0.34.0",
"description": "Eight configuration-health skills (plus setup) for a repo's Claude Code configuration: audit (settings.json / .mcp.json / hooks / plugins / permissions drift), audit-automation-gaps (evidence-gated verdicts on automation gaps), audit-permission-grants (allow-rule / allowed-tools grants for auto-mode durability and portability), audit-permission-state (which settings scopes exist and what rules each one holds — managed policy, user-global, project, local, and the pre-v2.1.211 start-directory copy), audit-instructions (locally-owned instruction surfaces vs current model capability — proposes removals/rewrites of instructions the model no longer needs, and detects cross-surface instruction conflicts), audit-prompting-postures (the additive lane — posture guidance the prompting guide says a component's purpose needs but the component does not carry), audit-pass (one coordinated, ordered, resumable pass over a named target — three-scope inventory, run-time-derived exclusion set, stable finding identity, suppression memory, resume, one human gate — delegating every check to the plugin that owns it), and unhobble (the empirical bare-baseline experiment: reversibly strip a repo's standing instructions, log real stumbles against the current model, re-add only what evidence earns).",
"author": {
"name": "Melodic Software",
Expand Down
72 changes: 72 additions & 0 deletions plugins/claude-config/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,78 @@
All notable changes to the `claude-config` plugin are documented here. Format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning.

## [0.34.0]

### Added

- **`audit` can now enumerate the hooks an enabled plugin ships.** It read settings-declared hooks
only, and three of its own surfaces said so in prose — `context/procedures.md` called the hook
question "a question to answer, not a lookup", and `reference/required-permissions.md` carried a
"Fail open where no hook inventory was taken" clause that, on most runs, resolved to hedging every
Category B finding. That gap was load-bearing in two places: Category D writes rules for
`${CLAUDE_PLUGIN_ROOT}` / `${CLAUDE_PLUGIN_DATA}` placeholders that only ever appear in a
plugin-provided hook, and Category B's third baseline narrowing turns on whether such a hook is
live. New `scripts/check-hook-coverage.sh` (with `.test.sh`) resolves each enabled plugin through
the installed-plugin registry — so no version-directory ordering is inferred — and reads its hook
config in all four documented shapes: `hooks/hooks.json`, a `hooks` path, an array of paths, and an
inline object in `plugin.json`. It also reports `disableAllHooks`, `allowManagedHooksOnly`, and
`strictPluginOnlyCustomization`, because a hook a setting has already switched off is not coverage.
The exit code is the contract: `0` complete, `1` partial with the unenumerated sources named, `2`
no inventory. The fail-open posture survives, narrowed to what a partial run could not read —
"could not look" is still never reportable as "looked and found nothing". Read-only: it never runs
a hook, and it never decides whether a hook *covers* a family, which stays Category B's judgment
against its three preconditions.
- **Two positive Category G evals.** The category had no positive case at all — its only two mentions
in `evals.json` were negative assertions inside the scope-filter case. One case now exercises the
headless overflow route, the other the lever-matches-the-roster rule.

### Changed

- **Category G is executable now, from either kind of session.** Its only overflow detector was
`/doctor`, which needs an interactive TTY, so the whole category yielded nothing in the harness's
own headless mode while the documented headless route went unmentioned. It now names `--debug` —
*"When the listing exceeds its budget, Claude Code also writes a warning to the debug log"* — as
the headless route, and `/context`'s Skills row as what it actually is: a second *interactive*
reading, not a substitute. A run that could measure nothing reports "not measured", never "no
overflow".
- **Category G states the budget it measures against.** The category named the per-entry cap and both
scaling settings and gave no budget value and no formula, so "overflowed, and by how much?" was
unanswerable by hand. It now carries `skillListingBudgetFraction` default `0.01`,
`SLASH_COMMAND_TOOL_CHAR_BUDGET`'s documented 8,000-character fallback,
`skillListingMaxDescChars` default `1536`, and the `200,000 × 4 × 0.01 = 8,000` arithmetic that
reconciles the first two.
- **Category G's lever list no longer recommends a lever the operator cannot pull.** The ordered
"cheapest first" list put `skillOverrides` second while carrying the caveat that it does not apply
to plugin skills, and never named the substitute upstream prescribes — so on a plugin-heavy roster
the list degenerated to the entry it labels "last resort". Levers are now split by roster origin,
`/plugin` is named for plugin skills, and a roster-composition count is required *before* any lever
is recommended. No per-skill `name-only` state is promised for `/plugin`; no page documents one.
- **Category G has a checklist table, like every other Phase 2 category.** It was the only letter
missing from `audit-checklist.md`, whose heading sequence ran A–F, H, I. The table also points at
`plugins/skill-quality/scripts/check-listing-budget.sh` — the aggregate measurement this
marketplace already ships — as an explicit **in-repo proxy**, with the population difference said
out loud: that script walks a repository's skills roots, while Category G asks about the listing a
consumer's session assembled. Measured cost is stated too (8 skills in 5.98s on Windows), because a
marketplace-wide invocation will not finish inside a default Bash tool timeout.
- **The `destructive-bash-deny` baseline is reported with its fragility, the way the Read deny table
already is.** Eight argument-constraining globs shipped rated `error` when absent with no caveat,
in a file that quotes the permissions page's *"Bash permission patterns that try to constrain
command arguments are fragile"* warning one section earlier against a different table. The concrete
hole is now stated: matching is prefix-anchored, so `Bash(git push --force *)` does not match
`git push origin main --force`. The patterns stay in the baseline — they raise the cost of an
accidental force push — but a finding no longer implies they bound a determined one.
- **The "a `PreToolUse` hook is a speed bump, not a boundary" ranking is scoped to the threat model it
was written for.** It is right for secret exfiltration, where an OS-level boundary exists and the
hook is strictly worse than it. It was stated unscoped, so it also governed destructive git — where
the sandbox's vocabulary is `filesystem.*` paths and `network.*` hosts, with no expression for a
command's arguments, so it does not separate `git push` from `git push --force` to the same remote.
Scoped, not deleted.
- **The read-it-verbatim guard now covers all of Phase 3, not one checklist row.** It was attached to
the Category F env-vars row while Phase 3.1 and Phase 3.4 fetched `settings` and `permissions` with
no such instruction — and `settings` is the page on which a summarizing fetch already reported three
present keys as absent. The guard now also says explicitly that a truncated read supports no finding
in *either* direction.

## [0.33.1]

### Fixed
Expand Down
36 changes: 35 additions & 1 deletion plugins/claude-config/skills/audit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,31 @@ Record the installed Claude Code version (`claude --version`) — Phase 3.2 comp
against it. Then run `bash "${CLAUDE_PLUGIN_ROOT}/skills/audit/scripts/check-structure.sh"`
before the table below.

### 1.0 Hook inventory

Run `bash "${CLAUDE_PLUGIN_ROOT}/skills/audit/scripts/check-hook-coverage.sh"` and keep its output for
the rest of the run. It enumerates settings-declared hooks **and** the hooks shipped by every enabled
plugin — resolved through the installed-plugin registry, so no version-directory guessing is
involved — plus the levers (`disableAllHooks`, `allowManagedHooksOnly`,
`strictPluginOnlyCustomization`) that switch hooks off wholesale.

Two categories depend on it and neither could take this inventory before:

- **Category D** writes rules for `${CLAUDE_PLUGIN_ROOT}` / `${CLAUDE_PLUGIN_DATA}` placeholders, which
only ever appear in a plugin-provided hook.
- **Category B's third narrowing** demotes a missing baseline deny to `info` when a *live* `PreToolUse`
hook already blocks the family. That narrowing is only takeable against a real inventory.

**Read the exit code, not just the table.** `0` means the inventory is complete. `1` means it is
**partial** — some enabled plugin did not resolve, or a hook config did not parse — and the "Not
enumerated" block names which. For anything a partial inventory could not see, keep the conditional
posture in [required-permissions.md](reference/required-permissions.md) "Fail open where the inventory
is incomplete". `2` is fatal (no readable settings scope, or `jq` missing); treat it as no inventory at
all rather than as an empty one.

The script never runs a hook and never decides whether a hook *covers* a family — that judgment stays
with Category B, against the three preconditions in "Narrowing the baseline".

### 1.1 JSON validity

`Valid JSON: no` blocks further analysis.
Expand Down Expand Up @@ -126,7 +151,7 @@ Nine categories — names + the question each answers below; **full per-check cr
- **D — Hooks**: paths resolve + readable, `timeout` in seconds and sane, valid matchers, quoted path placeholders in shell form, exec-form `command` resolvable on Windows, no duplicates, valid events
- **E — Plugins**: static checks (marketplace membership) + live upstream drift detection (`scripts/check-plugin-drift.sh` — ORPHAN/NEW/RENAME modes, auto-fix policy table in the context file)
- **F — Environment Variables**: documented/justified vars, secrets in `settings.local.json` only, forward-slash paths
- **G — Skill-listing budget**: `/doctor` overflow check and trim levers (description trimming, `skillOverrides`, budget settings)
- **G — Skill-listing budget**: overflow check by a route this run can actually take (`/doctor` interactively, `--debug` headless), the budget constant it was measured against, and trim levers scoped to the roster's composition (`skillOverrides` reaches project and user skills; plugin skills are managed through `/plugin`)
- **H — Model and effort settings**: `effortLevel`, `fallbackModel`, `availableModels`, `enforceAvailableModels` — values the harness accepts into the file but does not apply as written
- **I — Deep-link registration**: `disableDeepLinkRegistration` — the one documented value that takes effect, and a visible attempt at an enforcement requirement lodged in a scope that cannot enforce it

Expand All @@ -136,6 +161,15 @@ Nine categories — names + the question each answers below; **full per-check cr

External verification against current documentation.

**Read every page in this phase verbatim, not through a summarizer.** These pages are long — `settings`
and `env-vars` are hundreds of KB — and a summarizing fetch truncates, then reports the rows past the
cutoff as *absent*. That false negative has already been observed on the `settings` page: three keys
reported NOT FOUND that raw `curl` + `grep` found. So for each fetch below,
`curl https://code.claude.com/docs/en/<page>.md` to a file and grep the file, per the
[fetch route](https://github.com/melodic-software/claude-code-plugins/blob/main/docs/conventions/upstream-drift/README.md#reading-the-basis--the-fetch-route).
**A truncated read supports NO finding** — say so and move on, in either direction: neither "the key is
gone" nor "the key is unchanged" is reportable from a read that may have been cut.

### 3.1 Official docs check

Fetch [code.claude.com/docs/en/settings](https://code.claude.com/docs/en/settings) and compare:
Expand Down
7 changes: 5 additions & 2 deletions plugins/claude-config/skills/audit/context/procedures.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,11 @@ added, and neither is mechanical:
baseline", whose three preconditions govern: installed and enabled is not enough (`disableAllHooks`
and the managed `allowManagedHooksOnly` / `strictPluginOnlyCustomization` levers switch hooks off),
a `Bash` hook does not cover the `Read`-pattern family, and one command family's coverage says
nothing about another's. The audit reads settings-declared hooks only and cannot enumerate a
plugin's `hooks/hooks.json`, so this is a question to answer, not a lookup.
nothing about another's. **The inventory half is now a lookup**: Phase 1.0's
`scripts/check-hook-coverage.sh` enumerates settings-declared *and* plugin-declared hooks,
resolving each enabled plugin through the installed-plugin registry. What stays a judgment is
whether an enumerated hook covers *this* family — and where that script exited 1, the sources it
names as unenumerated remain a question, not an absence.
2. **Would the addition suppress a gate the project built on purpose?** Deny and ask rules are
evaluated regardless of what a `PreToolUse` hook returns, so adding a deny over a family a project
hook escalates to an *ask* replaces the prompt with an outright block and the human loses the
Expand Down
62 changes: 52 additions & 10 deletions plugins/claude-config/skills/audit/context/validation-categories.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,14 @@ Load the audit checklist alongside these: [audit-checklist.md](../reference/audi

## Category D: Hooks

**The inventory this category checks is Phase 1.0's**, from
`scripts/check-hook-coverage.sh` — settings-declared hooks *and* every enabled plugin's own hook
config, resolved through the installed-plugin registry. That matters for two of the rules below:
`${CLAUDE_PLUGIN_ROOT}` and `${CLAUDE_PLUGIN_DATA}` only ever appear in a plugin-provided hook, so
before Phase 1.0 existed those rules were written against a surface the audit never opened. Where the
script exited 1, say which sources went unenumerated rather than reporting the inventory as the
complete set.

- All hook script paths resolve to existing files on disk
- Scripts are readable (not permission-denied)
- `timeout` is a seconds value — flag a recognizably millisecond-scale figure (a round thousands
Expand Down Expand Up @@ -131,16 +139,50 @@ bash "${CLAUDE_PLUGIN_ROOT}/skills/audit/scripts/fix-plugin-drift.sh" --yes

## Category G: Skill-listing budget

- **Overflow check** — if `/doctor` reports dropped skill descriptions, the skill listing has exceeded
its budget and the least-invoked skills' trigger keywords are silenced (names still resolve;
auto-invocation degrades silently). `/doctor` needs an interactive TTY — prompt the user to run it.
Repos with large skill rosters overflow routinely
- **Levers, cheapest first** — trim `description` / `when_to_use` frontmatter (key use case first;
1,536-char cap per entry), `skillOverrides: { <skill>: "name-only" }` in a contributor's
`settings.local.json` (does NOT apply to plugin skills), then `skillListingBudgetFraction` /
`SLASH_COMMAND_TOOL_CHAR_BUDGET` in project settings as a last resort (costs context every turn)
- **Recommend, don't apply the list** — `skillOverrides` is contributor-scoped; surface the candidate
least-invoked skills, leave the actual name-only list to the developer
Row-by-row criteria are in [audit-checklist.md](../reference/audit-checklist.md) "G. Skill-listing
budget". What governs the category:

- **State the budget, or the finding is not computable.** The listing budget is
`skillListingBudgetFraction` of the model's context window — **default `0.01`, i.e. 1%** — and
`SLASH_COMMAND_TOOL_CHAR_BUDGET` overrides it with a fixed character count, **documented fallback
8,000 characters**. Each entry's combined `description` + `when_to_use` text is separately capped at
`skillListingMaxDescChars`, **default `1536`**. For a 200K-token window, `200,000 × 4 × 0.01 = 8,000`
characters, which is why the env var's fallback is that number. Without the constant a report can say
"overflowed" but not "by how much", so quote it. All three are upstream-owned: confirm them in Phase
3 against [settings](https://code.claude.com/docs/en/settings) and
[env-vars](https://code.claude.com/docs/en/env-vars) before publishing a number
- **Overflow check — two routes, and only one survives a headless run.** `/doctor` estimates the
listing's cost and its biggest contributors, and it needs an interactive TTY, so prompt the user to
run it. When this audit runs headless — `-p`, a spawned agent, a background job — use the documented
debug route instead: *"When the listing exceeds its budget, Claude Code also writes a warning to the
debug log, visible with `--debug`"*
([skills](https://code.claude.com/docs/en/skills), "Skill descriptions are cut short"). Report which
route was taken; a category that names only `/doctor` yields nothing in the harness's own headless
mode. `/context`'s Skills row reports the listing size after the budget is applied — a second
*interactive* reading, not a headless one. Overflow silences the least-invoked skills' trigger
keywords (names still resolve; auto-invocation degrades silently), and repos with large skill rosters
overflow routinely
- **Measure the roster composition before naming a lever.** Count listing entries by origin — plugin
skills, project skills (`.claude/skills/`), user skills (`${CLAUDE_CONFIG_DIR:-~/.claude}/skills/`).
This is the single input that decides which levers exist, and it is cheap. A run that skips it
recommends levers the operator cannot pull
- **Levers, cheapest first — and the ordering depends on that composition:**
- *Any origin* — trim `description` / `when_to_use` at the source, key use case first. Costs nothing
at runtime and is the only lever that helps every roster
- *Project and user skills* — `skillOverrides: { <skill>: "name-only" }` in a contributor's
`settings.local.json`
- *Plugin skills* — `skillOverrides` **does not reach them**: *"Does not apply to plugin skills,
which are managed through `/plugin`"* (settings) and *"Plugin skills are not affected by
`skillOverrides`. Manage those through `/plugin` instead"* (skills). So on a plugin-heavy roster
the lever is `/plugin` — disabling a plugin removes its skills from the listing — plus trimming
the descriptions upstream in the plugin that owns them. Neither page documents a per-skill
`name-only` state reachable from `/plugin`, so do not promise one
- *Last resort, any origin* — raise `skillListingBudgetFraction` / `SLASH_COMMAND_TOOL_CHAR_BUDGET`
in project settings. It costs context every turn, which is why it is last here even though the
docs present it first
- **Recommend, don't apply the list** — `skillOverrides` is contributor-scoped and `/plugin` is a
machine-level action; surface the candidate least-invoked skills, leave the actual list to the
developer

## Category H: Model and effort settings

Expand Down
Loading
Loading