Repository navigation
claude-ops: audit-skill-visibility never determines reachability in a live run and misattributes skillOverrides to plugin skills #4078
Copy link
Copy link
Closed
Labels
needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.
Description
Summary
The skill's headline promise is "which skills can the model actually see". In every live run the reachability field is unknown for all skills with cause enablement-undetermined, because collect_fleet_at hardcodes plugin_enabled: None and no collector reads enabledPlugins. Two of the classifier's four documented-settings paths are dead: plugin-not-enabled has no collector, and skill-override-off cannot fire because the frontmatter parser never emits skill_override, and would be wrong if it did, since skillOverrides does not apply to plugin skills at all. The SKILL.md description says a plugin skill can be hidden by skillOverrides; docs and binary say it cannot.
Evidence
- Live run in this repo, Claude Code 2.1.263: 258 of 258 skills
reachability: unknown, causeenablement-undetermined, while.claude/settings.jsoncarries anenabledPluginsmap with one plugin set tofalse. scripts/audit_skill_visibility.py:collect_fleet_at(around line 369) sets"plugin_enabled": Nonewith the comment "Enablement is not knowable from the filesystem alone";collect_installed(around line 569) never sets it either;reachability(around lines 989-1006) branches onskill_overrideandplugin_enabled, neither of which any live collector fills.- Official docs (settings-reference, skills):
enabledPluginsmapsplugin@marketplaceto a Boolean atAny filescope;falsemeans installed but disabled; "Project settings take precedence over user settings ... set it tofalsein.claude/settings.local.json... managed settings override local settings." And: "Plugin skills are not affected byskillOverrides." - Binary (2.1.263):
enabledPluginsschema text "Settings precedence is user < project < local < flag < policy", per-key merge, last defined scope wins. Listing resolver:if(e.type!=="prompt"||e.source==="plugin")return"on"so plugin-sourced skills never consultskillOverrides.
Expected behaviour
--installedmode: readenabledPluginsthrough the settings reader introduced by claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model #4076 (per-key merge, user < project < local < policy; flag scope reported as unread). A plugin resolved tofalsemakes every one of its skillsreachability: hidden, causeplugin-not-enabled, evidence naming the scope file that supplied thefalse. Absent from every scope resolves to enabled, matching the product's default.--plugins-root(checkout) mode: enablement is a category error for a checkout. Emit one report line "reachability not assessed: a checkout is not an install" and set each row's reachability tonot-assessedinstead of 258 rows ofunknown. Frontmatter-basedmisconfiguredstill fires in both modes.- Remove the
skillOverridespath: delete theskill-override-offbranch inreachabilityand anyskill_overridefield, and removeskillOverridesfrom the SKILL.mddescriptionand body as a cause a plugin skill can lose visibility to. State the current rule and its reason (plugin skills are governed byenabledPlugins;skillOverridesgoverns non-plugin skills this audit does not enumerate), per.claude/rules/skill-bodies-state-current-rules.md. - Keep
unknownonly for the case the sources genuinely cannot answer (a manifest entry whose settings scopes are all unreadable), and say why inremedy.
Acceptance criteria
- [event-driven] WHEN
.claude/settings.jsonsetsplaygrounds@melodic-software: falseand the script runs with--installed, everyplaygrounds:*row readsreachability: hidden, causeplugin-not-enabled, evidence.claude/settings.json. - [event-driven] WHEN user settings set a plugin to
falseand project settings set it totrue, its skills aremodel-reachable(project wins); WHEN local setsfalse, they arehidden(local wins). - [state-driven] WHILE running with
--plugins-root, no row readsunknownfor enablement; the report carries the single not-assessed line. - [unwanted-behaviour] IF a settings file is unreadable or malformed, THEN affected rows read
unknownwith aremedynaming the file, neverhiddenormodel-reachable. - [ubiquitous] No code path or SKILL.md sentence claims
skillOverridesaffects a plugin skill;ReachabilityTestcovers the three precedence cases and the checkout-mode line. scripts/affected-tests.sh --runis green.
Out of scope
- The settings reader itself and managed-scope vendoring (owned by claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model #4076; this issue depends on it).
- Non-plugin skills (
.claude/skills, user skills), which this audit does not enumerate.
Related
- Depends on: claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model #4076 (shared settings reader).
- Sibling: the unscored ranking presentation issue, linked in the first comment.
- Official sources: code.claude.com/docs/en/settings (
enabledPlugins,skillOverrides, precedence), /docs/en/skills ("Override skill visibility from settings": "Plugin skills are not affected byskillOverrides"), /docs/en/plugins-reference (plugin disable).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.
Activity
kyle-sexton commentedon Sep 11, 2026
Sibling issues from the same audit run: #4076 (settings-derived budget inputs, the dependency) and #4077 (unscored ranking presentation).
Generated by Claude Code
fix(claude-ops): determine plugin-skill reachability from enabledPlug…
fix(claude-ops): honor defaultEnabled and exempt disabled plugins fro…
feat(claude-ops): audit-skill-visibility names misconfigured skills, …