feat(skill-quality): enforce frontmatter name matches skill directory - #719
Conversation
PLUGIN-PHILOSOPHY has always required the frontmatter `name` to match the skill directory name, but nothing verified it — check 1 asserted only that `name:` was present and non-empty. The mismatch is not cosmetic. The directory name is what Claude Code namespaces the skill by, so a divergent frontmatter name silently relocates the invocation the doctrine says the skill has. And because the slash-command picker labels rows by the resolved leaf name, the drift never surfaced in the listing either. Lands blocking rather than as a warning: all 144 skills already conform, so there is no debt to grandfather and no baseline file to stale-guard. Closes #712 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ogVV7z2Heg9ojJ88nqNxh
|
Claude finished @kyle-sexton's task in 2m 17s —— View job PR Review: feat(skill-quality): enforce frontmatter name matches skill directoryTasks
Overall AssessmentSolid, well-scoped enforcement patch. The logic is correct, the
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 691a77ddd6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A trailing `# comment` is legal on a plain YAML scalar and is not part of the value, but skill_frontmatter::field returns the raw suffix — so `name: my-skill # migration note` compared as `my-skill # migration note` and failed an otherwise correctly named skill. Verified against the helper directly before fixing. Strips from the first whitespace-then-hash ahead of unquoting, so a quoted name carrying a comment resolves too. Skill names are kebab-case per the Agent Skills spec, so a '#' can never belong to the value. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ogVV7z2Heg9ojJ88nqNxh
|
Claude finished @kyle-sexton's task in 2m 37s —— View job Tasks
PR Review: feat(skill-quality): enforce frontmatter name matches skill directoryThis PR landed in two commits. The first (
|
| # | What | Direction |
|---|---|---|
| 14 | name: some-other-name in misnamed-skill/ |
Fail path: directory name appears in error message |
| 15 | name: "quoted-name" in quoted-name/ |
Pass path: double-quoted match doesn't trip gate |
| 16 | name: commented-name # kept for the migration note in commented-name/ |
Pass path: trailing comment is stripped |
| 17 | name: wrong-name # with a comment too in commented-bad/ |
Fail path: comment stripping doesn't mask real mismatch |
Cases 16 and 17 directly address the Codex P2 finding. Both the false-positive direction (16) and the false-negative direction (17) are verified — that's the right pair to test. ✓
Minor remaining gap — single-quoted YAML name
strip_quotes handles both "double" and 'single' via separate branches (skill-frontmatter.sh:62–70). The double-quoted path is exercised by case 15; the single-quoted path is not exercised for the name-matching gate specifically. The function is already proven for single quotes by the description/when_to_use checks, so the risk is low, but it is still a gap. Fix this →
No test for quoted-name-with-trailing-comment combination
name: "quoted-name" # comment — comment stripped first to "quoted-name", then unquoted to quoted-name. The combination is not tested, but the two halves are independently covered (case 15 proves unquoting, cases 16/17 prove comment stripping), so this is a very low-risk gap.
plugin.json — Version Bump
0.5.0 → 0.6.0 ✓. A new deterministic FAIL condition is user-visible behavior; minor bump is appropriate.
CHANGELOG.md
The [0.6.0] entry covers the enforcement logic and its rationale but does not mention trailing YAML comment stripping (added in the second commit). The omission is minor — comment stripping is an implementation detail of making the comparison robust rather than a separate feature — but the phrase "A quoted value is unquoted before comparison" could be expanded to "A quoted value is unquoted and a trailing YAML comment is stripped before comparison" to be complete.
Otherwise the entry is detailed and includes the blast-radius rationale, which is valuable for reviewers. ✓
Summary
The Codex P2 finding (false positive on name: foo # comment) has been correctly addressed in the second commit with a well-chosen pattern that respects the kebab-case constraint on skill names. The logic is sound, the test matrix covers all four critical directions (mismatch, quoted match, commented match, commented mismatch), and the version bump is appropriate.
Two minor notes remain from the prior review, both low-risk: a single-quoted YAML name test case, and a CHANGELOG line that could mention comment stripping. Neither is blocking. Safe to merge as-is.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b97bd362d1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A double-quoted YAML scalar may carry an escape — `"escaped\x2dname"` decodes to `escaped-name` — which strip_quotes leaves literal, so a correctly named skill would fail as a directory mismatch. Decoding YAML in bash to fix that would be the wrong trade. The Agent Skills spec already restricts a name to lowercase alphanumerics and hyphens, and PLUGIN-PHILOSOPHY defers to that charset in the same sentence that mandates the directory match — a half nothing enforced either. Validating it reports an escape sequence as the name defect it is, keeps the directory comparison working on literal text, and closes the other half of the doctrine's rule. All 144 skills conform, so this stays green alongside the match gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ogVV7z2Heg9ojJ88nqNxh
|
Claude finished @kyle-sexton's task in 3m 6s —— View job Tasks
PR Review: feat(skill-quality): enforce frontmatter name matches skill directoryThis review covers the full three-commit PR at HEAD ( Commit 3 (
|
CUR_NAME |
Charset valid? | Matches dir? | Outcome |
|---|---|---|---|
| empty | — | — | silent (existing missing-name error already fired) |
| non-empty | ✗ | — | is not kebab-case error |
| non-empty | ✓ | ✗ | does not match skill directory error |
| non-empty | ✓ | ✓ | clean |
Pre-existing quirk now resolved
Prior reviews flagged that name: #comment (no space before #) wasn't stripped by %%[[:space:]]#* and would land as CUR_NAME="#comment". Under commits 1–2 this would have fired a spurious mismatch error. Under commit 3, #comment fails the charset gate and reports "is not kebab-case" — a more accurate diagnosis. Not a regression, actually an improvement.
Test Coverage — Case 16b
The fixture correctly creates a directory named escaped-name (valid kebab-case) with frontmatter name: "escaped\x2dname". After strip_quotes removes the outer double-quotes, the literal \x2d fails the charset regex. The assertion checks both rc=1 and grep -q 'is not kebab-case' — the dual check is the right design, since rc=1 alone would pass even if the mismatch error had fired instead.
Complete test matrix for the name-matching gate across all three commits:
| Case | Direction | What |
|---|---|---|
| 14 | Fail | bare name mismatch |
| 15 | Pass | double-quoted match |
| 16 | Pass | trailing comment stripped (match) |
| 16b | Fail | YAML escape → charset defect, not mismatch |
| 17 | Fail | comment stripped but real mismatch still fires |
Both false-positive and false-negative directions are covered for each new mechanism. ✓
Remaining gap — single-quoted YAML name
Case 15 exercises name: "double-quoted". The strip_quotes single-quote branch (which handles name: 'single-quoted') remains untested in the name-matching gate. Proven via description/when_to_use checks elsewhere so risk is low — but the gap persists across all three commits. Fix this →
CHANGELOG — Incomplete Entry
A quoted value is unquoted before comparison
This line was written with commit 2 and not updated in commit 3. The [0.6.0] entry now understates what the gate actually does — it also strips trailing YAML comments (commit 2) and rejects names that aren't kebab-case (commit 3). A consumer upgrading to 0.6.0 with a skill named name: "escaped\x2d" would hit the charset error with no CHANGELOG hint that such a gate exists.
Suggested addition to the entry:
A quoted value is unquoted, trailing YAML comments are stripped, and the name must conform to the Agent Skills spec charset (lowercase alphanumerics and hyphens) — a YAML escape sequence is reported as a charset defect rather than a spurious directory mismatch.
plugin.json — Version Bump
0.5.0 → 0.6.0 ✓. All three commits ship together as one user-visible version; the minor bump is appropriate.
Summary
The third commit cleanly closes the Codex P2 finding (YAML escape sequences). Constraining the accepted charset rather than decoding YAML in bash is the correct trade — well-justified in the commit message and the inline comment. The logic is correct, the test matrix covers all four critical directions, and the overall PR delivers a sound enforcement of the name-matches-directory doctrine.
Two minor notes carry over: the single-quoted name test case, and a CHANGELOG entry that doesn't reflect comment stripping or the kebab-case constraint added in commits 2–3. Neither is blocking. Safe to merge as-is.
Closes #712
Summary
docs/PLUGIN-PHILOSOPHY.md:69requires the frontmatternameto match the skill directory name. Nothing enforced it — check 1 asserted only thatname:was present and non-empty. This closes the hole.Fix
plugins/skill-quality/scripts/check-skill.sh— check 1 gains a deterministic FAIL when frontmatternamediffers from the containing directory:The
-nguard keeps an absentnamereporting only the existing missing-namefailure rather than a spurious second one, and reuses the establishedstrip_quotes/fieldhelpers so a quoted value compares unquoted. Header check list updated to match.check-skill.test.sh— two new cases: a mismatching fixture must fail with the directory named in the message, and a quoted matching fixture must not trip the gate (the false-positive direction, which also proves quote stripping reaches the comparison).Blast radius is why this is worth stating plainly: the directory name is what Claude Code namespaces the skill by, so a divergent frontmatter
namesilently relocates the invocation the doctrine says the skill has. Because the picker labels rows by the resolved leaf name (see #710), the drift would not surface in the listing either — there is no way to notice it by eye.Verification
Landing this blocking rather than as a warning required proving the catalog already conforms, otherwise it would turn CI red on unrelated PRs. It does:
No baseline file, no grandfathering, no stale-guard needed.
Self-test — 15/15, including the two new cases:
Other gates run locally:
shellcheckon both changed scripts — clean.scripts/check-changelog-parity.sh --check— "Every versioned plugin has a CHANGELOG.md".markdownlint-cli2on the CHANGELOG — 0 errors.Version
0.5.0→0.6.0with a matching## [0.6.0]heading, satisfying--check-bump.Related