docs(topics): commit the skill-listing budget measurement for #1271 - #1286
docs(topics): commit the skill-listing budget measurement for #1271#1286kyle-sexton wants to merge 4 commits into
Conversation
Moves the listing-budget measurement out of the checkout-local .work/ memory tier into the committed contract tier so later lanes can read it. The measurement recovers the listing algorithm from the shipped Claude Code v2.1.219 binary rather than inferring it, and lands three results that reframe #1271: the when_to_use split costs +3 characters per skill rather than saving any; the named reduction sources total 2.4% of description mass; and the 130-skill listing is 83,270 characters against a 40,000-character budget at 1M context, so no metadata edit closes the gap. The work is authoring discipline and future-proofing, not a budget fix. Every load-bearing figure was re-derived by script against this tree before committing, and both documentation quotations were re-fetched from the live pages. Section 9 records what reproduced, the two corrections (file count 187 -> 188, median entry 556 -> 594), and the three figures carried without independent reproduction. The section 3 hand-transcription caveat is preserved: 73% dark remains an upper bound, not a measurement. Section 8 is new and records the incumbent gate: skill-quality:check's check 3 already enforces trigger-phrase preservation across description and when_to_use combined, which is the acceptance gate #1271 needs and does not need building. It also records that check 2 is a per-skill cap rather than the shared listing budget, that it omits the 3-character joiner, and that check 12 warns on all 15 proposed rewrites. Refs #1271 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 4m 27s —— View job PR Review: docs(topics): commit the skill-listing budget measurement for #1271Tasks
SummaryDocumentation-only PR: one file added, no skill files touched. The document is structured well and the arithmetic is internally consistent throughout — I verified the key figures. The re-verification record (§9) is exemplary transparency practice. That said, there are three substantive precision issues, one of which understates how much the two detectors diverge on a specific row. Findings1.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55d0ecd461
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| no usage history is dropped **first**. This is per-machine state, so *which* descriptions survive | ||
| differs between machines and between users of the same plugin. A newly published skill is | ||
| invisible-by-description on every machine until someone invokes it — which is hard to do if they | ||
| cannot see what it does. |
There was a problem hiding this comment.
A zero usage score only gives a new skill the lowest priority; it does not guarantee that its description is dropped. When the enabled listing is under budget, when residual space remains, or when the new entry is cheaper than other zero-priority entries, its description can survive—the skip-and-continue behavior described immediately below explicitly permits the last case. The current absolute claim is therefore misleading for users who install only part of the marketplace or configure a larger budget.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 1 | Qualify the claim that every new skill is hidden | VALID — fixed in d0af6701 |
Confirmed against the document's own facts. Fact 7 (L123-126) explicitly permits survival: "It walks the droppable set in descending priority and skips any entry whose description does not fit, continuing to consider cheaper entries. So a small low-priority description can survive while a large higher-priority one is dropped." Fact 6 only establishes drop ordering ("A skill with no usage history is dropped first"), not a guaranteed drop. The budget also scales with context (L92-94) and skillListingBudgetFraction is consumer-settable (L44), so a partial install or larger fraction can overflow nothing at all. |
👍 |
Fixed in d0af6701: the sentence is now conditioned on the enabled listing actually overflowing, and names both escape hatches (an under-budget listing drops nothing; fact 7's skip-and-continue lets a cheap zero-priority entry survive on residual space).
| Consistency check on the model of the algorithm: melodic's name-only floor is 3,140, the granted | ||
| descriptions add 20,306, for 23,446 of the 40,000 budget — leaving 16,554 for caveman, codex, |
There was a problem hiding this comment.
The stated figures cannot produce this consistency check: 79,653 total description characters minus 59,824 withheld leaves 19,829 retained. Relative to the documented name-only floor, 35 full entries can add at most those 19,829 characters plus 70 characters for ": ", all 211 when_to_use characters, and two 3-character joiners, totaling 20,116—not 20,306. Thus at least one hand-transcribed figure is wrong, so this paragraph currently does not independently corroborate the observation as claimed.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 2 | Reconcile the retained-description arithmetic | VALID — fixed in d0af6701 |
Confirmed, and the exact mechanism located. 79,653 − 59,824 = 19,829 retained; 19,829 + 211 + 6 = 20,046 = the sum of entry text over the 35 kept. 20,306 − 20,046 = 260 = 2 × 130 — the per-entry +2 bullet cost was added a second time for all 130 entries, when it is already inside the 3,140 name-only floor. That floor is verifiable exactly: Σlen(name) = 82,533 − 79,653 = 2,880, and 2,880 + 2 × 130 = 3,140. The correct term is 2 × 35 = 70. |
👍 |
Your independent figure of 20,116 is arithmetically correct — reproduced here exactly.
Fixed in d0af6701: 20,306 → 20,116, 23,446 → 23,256, 16,554 → 16,744, and the parenthetical now shows the derivation inline so the check is self-verifying. Confirmed via grep that no other figure in the repo depended on the three old numbers.
One caveat left unchanged because the fix does not touch it: the 20,046 term assumes both when_to_use carriers are among the 35 kept entries. §4 proves that for playwright (1 of 1 kept) but not for playbooks (1 of 3 kept). The exact +260 match shows the original arithmetic already made that assumption, so correcting 260 → 70 introduces no new one.
The "keep a literal Use when: prefix inside when_to_use" fix for the check 12 collision was recorded without its cost. At ~10 characters per skill on top of the joiner's +3, a fleet-wide migration pays roughly +1,700 characters across the 130 model-invocable skills, which exceeds most of the 1,906 the named reduction sources can offer. That makes the choice of fix decide whether the migration is character-neutral at all, so the number belongs beside the option rather than only in the issue discussion. Refs #1271 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 3m 51s —— View job PR Review: docs(topics): quantify the check-12 workaround's character costTasks
What this commit doesCommit This is a useful addition. The choice of check-12 fix genuinely determines whether the migration's budget headline is positive or negative, and that belongs in the evidence document. Finding:
|
| Finding | Status |
|---|---|
§2 totalChars formula: unconditional WTU in the sum |
open |
| §5 "up to 10%" understates per-row variance (28% for name-restating opener) | open |
| §5 "zeroing" language tension with +3-costs-more row | open |
§1.2 return that decompiler artifact unexplained |
open |
On the Codex inline comments
Line 121 (new skill visibility): The Codex note is technically correct — zero usage only guarantees lowest priority, not unconditional drop; the skip-and-continue loop at §1.2 item 7 permits a cheap low-priority entry to survive. However the §3 observation data shows entire plugin populations going name-only in a single 40k session, so "invisible-by-description" is the realistic outcome and the practical claim holds. A parenthetical hedge ("in practice, on any machine approaching the budget ceiling") would be accurate without over-engineering.
Line 224 (retained-description arithmetic): Codex's arithmetic is correct: 19,829 retained description chars + 211 WTU + 6 joiners + 70 (": " × 35) = 20,116, not 20,306. The 190-char gap means at least one hand-transcribed figure (59,824 withheld or 20,306 granted) is slightly off. The document marks this as an "order of magnitude" consistency check and explicitly flags the hand-transcription provenance, so the gap does not invalidate any conclusion — but it does undercut the value of the check. Worth correcting if the source figures can be re-verified against the system prompt record.
Verdict
The new commit's argument is correct and important. One precision fix is worth landing before merge: clarify that the 1,700 figure is the combined joiner-plus-prefix cost, not just the prefix alone. The remaining items are carry-overs from the prior pass.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dabb779830
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,673 @@ | |||
| # Skill-listing budget measurement for issue #1271 | |||
There was a problem hiding this comment.
If this PR merges as-is, this file becomes permanent under docs/topics/**, even though the repository's contract-slice lifecycle requires that content in this tier exist only on the task branch and that the final pre-merge commit remove the entire slice (docs/conventions/topic-docs/README.md:290-308). Since the PR is explicitly intended to land this evidence for later work, either promote the measurement to a durable location such as docs/specs/ or keep it branch-local and prune it before merge.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 3 | Relocate this measurement out of the contract tier | VALID (defer) — escalated, not fixed here | The convention text is exactly as cited. docs/conventions/topic-docs/README.md:33 — "| Contract | docs/topics/<slug>/ | Committed on the task branch only; pruned before merge |", and step 4 requires "A final commit prunes the contract slice". But this is a repo-wide policy divergence, not a defect this PR introduces: git ls-tree -r origin/main -- docs/topics already returns 59 committed files on main; step 5's "Enforcement: a required check that the net PR diff contains no path under docs/topics/**" is not implemented in any of the 8 workflows (the only ci.yml match is a path-gating comment); and the suggested promotion target does not exist — git ls-tree -r origin/main -- docs/specs is empty. |
👍 |
Deferring rather than acting, because resolving this correctly is a decision outside a measurement-record PR and outside what this automated pass should decide unilaterally. The three available resolutions are not interchangeable:
- prune this file before merge — but that defeats the PR's stated purpose (body: "so later lanes can read it"), and its net diff is exactly this one file;
- promote to
docs/specs/— but that directory does not exist yet, so this PR would be establishing the durable tier as a side effect; - amend the convention to match observed practice — the honest option if 59 already-merged files represent the real intent, but that is a convention change, not a docs fix.
Flagging for a maintainer decision. Whichever way it goes, the same call applies to the 59 files already on main, so it wants deciding once at the convention level rather than per-PR.
| - Keep a literal `Use when:` prefix inside `when_to_use`. No code change, but it costs ~10 | ||
| characters per skill **on top of** the joiner's +3. Across 130 model-invocable skills that is | ||
| roughly **+1,700 characters** — which would consume most of the 1,906 the named reduction sources | ||
| can offer (§5) and push a fleet-wide migration net-negative on characters. The "close to | ||
| character-neutral" framing therefore depends on which fix is chosen; this option is not neutral. |
There was a problem hiding this comment.
Use when: cost
For the actual migration from the current frontmatter, Use when: is already present in the 125 detected trigger blocks, so retaining that prefix does not add roughly 10 characters per skill. The existing boundary is " Use when:", while the migrated boundary would be " - Use when:", an increase of only 2 characters per skill (about 250 characters), not roughly 1,700. The larger figure compares against the already-rewritten no-prefix proposal, then incorrectly compares that delta with the 1,906-character savings measured from the current baseline, so the conclusion that this option consumes most savings and makes the migration net-negative is unsupported.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 4 | Count only the incremental Use when: cost |
VALID — fixed in d0af6701 |
Confirmed. The load-bearing defect is the basis mismatch: the +1,700 is measured against the §6 rewrite (which strips the marker — L384: "No Use when: block, no name-restating opener"), while the 1,906 it was compared against is measured by §5 against the current baseline (L334-336). Measured independently at this branch's HEAD: 123 of 130 model-invocable skills already carry a trigger marker, 100 the literal " Use when:" (Use when 107, Use for 16, Triggers on 0). Your per-skill delta of +2 is correct by construction — baseline Prose. Use when: 'a' vs migrated Prose. + " - " + Use when: 'a' turns one space into three characters. |
👍 |
Your correction is right and the conclusion it overturns was the important part. Baseline-relative the option costs 2 × 123 = 246 characters — 12.9% of the 1,906, not "most", and not net-negative.
Fixed in d0af6701: the bullet now states both bases explicitly rather than silently mixing them — ~10 chars/skill relative to the §6 rewrite (noting §6's measured −920 for re-anchor shrinks accordingly), and ~2 chars/skill (~250 fleet-wide, ~13% of the 1,906) relative to today's baseline. The unsupported "consumes most of the savings / pushes the migration net-negative" claim is removed; the bullet now says the character-neutral framing survives on the baseline basis while noting the option is still not free.
Small delta from your figures: I measure 123 skills where §5 recorded 125, because #1276 renamed re-anchor to discipline after that measurement. The per-skill deltas are constants, so this does not change the conclusion.
There was a problem hiding this comment.
Correction to my reply above. I attributed the 123-vs-125 marker-count gap to PR #1276 renaming re-anchor to discipline "after that measurement". That attribution was wrong, and I should have verified it before posting.
This branch's merge-base is 39880e3b and git diff --name-only 39880e3bb7 HEAD -- plugins is empty — plugins/discipline/ does not exist here and plugins/re-anchor/ still does. My count was therefore taken at the document's exact baseline tree, so the rename cannot explain the gap.
The real cause is a detector-definition difference: my frontmatter substring scan (Use when / Use for / Triggers on over parsed description values) is not §5's span detector, which the document already discloses as heuristic at L330-334 — §5 itself reports two independent detectors disagreeing (1,906 vs 1,989).
The verdict on finding 4 and the applied edit are unaffected: the per-skill delta is +2 either way, and 2 × 123 = 246 vs 2 × 125 = 250 does not move the "~13% of the 1,906, not net-negative" conclusion. The committed text says "123 of 130", which is what I measured directly at this branch's tree.
Addresses three Codex P2 review findings on the measurement record. The consistency check double-counted the per-entry +2 bullet cost across all 130 entries when it is already inside the 3,140 name-only floor (82,533 - 79,653 = 2,880; 2,880 + 2x130 = 3,140), so it belongs only to the 35 granted entries: 20,306 -> 20,116, 23,446 -> 23,256, 16,554 -> 16,744. Fact 6's invisible-by-description claim was absolute; fact 7's skip-and-continue and an under-budget listing both permit survival. The check-12 workaround's +1,700 was measured against the section 6 rewrite but compared with the 1,906 measured from the current baseline. Against today's baseline the marker already exists (123 of 130 model-invocable skills carry one, 100 the literal "Use when:"), so the incremental cost is ~2 characters per skill (~250 fleet-wide), which does not make the migration net-negative. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kb58mL6JnfEos5zrhgn4GD
|
Claude finished @kyle-sexton's task in 4m 36s —— View job PR Review: docs(topics): correct listing-budget arithmetic and qualify two claimsTasks
What this commit doesCommit What's resolved✅ Consistency-check arithmetic fixed (§3)
The correction from ✅ "Invisible-by-description" qualified correctly (§1.2 fact 6)
The absolute claim is now conditional on overflow ("Wherever the enabled listing overflows the budget…") with the two carve-outs correctly identified: under-budget listings and the skip-and-continue path. The qualification is accurate and consistent with fact 7's description. ✅ Check-12 workaround analysis substantially corrected (§8)
The prior commit's claim that the
The underlying reasoning — that 123 of 130 model-invocable skills already carry a trigger marker, 100 the literal "Use when:" — is the load-bearing fact that makes this work, and it's stated clearly. Remaining open findings from prior passesThe four items below were raised in prior passes and remain unchanged in this commit. 1. §2 formula notation still unconditionally includes
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d0af670128
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Concretely: **zeroing every character the three named sources can offer still leaves the listing | ||
| above 81,000 characters** (both detectors agree; the independent one lands at 81,410), against a | ||
| 40,000 budget at 1M context and 8,000 at 200k. The 73% drop rate observed in §3 would not move. |
There was a problem hiding this comment.
Raw listing cost remaining above the budget proves only that some descriptions will still be dropped, not that the observed 95 name-only entries remain unchanged. Under the skip-and-continue algorithm described in §1.2, removing characters from descriptions that currently survive can create enough residual capacity for previously dropped cheaper entries, so the 73% rate can move even while the uncapped total remains above 81,000; this conclusion needs the observation session's priority ordering to be recomputed or should be qualified.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 5 | Qualify the claim that the drop rate cannot improve | VALID — fixed in 46649727 |
Confirmed: "would not move" asserts an invariance the algorithm does not give, and the document's own fact 7 (L123-126) contradicts it — "skips any entry whose description does not fit, continuing to consider cheaper entries". I bounded the effect rather than recomputing the priority ordering: only characters cut from the 35 granted descriptions free any budget (the 95 already-dropped entries cost zero today), and the cheapest readmittable entry is ai-briefing:generate at 310 description characters — 312 over its name-only floor (name = 20, matching the document's 330 name+description). So 1,906 / 312 = 6.1 entries is the ceiling, and §5's narrower "genuinely available" 807 / 312 = 2.6. |
👍 |
Your correction is right on the absolute, and the quantification shows the section's conclusion survives: the ceiling is ~6 readmitted entries (95 → ~89, 73% → ~68%), realistically 1-2. So "barely move", not "would not move" — and the overflow itself is untouched either way.
Fixed in 46649727: the sentence now says the rate would barely move and states why the pool is too small to shift it, instead of asserting invariance. §3 already flagged the 73% as an upper bound (L218), so this is consistent with the rest of the document.
| 5. **`Actions:` enumerations on multi-verb skills.** `work-items:track`, `skill-quality:check`, | ||
| `re-anchor:setup`. These tell the model which verb to pass, which is selection information, not | ||
| documentation. Cutting them turns a correct invocation into a body read. |
There was a problem hiding this comment.
re-anchor:setup has disable-model-invocation: true and, as §6 already notes, never enters the model's listing; it also exposes only one action. Its Actions: enumeration therefore cannot tell the model which verb to pass, so including it in this must-not-trim category incorrectly preserves text that has no effect on the listing-budget problem.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| # | Finding | Classification | Evidence | Reacted |
|---|---|---|---|---|
| 6 | Remove the DMI setup skill from the listing-preservation rule | VALID — fixed in 46649727 |
Confirmed empirically on both counts, and the document already contradicts itself here. plugins/re-anchor/skills/setup/SKILL.md:6 carries disable-model-invocation: true, and :4 is argument-hint: "check"; its own description says Actions: check (read-only verification, **default and only action**…). The document states the governing rule twice — L130: "skills with disable-model-invocation are filtered out of the listing", and §6 L385: "setup is DMI and never enters the listing". So §7 item 5 was preserving text that is never in the listing and enumerates no choice. |
👍 |
Fixed in 46649727. I scoped it out with an explicit parenthetical rather than deleting the token silently — §7 is a must-not-flag list written for a future sweep, so a bare removal reads as an oversight and invites re-adding.
One correction to the finding's framing, which does not change the verdict: the live path on this branch is plugins/re-anchor/, not plugins/discipline/. PR #1276's rename is on origin/main but not here — this branch's merge-base is 39880e3b and git diff --name-only 39880e3bb7 HEAD -- plugins is empty.
Addresses two further Codex P2 findings raised against d0af670. Section 5's "the 73% drop rate would not move" asserted an invariance the algorithm does not give: fact 7's skip-and-continue readmits cheaper entries when characters are freed. The conclusion survives but the absolute does not, so the sentence now bounds the effect instead. Only the 35 granted descriptions free budget, and the full 1,906 buys about six entries at the cheapest entry's size (ai-briefing:generate, 310 description characters, 312 over its name-only floor). Section 7 item 5 listed re-anchor:setup as an Actions: enumeration that must not be trimmed, contradicting the same document's population rule (DMI skills are filtered out of the listing) and its own section 6 note. The skill carries disable-model-invocation: true and exposes only check, so it is scoped out with a note rather than silently dropped. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kb58mL6JnfEos5zrhgn4GD
|
Claude finished @kyle-sexton's task in 5m 21s —— View job PR Review: docs(topics): qualify the drop-rate claim and scope the Actions ruleTasks
What this commit doesCommit
What's resolved✅ Codex P2: "Qualify the drop-rate cannot improve claim" — addressed
The new paragraph correctly identifies the two-part mechanical argument: (a) the savings pool only affects the 35 granted entries, not the 95 already-dropped ones, and (b) the pool size translates to a handful of readmissions under skip-and-continue. The overflow is unchanged because 83,270 − 1,906 = 81,364 still far exceeds the 40,000 / 8,000 budgets. The qualification from Codex's finding is now properly incorporated. ✅ Codex P2: "Remove DMI setup skill from listing-preservation rule" — addressed
New-content accuracy check: 1-char discrepancy worth notingLine: §7 item 4 (line 560) reports Remaining open findings from prior passesFour findings raised in prior review passes remain unchanged in this commit. 1. §2 formula notation still unconditionally includes
|
| Finding | Status |
|---|---|
New: §5 ai-briefing:generate 310 vs implied 311 description chars |
open (minor) |
§2 totalChars formula: unconditional wtu in sum |
open |
| §5 "up to 10%" understates per-row variance (~28% for name-restating opener) | open |
| §5 "zeroing" language tension with +3-cost trigger-block row | open |
§1.2 return that decompiler artifact unexplained |
open |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4664972736
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| totalChars = Σ over included entries [ len(pluginName:skillName) + 4 + min(len(desc + " - " + wtu), 1536) ] | ||
| + (n − 1) |
There was a problem hiding this comment.
For every skill without when_to_use—128 of the 130 entries according to §4—this formula charges the three-character " - " joiner even though entryText at line 82 adds it only when that field is populated. Applying the printed formula therefore produces 83,783 separator-inclusive characters rather than the documented 83,399, preventing readers from reproducing the measurement; make the joiner conditional on nonempty when_to_use.
Useful? React with 👍 / 👎.
| dropping a phrase fails it. This is the "no discovery was lost" gate, already built and already | ||
| wired. Verified against the §6 proposal using the repo's own extractor | ||
| (`plugins/skill-quality/scripts/skill-frontmatter.sh:102`): **122 of 122 phrases preserved.** | ||
| Nothing new needs building; the acceptance criterion should cite this check. |
There was a problem hiding this comment.
When compression removes an unquoted but matchable term from the descriptive prose, check 3 cannot detect the regression because skill-frontmatter.sh:102 extracts only single-quoted substrings. It proves preservation of the enumerated quoted phrases, not that “no discovery was lost”; using it as the sole acceptance gate can therefore approve exactly the vocabulary loss that §5 acknowledges affects matching. Narrow this claim or add a check covering the remaining discovery vocabulary.
Useful? React with 👍 / 👎.
|
Closing this without merging. Recorded on #1271 in full; the short form: The document is a contract-tier measurement under Two things are routed rather than lost:
The §6 prepared 15-skill rewrite is not being revived. #1271 has since recorded the The eight open review threads on this PR are all on the pruned document and are moot with it. |
The execution record described lanes by what sat uncommitted in a worktree, which is unreachable from a clone and therefore not a citation. Every lane has since published, so each row now names the PR or issue that carries its result: L2 as #1343, L3 as #1349 with the UNBACKED-to-OPINION mapping stated so the claim set does not need to survive as a separate list, L4 as #1385 closed with recovery on #1403, L1 as #1286 closed with the fold on #1271 and gate defects on #1404. The source article is named with its URL rather than left as "a practitioner article", so a reader auditing this ADR's premises can reach the thing the digests measured. The digests themselves prune with the contract slice, which is why the pointer replaces them rather than supplementing them.
…n specifies Closes #1417 ## Summary `docs/conventions/topic-docs/README.md` specifies a required check that a merged PR carries no path under the contract-slice dir. The check was never built, so the convention has been unenforced for its entire life and 17 slices reached `main` — 6 of them on a single day. The only place `docs/topics/` reached CI at all was `scripts/docs-only-paths.txt`, as a docs-only ALLOWLIST entry, which makes such a PR cheaper to merge rather than blocking it. Evidence the rule is real and was being enforced by hand: PR #1286 was closed rather than merged, explicitly because its content was contract tier under `docs/topics/`. ## The deletion exemption The convention's own step 4 is a final commit that PRUNES the slice, so a literal "no path under the contract dir appears in the diff" reading would red-line the very commit that satisfies it. This gate keys on where a path LANDS: removals pass, a history-preserving `git mv` out of the contract dir (step 3's graduation) passes, and only an add, edit, or rename-into is red-lined. That requires knowing a path's status, which `--name-only` cannot express, so the gate reads `--name-status`. Deliberate deviation from the letter of the convention in service of its intent; the three-dot `base...HEAD` range is unchanged. ## Existing debt The 17 pre-existing slices are grandfathered by slug in `scripts/contract-slice-baseline.txt`, using the same stale-guarded idiom as `changelog-parity-baseline.txt` and `orphaned-fixtures-baseline.txt`: `--check` fails on an entry whose slice no longer exists, so an exemption cannot outlive its debt and a future slice cannot inherit a grandfathered slug. Graduating and pruning them is tracked separately. This is why the gate can land now instead of after a 71-file cleanup: it stops the bleed immediately while each slice graduates on its own PR, by whoever owns it. ## Verification The 11-case suite covers the add, pure-deletion, untouched, grandfathered, new-slug-despite-baseline, graduation-out, rename-into, unresolvable-base, live-baseline, stale-baseline, and usage paths. Measured against the four open PRs that carry `docs/topics/` paths, rather than asserted: #1318, #1252, and #1096 pass on their baseline exemptions; #1400 fails, correctly, because it adds two slices that are not pre-existing debt. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n specifies (#1429) Closes #1417 ## Problem `docs/conventions/topic-docs/README.md:305-309` specifies a required check that the net PR diff carries no path under the contract-slice dir: > 5. Enforcement: a required check that the net PR diff (`git diff --name-only base...head`) > contains no path under the resolved `<contract_dir>/**` (default `docs/topics/**`). It was never built. The convention has been unenforced for its entire life, and `main` currently holds **19** contract slices — **8 of them landed on 2026-07-25 alone**, two of those while this PR was open. The only place `docs/topics/` reaches CI today is `scripts/docs-only-paths.txt:43`, and it is there as a docs-only **allowlist** entry — which makes a PR confined to it *cheaper* to merge by skipping the heavy lanes. Nothing blocks it. The rule is real and people have been enforcing it by hand: **PR #1286 was closed rather than merged**, explicitly because its content was contract tier under `docs/topics/`. That is the cost of the missing gate — correct behaviour depending on whoever is looking remembering an unenforced rule, and 19 directories showing how reliably that scales. **#1400 merged while this PR was open**, landing two more slices. It was flagged there, and it is the sharpest available evidence for the gate: the failure mode is live, not historical. ## Why the gate permits deletions The convention's own step 4 is *a final commit that prunes the slice*. A literal reading — "no path under the contract dir appears in the diff" — would red-line the very commit that satisfies the convention. So the gate keys on where a path **lands**, not on whether it appears: | Change | Verdict | |---|---| | Delete under `docs/topics/` | pass — this is the prune step | | `git mv docs/topics/x/PLAN.md docs/adr/…` | pass — this is step 3's history-preserving graduation | | Add / edit under `docs/topics/` | **fail** | | Rename *into* `docs/topics/` | **fail** | | Diff never touches `docs/topics/` | pass | Knowing a path's status requires `--name-status`; `--name-only` cannot express it. That is a deliberate deviation from the letter of the convention in service of its intent, called out in the script header. The three-dot `base...HEAD` range is unchanged, so a slice `main` gained after a branch forked stays out of scope and no stale branch is forced to merge-from-main over someone else's violation. ## Why this lands before the cleanup `scripts/contract-slice-baseline.txt` grandfathers the 19 existing slugs, using the same stale-guarded idiom as `changelog-parity-baseline.txt` and `orphaned-fixtures-baseline.txt`: `--check` fails on an entry whose slice no longer exists, so an exemption cannot outlive its debt and a future slice cannot silently inherit a grandfathered slug. Exemptions are resolved from the **base revision**, not the working tree, so a PR cannot add a slice and grandfather its own slug in the same diff. The diff is judged against the union of the base and head contract roots, so a PR that relocates `contract_dir` cannot leave the root it selected uninspected either. Both bypasses were live in earlier pushes and were caught in review. The alternative — prune all 19 first, then gate — is a ~1.3 MB change requiring a graduation judgement on each slice by whoever owns it, and it would conflict every open PR that carries those paths. Gating first stops the bleed immediately while each slice graduates on its own PR at its own pace. The burn-down is #1419; each prune PR drops its own baseline line, and the stale guard means the debt cannot be quietly abandoned half-done. ## Verification `scripts/check-contract-slice-prune.test.sh` — 19 cases, all green: add, pure deletion, untouched tree, grandfathered slug, new slug despite a baseline, graduation out, rename in, unresolvable-base-ref (fail-closed, exit 2), live baseline entry, stale baseline entry, usage, self-grandfathering rejected, a pre-existing entry still exempting, `contract_dir` resolved from the concern file, a relocated root moving the gate's scope, a root-equivalent value exiting 2, a slug-less baseline surviving `set -u`, both base and head roots policed, and a grandfathered slice migrating to a relocated root. Four review findings were raised across two rounds and all four were reproduced before being fixed — two bypasses (self-grandfathering; a relocation leaving its own root uninspected) and two fail-open / crash defects (`contract_dir` ignoring the concern file; the gate aborting under `set -u` once the baseline empties, which is the exact end state #1419 drives toward). See the resolved threads; each carries its reproduction and the case that pins it. **Measured against the open PRs that actually carry `docs/topics/` paths, rather than asserted.** Because the gate reads the baseline from the base revision, these were run against a base that already carries it — the post-merge condition: | PR | Slice | Result | |---|---|---| | #1252 | `plugin-audit-port` | passes on baseline exemption | | #1096 | `fresh-eyes-checkpoint-audit` | passes on baseline exemption | | #1318 | `context-engineering-claude-5` + the two slices #1400 landed | fails until rebased onto a `main` carrying the updated baseline | #1318's failure is an artefact of it predating #1400's merge, not a defect: its branch adds those two files relative to its own fork point. Once rebased, they are on `main` and in the baseline, so they leave its diff entirely. The self-grandfathering bypass was verified closed by re-running the reviewer's own reproduction against the fix. Also verified: `shellcheck` clean, `actionlint` clean, `shfmt` clean, the org comment-hygiene policy reports zero violations in the new files, and both scripts carry the executable bit. ## Wiring `contract-slice-prune-gate` is added to `ci-status`'s `needs:` list. That aggregate derives its lane list from the needs graph, and `ci-status` is already a required status check on the ruleset, so the new gate becomes required with **no ruleset edit**. Job naming matches the existing precedent (`silent-skip-gate`, `orphaned-fixture-gate`, `changelog-parity-gate`). The self-test runs unconditionally so a broken gate cannot mask a regression; the PR-diff step is event-gated. ## Related - #1419 — graduate and prune the grandfathered slices. Each prune PR drops its own baseline line, and this gate's stale guard fails once an entry outlives its slice. Not closed by this PR. Its inventory needs updating to 19 once this lands. - #1400 — merged while this PR was open, landing two more slices; both added to the baseline as debt rather than treated as incoming work. Flagged there before it merged. - #1252, #1096 — open PRs carrying grandfathered slice paths; verified passing on their baseline exemptions. #1318 needs a rebase past #1400 (see Verification). - #1286 — closed by hand for carrying contract-tier content, which is the manual enforcement this gate replaces. - `docs/conventions/topic-docs/README.md` — the convention specifying this check as step 5. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
Summary
Moves the skill-listing budget measurement out of the checkout-local
.work/memory tier into the committed contract tier atdocs/topics/context-engineering-rightsizing/design/listing-budget-measurement.md, so later lanes can read it.The measurement recovers the listing algorithm from the shipped Claude Code v2.1.219 binary rather than inferring it. Three results reframe #1271:
when_to_useis appended todescriptionwith a literal" - "joiner, so migrating a trigger block into it costs +3 characters per skill. It saves nothing. skill metadata: trigger phrases crammed into description; when_to_use is unused and the shared listing budget silently drops them #1271 already says this; the implementation confirms it exactly.So this is authoring discipline and future-proofing, not a budget fix — and the PR is deliberately written that way. Overstating it would not survive contact with these numbers.
No skill file is modified. This PR is documentation only, one added file.
The
re-anchorrewrite is prepared, not applied§6 carries a complete rewritten frontmatter block for all 15 model-invocable
re-anchorskills, measured at −920 characters (−9.2%). It is not applied, because PR #1276 renames there-anchorplugin todisciplineacross 45 files and is open and unmerged. Racing a bulk metadata migration against a bulk rename of the same files is the conflict #1271's own "Blocked by" section names. The block ships as a ready-to-apply proposal against the renamed paths.follow-our-standardsgoes up by 13 characters under the rewrite. That asymmetry is the honest shape of the change on a short description and is stated as such.New: the incumbent gate (§8)
Read before proposing anything new.
skill-quality:checkalready does most of this:check-skill.sh:236-304(check 3) extracts trigger phrases fromdescriptionandwhen_to_usecombined on both sides of the diff. The exact migration skill metadata: trigger phrases crammed into description; when_to_use is unused and the shared listing budget silently drops them #1271 proposes therefore passes it, while genuinely dropping a phrase fails it. This is the "no discovery was lost" gate, already built. Nothing needs building — the acceptance criterion should cite this check.check-skill.sh:219-234(check 2) is a per-skill 1,536-character cap, not the shared listing budget its skill description advertises. There is no incumbent check on the shared budget at all — which is how it overflowed 10× unnoticed.DESC_LEN + WTU_LENatcheck-skill.sh:227, omitting the 3-character joiner. Not currently binding (longest entry is 1,161), but wrong at the root.check-skill.sh:440-448(check 12) warns unlessuse whenappears in either field. The §6 rewrite uses bare quoted phrases with no marker, so all 15 proposed rewrites WARN. Recorded with two honest fixes; the better one editscheck-skill.shand so sequences behind feat(skill-quality): fresh-eyes delegation doctrine + conformance gate (check 21) #1096, which is already editing that file. Nothing in this PR touchescheck-skill.shordocs/PLUGIN-PHILOSOPHY.md.Test plan
Every load-bearing figure was re-derived by script against this tree before committing, rather than transcribed from the source document. §9 is the full record.
name+description, 83,270 entry-cost sum, 108,460 and 109,581 for all 181, zero entries over the 1,536 cap, longest 1,161 (docs-hygiene:audit-derivability), 2 skills usingwhen_to_usecarrying 211 chars, 3,140 name-only floor, 40,000 / 8,000 budgets, and every term of the 192-entry decomposition.find plugins -name SKILL.mdis 188, not 187, and 7 nestedSKILL.mdfiles are excluded, not 6 — six undervendor/plus one underreferences/. Median entry is 594 (description) / 600 (entry text), not 556. Neither correction changes a conclusion.n − 1separators the measurement's own formula specifies gives 83,399. Both now appear on their stated basis.plugins/skill-quality/scripts/skill-frontmatter.sh:102, not a reimplementation — 122 of 122 phrases preserved, 0 lost.code.claude.com/docs/en/skills.md,.../settings.md) and matched character-for-character. A third-party SchemaStore divergence onskillListingMaxDescChars(8000 vs the official and shipped 1536) is recorded so a reader consulting it is not misled.markdownlint-cli2: 0 errors.typos: clean. Scanned for machine-specific absolute paths and usernames per the topic-docs redaction bar: none.Related
No linked issue — this PR deliberately closes nothing. It lands the evidence half of #1271 only; the metadata migration that issue asks for is blocked behind #1276, so #1271 stays open until that ships.
Refs #1271 — the reconciliation, the levers, and the incumbent-gate findings are posted as a comment there: #1271 (comment)
Blocked-on for the skill edits, not for this PR: #1276 (
re-anchor→disciplinerename).Sequencing: any fix to check 12 sequences behind #1096.