feat(claude-config,review): promote I8-b to unscoped and give severity.md enumerable tier criteria - #1881
Merged
kyle-sexton merged 3 commits intoAug 3, 2026
Conversation
…y.md enumerable tier criteria Promotes instruction-audit catalog row **I8-b** (conservative-reporting detection) from `Model scope: opus-5` to unscoped, and states the `review` plugin's severity tiers as decidable tests. `claude-config` 0.18.0 -> 0.19.0; `criteria.md` 1.5.0 -> 1.6.0; `review` 0.15.5 -> 0.16.0. ## The promotion, and the gate arm it meets I8-b's promotion gate is met on its **second arm** — multiple model guides converge. The Sonnet 5 prompting guide, "Code review harnesses", states the same claim as the Opus 5 guide about the same behavior, on the same trigger phrases (`source.md:140`): > When a review prompt says things like "only report high-severity > issues," "be conservative," or "don't nitpick," Claude Sonnet 5 may > follow that instruction more faithfully than earlier models did: it > may investigate the code just as thoroughly, identify the bugs, and > then not report findings it judges to be below your stated bar. The Sonnet 5 guide joins `## Sources`, because `criteria.md` carries its own invariant that "the trigger set is the source set." ## A citation defect the promotion exposed I8-b's Detect names three trigger phrases and attributes them to the Opus 5 guide. **"don't nitpick" appears nowhere in that guide** — `grep -cin nitpick` over the Opus 5 source returns `0`. The Opus 5 guide states only "only report high-severity issues" and "be conservative" (`opus-5-prompting/source.md:20`). The Sonnet 5 guide names all three verbatim, so it is the third phrase's only cited home, and the Source line now says so rather than leaving a trigger phrase attributed to a page that does not contain it. ## The severity.md change — and the premise it corrects **#1880 held this promotion back on the stated ground that it "makes the row fire on this repo's own `plugins/review/context/severity.md`." That premise is false**, on three independent grounds: 1. **No scanner candidate.** `severity.md` produces zero I8-b rows. Its only candidate under `instruction-scan.sh` is one I6 row. 2. **Excluded by I8-b's own carve-out.** The row states that "severity-based routing where everything is still reported somewhere ... are not findings." `severity.md` classifies findings; it withholds none. 3. **Outside the audited population**, exactly as #1880 established for `criteria.md` itself — the Phase A inventory covers `CLAUDE.md` / `rules/` / `skills/` / `agents/` / `output-styles/` under the user and project roots, and this repo has no `.claude/skills/` at all. The promotion could therefore have shipped alone. **The severity.md work is retained because it stands on its own source**, not as an I8-b remediation: the same Sonnet 5 guide section states, eight lines later (`source.md:149`), "be concrete about where the bar is rather than using qualitative terms like `important`" — and `important` is one of this file's own tier names. That is triage row RA-2, and it is a distinct claim from the one I8-b cites. Each tier now carries a test a reviewer can argue a novel finding against, in place of "Must fix" / "Should fix" / "Consider". **No finding changes tier** — the tests restate the existing bars and the example lists are unchanged. **The P1-P5 security fold now states its precedence over those tests.** CRITICAL's test names an unsafe result, which a P3 finding also satisfies read literally; without that precedence the criterion-stating change would have silently promoted every P3 from IMPORTANT to CRITICAL. ## Verification **Self-fire check, both trees, same command.** I8-b population over `origin/main` is **23 rows across 6 files**; over this branch, **28 rows across 7 files**. `severity.md` appears in neither — `grep -c` returns `0` on both. All five added rows are this commit's own quoting of the trigger phrases (`criteria.md` 2 -> 4, `CHANGELOG.md` 0 -> 3), fenced by I8-b's quoted/meta fence. Every one of the 23 pre-existing rows was already fenced: the restraint-clause shape (`code-tidying`'s tidyings catalog) and quoted/meta surfaces (this criteria file, the scanner and its tests, two model-adaptation delta chapters). **This repo has zero genuine I8-b findings.** **FAIL-case validation.** A planted file carrying the three phrases and `severity.md` were scanned in one invocation: the planted file emitted `I8-b` on all three lines, `severity.md` emitted none — proving the run was capable of detecting a hit rather than merely finding nothing. **Equivalence proven, not asserted.** The repo-wide enumeration uses `git grep` with the scanner's own `I8_B_ERE`, after the per-file scanner loop timed out. Running the real scanner over the 6 files `git grep` identified returns the identical 23-row `file:line` set. Detection logic is untouched: `instruction-scan.test.sh` reports 46/46. `markdownlint-cli2` reports 0 errors over all four changed markdown files. No linked issue ## Related - Phase 3b of the doc-corpus campaign. #1880 deferred this promotion; this commit ships it and corrects the ground on which it was deferred. - Source pass: `RA9-EXTEND-OR-CITE-PASS-2026-08-02.md`; triage row RA-2. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The promotion left one demonstrative behind. I8-b's Detect line read "which **this model** follows literally" — phrasing written when the row carried `Model scope: opus-5`, where "this model" resolved to the scoped one. An unscoped row has no scoped model, so the referent dangled. Now "which **current models** follow literally", which is what both cited guides state: the Opus 5 and Sonnet 5 guides describe the same literal-adherence behavior for their own models, and the row fires for every target. Text-only, in the row's Detect prose. Detection is untouched — the three trigger phrases and the two fences are unchanged, and `instruction-scan.test.sh` reports 46/46. `markdownlint-cli2` reports 0 errors. Folded into the unreleased `claude-config` 0.19.0 entry rather than opening a new version. No linked issue ## Related - Follows 3603c8a on this branch, which performed the promotion. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… a falsified page enumeration Two defects from independent verification of this branch. Both fold into the unreleased `claude-config` 0.19.0 and `review` 0.16.0 entries; no new version. ## D2 — the "no finding changes tier" guarantee did not hold CRITICAL's test read "a concrete input, caller, or subsequent change that the defect makes produce a wrong result." Read literally, **code duplication satisfies it**: the subsequent change is an edit to one copy, after which the copies diverge. That match was not merely available, it was decisive. The same section states the tests are applied in order and that "resemblance to a listed example is not that argument" — so a reviewer could neither stop at IMPORTANT nor rebut the CRITICAL match by pointing at duplication's place in the IMPORTANT example list, which is where the previous qualitative text pinned it. The criterion-stating change would have silently re-tiered duplication, which was out of scope. **Fix: the subsequent-change limb is now qualified `otherwise-correct`.** That draws the line the example lists already assumed — a **cascading architecture violation** breaks a future change whose author did everything right, so it stays CRITICAL; **duplication** bites only through a future edit that is itself incomplete, so it stays IMPORTANT. Adjudicated against all eleven listed examples, not just the two in tension. The four CRITICAL examples hold: correctness bugs, security vulnerabilities and broken contracts all satisfy the test on the *input* or *caller* limb, which the qualifier does not touch, and cascade holds on the amended limb. The four IMPORTANT and three SUGGESTION examples each still fail CRITICAL and land where they did. The leak existed only on the subsequent-change limb, which is why the qualifier is attached there rather than to the test as a whole. ## D3 — a closed enumeration this branch itself falsified The Recheck-triggers block read "Model-specific pages (the Fable 5 and Opus 5 guides) are superseded on each model generation." Adding the Sonnet 5 guide to `## Sources` made that list wrong on arrival. **Dropped rather than extended**, so it cannot go stale again: it now reads "the per-model prompting guides under Sources." Extending it would have reproduced the defect at the next model guide. The enumeration also contradicted its own paragraph three lines above, which argues that "naming a subset would leave the harness-behavior rows depending on pages nothing watches" — the block made exactly that mistake in its closing sentence. ## Verification `instruction-scan.test.sh` 46/46. `markdownlint-cli2` 0 errors over all four changed files. Detection logic untouched — D3 is prose in the Recheck-triggers block and D2 is in another plugin entirely. ## Not fixed here, deliberately A third defect is in commit 3603c8a's MESSAGE, which cites the concrete-bar quote as "eight lines later (source.md:149)". The correct citation is **`source.md:150`, ten lines after the three-phrase line at `source.md:140`**. Commit messages are immutable and this branch is append-only, so it is corrected in the PR body rather than by a rewrite. No tracked file carries the wrong number. No linked issue ## Related - Follows 3603c8a (the promotion) and ffad365 (the scoped-model referent) on this branch. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 4m 4s —— View job Review: PR #1881 — I8-b promotion + severity.md tier criteriaTasks
Scope notePer The changed criteria are prose-body content inside a skill's
Also spot-checked the scanner's own I8-b ERE ( FindingsNo correctness or design issues. The content is internally consistent:
One trivial nit, not worth a fix cycle on its own: Not run: |
|
Claude finished @kyle-sexton's task in 42s —— View job Security review complete
No security issues found. This PR's diff is entirely documentation content — two |
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
#1881 squash-merged to main while this branch was in flight, taking claude-config to 0.19.0 and criteria.md to 1.6.0 — the same numbers this branch had claimed. Merged rather than rebased: this branch is already pushed, and rewriting pushed history needs a force-push, which the campaign's append-only rule prohibits. The repo squash-merges PRs, so graph linearity buys nothing here. Version reconciliation. #1881 landed first and owns 0.19.0 / criteria 1.6.0. This branch renumbers to plugin 0.20.0 / criteria 1.7.0, and its CHANGELOG block moves under a new [0.20.0] heading above the merged [0.19.0] one. Both version literals auto-merged silently because the two sides wrote identical text — a clean merge that was still wrong, so both were corrected by hand rather than trusted. Conflict surface was narrower than expected: only CHANGELOG.md conflicted. The `## Sources` list auto-merged clean, keeping #1881's Prompting Claude Sonnet 5 entry alongside this branch's five (effort, thinking troubleshooting, settings, environment variables, prompt caching). Two semantic checks, since a clean textual merge proves nothing about meaning: - I17, I18 and I20 each defer to "the same audience test I8-b applies". #1881 promoted I8-b to unscoped and reworded its Detect and Source, but left fence 2 — the quoted/meta-surface carve-out those three cite — textually intact. All three references still resolve. - #1881's CHANGELOG asserts its I8-b population here is 23 candidate rows across 6 files, every one already fenced. That count was measured without this branch's rows. Re-checked on the merged tree: every I8-b candidate in criteria.md sits at lines 276-304, inside I8-b's own row, and the new rows begin at 530. This branch adds no I8-b candidates, so the claim survives unchanged. Detection re-swept over the merged tree, including #1881's severity.md rewrite: unchanged at 1 I17 candidate and 4 I19 candidates, the same files as before, every one landing on a documented fence. Zero adjudicated findings for all four rows. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…9, I20 (#1882) Ships the consumer-facing payload of the doc-corpus campaign: four new rows in the `claude-config:audit-instructions` catalog. `claude-config` 0.19.0 -> 0.20.0; `criteria.md` 1.6.0 -> 1.7.0. ## The rows - **I17** (base + I17-a + I17-b) — the thinking-disable x `xhigh`/`max` effort hazard. Stated at full strength: a per-request 400, real and **unguarded** (EC-9 replayed live 400s in the local OTEL store at `cc-logs.json:65283/:65291`, `model=claude-opus-5`; no documented pre-request guard). Statically checkable against settings files. Fences: `effortLevel` accepts `xhigh` but not `max` (auditors are not sent hunting a literal that cannot exist); `ultracode` covered explicitly ("sends `xhigh` to the model" — matches neither literal yet triggers the hazard). - **I18** — thinking blocks altered on the way back to the model: signature preservation, the `block.type == "thinking"` type-filter smell, within-turn echo integrity. Scope extends through the Agent SDK (owner decision Q3) and — per EC-2's structural corpus test (every thinking block in 3.6 GB carries `signature`, zero without) — local transcript handling and transcript-parsing tooling. - **I19** — restated external benchmark figure with no recheck trigger. Ships at **OPINION tier, off by default**: no official page states the requirement, and claiming ANTHROPIC-DOCS with `Source: none` would break the catalog's own authority invariant. The worked example is this repo's own RA-4 fix (PR #1876) and `docs/conventions/upstream-drift/README.md`. - **I20** — prefilled assistant response (model-delta standing row; RA-6 replayed `prefill` at 0 on this repo, expected hit rate near zero, cheap to carry). ## Disputes carried, resolved with evidence - **`redacted_thinking`:** the brief forbade any clause; literal obedience was impossible — the type-filter rule's decisive upstream sentence (`thinking.md:867`) states its failure mode *as* silently dropping `redacted_thinking` blocks. Shipped: the upstream sentence verbatim, plus a Must-NOT-flag fencing out any clause premised on those blocks being present locally (EC-2 found zero structural instances in 3.6 GB). - **IA-4's thinking-churn half dropped:** `prompt-caching.md` contains zero occurrences of "thinking" (replayed live twice) — a thinking-toggle cache-invalidation claim would have been a fabricated citation. I17-b covers effort churn only, quoting the page verbatim including the harness's own confirm-before-applying surface. - **Brief/source count discrepancy, recorded:** the brief cites 136,295 thinking blocks; EC-2's own record says 136,176. No shipped row depends on either (a local count is not a consumer fact). ## Verification Independently verified by a second model with the implementer's rationale withheld — two independent full passes (the first died with the session; the second re-derived everything from bytes and live raw fetches: 52,769 B `thinking.md`, 29,777 B `prompt-caching.md`). - **Standing gate (never ship a check this repo fails):** zero adjudicated findings across all four rows on the merged tree, twice replayed. Every candidate adjudicated by name: `opus-5.md:125` (model-adaptation delta, fenced), `orchestration.md` x3 (carry the RA-4 four-part drift record), `action-quality.md:9` (pointer, no figure). The vendor fence is load-bearing: unfenced, I19 fires 15 times, 11 inside `boris/vendor/` — the verbatim upstream baseline kept for byte-drift detection. - **Planted positives:** 7/8 greppable shapes fire on a constructed file; I18 shape 1 (signature dropped on reconstruction) has no greppable literal and is validated by reasoned read — recorded as a residual regression gap, not rounded up. - Post-review corrections shipped as follow-up commits (append-only): two quote defects found by review (a silently converted comma, a wrong section attribution), one truncation-marker fix (`c375d90739`), and the merge-reconciliation with #1881 (versions renumbered; `plugin.json`/frontmatter auto-merged "clean and wrong" on identical strings and were corrected by hand). - **Known inaccuracy in an immutable commit message:** `c375d90739`'s body claims the unmarked truncation "is the misquote-without-ellipsis defect the criteria file itself flags elsewhere" — `criteria.md` carries no quote-fidelity criterion; the classification came from the independent verifier, not the file. Recorded here rather than rewriting pushed history (same handling as #1881's citation defect). - `instruction-scan.test.sh` 46/46; markdownlint 0 errors; changelog parity/bump/order pass; `check-skill.sh` PASS. No linked issue ## Related - Phase 3b consumer payload of the doc-corpus campaign; sibling merges #1875-#1881. - Owner decisions Q3 (Agent SDK scope), Q4/IA-12 (agnosticism routing test), Q5. - EC-2 and EC-9 empirical records: `.work/_harness-snapshots-2026-07-31/EMPIRICAL-CHECKS-RESULTS-2026-08-02.md` (memory tier). - Follow-ups filed from this lane: mechanized pre-scan for I17-I20 (gate sweep is currently a scratchpad script), post-merge version-literal assertion for the silent auto-merge trap. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…tified Q8 split (#1884) Executes the owner-ratified Q8 SPLIT for the two rules the agnosticism test did not settle by itself: the agnostic-mechanism halves of IA-6 and IA-10-A2 ship here as catalog rows **I21** and **I22**; the consumer-state halves route to the dotfiles seam as drafts (nothing org-specific lands in the catalog — verified by org-term sweep). `claude-config` 0.20.0 -> 0.21.0; `criteria.md` 1.7.0 -> 1.8.0. ## The rows - **I21 — effort level pinned across a model change with no re-sweep** (`mechanical` / `ANTHROPIC-DOCS` / `warning`, unscoped). Grounded in the model-agnostic calibration sentence ("the same level name does not represent the same underlying value across models", `model-config.md:489`, no model qualifier — promotion gate met on the strong form). The model range is a **Detect condition** because the consequence varies: Fable 5 / Opus 4.8 / Opus 4.7 get a first-run default hold; Opus 5 has no hold — a previously set level carries over, which is where a stale pin actually reaches the request. Whether a settings-file `effortLevel` pin releases the hold is **unstated upstream and recorded as unresolved** rather than inferred. Four fences: bare `high` (with the Opus 4.7 `xhigh`-default carve-out stated), per-task choices, config keys (routes to `claude-config:audit`), and schema documentation with illustrative samples — bounded so that documenting a field *and then telling the reader which level to use* still fires. - **I22 — model-routing doctrine with no baseline named** (`mechanical` / `OPINION` / `info`, off by default, I19's exact precedent shape, `Source: none`). The contribution is the **delta-not-re-run** discipline: a trigger nobody can afford to re-run is not a control. Carries no baseline of its own by design — naming one would hand every consumer a foreign snapshot. Non-duplication argued in the row: I19 covers restated figures; the catalog-wide trigger governs this catalog's own staleness, not an audited surface's. ## Verification Independently verified by a second model with the implementer's rationale withheld; every quoted upstream sentence re-fetched raw and matched byte-exact (`model-config.md` 83,644 B, `effort.md` 21,744 B). - **Standing gate: zero adjudicated findings on this repo**, every candidate adjudicated by name (11 `effort: high` frontmatter files, 10 `model:` pins, the claude-ops lanes schema samples, the boris vendor baseline and transcribed-practice lines, `PLUGIN-PHILOSOPHY.md`'s dated-and-triggered lane rules). The verifier's independent hunt found no missed candidate. - **Fence 4 adversarially validated**: the verifier's constructed schema-doc-plus-prescription fixture still fires; the bare sample does not. The second commit exists because the first adjudication of `claude-ops/lanes/context/config.md:26` was wrong under the original fences — the fence gap is closed at the fence, not by stretching an existing one. - **Planted positives and negatives**: both rows fire on their positives; all ten negative near-miss lines land in distinct fences. - The second commit also drops an I22 over-claim (a sync stamp tracks transcription currency, never whether advice still names a live model) — fence now rests on the no-vet ground alone with the residual stated. - `instruction-scan.test.sh` 46/46; markdownlint 0 errors; changelog parity/bump/order pass; `validate-plugins.sh` pass. ## Deliberately not shipped here - The consumer-state halves (the dated org vet, the chezmoi fleet-pin coupling) — drafted to the campaign's memory tier for a separate dotfiles PR. The verifier substantiated the draft's own caution: the referenced vet issue (#1697) closed **without an executed-vet record**, so the dotfiles sentence will be narrowed or re-cited to the real record before that PR opens. - Any fence built on the first-run hold's settings-file behavior — unstated upstream, recorded as unresolved. No linked issue ## Related - Owner decisions Q4/IA-12 (agnosticism routing test) and Q8 (ownership carve-out, ratified) — the SPLIT verdict tables this executes. - Sibling merges: #1881, #1882 (I17-I20 precedent for Detect-condition model ranges); standards#311 + ADR-0002 (the carve-out's normative source); dotfiles#394. - Follow-up noted by the verifier, non-blocking: whether I22 should also fire on trigger-without-baseline doctrine (current neither/nor reading is the deliberate narrow form for an off-by-default info check). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…, effort lane rules, Haiku delegation boundary (#1885) Ships DOC bundle 1 of the doc-corpus campaign — the zero-gate set: DOC-19, DOC-11, DOC-13, DOC-9. `playbooks` 0.6.2 -> 0.6.3 (covers the DOC-19 chapter change; the `docs/PLUGIN-PHILOSOPHY.md` rows follow the repo's convention of not versioning repo-level doctrine — precedent a09efd7, 970d0cb). ## What ships where, and why there - **DOC-19** -> `plugins/playbooks/skills/fable-5/context/calibration.md`, new section "A claim's product surface travels with it": claude.ai/mobile consumer-prompt claims never transfer to the harness without a per-claim check. Placed in calibration (claim grading), not trust-and-authority (instruction authority). One distilled line added to fable-5 `SKILL.md` core doctrine. - **DOC-9, DOC-11, DOC-13** -> `docs/PLUGIN-PHILOSOPHY.md` §Model tiers and §Effort tiers — NOT the fable-5 context tree, by that tree's own rules: fable-5 `SKILL.md` excludes model-version claims from its chapters ("live only under `context/model-adaptation/`"), those files are per-model-version audience-scoped, and §Effort tiers already owns effort doctrine (a second home would violate the one-owner rule). DOC-9 gives the tier table's `Haiku 4.5` row its missing capability boundary (no interleaved thinking — reasoning-about-tool-results delegation needs an interleaving model; mechanical fan-out does not, and consecutive tool calls still chain without it). DOC-13's adaptive-thinking mental model grounds what a lane pin buys; DOC-11 adds the three genuinely missing lane rules. ## Deviations from the row text, each evidence-forced - **DOC-19's two "worked contradictions" were both stale and are shipped dated**: the cross-chat-retention sentence last appears under the Opus 4.1 entry (Aug 5 2025, `system-prompts.md:1427`); the cannot-open-URLs sentence under Sonnet 3.5 (Nov 22 2024, `:2180`); zero later occurrences or reworded survivors (471,033-byte snapshot, swept twice independently). Shipping them present-tense would have been the exact defect class this repo's own catalog flags. Also reworded "contradictions" -> "divergences" — the card's own thesis is that a claude.ai claim and a Claude Code doc were never two claims about one surface. - **DOC-9's citation was split and corrected**: `model-config.md:677` carries `interleaved_thinking` as an admin-declared capability string; the per-model Haiku fact lives at platform `thinking.md:758`, with the `:755` narrowing ("Consecutive tool calls do not require interleaved thinking") stated in the row's favor. - **DOC-11 reduced to its three genuine deltas** — the raise direction and cache caveat already shipped here, and the stability convention already lives in the consumer's dotfiles. Two restraints written into the prose: the cache guidance states the outcome and refuses the mechanism (platform and harness give incompatible mechanisms for the identical outcome), and it says "natural breaks", never `/clear` (unsourced; the harness Tip says `/compact`). - **DOC-13's routing-note half refused**: it routes between campaign-internal `.work/` slice directories no consumer has. The mental model ships; the navigation stays in the campaign's own index. ## Consumer-state halves Drafted, not committed: `.work` memory-tier drafts for the dotfiles seam (three CLAUDE.md additions with verbatim before-texts verified against dotfiles origin/main, routes, and the never-`chezmoi apply` constraint). They land in a separate batched dotfiles PR. ## Verification Independently verified by a second model with the implementer's rationale withheld: all four placements' cited grounds confirmed at the bytes; both staleness re-datings replayed (7 and 9 occurrences, zero later); all three DOC-11 deltas and both restraints verified at both platform and harness sources (live pages byte-identical to the corpus snapshots where claimed); the DOC-13 refusal and the cite-swap in the triage row confirmed; the versioning question adjudicated against repo precedent. Sources re-fetched raw this session with recorded lengths. markdownlint 0 errors; `check-changed-skills.sh` PASS; changelog parity/bump/order pass. No linked issue ## Related - Phase 3b DOC chapters of the doc-corpus campaign; sibling merges #1881, #1882; standards#311 + ADR-0002; dotfiles#394. - Bundle sequencing per the campaign scoping: bundle 2 (DOC-6/8/10/12) follows with EC-1-derived rewrites; DOC-7/20/23/43 remain gated on campaign decisions. - Optional polish noted by the verifier, non-blocking: add the harness cite beside the platform cite on the "natural breaks" sentence; site-local recheck triggers in the fable-5 tree are a future sweep (file-local precedent is stamp-only). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…undary in the docs profile (#1888) Lands two adopted Sitting-2 decisions in the anthropic-docs publisher profile — the campaign's consumer-facing tag-selection doctrine. `knowledge` 0.10.16 -> 0.10.17. ## PA-M — the harness-surface definition New bullet appended to the near-miss rule: **a harness surface is a surface a user can reach**, with three non-falsifying sub-shapes that harness-doc text can take without establishing one. Both the definition sentence and sub-shape (3) carry explicit `[campaign-owned amendment]` labels — the record shows the campaign making the selection-over-support choice, not inheriting an adjudication. Sub-shape (3) rests on its single attested instance (`env-vars.md:394`, a retry/fallback row) and carries its own over-broadening boundary: a doc line describing some *other* model's tier fails the sub-shape's own test. The landing site was derived three independent ways: the profile already uses "harness surface" as its negative-claim term (`:34`, `:54`); the adopted row's own text says "write into the profile"; and the 0.10.16 CHANGELOG deliberately held this exact question (J-12) for the dispositions interview this answers. ## PA-V — the bare-name boundary The `cc-applicable`/`mixed` boundary bullet now carries the fourth API surface (**model ID**) and the negative half the profile never stated: **bare names are not API surfaces** — a product name, display name, or docs-path slug never by itself triggers `mixed`. Ratified from the de facto standard 15+ rows already stood on (cross-vendor retag applied in-slice). Deliberately excluded: a `[campaign-owned amendment]` label (this ratifies an evidence-carried standard, not a campaign-invented definition) and "feature names" (present only in one slice's wording, not the adopted row — widening the enumeration would exceed the adopted authority). ## Verification Both commits independently verified by a second model with the implementer's rationale withheld: the PA-M landing audited across landing-site derivation, amendment fidelity (verbatim vs the adopted blockquote), self-fire (no profile or checklist conflict; no live campaign artifact violates the definition), and mechanics; the PA-V text was *authored* by that verifier from the adopted row and applied verbatim, with the producer re-confirming the authority citations at the bytes before editing. The one enumeration of the API-surface list repo-wide is this bullet — no drift introduced. markdownlint 0 errors; CHANGELOG newest-first, both entries folded into the unreleased 0.10.17. Downstream, already discharged against these rules: the four slice-local `api-only` ratifications (memory-tier) now cite the landed definition and route their three disclosed near-miss hits per these bullets. No linked issue ## Related - Sitting 2 of the doc-corpus decision block, ADOPTED 2026-08-03 after adversarial validation (21/21 CONFIRMED); PA-M ordered first carrying its amendment flags, exactly as the adoption block specifies. - Siblings this session: #1881, #1882, #1884, #1885, #1887 (open); standards#311 + ADR-0002; dotfiles#394, #399 (open). - The J-12 hold this closes: `knowledge` 0.10.16's CHANGELOG (PR #1879). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…1, effort dial seam (#1887) Ships DOC bundle 2 of the doc-corpus campaign: DOC-6 and DOC-12 (thinking cost doctrine, rewritten against the EC-1 finding), DOC-10 (effort extend-or-cite deltas), DOC-8 (the two-dials seam). `playbooks` 0.6.3 -> 0.6.4. ## The EC-1 rewrite — the load-bearing change The thinking cost doctrine (fable-5 `context-economy.md` + one distilled `SKILL.md` line) states the harness override as **build-pinned behavior**: Claude Code sends `clear_thinking_20251015 keep:"all"` on every thinking-enabled request, on keep-all AND last-turn-only models alike — prior-turn thinking is retained and input-billed everywhere. The per-model keep-all/last-turn split is presented only as what a raw API caller gets, beside the override. The full four-part record ships: claim, basis (wire-captured request bodies, build pinned by byte size), as-of 2026-08-03, recheck trigger (any CC upgrade — the override is a build-time constant, not a documented contract), plus the three gating conditions and both escapes (`CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1`; a gateway dropping the field) under which per-model defaults resume. Observation and inference kept apart: the wire proves retention; input-billing is upstream's rule applied to it. ## The other rows - **DOC-10:** genuine deltas only — the harness corroboration + confirmation-dialog fact into §Effort tiers (A3/A4 already shipped); `opus-5.md`'s campaign-internal "Phase 6 cross-check" deferral replaced with live Effort/model-config pages that actually own per-model starting levels, preserving the TRUNCATED finding. - **DOC-8:** thinking and effort as two separate dials (`adaptive` is a thinking mode, never an effort value; effort is soft guidance, `max_tokens` the only strict limit) landed in the effort-doctrine owner where the frontmatter `effort` trap is reachable. Pins untouched — they move only through the dotfiles seam, per the row's own text; the consumer-state drafts are in the campaign's memory tier. - **Self-corrections shipped as follow-up commits (append-only):** one retracted drift claim (the corpus sentence survives verbatim at the live page — the original dispute was wrong) and one carry of the build pin onto the always-loaded surface plus removal of a session-residue sentence. ## Verification Independently verified cold by a second model (the implementer's report was lost to a session interruption; the audit ran from the diff and primary bytes alone). Every quoted fragment re-verified verbatim against raw fetches with byte-size pins matching the commits' own records; the build pin re-verified live (same byte count, so the as-of stamp holds); zero surviving unqualified keep-all claims; nothing trips catalog rows I17-I22/I19/I8-b; playbooks 0.6.4 + CHANGELOG parity/order/bump green; markdownlint 0 errors. **Ledger note (immutable commit message):** `00dc30749d` says "DOC-12 supplies the three buckets"; the shipped section carries two of the pricing page's three — ordinary output-token billing is the pointer's job, not restated. Recorded here per the campaign's history-stays-immutable practice. No linked issue ## Related - Phase 3b DOC chapters; bundle 1 merged as #1885; siblings #1881, #1882, #1884; standards#311 + ADR-0002; dotfiles#394, #399 (open — bundle 2's dotfiles Draft 1 now carries an explicit apply-on-top-of-#399 dependency note). - EC-1 empirical record: `EC-1-PROCEDURE-REPAIR-2026-08-02.md` + interview-checklist ratification (memory tier). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 4, 2026
…tion audit with the live prompting guide (#1908) ## Summary Doc-alignment roster row 1: align repo doctrine and the instruction audit with the live **Prompting Claude Fable 5** page (platform.claude.com). The live page was re-fetched and is byte-identical to the repo's 2026-07-29 capture, so this ships what the page implies and the repo lacked — nothing in the repo misstated it. **playbooks 0.6.9** (fable-5 doctrine): - `context-economy.md` — the late-session decay ladder gains a fourth bullet naming a remaining-context number as a NON-signal (the page's "Rare cases of context-budget concern"). Scoped to the model's own initiative; an instructed stop outranks it under meta-rule 1, so sibling plugins that deliberately gate on the window keep working. - `communication.md` — new assessment-versus-change gate ("Assessment is a deliverable; a fix is a different one"): report-and-stop on problem/question framings, evidence bar before state-changing actions. Repairs `opus-4-8.md`'s pre-existing pointer to a section that did not exist. - `orchestration.md` — non-blocking dispatch: keep working while independent workers run; continue an oriented worker rather than respawn. - `execution.md` — bound on defensive over-building: boundary-only validation, no cleanup around a bug fix, no abstraction before the second real caller, no shim where changing the code is available. - `SKILL.md` — core-doctrine lines for all four (they fire before their chapters' load triggers plausibly would). **claude-config 0.21.1** (criteria 1.9.0, IA-2 model-delta cluster): - New row **I8-d** (short-turn assumptions, Model scope fable-5) with three fences keeping it off output-length instructions, surface-owned latency requirements, and documents about the pattern. - I8 base row gains the delegation throttle as a named worked instance, fenced for caps with non-model rationale. - SKILL.md discloses both as lane-only (not scanner-seeded). Deliberately NOT shipped, with verified reasoning: the IA-2(d) context-budget audit row — it fires on nine deliberate in-repo context-gating surfaces (context-guard exists on purpose), and its config half (`totalTokensReminder` / `CLAUDE_CODE_TOTAL_TOKENS_REMINDER`) appears on no official settings/env-var/model-config/context-window page. The finding shipped as the context-economy doctrine bullet instead. ## Test plan - Docs-only (skill prose, criteria, changelogs, version bumps). - Producer verification: both audit-instructions test suites pass (46 + 41 checks), skill-quality gate zero errors with warning counts identical to base, markdownlint clean, zero new I6 bare-prohibition candidates, all cross-references resolve. - Independent fresh-context Fable verifier, rationale withheld, 7 binary criteria — including its own live-page fetch and byte-diff, verbatim-quote checks on criteria sources, independent reproduction of every producer verification claim, both IA-2(d) non-ship legs verified (in-repo deliberate instances enumerated; env vars absent from live official docs), eval-convention check (does not extend to these surfaces — verified against the last five doctrine commits and #1881/#1882/#1884), and merge-readiness vs current main — **7/7 PASS, empty defect list**. ## Related - No linked issue. - Doc-alignment per-document loop, roster row 1 (memory-tier roster; rows 2+ follow). Companion context: #1899, #1906 (drift root-cause fixes that preceded the loop). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 8, 2026
…lity, and the re-verification stamp (#1997) No linked issue ## Summary Full conformance pass of this repository against the Opus 5 prompting guide, in two rounds. Round 1: verbatim guide read (byte-identical to the corpus pin, MD5 `8579d63fc9f793784b8c56320fd74e71`), directive inventory D1–D20, provenance check of the prior `opus-5-prompting-interview` operationalization (#1699 + follow-ups), grep-level sweep — repo clean, stamp refreshed. Round 2 (deep dive): a 42-agent workflow — 9 per-section semantic sweep agents over every instruction surface, 3 capability auditors (criteria coverage, chapter fidelity, consumer path), and 30 fresh-context adversarial verifiers briefed to refute. 33 raw candidates → 30 deduped → **6 confirmed, 24 refuted, 0 unverified**. All 6 fixed here, plus the fidelity and precision defects the capability auditors confirmed. ## Fix - **playbooks 0.6.22** — `opus-5.md`: four paraphrases restored to the guide's own strength (hedged "may" restored with the withholding mechanism re-attributed to the Sonnet 5 guide that states it; "often" restored; added "only" removed; scope fence completed with the routine-judgment-calls, request-seems-mistaken, and finish-whole-task clauses); new Vision section carrying the guide's re-validate-workarounds directive + tools-before-thinking lever; tool-heavy locality clause on the tool-call-leak artifact. `boris`: Tip 64's context-rot figure (300–400k on 1M) amended against the Opus 5 guide's consistency-throughout-the-window statement, same dated-blockquote shape as §72; SKILL.md routing row era-scoped. Plus the 0.6.21 re-verification stamp advance (2026-08-08, byte-identical). - **context-guard 0.4.8** — injected zone-crossing guidance no longer asserts degradation as a universal fact: degradation claim model-conditioned, bands named as tunable defaults, compaction-distance rationale kept unconditional. - **planning 0.27.4** — `prd` user-stories template drops "err on completeness over brevity" / "aim for exhaustive coverage" anti-brevity dials; keeps full coverage intent, adds the guide-calibrated no-padding bound. - **discovery 0.10.1** — `explore` sidecar bodies gain length calibration (the one uncalibrated surface in an otherwise tightly calibrated artifact set). - **education 0.5.5** — `quiz-me` report narrative sections gain length calibration. - **claude-config 0.21.10** (criteria 1.16.1) — scan-script header comments match I8-b's unscoped promotion; I8-a's truncated guide quote completed ("…for any non-trivial task"); I8-b's "same three trigger phrases" annotation reconciled with its own Source; SKILL.md documents the normalized version-token grammar (`opus-5` shape) the catalog matches against. - **prompts/loops/loop-lane-prompts.md** — volatile effort-default claim stamped with source, as-of date, and re-resolve trigger (verified live 2026-08-08 against model-config). ## Verification - Workflow: 42 agents, 0 errors; every confirmed finding verified by a fresh-context adversarial verifier with explicit refutation grounds (audience test, staleness-vs-self-check, independence classification, carve-out lanes, non-model rationale, quote accuracy, model-agnosticism); 24 candidates refuted on those grounds. - Fidelity claims verified against both texts verbatim before editing (guide "may follow that instruction literally" vs chapter "makes you report less"; "often longer" vs "run longer"; "step up to" vs "step up only"; full scope-fence diff). - Effort-default claim verified against the live model-config page (fetched 2026-08-08): "The default effort is `high` on every model that supports effort, except Opus 4.7, which defaults to `xhigh`." - Gates: `markdownlint-cli2` — 0 issues across all 15 changed .md; `scripts/check-changed-skills.sh origin/main` — 6 skills checked, 0 failed; `instruction-scan.test.sh` — 46/46; `bash -n` clean on both edited shell scripts. - Guide freshness: today's raw-`.md` fetch byte-identical to the 2026-07-25 corpus capture — no criteria Sources trigger fires; all prior vettings stand. ## Related - Refs #1996 (deferred audit-row + consumer-path design candidates from the capability auditors, recorded in a follow-up comment) - Refs #1699 (original operationalization), #1910, #1913, #1881, #1884 - ADR-0006 / ADR-0007 (model-scoped doctrine + seam address) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01E5qiaLiGqP6fUTHkRYS19R --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes instruction-audit catalog row I8-b (conservative-reporting detection) to unscoped and gives
plugins/review/context/severity.mdenumerable tier criteria.claude-config0.18.0 -> 0.19.0;review0.15.5 -> 0.16.0;criteria.md1.5.0 -> 1.6.0.The promotion
I8-b's gate is MET on its second arm (multiple model guides converge): the Sonnet 5 prompting guide states all three trigger phrases verbatim in one sentence (
source.md:140, "Code review harnesses"), converging with the Opus 5 guide. Annotated in row I7's met-gate precedent form; the Sonnet 5 guide URL added to## Sourcesper the trigger-set-is-the-source-set invariant.Source attribution corrected while citing: "don't nitpick" appears nowhere in the Opus 5 guide (
grep -cin nitpick-> 0); that guide states only the other two phrases (its line 20). The Sonnet 5 guide is the phrase's only cited home. This strengthens the convergence gate — three phrases now each attributed to a page that actually contains them.#1880's "Held back deliberately" premise was wrong
That PR deferred this promotion because unscoping I8-b would allegedly make it fire on
severity.md. It does not, on three independently verified grounds:severity.mdemits one I6 row and no I8-b; the I8-b ERE greps 0 on both trees.criteria.md:252-255) excludes "severity-based routing where everything is still reported somewhere" — severity.md classifies findings and withholds none.plugins/review/context/is none of those (same result feat(claude-config): corroborate I10 and concretize its remediation surfaces #1880 recorded forcriteria.mditself).The promotion could have shipped alone. The severity.md work ships here anyway, re-founded on its own source: nine lines below the three-phrase line, the same Sonnet 5 guide says to "be concrete about where the bar is rather than using qualitative terms like
important" (source.md:150) — andimportantis one of severity.md's own tier names. That is triage row RA-2, a distinct claim from the one I8-b cites.The severity rewrite
Each tier now carries a decidable test instead of a qualitative label; no finding changes tier. Guards added where the criterion-stating change could have silently re-tiered:
Verification
Independently verified by a second model with the implementer's rationale withheld, across two rounds. Scanner counts replayed from git refs both rounds: origin/main 23 rows / 6 files, all fenced; working tree 28 / 7, every addition this branch's own quoting. Planted-positive check: a constructed three-phrase file emitted I8-b on all three lines in the same invocation where severity.md emitted zero. Scanner-vs-git-grep equivalence proven on identical row sets.
instruction-scan.test.sh46/46; markdownlint 0 errors.Known citation defect in an immutable commit message:
3603c8a9b8says "eight lines later (source.md:149)". Both figures are wrong — the correct citation issource.md:150, ten lines after the three-phrase line atsource.md:140. No tracked file carries the wrong number; recorded here rather than rewriting pushed history.No linked issue
Related