feat(claude-config): corroborate I10 and concretize its remediation surfaces - #1880
Merged
Merged
Conversation
…n surfaces The RA-9 extend-or-cite pass adjudicated whether the Thinking page promotes I10 (reasoning-echo directives) out of `Model scope: fable-5`. It does not, and the row now carries the reason so the question is not re-litigated: the page's own section names both Claude Fable 5 and Claude Mythos 5 for the adjacent raw-chain-of-thought property (source :876), then names Fable 5 alone for the `reasoning_extraction` refusal (source :887). A sentence-adjacent chance to widen, declined — the narrower scope is deliberate, so the promotion gate stays unmet and `mythos-5` is deliberately not added. What the page does give I10 is a second, independent citation for the same Fable-5-scoped claim, from a feature page rather than a model guide, and the concrete surfaces its Remediate line only gestured at. One correction to the pass's own framing: the Thinking page does NOT name the Claude Code surfaces. `Ctrl+O` verbose mode and `showThinkingSummaries: true` are stated on the model configuration page (<https://code.claude.com/docs/en/model-config>); only `display: "summarized"` comes from Thinking. Both pages are therefore added to the catalog's `## Sources` list, because the Recheck-triggers block makes the trigger set the source set and a cited page nothing watches would leave the row depending on an unwatched source. Verified: live raw-markdown fetches of both pages (thinking.md 52769 bytes, byte-identical to the 2026-07-30 snapshot; model-config.md 83644 bytes), both quoted strings matched against source bytes, and `instruction-scan.test.sh` green at 46/46 — detection is untouched, only Source and Remediate change. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 50s —— View job Claude Code review complete
This is a docs-only change to Findings: none blocking. Everything I could verify checks out:
One thing I could not verify: the PR's "Verification" section claims a second-model pass re-fetched both source pages raw and confirmed the quoted strings byte-for-byte. I don't have network access to independently re-fetch I also couldn't run |
|
Claude finished @kyle-sexton's task in 1m 2s —— View job Security Review — Complete
No security issues found. This PR is a pure documentation/reference-data change to the
Specific checks performed (all clean):
Nothing in scope for this security-focused lane (no injection surfaces, no auth/access-control logic, no secret handling, no CI/workflow changes). |
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…y.md enumerable tier criteria (#1881) Promotes instruction-audit catalog row **I8-b** (conservative-reporting detection) to unscoped and gives `plugins/review/context/severity.md` enumerable tier criteria. `claude-config` 0.18.0 -> 0.19.0; `review` 0.15.5 -> 0.16.0; `criteria.md` 1.5.0 -> 1.6.0. ## The promotion I8-b's gate is MET on its second arm (multiple model guides converge): the Sonnet 5 prompting guide states all three trigger phrases verbatim in one sentence (`source.md:140`, "Code review harnesses"), converging with the Opus 5 guide. Annotated in row I7's met-gate precedent form; the Sonnet 5 guide URL added to `## Sources` per the trigger-set-is-the-source-set invariant. **Source attribution corrected while citing:** "don't nitpick" appears nowhere in the Opus 5 guide (`grep -cin nitpick` -> 0); that guide states only the other two phrases (its line 20). The Sonnet 5 guide is the phrase's only cited home. This strengthens the convergence gate — three phrases now each attributed to a page that actually contains them. ## #1880's "Held back deliberately" premise was wrong That PR deferred this promotion because unscoping I8-b would allegedly make it fire on `severity.md`. It does not, on three independently verified grounds: 1. **Zero scanner candidates** — the real scanner over `severity.md` emits one I6 row and no I8-b; the I8-b ERE greps 0 on both trees. 2. **I8-b's own carve-out** (`criteria.md:252-255`) excludes "severity-based routing where everything is still reported somewhere" — severity.md classifies findings and withholds none. 3. **Outside the audited population** — SKILL.md Phase A inventories CLAUDE.md / rules/ / skills/ / agents/ / output-styles/ under user and project roots plus hook text; `plugins/review/context/` is none of those (same result #1880 recorded for `criteria.md` itself). The promotion could have shipped alone. The severity.md work ships here anyway, **re-founded on its own source**: nine lines below the three-phrase line, the same Sonnet 5 guide says to "be concrete about where the bar is rather than using qualitative terms like `important`" (`source.md:150`) — and `important` is one of severity.md's own tier names. That is triage row RA-2, a distinct claim from the one I8-b cites. ## The severity rewrite Each tier now carries a decidable test instead of a qualitative label; no finding changes tier. Guards added where the criterion-stating change could have silently re-tiered: - The P1-P5 fold explicitly takes precedence for P-scored findings (otherwise every P3 security finding would have read into the new CRITICAL test). - CRITICAL's subsequent-change limb reads "otherwise-correct change", so cascade architecture violations (break a *correct* future change) stay CRITICAL while code duplication (bites only through an *incomplete* future edit) stays IMPORTANT. All eleven tier examples adjudicated against the new tests — twice, independently. ## Verification Independently verified by a second model with the implementer's rationale withheld, across two rounds. Scanner counts replayed from git refs both rounds: origin/main 23 rows / 6 files, all fenced; working tree 28 / 7, every addition this branch's own quoting. Planted-positive check: a constructed three-phrase file emitted I8-b on all three lines in the same invocation where severity.md emitted zero. Scanner-vs-git-grep equivalence proven on identical row sets. `instruction-scan.test.sh` 46/46; markdownlint 0 errors. **Known citation defect in an immutable commit message:** `3603c8a9b8` says "eight lines later (`source.md:149`)". Both figures are wrong — the correct citation is `source.md:150`, ten lines after the three-phrase line at `source.md:140`. No tracked file carries the wrong number; recorded here rather than rewriting pushed history. No linked issue ## Related - Phase 3b of the doc-corpus campaign; Phase 3a merged in #1875, #1876, #1877, #1878, #1879, #1880. - Reverses #1880's "Held back deliberately" reasoning with evidence (above). - Triage rows: RA-2 (severity criteria), RA-9/Q6 (I8-b promotion), owner decision Q9 (bundling). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies the evidence-forced half of the RA-9 extend-or-cite pass to instruction-audit catalog row I10 (reasoning-echo).
claude-config0.17.0 -> 0.18.0;criteria.md1.4.0 -> 1.5.0.What this does NOT do, and why that is the point
I10 keeps
Model scope: fable-5. It is not promoted, andmythos-5is not added.The campaign triage ranked this promotion its #1 item, on the theory that the extended-thinking platform page is a second, model-independent source that would satisfy I10's promotion gate. It is not, and the page's own structure is what settles it:
:874— the H2 reads "Thinking output on Claude Fable 5 and Claude Mythos 5":876— names both models for the adjacent raw-chain-of-thought property:887— names only Claude Fable 5 forstop_details.category: "reasoning_extraction"Eleven lines apart, in one section. The page had a sentence-adjacent opportunity to widen the refusal and declined it. That is deliberate scoping, not loose phrasing. Adding
mythos-5would fabricate scope from a claim about a different property — exactly what the catalog's own model-scoping block warns against.That reasoning is now recorded in the row itself, so it is not re-litigated a fourth time.
What it does
thinkingblocks or use a send-to-user tool. It now names the actual surfaces:Ctrl+Overbose mode andshowThinkingSummaries: true, anddisplay: "summarized".Each surface is attributed to the page that actually states it. The pass this work came from asserted all three were on the thinking page; two of them are not —
Ctrl+OandshowThinkingSummariesare stated atmodel-config:532, and onlydisplay: "summarized"is on the thinking page. Both pages are therefore added to## Sources, becausecriteria.mdcarries its own invariant that "the trigger set is the source set — naming a subset would leave the harness-behavior rows depending on pages nothing watches."Verification
Independently verified by a second model, with the implementer's rationale withheld. Both pages re-fetched raw (
model-config.md83,644 bytes;thinking.md52,769 bytes, byte-identical to the frozen snapshot), all three surface names confirmed verbatim, and the promotion-gate facts re-confirmed at both snapshot and live bytes.Self-fire check. Because a catalog row that fires on this repo would break the campaign's own rule against shipping a consumer check we fail: the scanner was run against both trees. I10 candidates in
criteria.mdgo 2 -> 4, and all four are inert —criteria.mdis a skill reference file, outside the audited population (CLAUDE.md/rules//skills//agents//output-styles/under the user and project roots). No new class of self-hit is introduced.Detection is untouched:
instruction-scan.test.shreports 46/46, and this edit changes Source and Remediate only.Held back deliberately
The I8-b promotion, whose gate genuinely IS met by verbatim two-guide convergence, is not here. Promoting it makes the row fire on this repo's own
plugins/review/context/severity.md— the same work as triage row RA-2, an open owner decision. Shipping the promotion first would make the next audit run flag this repository.No linked issue
Related
RA9-EXTEND-OR-CITE-PASS-2026-08-02.md(23 rows adjudicated; the triage had checked 6).