chore(knowledge): land the near-miss disclosure rule and the blog-channel extraction artifacts - #1879
Merged
kyle-sexton merged 4 commits intoAug 2, 2026
Conversation
…mendments Second and final batch from the eleven-run digest campaign. Every rule states its evidence inline, and every figure in it was re-verified against the slice bytes in this session rather than carried over from the triage. Anthropic profile: - What falsifies `api-only`, as one graded rule rather than a boundary re-derived per row. Only the corpus documenting the claim's own specific assertion falsifies the tag. A harness page covering the subject without stating the rule is a near-miss — tag survives, row must name it by page and line. Weaker still, and not even a near-miss: a counterpart artifact on the other property, and a workload named only as an example beside a guide that teaches it. - A positive tag's row answers two questions as two bullets: is the assertion documented, and does the harness have the surface the guidance operates on. - Vendor-voice attestation follows the voice, not the containing page, so a blog passage embedded in a system prompt carries the marker too. - Absence checks select their pages from the publisher's own `llms.txt` index rather than a guessed slug, and any non-zero hit is read at its match site — a corpus-snapshot page can be rendered HTML, making its hits markup. - The two reproducible `claude.com/blog` extraction artifacts are recorded with reconstruction guidance; both reproduced exactly across two blog runs, which is the condition the earlier deferral named. SKILL.md Phase 2: site-injected page chrome stays in the immutable snapshot and is flagged non-prose in INDEX.md, so the Phase 3 agent does not digest it as content. Three further evidence-forced items are deliberately not applied: they change instruments that live in the campaign's untracked work root, not in the shipped plugin, and whether any graduates into the skill is a scope decision. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s already shipped Three corrections to the batch, all found by independent review of the file rather than of the diff. - The index-hit rule said "never counted" flat, which contradicted the sampling clause the applicability filter already carries and whose own evidence is a 199-line hit set. It now subordinates to that clause instead of restating the boundary, and its scope narrows to what the evidence reaches: the index itself and snapshot pages that turn out to be rendered HTML, not any corpus snapshot. - The blog title reconstruction leaned on the canonical URL slug without saying what a slug loses. It recovers word boundaries only — the models-explained slug cannot yield the colon in its own title — so a title recovered that way is labelled reconstructed. - The `api-only` bullet's own gloss still read "no harness surface exists" unqualified, which is the broad reading the graded rule below it exists to correct. It now points at that rule. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The batch was built from the triage's `evidence-forced` column without cross-checking
`PHASE2-JUDGMENT-AMENDMENTS-2026-08-01.md`, which writes five of these same items up as
judgment calls held for the dispositions interview, under a header stating "None applied."
The judgment file governs, on its own reasoning: J-14 says it is held "only because no
defect was demonstrated, so the bar for the applied subset is not met", and J-7 says it
should be decided AFTER the Decision-A ordering question the maintainer has already
reserved — shipping it pre-empts that decision.
Withdrawn from the shipped text:
- PA-I / J-6 — attestation following the vendor voice into a non-blog page.
- PA-Q / J-7 — splitting the two questions a positive tag's row collapses.
- PA-N / J-8 — naming the publisher's `llms.txt` index as the page-selection instrument,
and the previous commit's repair of its non-zero-hit clause, which goes with it.
- PA-M / J-12 — counterpart artifacts and same-workload mentions as weaker than a
near-miss.
- PA-AD / J-14 — the SKILL.md Phase 2 convention for site-injected matter, which returns
that file to its 0.10.15 text.
Still shipped: PA-Y (the near-miss rule) and PA-Z (the blog-channel extraction artifacts).
PA-AG was verify-and-close and never edited anything.
PA-Y and PA-M shipped as one graded clause with two grades below the falsifying line.
Separating them leaves PA-Y whole rather than half a rule: Ruling 1 Decision A states the
falsification standard ("the claim's own specific assertion, not merely its general topic")
and the near-miss grade in one paragraph, and that ruling is PA-Y's own cited source. Only
the second sub-bullet was PA-M's. The clause is reworded from two grades to one.
For the same reason the previous commit's `api-only` gloss repair stands — it points at the
falsification standard, which is PA-Y's — with its pointer reworded from the graded rule to
the near-miss rule.
Version stays 0.10.16; the CHANGELOG entry now describes only what ships, and separates the
two reasons items are held: three that change untracked campaign instruments, and these five
that the judgment file holds for the interview.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Independent review of the split entry, not of the diff. The lead still read "the second and final batch of the campaign's evidence-forced amendments" — inherited from before the split and false after it. Five items the triage classifies `evidence-forced` are now held for the dispositions interview, which the entry's own closing paragraph says two paragraphs later. The lead and the close contradicted each other. It now claims only "a second batch", points at the two held classes, and states the standard as a defect the runs produced rather than as a count of what remains. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 38s —— View job Review: PR #1879 —
|
|
Claude finished @kyle-sexton's task in 1m 15s —— View job Security review complete
No security issues found. This PR's net diff (commits
None of these introduce or modify:
The added markdown is guidance text authored by the maintainer for an LLM-driven digest pipeline (how to reconstruct a blog title from a URL slug, how to disclose near-miss citations) — it doesn't change what gets fetched, executed, or trusted, so there's no new injection, authorization, or supply-chain surface here. Per instructions, I also did not re-flag anything in zizmor's static-analysis lane (unpinned actions, dangerous triggers, excessive permissions) — moot here anyway since no workflow files are in scope. |
|
Claude has reviewed this PR 1 time. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
kyle-sexton
deleted the
chore/docpage-digest-remaining-evidence-forced-amendments
branch
August 2, 2026 22:28
kyle-sexton
added a commit
that referenced
this pull request
Aug 2, 2026
…urfaces (#1880) Applies the evidence-forced half of the RA-9 extend-or-cite pass to instruction-audit catalog row **I10** (reasoning-echo). `claude-config` 0.17.0 -> 0.18.0; `criteria.md` 1.4.0 -> 1.5.0. ## What this does NOT do, and why that is the point **I10 keeps `Model scope: fable-5`. It is not promoted, and `mythos-5` is not added.** The campaign triage ranked this promotion its **#1 item**, on the theory that the extended-thinking platform page is a second, model-independent source that would satisfy I10's promotion gate. It is not, and the page's own structure is what settles it: - `:874` — the H2 reads "Thinking output on Claude Fable 5 **and Claude Mythos 5**" - `:876` — names **both** models for the adjacent raw-chain-of-thought property - `:887` — names **only** Claude Fable 5 for `stop_details.category: "reasoning_extraction"` Eleven lines apart, in one section. The page had a sentence-adjacent opportunity to widen the refusal and declined it. That is deliberate scoping, not loose phrasing. Adding `mythos-5` would fabricate scope from a claim about a different property — exactly what the catalog's own model-scoping block warns against. That reasoning is now recorded **in the row itself**, so it is not re-litigated a fourth time. ## What it does - **A corroborating second Source** — the extended-thinking page, independent of the Fable 5 model guide already cited. - **A concretized Remediate line.** It previously said, abstractly, to read structured `thinking` blocks or use a send-to-user tool. It now names the actual surfaces: `Ctrl+O` verbose mode and `showThinkingSummaries: true`, and `display: "summarized"`. **Each surface is attributed to the page that actually states it.** The pass this work came from asserted all three were on the thinking page; two of them are not — `Ctrl+O` and `showThinkingSummaries` are stated at `model-config:532`, and only `display: "summarized"` is on the thinking page. Both pages are therefore added to `## Sources`, because `criteria.md` carries its own invariant that "the trigger set is the source set — naming a subset would leave the harness-behavior rows depending on pages nothing watches." ## Verification Independently verified by a second model, with the implementer's rationale withheld. Both pages re-fetched raw (`model-config.md` 83,644 bytes; `thinking.md` 52,769 bytes, byte-identical to the frozen snapshot), all three surface names confirmed verbatim, and the promotion-gate facts re-confirmed at both snapshot and live bytes. **Self-fire check.** Because a catalog row that fires on this repo would break the campaign's own rule against shipping a consumer check we fail: the scanner was run against both trees. I10 candidates in `criteria.md` go 2 -> 4, and all four are inert — `criteria.md` is a skill reference file, outside the audited population (`CLAUDE.md` / `rules/` / `skills/` / `agents/` / `output-styles/` under the user and project roots). No new class of self-hit is introduced. Detection is untouched: `instruction-scan.test.sh` reports 46/46, and this edit changes Source and Remediate only. ## Held back deliberately The **I8-b promotion**, whose gate genuinely IS met by verbatim two-guide convergence, is not here. Promoting it makes the row fire on this repo's own `plugins/review/context/severity.md` — the same work as triage row RA-2, an open owner decision. Shipping the promotion first would make the next audit run flag this repository. No linked issue ## Related - Phase 3b of the doc-corpus campaign; Phase 3a self-alignment merged in #1875, #1876, #1877, #1878, #1879. - Source pass: `RA9-EXTEND-OR-CITE-PASS-2026-08-02.md` (23 rows adjudicated; the triage had checked 6). Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…y.md enumerable tier criteria (#1881) Promotes instruction-audit catalog row **I8-b** (conservative-reporting detection) to unscoped and gives `plugins/review/context/severity.md` enumerable tier criteria. `claude-config` 0.18.0 -> 0.19.0; `review` 0.15.5 -> 0.16.0; `criteria.md` 1.5.0 -> 1.6.0. ## The promotion I8-b's gate is MET on its second arm (multiple model guides converge): the Sonnet 5 prompting guide states all three trigger phrases verbatim in one sentence (`source.md:140`, "Code review harnesses"), converging with the Opus 5 guide. Annotated in row I7's met-gate precedent form; the Sonnet 5 guide URL added to `## Sources` per the trigger-set-is-the-source-set invariant. **Source attribution corrected while citing:** "don't nitpick" appears nowhere in the Opus 5 guide (`grep -cin nitpick` -> 0); that guide states only the other two phrases (its line 20). The Sonnet 5 guide is the phrase's only cited home. This strengthens the convergence gate — three phrases now each attributed to a page that actually contains them. ## #1880's "Held back deliberately" premise was wrong That PR deferred this promotion because unscoping I8-b would allegedly make it fire on `severity.md`. It does not, on three independently verified grounds: 1. **Zero scanner candidates** — the real scanner over `severity.md` emits one I6 row and no I8-b; the I8-b ERE greps 0 on both trees. 2. **I8-b's own carve-out** (`criteria.md:252-255`) excludes "severity-based routing where everything is still reported somewhere" — severity.md classifies findings and withholds none. 3. **Outside the audited population** — SKILL.md Phase A inventories CLAUDE.md / rules/ / skills/ / agents/ / output-styles/ under user and project roots plus hook text; `plugins/review/context/` is none of those (same result #1880 recorded for `criteria.md` itself). The promotion could have shipped alone. The severity.md work ships here anyway, **re-founded on its own source**: nine lines below the three-phrase line, the same Sonnet 5 guide says to "be concrete about where the bar is rather than using qualitative terms like `important`" (`source.md:150`) — and `important` is one of severity.md's own tier names. That is triage row RA-2, a distinct claim from the one I8-b cites. ## The severity rewrite Each tier now carries a decidable test instead of a qualitative label; no finding changes tier. Guards added where the criterion-stating change could have silently re-tiered: - The P1-P5 fold explicitly takes precedence for P-scored findings (otherwise every P3 security finding would have read into the new CRITICAL test). - CRITICAL's subsequent-change limb reads "otherwise-correct change", so cascade architecture violations (break a *correct* future change) stay CRITICAL while code duplication (bites only through an *incomplete* future edit) stays IMPORTANT. All eleven tier examples adjudicated against the new tests — twice, independently. ## Verification Independently verified by a second model with the implementer's rationale withheld, across two rounds. Scanner counts replayed from git refs both rounds: origin/main 23 rows / 6 files, all fenced; working tree 28 / 7, every addition this branch's own quoting. Planted-positive check: a constructed three-phrase file emitted I8-b on all three lines in the same invocation where severity.md emitted zero. Scanner-vs-git-grep equivalence proven on identical row sets. `instruction-scan.test.sh` 46/46; markdownlint 0 errors. **Known citation defect in an immutable commit message:** `3603c8a9b8` says "eight lines later (`source.md:149`)". Both figures are wrong — the correct citation is `source.md:150`, ten lines after the three-phrase line at `source.md:140`. No tracked file carries the wrong number; recorded here rather than rewriting pushed history. No linked issue ## Related - Phase 3b of the doc-corpus campaign; Phase 3a merged in #1875, #1876, #1877, #1878, #1879, #1880. - Reverses #1880's "Held back deliberately" reasoning with evidence (above). - Triage rows: RA-2 (severity criteria), RA-9/Q6 (I8-b promotion), owner decision Q9 (bundling). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…undary in the docs profile (#1888) Lands two adopted Sitting-2 decisions in the anthropic-docs publisher profile — the campaign's consumer-facing tag-selection doctrine. `knowledge` 0.10.16 -> 0.10.17. ## PA-M — the harness-surface definition New bullet appended to the near-miss rule: **a harness surface is a surface a user can reach**, with three non-falsifying sub-shapes that harness-doc text can take without establishing one. Both the definition sentence and sub-shape (3) carry explicit `[campaign-owned amendment]` labels — the record shows the campaign making the selection-over-support choice, not inheriting an adjudication. Sub-shape (3) rests on its single attested instance (`env-vars.md:394`, a retry/fallback row) and carries its own over-broadening boundary: a doc line describing some *other* model's tier fails the sub-shape's own test. The landing site was derived three independent ways: the profile already uses "harness surface" as its negative-claim term (`:34`, `:54`); the adopted row's own text says "write into the profile"; and the 0.10.16 CHANGELOG deliberately held this exact question (J-12) for the dispositions interview this answers. ## PA-V — the bare-name boundary The `cc-applicable`/`mixed` boundary bullet now carries the fourth API surface (**model ID**) and the negative half the profile never stated: **bare names are not API surfaces** — a product name, display name, or docs-path slug never by itself triggers `mixed`. Ratified from the de facto standard 15+ rows already stood on (cross-vendor retag applied in-slice). Deliberately excluded: a `[campaign-owned amendment]` label (this ratifies an evidence-carried standard, not a campaign-invented definition) and "feature names" (present only in one slice's wording, not the adopted row — widening the enumeration would exceed the adopted authority). ## Verification Both commits independently verified by a second model with the implementer's rationale withheld: the PA-M landing audited across landing-site derivation, amendment fidelity (verbatim vs the adopted blockquote), self-fire (no profile or checklist conflict; no live campaign artifact violates the definition), and mechanics; the PA-V text was *authored* by that verifier from the adopted row and applied verbatim, with the producer re-confirming the authority citations at the bytes before editing. The one enumeration of the API-surface list repo-wide is this bullet — no drift introduced. markdownlint 0 errors; CHANGELOG newest-first, both entries folded into the unreleased 0.10.17. Downstream, already discharged against these rules: the four slice-local `api-only` ratifications (memory-tier) now cite the landed definition and route their three disclosed near-miss hits per these bullets. No linked issue ## Related - Sitting 2 of the doc-corpus decision block, ADOPTED 2026-08-03 after adversarial validation (21/21 CONFIRMED); PA-M ordered first carrying its amendment flags, exactly as the adoption block specifies. - Siblings this session: #1881, #1882, #1884, #1885, #1887 (open); standards#311 + ADR-0002; dotfiles#394, #399 (open). - The J-12 hold this closes: `knowledge` 0.10.16's CHANGELOG (PR #1879). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second batch of evidence-forced docpage-digest amendments.
knowledge0.10.15 -> 0.10.16. Two rules ship; six items are deliberately held, each with its reason stated.What ships
claude.com/blogrun reproduced both exactly, re-verified byte-exact in both slices. The guidance leads with the canonical URL slug and states what a slug loses — word boundaries recover, punctuation and casing do not — so a slug-recovered title is labelled reconstructed.cc-applicable/mixedboundary rule already in the profile matches what its source asked for.What is held, and why — two different reasons
Five held because the campaign's judgment file holds them. PA-I (J-6), PA-Q (J-7), PA-N (J-8), PA-M (J-12) and PA-AD (J-14) were shipped in the first draft of this branch and then withdrawn. The triage marks them
evidence-forced;PHASE2-JUDGMENT-AMENDMENTSwrites them up as judgment calls under a header stating "None applied." Two records disagreed, and the judgment file governs on its own reasoning:They are now full rows in the owner-decision artifact, each restating the triage-vs-judgment conflict rather than resolving it silently. Between the two readings they would otherwise have landed in neither place.
Three held because they change untracked instruments. PA-B's second half, PA-P and PA-AK all modify scripts under the memory-tier
.work/root, which is never committed and is not part of the shipped plugin.Verification
Independently verified twice. The first round FAILED on the five judgment-held rules. The second confirmed the split is clean:
SKILL.mdis byte-identical to its 0.10.15 state, grep finds no residue of the withdrawn rules, and PA-Y reads as a whole rule rather than half of the graded pair it was originally merged into.The implementer also ran a second pass over the whole file rather than the diff, and caught three defects in its own first commit — including a new rule that contradicted one shipped in #1875, which is exactly the two-rules-for-one-boundary drift this pipeline exists to prevent.
No linked issue
Related
knowledge0.10.15), which shipped the first evidence-forced batch.