docs(topics): absorb the context-engineering source into a designed, dispatched instruction-surface pass - #1322
Merged
Conversation
Contract directory for the context-engineering pass over this repository's plugin surface, derived from Anthropic's context-engineering guidance and audited against the 60 plugins already in the marketplace. Two prior sessions of planning and one adversarial pass existed only in the working tree; this commit makes them durable. ## Contents - `PLAN.md` — the contract being executed: 13 phases, all `[TODO]`, plus acceptance criteria split into mechanical and behavioral tiers, a risks-and-mitigations table, and an execution-shape table naming which phases run in the main session versus dispatched sub-agents. - `design/article-sections.md` — the source decomposed into the units (§S1–§S15) every later document anchors to. - `design/coverage-matrix.md` — each unit mapped to the plugin surface it would touch. - `design/skill-inventory.md` — the existing-skill overlap audit that decided what is new work versus an extension. - `design/official-corroboration.md` — the fresh-docs sweep ledger; four slugs fetched, the rest outstanding as Phase 1. - `design/design-resolution.md` — the cross-plugin criteria seam: four candidate shapes scored, with a canonical repo-level source materialized per carrying plugin by a sync script as the starting position. ## State Nothing is implemented. Phase 1 (fresh-docs sweep) gates every later phase, and two operator decisions stand in front of execution: the seam choice in Phase 2, and a proportionality gate in Phase 2.5 that decides how many of the seven candidate detectors justify the machinery the plan proposes. Per `docs/conventions/topic-docs/`, this directory lives on the task branch only and is pruned before merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All eleven page slugs the plan named are fetched and recorded with their
URLs, and the falsifiability walk over `llms.txt` accounts for all 172
listed pages as fetched, deferred with a trigger, or out of scope with a
reason. Both of Phase 1's sanity checks pass.
## Findings that change later phases
- `features-overview` already prescribes D7's routing. Its "Compare
similar features" section is official guidance on choosing between
`CLAUDE.md`, `.claude/rules/`, and skills, carries the 200-line rule,
and states the instruction-versus-hook enforcement boundary verbatim.
D7 now has to show what it detects beyond restating that page, which
is Phase 2.5's question.
- Output styles are an instruction surface nobody enumerated. They modify
the system prompt directly, default to removing Claude Code's built-in
software-engineering instructions, ship from plugins in an
`output-styles/` directory, and `force-for-plugin` lets a plugin
override the operator's selection. D1's surface partition is incomplete
without them.
- The compaction table makes progressive disclosure a trade rather than a
free win: path-scoped rules and nested `CLAUDE.md` files are lost after
compaction until a matching file is read again, while root `CLAUDE.md`,
unscoped rules, and auto memory are re-injected from disk.
- Deferred tool loading is owned by `mcp#scale-with-mcp-tool-search`, not
`tools-reference`, which carries only the `ToolSearch` row.
- `skillOverrides` does not reach plugin skills, so it is not a
remediation lever for the surface this pass audits.
- `/doctor` presence is a three-part prerequisite: the v2.1.205 skill
cutover, `DISABLE_DOCTOR_COMMAND`, and a `skillOverrides` entry.
## Also recorded
Subagent startup loading and the Explore/Plan carve-out, the native
diagnostic surface beyond `/doctor`, `claudeMdExcludes` scope-merge rules
and its managed-policy floor, launch-directory-dependent startup scope,
and the documented `userConfig` `default` that this repository already
established empirically does not resolve through `${user_config.*}`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… read The sanity check enumerated eleven slugs while the sweep fetched fourteen, so a re-run would verify the original list and silently skip `features-overview`, `output-styles`, and `mcp` — the three pages the falsifiability walk surfaced and the Outcome block calls load-bearing. The check also needed the autolink's closing `>`: a bare `docs/en/plugins` matches `docs/en/plugins-reference` and `docs/en/mcp` matches `docs/en/mcp-quickstart`, so both slugs could pass without their own page ever having been fetched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nothing Phase 2.5 ran ahead of Phase 2. design-resolution.md says the Tier A classification and Phase 2's own existence rest on "a versioned criteria catalog consumed by more than one plugin", and Phase 2.5 is what decides whether there is one, so the planned order would have settled the seam against a premise the gate deletes. One officially-backed check survives. D1 lands as I12 inside the catalog that already exists; D3 ships OPINION-tier and default-off; D2, D4, D6, and D7 are edits to checks that already exist; D5 defers with a trigger. The cross-plugin catalog, its convention-registry owner doc, and the sync-script materialization are dropped. The sweep and the re-run contract survive, because their justification was never detector count — the checks are delegated, the run semantics are not. Phase 2 then resolves to no shared artifact at all. Shape 4 was the starting position and reverses on four findings verified against the tree: check-cross-plugin-source-drift.sh clusters on the full path-within-plugin, so a criteria catalog under differing skill names forms zero clusters and its cited CI guarantee never fires; relocating the file breaks check-skill.sh's ref check and then escapes it entirely; adoption costs six to seven registration points, one of them unreachable; and the catalog has no frontmatter to bump. The strongest argument is one no shape analysis had — no plugin in this corpus reads another's reference/criteria.md, and audit-encapsulation classifies reference/ as private surface. Homing was reassigned against the incumbents' actual bodies rather than their listing descriptions. skill-quality:check hosts nothing: its contract is "NO model invocation ... reproducible in CI", and argument-hint is read by nothing in the plugin. docs-hygiene hosts nothing: every trimmer there already owns a stopping condition, which leaves D4's carve-out with exactly one consumer. Two host plugins, not four. Two independent reviews gate the verdict and both are recorded. A blind derivation reached the same count from the source documents alone and caught an inversion: D4 is OPINION-tier and withholds findings, so defaulting it off deletes the mitigation for the plan's own High/High over-trimming risk. A cross-vendor review found the re-derived shape contradicted a settled Brief criterion it never cited, and that D2's demotion failed the gate's own stated test — the gate now separates the coverage test from the evidence test and every row names which it fails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t exercise Phase 7's audit ran fenced and read-only over all 43 setup skills: 41 conforming, 2 partial, 0 fully non-conforming, 4 legitimately check-only with their premises settled from plugin manifests rather than from the skills' own prose about themselves. No setup skill reads a sibling plugin's files, so the per-plugin-by-construction claim holds where it matters. The two most useful findings are owner-doc gaps rather than plugin defects. context-guard and rate-limit-guard share one configuration shape — the user's own settings.json plus a plugin-owned machine file — and resolved it two different ways, with PLUGIN-PHILOSOPHY sanctioning neither, because its check-only carve-out is written for a surface that is nothing but userConfig. And "non-trivial userConfig", which the requirement gate turns on, is never defined, which leaves education, repo-hygiene, and visualization unanswerable from the doc's own text. Grading them against what the fleet happens to do would move the audit's owner from the doc to the corpus, so they are reported undetermined. Phase 10 gains a measured bound rather than an assumption. This repository has zero @-imports, zero nested CLAUDE.md files, zero files under .claude/rules, and zero files carrying paths: frontmatter, so D6's two target defects have zero instances here — the destination class is empty. A green dogfood run would therefore be evidence the repository is clean, not evidence the rule works, and D6 needs a synthetic fixture to be validated at all. That is a Phase 6 obligation, not a Phase 10 discovery. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lly costs The whole progressive-disclosure remediation rests on one premise — that moving content into a path-scoped rule defers its load. A live plugin reference asserted the opposite, dated 2026-04-01, citing four issues. If it held, the remediation would send operators to a destination that saves nothing and the compaction table would price a cost never incurred. It does not hold. A first-party repro on Claude Code 2.1.219 shows a `paths:`-scoped rule absent at session start, present after reading a matching file, and still absent after reading a non-matching one. Deferral works cleanly in both directions, exactly as the memory page describes. The cited evidence failed twice over, which is recorded because the same failure could recur in this work's own citations. Two of the four issues are closed NOT_PLANNED and neither ever supported the claim — one asks that Write/Edit also trigger injection, which presupposes deferral works, and the other is a docs issue about subagents. The two still open assert opposite failure modes, so they cannot jointly support one conclusion, and the stale-open one is argued against by its own last comment: adding `paths:` frontmatter to 75 rule files dropped a session baseline from 139.5k to roughly 40k tokens. Four costs the remediation must price, all verified, one of them new and the sharpest thing D6 will detect: an `@import` inside a path-scoped rule inlines at session start while the rule's own body correctly stays out, so an operator who moves content into a scoped rule but pulls it in by import saves nothing. Path-scoped content is also invisible to subagents and skill-forked contexts, a new-file Write does not trigger the rule, and multi-entry `paths:` lists may silently fail to match — that last one flagged rather than verified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rrect the record Phase 4 lands as sixteen numbered assertions rather than prose intent, because the gate's argument for the sweep existing at all is that the run semantics are the product — if the contract is intent, that argument has nothing behind it. The load-bearing choices: the finding anchor is content-derived rather than line-derived, so an unrelated edit above a finding cannot churn the report; a run never writes into its own scan set, which reduces to `git status --porcelain` being empty afterward; state is keyed by canonical repository identity plus a worktree discriminator, never by working directory; applying runs refuse rather than queue, because a long sweep behind a silent lock looks like a hang; inline suppression is permitted only where the pass may write, so a cluster copy is suppressed at its canonical source; and the behavioral tolerance carries a floor of 2 so a small set cannot round it to zero and smuggle identity back in. Phase 6 designs D1 with five must-not-flag cases drawn from this repo and the official docs, a remediation that splits by scope, and a native-first inventory ruling per mechanism. Drafting it forced a correction back upstream: the first statement of D1's scope excluded skills, subagents, and MCP servers wholesale because they "override by name". That misreads the rule. Override-by-name resolves a collision between two entities sharing a name, where one is simply inert; it says nothing about a skill body contradicting CLAUDE.md — which is the source article's own headline example, system prompt against skill against user request. Excluding skill bodies would have excluded the failure D1 exists to detect. The exclusion is now narrow and everything else that holds instruction text is in the comparison set. The record is corrected where later work would otherwise anchor into stale claims. article-sections.md's traceability table routed every section to a detector that no longer exists, and it is the input to Phase 3's coverage gate and to acceptance criterion 2. The authority axis is now written per row, which is what catches S14 having gone untagged while S10's identical claim was tagged. S8 is split rather than flipped: features-overview states surface routing but not definition-site locality, so D3 stays OPINION-tier. S5 drops from COVERED to PARTIAL, since a trimming pass with no stopping condition is not covered. The unsourced /checkup alias is gone, and design-resolution.md's three refuted claims are corrected in place rather than rewritten away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A corrections list that never says whether it was acted on is how the same defect gets rediscovered a third time. All six are applied in this branch, so each is marked with what was actually done. One of the six was wrong about its own scope. Correction 4 named S5's COVERED verdict in coverage-matrix.md; the same contradiction also stood in article-sections.md's traceability table, so fixing the matrix alone would have left it standing one file over. Both now read PARTIAL. The `/checkup` mention that survives in this document is the record of the defect, not a second live instance, and the note now says so — a grep hit there is not a miss. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s branch The re-derivation changed what Phase 3 produces without changing where it sits. The original phase built a design artifact; the re-derived one edits plugins/claude-config and plugins/claude-memory, which are live plugin files. Phase 8 is where the branch splits and Phase 11 requires documentation and implementation to land as separate PRs, so performing those edits in Phase 3 would put implementation on the docs branch. Phase 3 is therefore discharged as a specification — the per-file edit list is the deliverable — and execution moves to Phase 8. Its coverage sanity check splits accordingly: the traceability half is assertable against the specification now, and the three gates that need the edits on disk are carried forward. Two further live-file corrections found during design ride with them rather than leaking onto this branch: the stale path-scoping claim in claude-memory's reference, and the setup-corpus fixes whose first two items change PLUGIN-PHILOSOPHY itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…port shape A same-named skill, subagent, or MCP server shadowed at a higher- precedence scope is not a conflict — exactly one is live — but it is worth telling an operator about, because an inert definition that looks live is its own trap. It becomes a separate info-severity finding rather than part of D1. The split earns a property rather than just tidying. Detecting a shadowing is name comparison across a known precedence order: mechanical and fully deterministic, where D1 proper is behavioral. So the shadowing finding lands in the mechanical tier and contributes to the diff-clean gate, while D1's conflict findings sit in the behavioral tier under the stability tolerance. Merging them would have dragged a deterministic check into a non-deterministic section and weakened the determinism property for nothing. The report is two files because incremental persistence and a sectioned report want different shapes: an append-only JSONL during the run, since a single document would have to be rewritten whole on every append and that is exactly the operation an interruption leaves half-done, and an assembled findings.json at the end. Lanes are keyed by check crossed with surface class — per-check lanes serialize across the tree, per-file lanes multiply manifest overhead, and surface class is the granularity the exclusion set and the three-scope inventory already use. The suppression record's location and format are deliberately left open rather than invented. This repository has config-cascade and consumer-config-layering conventions and a live consumer-side config file, and inventing a new consumer surface against existing convention is the failure this work audits others for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The name is the operator's call, but the naming pass settled something that binds regardless of which one wins, and it matters more than the name does. audit-instructions' description already opens with this skill's exact surface list — user + project CLAUDE.md, .claude/rules, skill bodies, agent definitions, prompt-type hooks, output styles — character for character. So the surface cannot be the distinguisher and no name can carry the distinction alone. The picker labels rows by short name and readers scan the first clause of the description, so this skill's description must open with the run semantics — a coordinated cross-scope pass over a named target, fix-capable behind an explicit override — and must not open with the surface. Open with the surface and the two skills are indistinguishable in the picker whichever name is chosen. Two mechanical results carried so they are not rediscovered: no leaf-name collision exists for any candidate, checked by exact match against all 125 unique leaf names, so no registry entry is needed — and adding one speculatively would fail --check, since an entry that no longer collides is itself a failure. CATALOG-TAXONOMY does not reach the decision; its form rule governs category values, which are the inverse of the skill-name grammar. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four places still described machinery the proportionality gate deleted, and each would have misled whoever executes the remaining phases. The convention-registry grounding finding was carried as a constraint on this work because a shared catalog would have been a new cross-plugin convention. Phase 2 chose no shared artifact, so it no longer binds and no registry row is owed. The gate itself stands, and ground truth found it currently unheld in practice — 17 rows covering 2 of the repository's 5 materialization mechanisms — which is a finding about the repository, not about this work. Two risk rows are resolved rather than deleted, because a risk table is a record of what was foreseen. The catalog-adopted-without-an-owner-doc row is retired outright. The machinery-outweighs-the-payload row FIRED, and its mitigation worked exactly as written: one detector survived, the operator approved re-deriving, and the catalog, owner doc, registry row, and materialization are gone. Blast radius stays HIGH but loses two of its four contributors — two plugins are modified as instruments rather than four, and no new contract surface crosses a plugin boundary. Re-rated in place with the reason, because a blast radius that never moves is one nobody reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cross-vendor review found that claude-memory:audit already ships C6 Consistency [FAIL] — "Do any instructions contradict each other across CLAUDE.md, CLAUDE.local.md, and rules files?" — citing the SAME official line this plan cites as evidence that no tool performs the review. Verified verbatim. That premise was the foundation of D1, the deliverable's only officially-backed check, and it is the third time this work has made a negative claim about a body nobody read. D1 survives, scoped by routing rather than by assertion. C6 compares within the memory layer only; it does not reach skill bodies, agent definitions, prompt-type hooks, or output styles, does not compare across layers, and does not know the managed-policy tier. The source article's own headline example — a skill body against CLAUDE.md — is cross-layer and unowned. So I12 routes a wholly-memory-layer contradiction to claude-memory's C6 and keeps the cross-layer, non-memory, and managed cases, using the same routing convention I1-I5 already run. That also removes the duplicate-finding problem the review raised: two plugins would otherwise report one contradiction with no reconciliation rule. Separately, the exclusion-set paragraph asserted "three worktrees exist under .claude/worktrees/". That directory does not exist on this machine, and git worktree list reports 26, in a sibling directory outside the repository. Wrong twice, in the paragraph arguing for deriving counts instead of transcribing them. The derivation mechanism is robust to both errors; the narrative was not. Counts are now cited by command or not at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every cross-document citation in the topic tree pointed at a line number, into files this work keeps editing. Nothing catches the drift: link-check.yml is weekly, advisory, and not wired into ci.yml; markdownlint does no existence validation; and check-skill.sh scans only SKILL.md. One citation had already drifted far enough to point a reader at an unrelated passage. Zero live line-number citations remain across the six files. Each was converted by opening the target, reading the cited lines, confirming the citing sentence's claim still holds there, and naming the section that actually contains it. Two line citations survive deliberately, both inside the corrections record. They are historical accounts — "S5's COVERED contradicted S13's remainder", "it had been described as (built-in, alias /checkup)" — describing where a defect was when it was found, not pointers to live content. Rewriting those to headings would falsify the record. Verified after: markdownlint clean, and all 41 relative links across the tree resolve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The determinism property was scoped to the `mechanical` tier. Verified against the implementations rather than the labels, that tier cannot carry it, for two independently sufficient reasons. No dispatched check reaches the report without model judgment. audit-instructions says its deterministic pre-scan "is advisory and a grep cannot judge whether a rationale is genuinely present, so the lane refines every candidate rather than reporting it verbatim", and Phase C then re-judges EVERY removal or rewrite proposal — not only behavioral ones — where "a proposal the verifier defends is demoted to info or dropped". So a check the catalog labels mechanical passes through two model stages before it is a finding. And half the dispatched catalog was never in the vocabulary. Verified by count: claude-memory's criteria carries 17 checks and ZERO occurrences of mechanical or behavioral. It labels FAIL/WARN/INFO. Three tiers replace two. Derived — the three-scope inventory, the exclusion set, shadowed-definition findings, and raw script candidate rows — has no model in its path and carries exact equality. Judged carries the stability tolerance. Delegated is /doctor's output and carries neither. Two properties are stronger than what they replace. The inventory and exclusion set are IN the derived tier rather than scaffolding beneath it, so a surface that silently leaves scope now fails the gate — a shrinking scope otherwise reads as an improving report. And exceeding the judged tolerance fails the run's self-check and is reported as an instability finding against the sweep itself; the tolerance may be revised only by an explicit recorded decision, never as an implicit response to its own violation. D1's exclusion from the determinism gate stops being a mark against it. No catalog check contributes, so the exclusion is the norm. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ion surface The sweep is `audit-pass`, chosen from a 32-candidate five-lens tournament. What the name pays is recorded at the naming site so it is not re-litigated: a thin qualifier, a pass/fail connotation the design disclaims, an object-prior that lets it misparse as "audit the pass", and a compiler-usage level mismatch where a pass is a member of a sequence and the coordinator is the pass manager. Those are underspecification, which is why it beat audit-battery, audit-campaign, and audit-cycle — each of whose verified field meaning actively contradicts the design. The runner-up and its reason are recorded too. ISO 19011's combined audit — one audit at a single auditee covering two or more otherwise- separate management systems — is literally this artifact's coordination difference, and audit-combined was the only candidate whose established field meaning matched rather than approximated. It lost on readability, and because the accuracy it bought is the distinction the description already carries. An earlier framing is recorded as rejected so it does not return. "Auditing the instructions to make sure they are aligned with the latest guidance" produced no survivor across eight candidates: D1, the entire officially-backed payload, detects two of the target's OWN instructions that cannot both be satisfied — internal self-consistency consulting no external guidance at all — while "alignment with current guidance" is a property of the delegated catalogs' recheck triggers, and the phrasing is re-anchor:recheck-against-upstream's trigger space nearly verbatim. Scope narrows from "a repo or folder" to a git repository. Every mechanism the re-run contract rests on is git-derived, including the headline proof that a run never wrote into its own scan set. A smaller true promise beats a larger false one. The suppression record resolves to two artifacts, per config-cascade's own boundary: an owner doc under docs/conventions/ declaring the keys, and the instance at .claude/audit-pass.md. Keyed per finding id rather than as a list, since a closed list is taken whole and one personal suppression would silently discard every team suppression. Team layer wins on conflict. Reason and date required — no precedent exists on any suppress path here, and the closest analogue's bare-id form is not transferable, because it justifies itself by arguing its opt-out can only cause junk to be missed, never removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…at carried it claude-memory:audit ships C6 Consistency [FAIL] — "Do any instructions contradict each other across CLAUDE.md, CLAUDE.local.md, and rules files?" — citing the same official line four of this work's documents cite as evidence that nothing performs the review. Verified verbatim. Four statements were false and are corrected in place, each saying what it previously claimed rather than being quietly swapped: - coverage-matrix.md's S3 row read GAP with incumbent "none". Now PARTIAL, naming C6 as the incumbent and the cross-layer and non-memory surfaces as the remainder. - The Brief read "Nothing does it. This is the strongest gap." - official-corroboration.md's S3 row read "No tool performs this review". - proportionality-gate.md's D1 row read "No incumbent compares two instruction surfaces against each other". D1 survives and remains the payload. C6 compares within the memory layer only; it does not reach skill bodies, agent definitions, prompt-type hooks, or output styles, does not compare across layers, and does not know the managed-policy tier. The source article's own headline example — a skill body against CLAUDE.md — is cross-layer and unowned. I12 is scoped by routing, using the convention I1-I5 already run. Naming closes in the Brief's Open list too: the sweep is audit-pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…heck 21 The homing decision that skill-quality hosts nothing rests on its "NO model invocation" contract. PR #1096 adds a twenty-first check, so the contract was read rather than assumed: check 21 is bash plus awk, no model in the path, and the PR's own spec calls it a deterministic scanner. No homing reversal. Sharpens the one clause that needed it — check 21 is the gate's first judgement-shaped rule, so the discriminator is now stated as whether a mechanical proxy exists rather than as a blanket bar on judgement.
…ing it This work had independently formulated the fresh-context checkpoint rule that #1096 codifies and gates. Parallel formulations of one doctrine drift, so the language now cites the spec owner and adopts its vocabulary. #1096 is stricter on five counts this work had not assumed: the generic rung is the default, the named-agent bar, artifact-not-story, session-tier-or-above for consequential verdicts, and greppable declaration conformance. Each is mapped to the site it binds. Also records the conformance obligation check 21's scan surface places on the criteria file and the sweep's own SKILL.md.
D1 was classified behavioral without the delete-and-watch loop that grounds every other behavioral check, and its own Remediate forbids the deletion that loop verifies. The catalog's tier definition bundles an evidence claim with a removal-remediation protocol; D1 meets the first and the second is inapplicable rather than skipped. Defines D1's own loop from machinery already specified — Phase C refutation retargeted at reconcilability, operator resolution, P2 convergence as the watch step — and names what that loop cannot establish.
…'s side The Brief headlined "Shape: a runbook" while the proportionality gate poses that exact question and answers "a component". The Brief's stated reason — distributed concerns, all of them get applied — argues for delegation, which a delegating component satisfies equally, and never reaches the run semantics the gate's argument rests on. So the Brief loses and says why in place. Renames the surviving descriptive uses to "sweep"; the sites that pose or answer the question keep the word.
…ure's level OPINION content — built, reachable, one argument away — was granted a loud per-run disclosure on the reasoning that a quiet default is unreachable. /doctor-absence leaves half the subject with no incumbent and no replacement, which is strictly worse by that reasoning, and got one machine-readable row. Escalates it to a matching prose line plus a warning finding against the run rather than the target, keeps the skipped row, and states the rule at the level it operates on so a third case does not re-litigate it.
P3 licensed finding-set growth on a catalog version bump, but I6's and I10's actual detection behavior lives in audit-instructions' SKILL.md Phase B and Phase C prompt text and in the pre-scan script, neither of which carries a version. A prompt rewording moved the finding set with no version accountability. P3 now keys on (catalog version, host plugin semver, prompt digest). The plugin semver is the seam that already moves on a content change; the digest fails closed where that convention fails open. Records the false-permission direction the digest introduces and widens task #43 accordingly.
The sweep reads arbitrary instruction text — text whose purpose is to steer a model — and then mutates files. That is the indirect-prompt-injection shape, and Phase 11's tagged security review is the plugin-acceptance gate, which checks distribution hygiene and would surface none of it. Three threats grounded in fetched sources (Claude Code security page, OWASP LLM01:2025, and the Claude Security plugin's never-auto-apply posture): instruction text targeting the auditor, the suppression record as a durable silencer, and the per-run human gate's real limits. Mitigations map to mechanisms the design already has where possible, and the one place no authoritative source supports the argument is labelled as argued from this design's own numbers.
The mitigations silently introduced four constraints on artifacts task #28 owns and one amendment to the re-run contract. Listing them stops a sibling lane from inheriting a posture change by reference. Amends rerun-contract §4 so writing a suppression is never an apply — §4 said where a suppression lives, not who may author one, which left the T2 laundering path open. Grounds the /doctor warning in P4a's existing finding-against-the-run shape, and records the Phase C verifier prompt as a T1 hit on a file this branch does not own.
…fiable properties The identity tuple carried one surface and one anchor, which cannot express D1/I12 — inherently pairwise across two surfaces. An ordered pair fails on symmetry; two linked findings fails SARIF 3.27.12, since fixing either side retires a contradiction. Identity becomes (check, claim, sites) over a canonically-sorted site set, with primary/related carried outside the hash. Adds what the contract lacked: versioned anchor names, a granularity discriminator so whole-surface findings are content-free, per-surface-class excerpt extraction, a tiered match table that finally gives P2 a detector, and cluster-copy canonicalization before identity rather than only before write. Scopes two properties that were falsifiable as written. Assertion 1.1 cannot hold for liveness-dependent findings, because liveness depends on launch directory, merged claudeMdExcludes, declined imports, and setting-sources — none of which are the tree. P1 gains a liveness-basis clause and P3 a harness version clause, since a dead-surface finding's truth is a property of the harness, not the corpus. Justifies the hash truncations by collision domain rather than assertion, and reconciles their non-adversarial threat model against the injection section.
… two ratified calls Task #56 settled empirically on v2.1.219: Claude Code does not read AGENTS.md. Excluded from D1 affirmatively rather than by omission — an un-imported file is dead text, and including it would flag CLAUDE.md against itself in exactly the repos following Anthropic's recommended import pattern. Stated as "not a surface in a stock install" since a third-party plugin claims the capability unexamined. The more valuable half is a new build requirement: D1 must expand @path imports and resolve symlinks, because an imported file is part of the CLAUDE.md surface by Claude Code's own loading semantics. Without it every @docs/foo.md is invisible and AGENTS.md was only one instance of a general blindness. Re-derives the .claude/ storage argument, which failed for a reason #56 was not needed for: root CLAUDE.md and CLAUDE.local.md are live surfaces outside .claude/. The record belongs there as skill configuration, not by surface locality. Declines to lean on /init's generation-time reads. Records both ratified operator decisions with the losing arguments intact: the router ships with the #1225 reconciliation owed before Phase 9 and its falsifier stated, and Type B ships with the safety-critical allowlist binding rather than advisory.
… do not graduate The contract slice is pruned before merge, so every catalog pointer into docs/topics/ dangles and link-check.yml fails. Resolved by derivation from topic-docs' own "Pointer discipline on durable surfaces" RATIONALE, not from its enumeration — a shipped plugin catalog is not among the surfaces that rule lists, and claiming otherwise would be the transcribe-don't-derive failure this effort keeps hitting. The rationale reaches it a fortiori: a shipped catalog outlives the prune permanently. claude-memory's criteria.md already ships the shape and greps clean for both design/ and docs/topics, so the cost is zero. Design traceability rides the PR. This is what unblocks the docs PR.
…ourth instance skill-inventory.md asserted a SessionStart hook injecting a persistent ruleset on every prompt. Enumerating every hook event across all 15 hooks.json files refutes it: the one SessionStart arms a detached observer, emits no additionalContext or systemMessage, and no-ops unless its plugin option is set. No UserPromptSubmit exists anywhere. Bounded honestly — settings.local.json, managed settings, and the marketplace copy were not read. PLAN.md's three-scope argument cited it as a second example and now stands on the one that survives. The same paragraph carried "50 lines / 9,825 characters" for the user CLAUDE.md. Measured by command: 69 lines / 10,550 bytes. Noted that a non-blank-line count returns 51, close enough to the stale figure to read as confirmation. Named as the fourth instance of this effort's own recorded pattern — negative or numeric claims about bodies nobody read. skill-inventory.md predates the counts-by-command rule and was never swept against it. Also marks the /doctor interactive-TTY assertion unverified rather than inheriting it, and records that /doctor is narrower than the no-overlap constraint has been read to mean.
…xclusion reason The traceability table's claim is that every section of the source resolves to a check, an incumbent, or a recorded exclusion. Five findings from the blind independent derivation had no row at all, and S1's exclusion reason was wrong as written. S1 read "Excluded — framing. No rule to enforce... nothing is auditable from it." The blind reader found an auditable rule there three times independently. The assembly claim still yields no rule, so S1 stays excluded on that half; the reason now says what is actually true and points at the remainder rather than denying it exists. The five remainders each get a row, and each row states its disposition rather than leaving the reader to infer one: - S1-a, cross-tool split-brain: this repo ships a codex plugin handing work to Codex CLI, which does read AGENTS.md, so two agents work one repo under two rule sets. Cross-tool, not cross-surface, so it is outside D1's shape — task #56 settled that Claude Code does not read the file. Recorded as a row rather than as silence, with no owner claimed. - S1-b, unreachable support files: 11 of 395 across 6 of 181 skills, one a false positive. Unreachable is not deferred, it is dead. Mechanically checkable and S7's own subject, but not built here — named as a candidate for skill-quality, the gate that already walks skill-internal refs. - S1-c, the re-attach cap: recorded as a mechanism and never counted. 13 of 181 bodies at or above it, and the shared budget holds exactly five, so a chain of six evicts its own earliest member. Phase 10 counts the line-based figure, where 0/181 over 500 lines reads as conformance and 14/181 over the truncation cap says otherwise. Calibration input, not a check. - S1-d, "less constrained" is not "less specific": the official pages tell authors to be more concrete and warn that adherence drops when instructions are vague. A pass reading the article literally produces that exact failure, and I6's "state what to do instead" is the move that can trade an absolute for a vague replacement. A constraint on D4 and I6. - S1-e, issue #1225: structurally a second repo-wide sweep, ratified to ship anyway, reconciliation open as task #61 before Phase 9. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review threads against the design record, addressed where they were right. The load-bearing ones are logical holes in the re-run contract rather than wording: - P2 asserted a strict subset over the derived tier, so a successful fix to D1 -- a judged finding that contributes nothing derived -- failed the convergence gate. Closure is now measured in the tier the fix acted on. - P3 was missing the liveness-basis clause P1 already carries, so correct environment-dependent growth read as a sweep defect. - Assertion 1.5 failed the run on any judged disappearance that P4 explicitly tolerates; judged closures now land in P4, whose metric becomes the symmetric difference so removals cannot pass unremarked. - The dead-surface rule tested ancestor chains only, so an ordinary nested memory file classified as dead, contradicting the paragraph above it and producing a perfectly reproducible false finding. - The resume digest covered files only, so a resume across a changed liveness basis or harness version silently mixed stale lanes with fresh ones. - One tagged anchor per site leaves no greatest common version to match on, so the first algorithm bump discards the record versioning was adopted to protect. Both halves fixed: emission and storage. Also: escaped symlink targets are read-only, redirect destinations are refused rather than overwritten, read-only runs serialize against applying ones, stale locks always have a recovery path, non-commentable surfaces route suppressions centrally, the one-site matching transition is stated beside the pairwise table rather than by rewording a cross-lane contract, partial-log records carry attempt identity and survive a torn write, and a personal-only suppression key no longer defeats the team policy floor. Counts re-derived by command: the SKILL.md remainder is one nested worktree, not three; the assertion tally was stale before this round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdF74i18gkGtHC9fjZyDrU
kyle-sexton
added a commit
that referenced
this pull request
Jul 25, 2026
… reference claims (#1393) Supersedes #1377 (which superseded #1315). Same work, rebased onto current `main` with five version collisions resolved, plus both review findings from #1377 closed and a pre-existing broken-reference failure fixed. No linked issue. ## Why the owner doc moves first Two plugin-level findings dissolved on inspection into owner-doc gaps rather than plugin defects, so the definitions are fixed at their definition site and the plugins then conform without edits. - **`userConfig` non-triviality had no definition**, leaving criterion (c) unfalsifiable. The line now follows from what Claude Code's native configuration prompt *is* — a collector, not a verifier. It stores what the consumer typed; it never confirms the path exists, the token authenticates, or two options agree. Resolves `education`, `repo-hygiene`, and `visualization` at once. - **The check-only carve-out was scoped to plugins whose entire configuration is `userConfig`**, excluding a shape the fleet actually ships: a plugin whose behavior is delivered through Claude Code settings this contract forbids setup to mutate. The carve-out now names its real condition — no writable owned artifact — and enumerates three qualifying surfaces. Silence is not the conforming response: `check` prints the exact edit, states that it is the operator's to apply, and names what re-invalidates it. Resolves `context-guard` and `rate-limit-guard` together. ## Two reference claims that were false - **`claude-memory`'s path-scoping status.** The reference asserted `.claude/rules/` files load unconditionally regardless of `paths:`. A first-party repro on 2.1.219 disproved it — a rule scoped `paths: ["**/*.tsx"]` was absent at session start, present after reading a matching file, absent again after a non-matching one. The cited evidence failed independently: two of four issues are closed NOT_PLANNED and never supported the claim; the two still open assert opposite failure modes. - **The `CLAUDE.md / AGENTS.md` compaction row.** Claude Code does not read `AGENTS.md` — [official docs](https://code.claude.com/docs/en/memory) say so, and it is absent from `/context`'s Memory Files enumeration even with a `CLAUDE.md` beside it. ## Two review findings from #1377, both real, both closed here They are the same shape: a documented fallback that reads as a safe default and is in fact a silent wrong answer. - **`machine-health`** told the caller to omit `-StateBase` when `${CLAUDE_PLUGIN_DATA}` does not expand. But a skill-invoked tool subprocess can inherit an **unrelated installed plugin's** `CLAUDE_PLUGIN_DATA` — the extensibility contract's own smoke tests document this — so the fallback can write this plugin's state, logs, and catalog overlay into another plugin's directory. It now passes the report root explicitly and reports the unresolved root. Colocating state with reports is wrong-but-visible; the inherited variable is wrong-and-silent, and only one of those is recoverable by a reader. - **`session-flow`** told a headless consumer to supply "every key to change" on reinstall. Uninstalling drops the stored `pluginConfigs` entry, so every omitted key reverts to its manifest default — reinstalling purely to enable the observer silently resets a customized analysis model, idle threshold, bare mode, and maximum lifetime. The rule is every key whose value should be **non-default**, and `check` must run **before** the uninstall, because afterwards there is nothing left to read the values from. ## A pre-existing gate failure this PR surfaced `machine-health`'s setup cited five of the audit skill's reference files by bare relative path, which resolve to nothing from the setup skill's own directory — three of them fail `skill-quality:check`. The gate only surfaced them because this PR touches the file. The correct form was already in the same document twice; the remaining five now match it. The files stay where they are, under the audit skill that owns them: approvals and the output schema are the audit's contracts, and copying them would create a second owner. ## Ranked fix 4 — decided, and the answer is "accept the drift" The path-shape question came out **both ways**. In place, registration is impossible: `SKILL.md` is in the drift checker's `skip_basenames` and the checker hashes whole files, never line ranges. After extraction it *would* work, since both skills are named `setup`. So registration is available only at the cost of creating the artifact in dispute. Accepted on merits: the block substantially restates rules two owner docs already carry, so a shared fragment would be a second owner for them. Restating is what a `SKILL.md` must do — it is the surface a session loads and cannot defer at runtime to a document the consuming repo does not have. Three parts are plugin-authored rather than contract-derived, and `planning`, a third implementer, contradicts two of them outright. ## Rebase resolutions Five version collisions with `main`, each laddered rather than clobbered, with `main`'s entry preserved intact above ours: `discovery` → **0.8.4**, `session-flow` → **0.15.3**, `rate-limit-guard` → **0.2.1**, plus `claude-config` at **0.9.3** and `claude-memory` at **0.4.1**, which are the first rungs of the ladder #1316 and #1318 build on. ## Related - #1377, #1315 — the branches this supersedes; both closed, neither could be rebased in place - #1316 — criteria payload (`claude-config` 0.10.0) - #1318 — the `audit-pass` skill (`claude-config` 0.11.0) - #1322 — the design record - #1278, #1279 — couplings scoped out to their own issues 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both raised by the cross-vendor review and both real: a check could not emit its own headline finding, and a routed capability had no way into the report. - Lanes keyed (check x surface class) partitioned D1/I12 by exactly the boundary it compares across. A skill body contradicting CLAUDE.md put the two instructions in two lanes; identity is (check, claim, sites) over a sorted set, so emitting the pair needs both anchors at once, and no lane held them. Checks now declare within-class or cross-class. A cross-class check is one lane scoped to its declared comparison set -- for I12, the all-surface read-only inventory the catalog already defines -- and its resume digest covers that whole set, so a moved counterpart class invalidates the lane instead of resuming clean. - The /doctor handoff wrote no lane record and no terminator, so the trim-and-migrate result was silently absent from a run that claims to coordinate it, while `delegated` is defined as that output. The delegation is now an ordinary lane that always terminates: handed-back, declined, or open. An open lane is reported by name and is resumable by re-prompt rather than re-scan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdF74i18gkGtHC9fjZyDrU
kyle-sexton
added a commit
that referenced
this pull request
Jul 25, 2026
…1323) ## What this PR lands One file: `docs/adr/0004-rightsize-instruction-surfaces-by-incumbent-first-arbitration.md`. The branch carried the full contract slice for this effort while the work was in flight — 13 blind section digests, the ratified decision set, the collision register, the agent brief, and the source article, all under `docs/topics/context-engineering-rightsizing/`. Per `docs/conventions/topic-docs/README.md:290-309` that slice is task-branch-only: durable outcomes graduate, follow-ups route to the tracker, and a final commit prunes the slice. That lifecycle is now complete on this branch: - **Graduated** — the ratified decision set, the execution errata, the conflict definition D-4 ships against, and the incumbent-gate blind spot are promoted into ADR 0004. - **Routed** — the D-6 fable-5 follow-up is filed as #1324. - **Pruned** — `22744fd664` removes the contract slice. `git diff --name-only main...HEAD` returns exactly the one ADR path, satisfying the enforcement rule at `README.md:305-309`. ## The digests were produced blind Thirteen agents each read one section of the source article with no sight of the others' output and no sight of this repository's prior conclusions. All thirteen reported clean fences. The convergences between them are therefore independent measurements rather than echoes, which is what makes the repeated findings load-bearing — and what the ADR's "Context" section rests on. The digests themselves are working material, not durable knowledge, so they prune rather than graduate. What survives is the arbitration posture they converged on (D-15) and the incumbent-first gate (D-1) that the pass proved was worth paying for. ## Three decisions were refuted by measurement — recorded, not executed The ratified text of all nineteen decisions is preserved verbatim in the ADR. Where first-hand measurement contradicted a decision's supporting evidence, the correction sits beside it as errata and the lane **stopped rather than re-deciding**: - **D-12** — its cited 2 ms control never ran; both comparators failed at launch. The directive (fix the root cause) stands and only the citation is corrected. The real defect is worse than the decision assumed: the guards are substantially fail-open. - **D-13** — **deferred, not executed.** The removal set is empty. All ten zero-invocation seeded plugins are hook plugins, for which zero transcript invocations is the expected reading of a correctly functioning one. - **D-17** — **deferred, not executed.** Diagnosis confirmed, prescription unexecutable: the move aborts `chezmoi apply` fleet-wide with a hard template error. Both deferrals await operator re-decision. Nothing was committed on their behalf in any repository. ## Deliberate, disclosed collision Branch `docs/context-engineering-claude-5-topic` (PR #1322) covers the same source article and is worked by a parallel session. The operator ruled that this pass runs independently and that the collision is resolved at merge, not by folding. The ADR records this as a standing constraint rather than as a register entry, because the register was working material and prunes with the slice. ## What the pruned register taught, and why only the lesson survives The collision register presented three writers on `dot_claude/CLAUDE.md` as the complete set when there were four (dotfiles PR #319, open and editing that file mid-pass). The missing row cost nothing — it merged before colliding — but the false completeness is the finding, and two lanes were dispatched trusting it. The register is not preserved; the generalizable lesson is, in the ADR: re-derive collisions from `gh pr list` at the moment of acting rather than trusting a snapshot. ## Verification Gates run locally on the changed slice before pushing: `markdownlint-cli2`, `typos`, `editorconfig-checker`, `lychee`, `gitleaks`, and `scripts/check-docs-only.sh`. All clean, nothing suppressed. No machine-specific paths, no `.work/` citations, and no shell or shebang files in the committed result. ## Related **No linked issue** — this PR closes nothing. It lands the durable design record; the execution lanes it dispatched carry their own issues and PRs. Refs #1324 — the D-6 follow-up covering `plugins/playbooks/skills/fable-5/**`, excluded from this pass while PR #1261 rewrote it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 3m 42s —— View job Review complete
Two commits since the last review (
All prior open findings resolvedEvery finding carried across from the last review ( P3 liveness-basis gap — P2 tier scope — Assertion 1.5 / P4 judged-disappearance conflict — Single-site matching table gap — Resume digest missing non-file inputs — Stale worktree claim in Coverage matrix S3 "no incumbent" summary — Codex findings also resolved in these commits
One finding still openForward reference to
Convention conflict — unchangedFifteen topic directories are already on Cross-PR semver collision — unchangedBoth #1316 and #1318 bump Summary: The two new commits comprehensively resolve all seven prior open findings and address twelve additional Codex findings, closing out the cross-vendor review sweep in full. One item remains open: the present-tense forward reference to |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 619acc52b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Second review round. Two of the three findings were the same defect: the run-basis precondition was stated per-property, so P3 carried all four inputs, P1 carried only tree and liveness, and P4a carried only "unchanged tree". Restating a clause three times is what let two copies drift, so it is now defined once -- runs are comparable when tree, liveness basis, detection version triple, and harness version all match -- and the properties cite it. A non-comparable pair is reported as non-comparable, naming the input that moved, rather than as a pass or a failure. P1's own text already conceded that a harness update breaks its unqualified form, since the derived tier reads a versioned registry of harness behavior; P4a's case is the more likely one, because a catalog revision is the ordinary way this deliverable evolves and it moves the judged set far past 10% while every run is correct. Third finding, separately: the lock's no-start-identity fallback reclaimed by age alone, so an applying run legitimately exceeding 30 minutes lost its lock while still mutating and a second applier proceeded -- Assertion 3.1 violated by the mechanism written to uphold it. Age cannot discriminate a dead holder from a slow one on any platform. The holder now refreshes a heartbeat and staleness means the heartbeat stopped advancing, with a start identity short-circuiting the wait where available and an operator override behind both. Two properties held at once: no lock is unreclaimable, and no live holder is reclaimed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdF74i18gkGtHC9fjZyDrU
|
Claude finished @kyle-sexton's task in 2m 15s —— View job Review complete
Single commit since the last review ( All three Codex P1 findings from the prior review resolved"Preserve active locks without a start identity" (Codex P1 at The no-start-identity fallback (age-based reclamation alone) is replaced with a heartbeat mechanism. A run that holds its lock for hours keeps it — the heartbeat keeps advancing; a crashed run stops refreshing and is reclaimed within a small multiple of the interval. Three rungs: heartbeat staleness (always available), start-identity short-circuit where the platform provides one, and an explicit operator override reported when used. The reasoning — "reclaiming a dead holder and reclaiming a slow one are indistinguishable by age" — is correct and the fix is sound. ✅ "Condition derived equality on detection and harness versions" (Codex P1 at "Compare judged stability only on the same run basis" (Codex P1 at Resolved: Shared comparability preconditionThe commit consolidates into one place the four-input precondition that three prior review passes found drifting per-property. The reasoning is well-grounded: P3 spelled all four out, P1 carried two of the four, P4a carried one, and the divergence produced two confirmed review findings. Stating the precondition once removes the mechanism by which it drifted. ✅ Resolved: P3b closes the detection-version scope holeP3 originally keyed on The P3b sub-property's Still open: Forward reference to
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 641a97f8ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…t list Third review round, and two of the three findings are the same failure mode: a set that must be complete was written as an enumeration, and the enumeration was short. - `prompt_digest` covered "the host SKILL.md and any script it names", omitting the criteria catalog the detection rules actually live in. A criteria.md edit that skipped the manual catalog bump left the digest unchanged -- worse than the version-bump gap the digest exists to close, because resume then skips the completed lane and carries forward findings produced under superseded criteria, reporting two rule sets as one. Coverage is now stated as the criterion: every input that can change what the check detects, with the catalog and its imports named because they were the ones missed. - The liveness basis omitted the additional-directory set (--add-dir and CLAUDE_CODE_ADDITIONAL_DIRECTORIES_CLAUDE_MD) while the liveness section above already recorded that it adds memory files -- the document contradicting itself in the false-alarm direction. Added to the basis and to the resume digest, and the basis is likewise governed by its criterion. Third, separately: the Terms block still defined a lane as one check applied to one surface class, after checks-and-sweep.md introduced the cross-class lane that lets I12 see both anchors. An implementer following the governing contract would have recreated the split. The two documents now state one definition. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdF74i18gkGtHC9fjZyDrU
|
Claude encountered an error after 2s —— View job I'll analyze this and get back to you. |
|
Warning Automated review did not complete — this is an infrastructure failure, not a review verdict. Treat any Claude comment on this PR (including a placeholder like "I'll analyze this and get back to you") as incomplete, not "no findings."
Re-running the job, or pushing a new commit, will retry the review. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8c25a50049
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+695
to
+698
| **Every property below is conditioned on one shared precondition, stated here once rather than | ||
| per-property.** The cross-vendor review found P1 and P4a missing a clause P3 already carried, and | ||
| stating it three times is what allowed two of the three to drift — so the **comparable-runs** | ||
| precondition is defined in one place and the properties cite it: |
There was a problem hiding this comment.
Applying this shared precondition to P2 makes the convergence property vacuous: an accepted fix normally changes the target tree, so the before-and-after runs are non-comparable and P2 asserts nothing about whether the accepted finding disappeared. Exempt P2 from the unchanged-tree requirement or define a separate fix-aware comparison basis.
Useful? React with 👍 / 👎.
Comment on lines
+700
to
+701
| > `R1` and `R2` are **comparable** when their **target tree**, **liveness basis**, **detection | ||
| > version triple** of every check consulted, and **harness version** are all equal. |
There was a problem hiding this comment.
When ~/.claude/CLAUDE.md or a managed-policy instruction file is added, removed, or edited, the repository's target tree and every listed liveness selector can remain equal even though the three-scope inventory, raw candidates, and judged conflicts legitimately change. These runs are consequently classified as comparable and can falsely fail P1 or P4; include existence and content digests for every non-repository scan surface in the basis.
Useful? React with 👍 / 👎.
Comment on lines
+825
to
+826
| - `|J(R1) △ J(R2)| ≤ max(2, ceil(0.10 × |J(R1)|))` over a **comparable** pair — **the stated | ||
| tolerance**, measured across three consecutive runs, with the worst pair taken. The tolerance |
There was a problem hiding this comment.
If the operator changes the selected model or its effort/sampling configuration between runs, the tree, liveness basis, check digests, and Claude Code harness version can all remain equal while the model-judged set legitimately moves beyond this tolerance. P4a then reports correct cross-model variation as sweep instability, and resume can combine findings produced by different judges; record the judging model configuration in comparability and lane digests.
Useful? React with 👍 / 👎.
Comment on lines
+700
to
+701
| > `R1` and `R2` are **comparable** when their **target tree**, **liveness basis**, **detection | ||
| > version triple** of every check consulted, and **harness version** are all equal. |
There was a problem hiding this comment.
A change to audit-pass's anchor normalization, finding-ID construction, inventory logic, or report schema need not change any consulted check's detection triple or the harness version, so these runs remain comparable even though derived identities can legitimately change. Per-lane resume is worse for lanes hosted outside claude-config, because it can reuse records produced by the old sweep implementation; include a digest/version of the sweep and identity/schema machinery in both comparability and resume inputs.
Useful? React with 👍 / 👎.
Comment on lines
+143
to
+144
| - A rerun over an unchanged tree produces the same **derived-tier** result, never a different one. A | ||
| rerun after accepted fixes produces a strictly smaller one. Absent a change to the tree, it grows |
There was a problem hiding this comment.
Fresh evidence after the earlier convergence thread is that this governing acceptance criterion still requires the derived result to become strictly smaller after every accepted fix, even though the corrected P2 now explains that resolving D1—a judged-only finding—legitimately leaves the derived set unchanged. An implementation can therefore satisfy the revised contract and still fail this acceptance gate; make the criterion require disappearance in the tier containing the accepted finding.
Useful? React with 👍 / 👎.
kyle-sexton
added a commit
that referenced
this pull request
Jul 25, 2026
…parability Reconciles the shipped run contract with the design record #1322 landed, and corrects that record where it had gone stale about this branch. The real gap: P1 was conditioned on tree and live surface set alone, so a harness or detection-version change read as a determinism failure -- the derived tier reads a versioned registry of harness behavior, so it is a function of the harness version by construction. The shipped contract now carries the same single comparability precondition the design does, stated once and cited by every property rather than restated per-property, which is what let the clause drift in the first place: tree, live surface set, detection version (catalog version plus a digest over the check's own detection-behavior inputs, including the criteria catalog and its imports), and harness version. A non-comparable pair is reported as non-comparable naming the input that moved, never as a pass or a failure. PLAN.md's "two divergences between this branch's design and what #1318 ships" was verified against the branch and is stale on both counts -- the superseded identity tuple was replaced by (check, claim, sites) with the anchor version tag and granularity prefixes, and liveness is present in both Assertion 1.1 and P1. Both are struck through with what closed them rather than deleted, since the record of what the walk found is the traceability the gate rests on. A stale divergence claim left standing on a durable surface is the exact defect this deliverable detects in other people's instruction files. Also: the 0.11.0 changelog still described the determinism gate as capturing a dirty-file count, which an earlier commit on this branch had already replaced with the worktree digest. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdF74i18gkGtHC9fjZyDrU
This was referenced Jul 25, 2026
Merged
Closed
kyle-sexton
added a commit
that referenced
this pull request
Jul 30, 2026
…uests (#1781) ## Why A review that lands **after** a merge has nowhere to go: - the ruleset's `required_review_thread_resolution` is a **merge-time predicate** that already passed; - the babysit lane works *open* PRs, and a merged PR leaves that queue; - nothing on a merged PR surfaces its open threads — GitHub shows the merge, not the findings. Six findings — one **P1** — posted **46 seconds after #1720 merged** and sat unread for a day. They surfaced only because a later session happened to audit that merge batch. Nothing was bypassed; the gate was satisfied *because the threads did not yet exist*. The morning brief is the right home: read-only, unattended, and already where attention signals land. ## What it does Compares each unresolved thread's **first-comment timestamp** against the PR's `mergedAt`, and reports only threads the gate could never have seen. A thread that predates the merge was visible to the gate — that is an ordinary unresolved thread, not this failure mode, and it stays out. - **One line per PR, at that PR's worst severity, with a finding count.** Several findings on one PR are one thing to go look at; repeating the title per thread buries every other PR. Collapsing on the *worst* severity means a P0 sitting beside advisory findings can never be softened. - **Severity survives to the operator** — a stranded P1 must not read like a P3. - **`--stranded-days`** (default 3) — wide enough to cover slow bot review *and* an operator-absent weekend. ## It fails loud, not clear A GraphQL error document is well-formed JSON that simply carries no `data`. The extraction would yield an empty list and render **"every merged PR in the window is clear"** — an all-clear asserted from an answer never received, which is the same fail-open shape this section exists to catch. This is not hypothetical: a rate-limit error did exactly that during development. An API error now says explicitly that it is *not* an all-clear, and prints the message. Covered by a regression case. ## This is a standing leak, not a one-off Its **first live run** against this repository immediately surfaced four more stranded findings on other merged PRs — including a **P1 on #1694** (merged `05:04:45Z`, finding posted `05:05:20Z`, 35 seconds later) recording that a shipped `autonomy` cell **never reached installations**. ## Verification - `morning-brief.test.sh`: **30 → 63 cases, 0 failures.** - The **negative** cases carry the weight — a pre-merge thread, an already-resolved post-merge thread, and a merge outside the window must all stay silent, or the section is noise rather than signal. Plus: collapse-does-not-soften-severity, highest-severity-first, window-widening, and the API-error case above. - The fixture mirrors the real #1720 shape, including the 46-second gap. - `shellcheck -x` on script and test — clean. One `SC2016` is declared, not blanket-suppressed: the `$owner`/`$name`/`$endCursor` in the GraphQL query are server-side variables bound by `-F` and **must** reach the server unexpanded. - `node scripts/validate-plugin-contracts.mjs` — 43 setup skills, 2153 files, pass. - `npx markdownlint-cli2` on both changed markdown files — 0 errors. ### Live run — posted in full in the comments below A live run on the current branch found **44 merged PRs carrying post-merge findings in a five-day window: 0 P0, 10 P1, 34 P2.** Among the P1s: **#1503**, a guardrail-bypass fix whose own review landed unread, and **#1322** with 5 findings. Read the **second** comment for the authoritative figures — the first was produced by the pre-review severity logic and reported a false P0, which review then caught. No truncation warning fired, so the read is complete. The five-day window filter was spot-checked against `mergedAt` (a PR numbered #969 in a 5-day window looks wrong until you check: it merged `2026-07-25`, 4.2 days before the run). This is a far larger leak than the six findings that exposed it. ## Related Closes #1777 Refs #1720 Refs #1759 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 3, 2026
…and #1425 (#1886) ## Summary Eight review threads were posted on #1322 and #1425 **after** those PRs merged, so the merge gate never saw them and they stranded unresolved. All eight were verified against `origin/main` and all eight were still real — none had been fixed by a later PR. Two of them, though, were not design gaps. They were the design doc (`docs/topics/context-engineering-claude-5/design/rerun-contract.md`) having gone **stale against a shipped contract that already solved them** — `plugins/claude-config/skills/audit-pass/reference/run-contract.md` already carries both `fix-comparable` and the widened `scan-baseline state digest`. Those two adopt the shipped vocabulary and semantics rather than minting a parallel set beside it. ## Findings and disposition | # | Finding | Verdict | Fix | |---|---|---|---| | 1 | P2 vacuous under the shared unchanged-tree precondition | Real; solved upstream | Adopt shipped `fix-comparable` (attribution-based) | | 2 | External instruction surfaces missing from the run basis | Real; solved upstream | Adopt shipped `scan-baseline state digest` | | 3 | Judging model config missing from the run basis | Real; uncovered in both | Precondition on P4/P4a only, not the shared list | | 4 | Sweep machinery version missing from the run basis | Real; uncovered in both | Added to comparability and the resume digest | | 5 | `PLAN.md` acceptance gate demands a strictly smaller derived set | Real | Disappearance in the tier that reported it | | 6 | `/research-deep` single-topic Tier 2 returns an ungraded artifact | Real | One boundary section every dispatching tier cites | | 7 | Root `.worktreeinclude` missing nested sub-slice patterns | Real | Materialized; verified empirically | | 8 | Multi-topic eval encodes the retired artifact layout | Real | All six evals realigned | ## Notable design decisions - **F3 is deliberately narrower than the finding asked.** The finding said "record the judging model configuration in comparability and lane digests". Putting it in the *shared* precondition would make P1/P3/P3a unfalsifiable across a model swap while asserting nothing extra, since `D` contains no model judgement — P1 is assertable precisely *because* nothing in the derived tier passes through a model. It conditions P4's tolerance and P4a only, plus the resume digest for judged lanes. - **F1's fix carries two bounds the finding did not name**, both surfaced by review: an identity-machinery change is never attributable (it recomputes every `finding_id`, so the pair is not fix-comparable at all), and attribution requires a recorded applied-set the design doc does not yet name. Phase 6 owes that capture; until it exists P2 abstains on operator-applied routed findings rather than claiming them. ## Verification - Two independent fresh-context audit rounds with rationale withheld. Round 1 found the shipped contract I had missed and the parallel-vocabulary problem; round 2 found nine defects in the revision, of which the ones this diff caused are fixed here (lane-digest scope falsifying Assertion 5.2, the judging-configuration clause sitting in P4's consequent, the sweep-version attribution hole, the orphaned containment half, an over-reaching user-scope claim, and PLAN.md collateral). - `.worktreeinclude` verified empirically, not by reading: `git ls-files -o -i --exclude-from` intersected with `git check-ignore` in a throwaway repo. Before: nested sub-slice files silently dropped. After: carried, with baselines and raw scratch still excluded. The tracked file now matches the documented block in `docs/conventions/topic-docs/README.md` byte for byte. - Gates: `check-changed-skills.sh origin/main` PASS (0 errors, 0 warnings — a pre-existing warning cleared as a side effect), `check-changelog-parity.sh` `--check` / `--check-order` / `--check-bump` all clean, markdownlint clean, `evals.json` valid. ## Known gaps, recorded not fixed - The design doc and the shipped contract now diverge in **both** directions: the design has `sweep version`, the shipped has `behavior-affecting arguments` and an `observable detection version` (a form it adopted after finding the design's triple unestablishable from outside a delegate plugin). Recorded in `PLAN.md` as Phase 8 reconciliation work, because changing either predicate changes what the shipped pass reports. - The design doc's new basis inputs land as prose without named assertions. So do the four incumbent inputs — this is a pre-existing structural gap, not one this diff introduces. - `PLAN.md` marks Phases 6, 8 and 9 TODO while `audit-pass` ships a more advanced run contract. Implementation ran ahead of the plan; flagged, not reconciled here. No linked issue ## Related - Refs #1322 — five stranded threads (findings 1-5) - Refs #1425 — three stranded threads (findings 6-8) - Adopts vocabulary from `plugins/claude-config/skills/audit-pass/reference/run-contract.md` --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 12, 2026
…ckRollup running-check pitfall (#2513) ## Summary Lands the two PR-lifecycle facts #1436 recorded from driving #1393/#1316/#1318/#1322 to merge, in the document that was prescribing the expensive path: - **Stale-branch recovery now defaults to merge-forward.** `monitor.md`'s mergeable pre-check (§3.1 step 1) and stale-branch recovery rule (§3.2) both prescribed "force-push with lease" — but auto-mode permission classifiers commonly deny force-push, and the recorded consequence was a fresh branch + fresh PR + closing the old one per rebase (the #1315 → #1377 → #1393 churn, with every review thread re-opened on the successor). Merging the default branch *into* the PR branch resolves staleness and pushes **fast-forward** — no force-push — and under a squash-only default branch the merge commits collapse to one commit on merge, so linear-history requirements stay satisfied. Verified in the issue's own record: #1393 landed that way and #1318 was merge-forwarded five times without needing a new branch. Rebase stays available as the exception for projects requiring a linear PR branch where force-push is actually permitted. - **`statusCheckRollup` reports a running check as `conclusion: ""` (empty string), not `null`.** The complement-shaped filter (`conclusion != null and != "SUCCESS"`) therefore counts every in-progress check as a failure — the exact misreport in the issue (two "failing" checks that were simply still running). The multi-PR scan section (§3.0.6, the one place this skill reads `statusCheckRollup`) now documents the pitfall with value-positive jq selectors for "failed" and "still running". Version `0.53.11` → `0.53.14` (patch; `0.53.12`/`0.53.13` are claimed by in-flight PRs #2450/#2453/#2483/#2510 and #2469 — skipping past them per the #1746 collision pattern). ## Test plan - `npx markdownlint-cli2@0.23.2` on both edited markdown files — 0 issues. - Docs-only change to skill reference text; no scripts or hooks touched. The jq forms added are the ones from the issue, verified against `gh pr view --json statusCheckRollup` semantics. ## Related Fixes #1436 Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
cursor Bot
pushed a commit
that referenced
this pull request
Aug 12, 2026
…ckRollup running-check pitfall (#2513) ## Summary Lands the two PR-lifecycle facts #1436 recorded from driving #1393/#1316/#1318/#1322 to merge, in the document that was prescribing the expensive path: - **Stale-branch recovery now defaults to merge-forward.** `monitor.md`'s mergeable pre-check (§3.1 step 1) and stale-branch recovery rule (§3.2) both prescribed "force-push with lease" — but auto-mode permission classifiers commonly deny force-push, and the recorded consequence was a fresh branch + fresh PR + closing the old one per rebase (the #1315 → #1377 → #1393 churn, with every review thread re-opened on the successor). Merging the default branch *into* the PR branch resolves staleness and pushes **fast-forward** — no force-push — and under a squash-only default branch the merge commits collapse to one commit on merge, so linear-history requirements stay satisfied. Verified in the issue's own record: #1393 landed that way and #1318 was merge-forwarded five times without needing a new branch. Rebase stays available as the exception for projects requiring a linear PR branch where force-push is actually permitted. - **`statusCheckRollup` reports a running check as `conclusion: ""` (empty string), not `null`.** The complement-shaped filter (`conclusion != null and != "SUCCESS"`) therefore counts every in-progress check as a failure — the exact misreport in the issue (two "failing" checks that were simply still running). The multi-PR scan section (§3.0.6, the one place this skill reads `statusCheckRollup`) now documents the pitfall with value-positive jq selectors for "failed" and "still running". Version `0.53.11` → `0.53.14` (patch; `0.53.12`/`0.53.13` are claimed by in-flight PRs #2450/#2453/#2483/#2510 and #2469 — skipping past them per the #1746 collision pattern). ## Test plan - `npx markdownlint-cli2@0.23.2` on both edited markdown files — 0 issues. - Docs-only change to skill reference text; no scripts or hooks touched. The jq forms added are the ones from the issue, verified against `gh pr view --json statusCheckRollup` semantics. ## Related Fixes #1436 Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The design record behind #1315, #1316, and #1318 — the decomposition of the source article, the proportionality gate that decided what to build, and the contracts the shipped code implements.
No linked issue.
What it decides
Seven candidate detectors (D1–D7) went through a proportionality gate. One survived as a new check — cross-surface instruction conflict, the only finding with no incumbent. Every other candidate became an edit to a check that already owns its surface, or an explicit exclusion with a recorded reason. The gate's own working is here, including the parts that went against the first answer.
The corrections are the point
This branch documents four occasions where the effort asserted "nothing covers this" and was wrong each time, because nobody read the body of the incumbent —
skill-quality:check,mcp-tools:audit,claude-memory's C6 (on the deliverable's own primary check, after the failure mode had been named), and an eval runner that already existed as working code. Counts had the same disease. The standing rule that came out of it — derive counts by command, never transcribe them — is recorded with the instances that produced it.Independent verification
A second derivation of the same source article was made deliberately blind to this one, and the two were compared. They substantially agree: the blind reader independently reached "one detector with no incumbent, everything else an edit inside the plugin that already owns the surface", and independently routed its own new rule to the same catalog at the same check number. Six findings moved a disposition and are recorded, along with seven things this derivation found that the blind one missed. Where they genuinely diverge — the blind derivation ratified "no new router" — the disagreement is recorded rather than resolved in this work's favour, with the reconciliation left open as a row due before Phase 9.
A convention conflict this PR cannot resolve on its own
docs/conventions/topic-docs/placesdocs/topics/<slug>/in the contract tier: "committed on the task branch only; pruned before merge." This PR does not prune, because fifteen topic directories are already onmain, including one merged today. Practice and the written convention disagree wholesale, and this branch follows practice rather than silently picking a side.Reviewer call: merge as-is (consistent with every prior topic branch), or prune first (consistent with the written convention, which would then need every existing topic directory explained). Either way the convention and practice should stop disagreeing.
Task #37 was resolved on the assumption the prune is real, and that work stands regardless — the shipped catalog carries its own source URLs and recheck triggers rather than pointing into this tree, so nothing in #1316 or #1318 dangles whichever way this goes.
Also in here
The prompt-injection threat model the design entirely lacked —
audit-passreads instruction surfaces that are attacker-controlled text in any repository the operator did not write. Plus the re-run contract, the determinism tiering, the seam resolution, and the naming record foraudit-pass(32 candidates, five lenses, with what the chosen name costs recorded so it is not re-litigated).Rebased onto current
main, including there-anchor→disciplineplugin rename from #1276.Related
audit-passskill🤖 Generated with Claude Code