docs(philosophy): pass (b) doctrine currency — correct two model/effort claims, record upstream corroboration - #2184
Conversation
…rt claims, record upstream corroboration Pass (a) asked whether the repo's claims about Claude Code's BEHAVIOR were current. This pass asks whether the doctrine those gates enforce is still the right rule against current authoritative guidance. Every source was asked to state its own position; no claim of supersession is made without a verbatim quote from the page itself. Two corrections. Effort tiers: the model ladder's bottom rung is Haiku 4.5, and the effort lane rules told a bulk mechanical sweep to pin `low` there. Those two ladders do not compose. model-config's effort table says "Models not listed here do not support effort" and lists no Haiku; the model roster corroborates independently with adaptive thinking off for Claude Haiku 4.5. This is not the documented unsupported-level fallback, because no supported level exists to fall back to — the pin is inert, and silently. Nothing in the fleet pins `haiku` today, so the defect was doctrinal rather than live. availableModels: the blocked-pin paragraph recorded the fallback branch as "unresolved upstream, and stated here as such", and derived a worst case from `enforceAvailableModels`' Default-resolution ordering. Upstream now resolves it outright, per surface and differently for each: a blocked subagent override falls back to the inherited model, except that a blocked family alias on the Anthropic API substitutes to the newest permitted version of its family (a v2.1.222 change the page dates); a blocked skill or command override is ignored and runs on the session model. The conclusion survives — a blocked subagent alias can still land below the session, and a blocked cheap pin lands on the session's own model — but the basis and the recheck trigger were pointing at a sentence that no longer carries the ambiguity. The skill/command branch is now bounded and is stated separately, which the old paragraph lumped together. Recorded corroboration, where doctrine that stood on its own reasoning is now stated upstream in upstream's words: - Fresh-eyes checkpoints. best-practices now says a fresh context "improves code review since Claude won't be biased toward code it just wrote" and that a verification subagent exists "so the agent doing the work isn't the one grading it". Its "Add an adversarial review step" section also names a failure this document did not carry — a reviewer prompted to find gaps "will usually report some, even when the work is sound" — so the inline-template conventions gain a fourth bullet bounding what counts as a finding. - Naming. The gerund-deviation sentence characterized the upstream guidance instead of quoting it; it now quotes it. Gerunds are what that page says to "consider using", `process-pdfs`-style action-oriented names sit under "Acceptable alternatives", and "inconsistent patterns within your skill collection" is what it lists under Avoid. - Setup contract. `disable-model-invocation: true` matches upstream's own rule for the flag. The rest of the contract has no upstream position, and now says so plainly rather than sitting adjacent to cited material. - Design boundary. The vague appeal to "Claude Code's distinction" now carries the citation and the two phrases the distinction is actually drawn in. Stamp refreshes, only on pages actually fetched this session: sub-agents, model-config, code.claude.com best-practices, plugins, agent-skills best-practices, platform thinking, models overview. Also recorded: `CLAUDE_CODE_SUBAGENT_MODEL` has an `inherit` off position since v2.1.196, which the unqualified "overrides both" claim missed; and the model-tier recheck trigger re-tested negative — Claude Mythos 5 exists upstream but "is not generally available". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdBsorqGjCU4Mg5nrmgz4m
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Claude finished @kyle-sexton's task in 4m 14s —— View job Review complete ✅
SummaryThis is a docs-only change to
I could not independently run the PR's listed verification commands ( Finding (minor — doc-currency stamp consistency)docs/PLUGIN-PHILOSOPHY.md#L751 — the citation to AssessmentNo correctness bugs found. The two substantive doctrine corrections (the |
|
Claude has reviewed this PR 3 times. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1616fe6900
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…sweep Review caught an inconsistency: the `interleaved_thinking` capability-value citation still read 2026-08-03 while two other citations to the same model-config page were bumped to 2026-08-10 in the same diff. The section was in fact covered by this session's fetch and both halves of the claim still hold — "`interleaved_thinking` | Thinking between tool calls" is present in the capability table, and "listed capabilities are enabled and unlisted capabilities are disabled for the matching pinned model" is verbatim. Refreshing rather than annotating, because the check was genuinely run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdBsorqGjCU4Mg5nrmgz4m
|
Claude finished @kyle-sexton's task in 2m 50s —— View job Review complete ✅
SummaryDocs-only change to I independently re-fetched every upstream page this diff newly cites or re-quotes —
FindingOne inline comment posted on the effort-tiers section (docs/PLUGIN-PHILOSOPHY.md#L850-L854): the new sentence "the pin is inert rather than downgraded, and silently so" is an inference from the absence of upstream documentation, not a verbatim-quoted fact like the rest of the diff. I re-fetched AssessmentNo other correctness issues found. The two substantive doctrine corrections in this PR (the |
|
Thanks — the stamp finding is correct and is fixed in 358710a. You were right that it was a miss rather than a deliberate omission. The On the verification commands you couldn't run: fair to treat as author-claimed from your side, but they are independently reproduced by CI on this PR — One incidental note for anyone reading the diff later, since it touches the same section: that capability table is also the reason the effort/Haiku correction is scoped to the model rather than to every Haiku deployment. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…verified Review caught the one sentence in this PR that broke its own method rule. The effort/Haiku correction asserted that a `low` pin on a model with no effort support "is inert rather than downgraded, and silently so". The cited pages establish the missing capability, not the runtime handling: model-config's fallback sentence — "If you set a level the active model does not support, Claude Code falls back to the highest supported level at or below the one you set" — presupposes some supported level exists, and upstream is simply silent on the zero-support case. That was an inference from doc silence stated in the same declarative voice as the verbatim-quoted claims around it, which is exactly what "no verbatim quote, no claim" is meant to prevent. The finding itself is unaffected, because it never depended on the gap: the capability is absent either way, so a lane wanting the cheapest tier takes it by model and omits the pin. The passage now says the absence is documented and the handling is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FdBsorqGjCU4Mg5nrmgz4m
|
Claude finished @kyle-sexton's task in 2m 57s —— View job Code review complete ✅
SummaryDocs-only change (+83/-31, one file). No plugin manifest, hook contract, or marketplace schema is touched, so CLAUDE.md's fresh-docs mandate doesn't formally gate this file — but the PR's own methodology ("no supersession without a verbatim quote") is the higher bar, and I held it to that. Prior finding, independently re-checked as fixed. Two earlier automated reviews (and a matching Codex comment) flagged that the original effort-tiers text asserted the zero-supported-level pin behavior as settled fact ("inert…and silently so") when upstream only documents the fallback-between-supported-levels rule, not the zero-level case. I re-read the current text at docs/PLUGIN-PHILOSOPHY.md#L853-L859 (commit 80bd47d) and confirm it's now correctly hedged: "What the harness then does with the pin — ignore it, warn, or fail — is undocumented, and unverified here." The recommendation ("omit the pin, take the tier by model") is preserved. This resolves the open finding. Independent verification of the new/changed citations. Rather than re-trust the PR body's or prior reviews' claims, I fetched the live pages myself this session and grep'd/read the relevant sections:
Every quote I checked was accurate and in context. I found no new correctness issues in this diff. Not independently re-run: the PR's listed verification commands ( AssessmentNo correctness bugs found, no new inline findings to raise. The doctrine corrections ( |
#2187) No linked issue ## Summary The repo carries **327 dated verification stamps** (`verified` / `fetched` / `as of` 2026-MM-DD), spanning 2026-07-15 → 2026-08-08. PR #2169 re-verified only the ~8 claims a gate or check skill directly enforces. The rest had gone unread for up to four weeks — which is exactly the drift a dated stamp exists to surface. A stamp nobody re-reads is worse than no stamp: it advertises a currency the claim does not have. This PR re-verifies **72 stamps** against the live upstream pages and reports honestly on the 49 upstream claims it did not reach. ### The method change that makes this pass different Earlier passes fetched docs through `WebFetch`, which summarizes through a small model and **truncates long pages**. Under that route "the page doesn't say that" is a routine false negative, so no negative claim can be settled and no drift classification is safe. This pass reads **`https://code.claude.com/docs/en/<page>.md`**, which returns the complete raw markdown source (`content-type: text/markdown`), and greps it: | page | raw bytes | | :-- | --: | | `hooks.md` | 249,026 | | `settings.md` | 290,489 | | `changelog.md` | 525,429 | | `sub-agents.md` | 96,094 | | `plugins-reference.md` | 90,982 | Every CONFIRMED claim below is backed by a verbatim `grep -F` match against the full page. Two **negative** claims are now assertable for the first time, because the whole page was in hand rather than a truncated slice: - `docs/conventions/hook-observability/README.md` — no native "verbose hooks" toggle exists. - `plugins/claude-memory/skills/stateless/reference/official-guidance.md` — no settings-precedence exception bullet names `autoMemoryEnabled`, `CLAUDE_CODE_DISABLE_AUTO_MEMORY`, or auto memory. One caution for the next person: markdown **link flattening** causes false misses. Four `sub-agents` quotes appeared to have vanished until inspection showed the only difference was `[fork](#fork-the-current-conversation)` becoming `a fork`. All four were verbatim-intact. Grep on a link-free fragment. ## Fix ### Drift found and corrected (3) **1. `scripts/check-silent-skips.sh` — exit-0 stderr is not "discarded entirely".** The gate comment claimed exit-0 hook stderr is "discarded entirely". The hooks reference says: > Stderr from a hook that exits 0 goes to the debug log only, never the transcript, and Claude never sees it. The debug log is the one place it survives. **The gate's conclusion is untouched** — a debug-only sink is not a visibility surface, so a `>&2`-only notice on an exit-0 skip path is still invisible to both user and agent. The comment now states what upstream states, and records the correction so the next reader does not re-derive it. **2. `context-guard` — the 2.1.132 token-semantics floor lost its upstream source.** `reference/reader-contract.md` quoted the statusline page as saying *"Before v2.1.132 these were cumulative session totals"*. That sentence **is no longer on the page**, and the version number went with it. What the page still states is only the present-tense semantics the floor depends on — "Token counts currently in the context window, from the most recent API response". The dead quote is replaced with an explicit sourcing-status note; `scripts/statusline-tee.sh` carries the same note at its `cli_version` comment. **The floor itself is deliberately unchanged.** `TOKEN_SEMANTICS_MIN_VERSION` still gates the token shape at `>= 2.1.132`; no behaviour, test, or zone result moves. Dropping it could only *widen* which payloads the token shape trusts, and the misfire it guards against is silent — a pre-2.1.132 cumulative 170k in a 200k window reads as a perfectly plausible current occupancy and resolves `dumb` while the live context may be smart-zone. It stays as a conservative lower bound, now labelled as retained-not-doc-backed. Re-source before any change that relaxes it. **3. Both `statusline-shim.sh` copies cited a plugins-reference section that no longer exists.** The 14-day orphaned-cache grace period was attributed to a section titled *"Plugin cache and file access"*. Upstream renamed it to **"Plugin caching and file resolution"** (cache root `~/.claude/plugins/cache`). The behaviour cited is unchanged and still stated verbatim — but a citation whose section title has vanished cannot be followed, which is precisely the rot a dated stamp exists to expose. Both comments now name the current title and record the former one. ### Scope correction (1) `docs/PLUGIN-PHILOSOPHY.md` — the "Pages load-bearing for this document" stamp covered seven bullets, one of which is a `melodic-software/standards` file, not a Claude Code page. The refreshed stamp now says so explicitly instead of implying the whole list was re-checked. The six Claude Code pages were each re-fetched and confirmed to still carry the topics named beside them (`Plugin manifest schema`, `Version management`, `Plugin caching and file resolution`, `Persistent data directory`, `Hooks in skills and agents`, `Tag plugin releases for version resolution`, `pluginConfigs` read scopes). ### Confirmed and re-dated (no claim changed) `docs/PLUGIN-PHILOSOPHY.md` · `docs/conventions/hook-observability/README.md` · gate/hook/script comments in `check-silent-skips.sh`, `post-compact-mark.sh`, `record-rate-limit-stop.sh`, `packet-seal.sh`, `destructive_guard.py`, `skill-reference-verify.sh`, `markdown-format.sh`, `typos-format.sh`, `check-structure.sh`, both `statusline-shim.sh` · and skill bodies/references across `claude-config`, `claude-memory`, `claude-ops`, `context-guard`, `discipline`, `disk-hygiene`, `guardrails`, `implementation`, `plugin-quality`, `rate-limit-guard`, `session-flow`, `work-items`. The densest single block, `session-flow/skills/orchestrate/context/sources.md` (13 stamps), was re-checked quote by quote against `sub-agents`, `workflows`, `changelog`, and `whats-new/2026-w32`. All still match. Its 0.22.2 finding is independently re-confirmed: the sub-agents page now states outright *"There's no limit on the total number of subagents Claude can spawn over a session"*, and `CLAUDE_CODE_MAX_SUBAGENTS_PER_SESSION` appears nowhere on it. ## Verification ### Coverage — all 327 stamps accounted for **The first version of this table was wrong, and review caught it.** It was built from a single-line regex, `(verified|fetched|as of) 2026-[0-9]{2}-[0-9]{2}`. This repo hard-wraps prose, so a stamp whose keyword ends one line and whose date begins the next is invisible to it — **30 stamps were hidden that way**, and the denominator I published (291) was itself understated. The wrap-aware pattern is: ``` (verified|fetched|as of)\s*(?: \s*(?:[#>*-]\s*)?)?(2026-\d{2}-\d{2}) ``` Buckets below are disjoint and sum to the total; nothing is hidden in a residual. | Bucket | Count | Treatment | | :-- | --: | :-- | | **Dated 2026-08-10 (current)** | **104** | 72 refreshed by this PR after verbatim confirmation; 32 already current from PR #2169, #2184 and other in-flight work | | CHANGELOG entries | 69 | **Left alone.** A CHANGELOG entry describes what was true when written — re-dating it would falsify the record | | Upstream Claude Code doc — **not re-checked** | **49** | Honest gap. See below | | Repo-internal claims (no external page) | 47 | Out of scope for a docs-currency pass — recorded gate runs, this repo's own probes | | Non-upstream sources | 25 | `platform.claude.com`, `claude.com/blog`, `github.com`, `git-scm.com` — different corpus | | `docs/topics/**` | 29 | **Left alone.** Records of what was true when written; Contract tier, gated | | `env-vars`-cited | 4 | Skipped by assignment — a parallel agent owns that route | | **TOTAL** | **327** | | The brief estimated ~260; the wrap-aware regex above is the number. ### The line I drew, stated before the work started Verify, in order: (1) every `docs/PLUGIN-PHILOSOPHY.md` claim; (2) every code-comment claim inside a gate, hook, or script, since those are load-bearing for behaviour; (3) skill-body and reference claims on pages obtained in full. Deprioritize READMEs and CHANGELOGs as historical record, and leave `docs/topics/**` alone. **What that leaves unchecked: 49 upstream claims**, thinly spread — `docs/conventions/loop-lane/README.md` (8), `docs/MIGRATION-PLAYBOOK.md` (4), `docs/PLUGIN-PHILOSOPHY.md` (3, each a stamp spanning a `platform.claude.com` page alongside Claude Code pages — see below), and `playbooks/**` (9, mostly `platform.claude.com`-adjacent model-calibration notes). No other file has more than 2. These are reachable by the same method and are the natural next slice; they were cut for depth over breadth, not because they are lower risk. Three `PLUGIN-PHILOSOPHY` stamps cover several pages at once, at least one of them outside this corpus. Where I verified the Claude Code pages but not the `platform.claude.com` one under the same stamp, I left the date alone rather than refresh a stamp that also vouches for a page I did not open. The same rule produced the scope correction above. **No stamp was re-dated without a verbatim quote retrieved this session.** Every one of the 12 stamps added in the review round was already backed by a quote pulled in the original sweep; none was newly assumed. **Every stamp in a file this PR claims to have verified now carries the new date.** That was not true in the first push — review caught CHANGELOG entries asserting more than the diff delivered — and the wrap-aware regex above is what makes the claim checkable rather than assertable. Bumping a date on a claim I could not locate would launder an unverified claim into a fresh stamp — strictly worse than leaving it stale, and invisible to review. Where a claim could not be confirmed it kept its old date and sits in the 46. ### Gates (CI form, committed tree) - `bash scripts/check-contract-slice-prune.sh --check-diff origin/main` — pass - `bash scripts/check-changelog-parity.sh --check-bump origin/main` — pass - `bash scripts/check-skill-portability.sh origin/main` — pass - `npx markdownlint-cli2 <43 changed md files>` — 0 errors - `bash scripts/check-changed-skills.sh origin/main` — pass (10 skills checked, 0 failed) 14 plugins touched, each patch-bumped with a `## [x.y.z]` CHANGELOG entry. Pure-`docs/` changes carry none. No `docs/topics/<slug>/` created. ## Related - PR #2184 — landed on `docs/PLUGIN-PHILOSOPHY.md` while this was in flight; rebased onto it, and its richer model/effort corrections were kept over this PR's date-only edits at every conflict - PR #2169 — the prior audit whose ~8 gate-enforced claims are treated here as fixed points and not redone - The `.md`-endpoint route was shared with the parallel `env-vars` effort, whose truncation problem it also solves --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
No linked issue
Summary
Doc-currency pass (b), never run before. Pass (a) (#2169) asked whether the repo's claims about how Claude Code behaves were still factually current. This pass asks the harder question: is the doctrine these gates enforce still the right rule, given current authoritative guidance?
Scope was doctrine a gate, check skill, or CI script actually enforces. Every source was asked to state its own position; no supersession is claimed without a verbatim quote from the page itself.
Two things had quietly diverged, both in
### Model tiers/### Effort tiers. The rest held, and several items are now backed by upstream text that did not exist — or was not cited — when they were written.Per-item verdicts
sub-agents: "Each subagent starts with a fresh, isolated context window… The exception is a fork, which inherits the parent conversation instead of starting fresh."best-practicesnow states the doctrine too: a fresh context "improves code review since Claude won't be biased toward code it just wrote"; a verification subagent exists "so the agent doing the work isn't the one grading it"best-practices"Add an adversarial review step" names a failure the doc did not carry: "A reviewer prompted to find gaps will usually report some, even when the work is sound", and chasing all of them "leads to over-engineering"best-practicesranks a Stop hook and a verification subagent as distinct rungs of the same "give Claude a way to verify its work" ladder; nothing contradicts the narrow reachprocess-pdfssits under "Acceptable alternatives", and "inconsistent patterns within your skill collection" is under Avoidcheck+apply, closed verb setsetupskill.disable-model-invocation: trueis backed: upstream's own rule is "for workflows with side effects that you want to trigger manually"Setuphook event,SessionStart)Setupevent and its--init-only/--init/--maintenanceframing unchanged on the hooks referencecheck-skill.shcheck 4). The 200-line soft target is house convention, correctly labeled advisory. "Avoid deeply nested references" is carried, correctly attributedbest-practicestoday, unchangedtoolsfrontmatter unchanged; the Agent tool still exposesmodeland no effort counterpart (live schema, 2026-08-10)availableModelsblocked pinlowat the bulk runglow. "Models not listed here do not support effort" and no Haiku is in that table — corroborated independently by the roster's adaptive-thinking column reading No for Claude Haiku 4.5CLAUDE_CODE_SUBAGENT_MODELinheritto use normal model resolution instead" (v2.1.196)sub-agentsfrontmattereffortconfirmed with all five levels; the unsupported-level fallback sentence confirmed verbatimpluginsdraws exactly this line: standalone for "project-specific customizations", plugins for "versioned releases, reusable across projects"dependenciesand namespacing but takes no stance on optional cross-plugin coupling. Stands on its own reasoningmelodic-software/standards, not to upstream. No masqueradebest-practicesindependently asks for evidence over assertion: "Have Claude show evidence rather than asserting success"Fix
Two corrections and four recorded corroborations, all in
docs/PLUGIN-PHILOSOPHY.md. No plugin touched, so no version bump or CHANGELOG entry is owed.Corrections
haikutoday, so the defect was doctrinal, not live.availableModelsblocked pin. The "unresolved upstream" framing and theenforceAvailableModels-ordering derivation are replaced by the now-documented per-surface rule, split into its subagent and skill/command branches, which the old paragraph lumped together. The conclusion survives intact — a blocked subagent alias can still land below the session, and a blocked cheap pin lands on the session's own model — but the recheck trigger was pointing at a sentence that no longer carries the ambiguity, so it is rewritten too.Corroboration recorded
disable-model-invocationand says plainly that the rest has no upstream position.Stamps refreshed only on pages actually fetched this session:
sub-agents,model-config,code.claude.combest-practices,plugins,hooks,agent-skills/best-practices, platformthinking,models/overview.Verification
Fleet grep confirms the effort finding is doctrinal only:
git grep -n '^model:' -- 'plugins/*/agents/*.md'returns nohaikupin, and everyeffort:pin sits on anopus,sonnet, orinheritlane.Could not verify. Stamps left untouched on pages not fetched:
plugins-reference,code.claude.comskills,feature-availability,agents,agent-teams,cross-session-messaging,sessions,checkpointing,prompt-caching,settings, and the platformeffort/thinking-steering-and-costpages. The Recorded gate runs and Component stances tables therefore keep their existing dates. TheSessionStart-plus-${CLAUDE_PLUGIN_DATA}half of the native-init stamp was not re-verified againstplugins-reference, so that 2026-07-17 stamp stands even though theSetuphalf was confirmed.One upstream rule this repo does not carry, deliberately left alone as out of scope for a currency pass:
agent-skills/best-practicesasks for a table of contents in reference files longer than 100 lines. Adding it would be new doctrine, not a currency correction.Related
🤖 Generated with Claude Code
https://claude.ai/code/session_01FdBsorqGjCU4Mg5nrmgz4m