fix(playbooks,docs-hygiene): correct two caching claims against the live pages - #1920
Merged
Conversation
… surface that owns them Roster row 16 (Prompt caching, API) verified two families of caching claim against the live pages. Both failed, in opposite directions: one stated a harness mechanism that its own modal case defeats, the other cited the API page for a claim whose consumers are Claude Code sessions. playbooks 0.6.14 -> 0.6.15. fable-5's orchestration chapter argued that continuing an oriented worker is cheap because "accumulated context is a cache read rather than a re-derivation". Subagents "use the five-minute TTL even on a subscription", so a worker resumed after a fan-out wave longer than five minutes re-writes its whole context at the five-minute cache-write rate -- "1.25 times the base input tokens price" -- not the cache-read rate. The recommendation stands; its reason is now the re-derivation saved, not the tokens. docs-hygiene 0.9.4 -> 0.9.5. extract-ssot's anti-pattern #9 was "Cache invalidation cascade", and decision-framework carried the same dead mechanism in two places: test #3's rationale, cited to the API prompt-caching page glossed as "cache TTL hinges on stability", and KEEP-INLINE row C. The API page says no such thing and is the wrong surface: this skill scopes to tracked markdown consumed by Claude Code sessions, where editing an always-loaded file mid-session "does not invalidate the cache, but the edit also doesn't apply", and sequential sessions "share the prefix only when the git status snapshot at startup matches" -- that snapshot "captures branch and recent commits", so committing the edit breaks sharing exactly as any other commit does and SSOT edit frequency is not a distinct driver. The real cost is propagation, and it still earns a gate: #9 is now "Always-loaded SSOT propagation lag" -- a correction lands while every running session keeps following the superseded version until its next /clear, /compact, or restart. Test #3 and row C now state that mechanism. The pattern is scoped to CLAUDE.md and UNSCOPED rules files, since rules with paths: frontmatter "load later, when Claude first reads a matching file" and a pre-load edit "does take effect". A scope fence keeps API-surface caching separate, where prefix volatility genuinely costs an Agent SDK fleet sharing one prefix across machines. Slot 9 is preserved so the by-number citations to #10-#13 stay intact. Sources verified 2026-08-04, fetched as raw markdown: platform.claude.com/docs/en/build-with-claude/prompt-caching (152,223 B, MD5 99507693631240bea1d10fc75f07c78f) and code.claude.com/docs/en/prompt-caching (29,721 B, MD5 3c33a5402e05a5e18ea8c5177780bbea). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Claude finished @kyle-sexton's task in 2m 30s —— View job Code Review
Verdict: no blocking findings. This is a well-scoped, well-verified fix. I re-fetched both live doc pages this session (
Other things checked and clean:
One minor, non-blocking observation: No code/logic to review here beyond markdown — didn't find architecture, error-handling, or test-coverage concerns since this PR only touches documentation content and versioning. |
|
Claude finished @kyle-sexton's task in 50s —— View job Security review — PR #1920No security issues found. This PR is documentation-only: two markdown skill/context files ( Checked and confirmed out of scope for this review:
Nothing here touches trust boundaries, permissions, or data flow. Clean from a security-review standpoint.
|
|
Claude has reviewed this PR 2 times. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 16ecc8fe32
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…t mitigation Codex finding: the "goes back inline" branch is a no-op when the original home was itself always-loaded - inline in CLAUDE.md or an unscoped rule carries the same propagation lag the mitigation exists to reduce. The split now names late-enough-loading targets (skill body, paths:-scoped rule, on-demand file) and states when inline genuinely helps. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 36s —— View job Code Review — PR #1920
Verdict: no blocking findings. This synchronize re-review picks up from the two earlier passes on this PR (general code review + security review, both clean). The only change since then is d08a720, which fixes the one open item: Codex's P2 finding that the volatile-split mitigation's "goes back inline" branch could leave content on the same always-loaded surface it was extracted from, undoing the extraction without curing the propagation lag that motivated it. Confirmed the fix lands correctly at
This is the right fix — it doesn't just gate the new target, it also disqualifies the "return to inline" path when the source was itself always-loaded, which is exactly the case Codex's example (content originally inline in Re-checked for regressions from this one-line-scoped change:
Couldn't run |
|
Claude finished @kyle-sexton's task in 52s —— View job Security review — PR #1920No security issues found. Reviewed the full diff (
Checked and confirmed out of scope for this review:
The latest commit ( Nothing in this diff touches trust boundaries, permissions, or data flow. Clean from a security-review standpoint.
|
kyle-sexton
added a commit
that referenced
this pull request
Aug 4, 2026
…fleet acts on (#1922) ## Summary Doc-alignment roster row 17: **How Claude Code uses prompt caching** (the harness half of the caching pair; page unmoved vs row 16's capture — MD5 re-confirmed three times). Most of the page was already verified-in-use this session (I17-b's authority chain, row 16's sweep); this row closed the residue. **claude-ops 0.27.0** — the observability skill stored the cache half of the cost signal but never surfaced it: `cc_metrics` has always split token usage into input/output/cacheRead/cacheCreation, yet no report section rendered it. Now shipped: - A **Cache health** report section — deliberately its own OTEL-sourced section rather than new columns on the ccusage-sourced Token/cost table, fixing a grain mismatch *and* a provenance mismatch in one move (per-model windowed data vs a session-scoped query; OTEL tokens vs ccusage billing). - A new per-model windowed query in `otel-queries.md`, **verified by execution** against a live store (returns real per-model rows; the live data even demonstrates the diagnostic — an inverted read/creation ratio on short-session Haiku subagents). Deliberately hot-tier-only: the cold-tier union was built, tested, and **fails** with an IO error on any fresh install lacking cold parquet — the constraint is documented at the point of use rather than shipping a query that breaks for most consumers. - The upstream reading ("a high read-to-creation ratio means caching is working well…") in-fence as report copy; the **reported-never-graded** rule outside the skeleton fence with only the non-derivable rationale (upstream publishes direction, no threshold — a severity band here would be an invented number cited as sourced). - A Cache-health routing row distinguishable from the existing historical-token row. **`docs/PLUGIN-PHILOSOPHY.md`** (one line, no bump per precedent) — the component-stances MCP-servers row gains its second adoption cost, previously omitted: the only component type that can cost a consumer their prompt cache — with the load-bearing deferred-tools qualifier (the cache survives when the server's tools are deferred, the default on supported models), conclusion-plus-pointer, no enumeration. The row's Verified column deliberately not advanced (it dates the MCP link, which was not re-verified; the new claim carries its own inline stamp). Residue dispositions with evidence: 7 sections verified-this-session, 3 already satisfied in-repo, the rest declined on the no-decision-turns-on-it test (provider table, TTL env-var enumeration, worktree cache scope, tool-deny mid-session cost, compaction mechanics). Recorded for the roster: the session-flow handoff "fork" naming collision with the harness feature of the same name (skill defines its term inline; nothing false). ## Test plan - markdownlint 0 errors; changelog parity all three modes; manifest presence; skill portability; check-changed-skills PASS. - Producer-side independent reviewer (rationale withheld) found 2 blockers + 1 major on the first commit — a rendering directive inside the report skeleton's fence, an unfillable-at-grain data path, an overstated routing-gap claim — all fixed, with the fill-path fix going beyond the reviewer's options after the producer found the underlying provenance mismatch. - Orchestrator-commissioned Fable verifier over the final state (which no prior review had covered end-to-end): live page fetch, fence-boundary check, **query extracted verbatim and executed read-only against the live OTEL store**, cold-union failure reproduced first-hand, provenance separation judged, philosophy-row accuracy and table integrity, claim accuracy against origin/main — **7/7 PASS, empty defect list**. ## Related - No linked issue. - Doc-alignment loop, roster row 17. Predecessors: #1908–#1920. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 4, 2026
…ss accepts but declines to honor (#1923) ## Summary Doc-alignment roster row 18 — the priority head's final row: **Model configuration** (code.claude.com/docs/en/model-config), the corpus's most-cited harness page, first formally captured here (82,975 B, MD5 `5c9e06c5…`) with a **semantic-stability proof**: normalized token streams of the live page and the frozen 2026-07-31 snapshot are identical (10,222 tokens each, control-probed) — retroactively confirming seven earlier rows' model-config-dependent work ran against an unchanged page. **claude-config 0.21.7 / criteria 1.15.0**: - The audit skill advertised settings auditing yet had zero coverage of the model-configuration keys. New **category H (Model and effort settings)** — four rows, each a value the harness accepts into a settings file and then declines to honor: `effortLevel: max/ultracode` (not accepted there), `fallbackModel` beyond three after dedup (with the raw-`maxItems:3` schema test reported separately — the two tests genuinely diverge, verified empirically), a specific entry disabling its family wildcard (including the Mantle-ID and custom-model-option arms), and `enforceAvailableModels` against an unset/empty list (rated **error** per the skill's own enforcement-bypass rubric). Visibility stated per row from the page's own text (row 3's substitution notice; row 2's documented silence). Category H is wired into the skill's phase flow: "Eight categories", a Category H section, and a MANDATORY Phase 3.3 live model-config fetch gating every H finding. `settings.local.json` routed through the safe-read path. - **criteria I17's carve-out justification corrected** (1.15.0): it claimed the schema makes `effortLevel: max` "unreachable" in settings — false (JSON Schema is advisory; the harness reads a violating file). The directive is unchanged; the justification now states the true reason and points at the sibling that owns the file-level check, making the two halves of one plugin cohere. - Loop-lane alias stamp refreshed in place (prescribed by that convention's own no-drift rule; claim re-verified against today's bytes). CLOSE-2 re-confirmed discharged. The roster's named component (dotfiles pin audit) declined — machine-scope, not this repo's to own. Recorded upstream: schemastore's `effortLevel` description omits Opus 5/Sonnet 5 from its effort-support list — stale against the live page; their fix, flagged because this PR now cites that schema. ## Test plan - markdownlint 0 errors; changelog parity all modes; `instruction-scan.test.sh` 46/46; `conflict-scan.test.sh` 41/41; scripted quote fidelity (6 literals exact, 3 near-miss control probes absent); schema constraints script-verified including the no-`uniqueItems` fact the divergence rests on. - Two full independent passes converged: the producer's reviewer (found the blocking category-letter collision that made the section unreachable, plus 8 accuracy/coherence defects — all fixed) and an orchestrator-commissioned Fable verifier (upstream fidelity, the stability-proof reproduction, IA-10-boundary and quote-vs-cite adjudications, then a delta re-verify of the fix round: **every item PASS, empty defect list**, both its residual observations non-blocking). - Producer self-caught one defect pre-review (a sorted dedup misidentifying dead chain entries) and corrected one of its own premises when tested (A–F never quote upstream; the quote call stands on the consumer-repo ground instead, disclosed as a deliberate departure). ## Related - No linked issue. - Doc-alignment loop, roster row 18 — completes the 18-row priority head. Predecessors: #1908–#1920, #1922. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 4, 2026
…and its fences (#1924) ## Summary Doc-alignment roster row 241: **Choosing a Claude model and effort level in Claude Code** (claude.com blog) — the post the owner's model-routing lane and two main-tree surfaces cite for the model-vs-effort dial discrimination. First custody baseline taken (article-body-only hash, so site chrome can't fake drift). The roster's premise was corrected en route: the post was *not* "digested nowhere" — planning's session-config already carried a faithful two-knob digest; what it lacked was custody and the post's own qualifiers. **planning 0.27.3** — the "Two orthogonal knobs" section gains: - Its citation, led by the strongest ground available: model-config's own delegation sentence ("For guidance on which model and effort level fit different kinds of work, see [the post] on the blog") — the reference page points *at* the post; the narrowed negative is the supporting ground: no reference page states the try-versus-know **diagnostic** (choosing-a-model *orders* the levers — "Tuning effort is often a better lever than switching models" — and the effort page pairs effort against prompting; ordering a lever is not diagnosing which failure you have). - The post's **context-first prior step** neither knob had: on a wrong answer, examine the provided context before touching a dial — sharpened here because the step is this skill's own product (the Brief *is* the context fix). - Two missing fences: the discriminator is "a starting point, not a hard rule" (the post's figure caption, disclosed as such), and raising effort is sharpest below the model's default. - The ambiguity signal: ambiguity the rounds could not retire argues up; a Brief precise enough to execute from argues down. Both pre-existing main-tree citations verified holding against the live post. Owner-facing findings routed to the dotfiles ledger (not this repo's to edit): the CLAUDE.md lane's dropped "clearly tried" (making the two branches overlap), the Fable-reservation grounds contradiction, the missing context-first step — with one earlier finding partially withdrawn after the verifier surfaced the choosing-a-model source. ## Test plan - Docs-only; markdownlint 0 errors; check-skill PASS identical to base; scripted quote fidelity across four live sources (post, effort, model-config, choosing-a-model) — after the producer caught and fixed its own circular self-match, all spans verbatim with one disclosed editorial elision. - Independent fresh-context Fable verifier (rationale withheld, 8 binary criteria incl. an adversarial test of the load-bearing negative): 7/8 PASS; its one FAIL (the negative stated too broadly) fixed with its prescribed wording, the fix cross-verified verbatim against both live pages before amending. ## Related - No linked issue. - Doc-alignment loop, roster row 241. Predecessors: #1908–#1920, #1922, #1923. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Aug 4, 2026
…ance and the C9 presence check (#1925) ## Summary Doc-alignment roster row 236 — **Building verification loops in Claude Code with skills** (the DOC-37 row: the roster's densest component-opportunity row, named in the campaign's completion criteria). Live post verified unmoved on every load-bearing point (three converging checks + a firecrawl body diff; minor additive drift only). **Zero new skills were minted — deliberately.** The slice's own Target typing makes two of the seven "skill candidates" rule rows and one a doctrine chapter; three shipped as content on surfaces that already own the shape, four deferred with a build-on-demand trigger (all four zero-instance by grep; one already correctly served by the shipped verification-surfaces table). **playbooks 0.6.16** — new skill-authoring spoke `reference/verification-loops-in-skills.md`: - Three skill-creation routes ranked with the documented dependency-free direct-ask first (the platform page's own "you don't need a 'writing skills' skill" disclaimer), the skill-creator plugin second on needs-no-install grounds — with creation *including its interview flow* correctly credited as first-party documented (README + the plugin's own SKILL.md "Interview and Research" phase). - The **conditional bare-resolution rule**, stated accurately: a plugin skill's bare name also resolves *unless another command claims it* (skills.md's own sentence, v2.1.216+ noted); write the namespaced form because it is unconditional — not because bare fails. This corrects an inference the campaign's frozen EC-4 record had closed wrongly (its cited evidence was about collision, not resolution; its live test was never run) — **EC-4 should be treated as superseded**, and rows that consumed its answer re-checked. - The embed-debugging checklist leading with the documented diagnosis (prominence/wording, then unfollowed references, tested on real work not test scenarios), the blog's description-diagnosis as a clearly-marked second hypothesis; the shadowing note with its true replace-not-extend semantics (the blog omits shadowing entirely); the three senses of "chaining" disambiguated. **claude-memory 0.5.3 / criteria 1.4.0** — new criterion **C9**: project CLAUDE.md build/test-command presence, FAIL-headed with branches per the file's own convention, an explicit C7 boundary (C7 owns references; C9 owns commands; never double-report), sourced quotes added to official-guidance.md so the update action can't orphan it — and a **step-0 false-positive guard** the producer added against its own citation: memory.md's table puts "Build commands" in the *auto-memory* column, so commands reachable on any loaded surface route to C3 placement, not a C9 WARN, with the counter-evidence disclosed in the Why. Eval 8 exercises all three regression-prone behaviors. ## Test plan - check-changed-skills 2 skills 0 failed; markdownlint 0 errors; changelog parity + order; manifests/catalog/contracts/portability all pass; eval 8 schema-validated. - Two full independent passes ran to convergence. The producer's reviewer falsified **three successive coverage-negatives** ("bare doesn't resolve", "blog alone", "stated by neither") — each corrected against a wider primary-source fetch, and the lesson (phrase coverage claims positively) applied across the artifact. The orchestrator-commissioned Fable verifier independently fetched every load-bearing source (including the upstream skill-creator SKILL.md to adjudicate the final conflict), validated the no-skills-minted call against the slice's own typing, and pinned the final SHA: **verified end to end, no content defects**. ## Related - No linked issue. - Doc-alignment loop, roster row 236 (DOC-37). Predecessors: #1908–#1920, #1922–#1924. Campaign record note: EC-4 (EMPIRICAL-CHECKS-RESULTS-2026-08-02) is superseded by this PR's documented conditional-resolution rule; the frozen record stays frozen, the supersession is recorded on the roster. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Doc-alignment roster row 16: Prompt caching (API) — first captures of both the API page (152,223 B) and its harness sibling (29,721 B), with a hard surface-scoping discipline (API vs harness caching semantics are different products' claims). Seven repo surfaces stating caching facts swept; five verified clean and left alone (including boris's $12.50/$1 figures — confirmed contextually right for subagent orchestration, where the five-minute TTL governs); two carried real defects:
playbooks 0.6.15 —
orchestration.md's continue-an-oriented-worker rationale claimed "accumulated context is a cache read". Wrong in the chapter's own modal case: the harness page states subagents build their own cache and use the five-minute TTL even on subscription, so a worker resumed after a longer fan-out wave re-writes its whole context at the five-minute cache-write rate ("1.25 times the base input tokens price"), not a cache-read rate. The recommendation stands; the reason is now the re-derivation saved (a replacement pays the same tokens plus the rediscovery tool turns), with the TTL and pricing anchors cited.docs-hygiene 0.9.5 —
extract-ssot's anti-pattern #9 ("Cache invalidation cascade") rested on a mechanism that is dead on the skill's own declared surface: the harness page states mid-session edits of always-loaded files keep the cache (the edit just doesn't apply), and cross-session sharing keys on the git-status snapshot, which any commit breaks. Rewritten in place as "Always-loaded SSOT propagation lag" — corrections ship that live sessions don't see until/clear//compact/restart — with a scope fence for the API surface (where prefix volatility genuinely costs an Agent SDK fleet), scoped to unscoped rules files (path-scoped rules load lazily; pre-load edits apply), and slot 9 preserved because #10–#13 are cited by number in eight places. The dead vocabulary survives only in the changelog, quoted as removed.Routed, not acted on: the owner's dotfiles CLAUDE.md caching claim verified correct with one additive omission (fast mode is a third cache-key element) — recorded for dotfiles routing; the API page's explicit-default-effort no-invalidate row flagged as a future I17-b enrichment parallel.
Test plan
Related
🤖 Generated with Claude Code
https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X