feat(playbooks,claude-memory): ship the DOC-37 verification-loop guidance and the C9 presence check

[kyle-sexton]

Summary

Doc-alignment roster row 236 — Building verification loops in Claude Code with skills (the DOC-37 row: the roster's densest component-opportunity row, named in the campaign's completion criteria). Live post verified unmoved on every load-bearing point (three converging checks + a firecrawl body diff; minor additive drift only).

Zero new skills were minted — deliberately. The slice's own Target typing makes two of the seven "skill candidates" rule rows and one a doctrine chapter; three shipped as content on surfaces that already own the shape, four deferred with a build-on-demand trigger (all four zero-instance by grep; one already correctly served by the shipped verification-surfaces table).

playbooks 0.6.16 — new skill-authoring spoke reference/verification-loops-in-skills.md:

• Three skill-creation routes ranked with the documented dependency-free direct-ask first (the platform page's own "you don't need a 'writing skills' skill" disclaimer), the skill-creator plugin second on needs-no-install grounds — with creation including its interview flow correctly credited as first-party documented (README + the plugin's own SKILL.md "Interview and Research" phase).
• The conditional bare-resolution rule, stated accurately: a plugin skill's bare name also resolves unless another command claims it (skills.md's own sentence, v2.1.216+ noted); write the namespaced form because it is unconditional — not because bare fails. This corrects an inference the campaign's frozen EC-4 record had closed wrongly (its cited evidence was about collision, not resolution; its live test was never run) — EC-4 should be treated as superseded, and rows that consumed its answer re-checked.
• The embed-debugging checklist leading with the documented diagnosis (prominence/wording, then unfollowed references, tested on real work not test scenarios), the blog's description-diagnosis as a clearly-marked second hypothesis; the shadowing note with its true replace-not-extend semantics (the blog omits shadowing entirely); the three senses of "chaining" disambiguated.

claude-memory 0.5.3 / criteria 1.4.0 — new criterion C9: project CLAUDE.md build/test-command presence, FAIL-headed with branches per the file's own convention, an explicit C7 boundary (C7 owns references; C9 owns commands; never double-report), sourced quotes added to official-guidance.md so the update action can't orphan it — and a step-0 false-positive guard the producer added against its own citation: memory.md's table puts "Build commands" in the auto-memory column, so commands reachable on any loaded surface route to C3 placement, not a C9 WARN, with the counter-evidence disclosed in the Why. Eval 8 exercises all three regression-prone behaviors.

Test plan

• check-changed-skills 2 skills 0 failed; markdownlint 0 errors; changelog parity + order; manifests/catalog/contracts/portability all pass; eval 8 schema-validated.
• Two full independent passes ran to convergence. The producer's reviewer falsified three successive coverage-negatives ("bare doesn't resolve", "blog alone", "stated by neither") — each corrected against a wider primary-source fetch, and the lesson (phrase coverage claims positively) applied across the artifact. The orchestrator-commissioned Fable verifier independently fetched every load-bearing source (including the upstream skill-creator SKILL.md to adjudicate the final conflict), validated the no-skills-minted call against the slice's own typing, and pinned the final SHA: verified end to end, no content defects.

Related

• No linked issue.
• Doc-alignment loop, roster row 236 (DOC-37). Predecessors: feat(playbooks,claude-config): align Fable 5 doctrine and the instruction audit with the live prompting guide #1908–fix(playbooks,docs-hygiene): correct two caching claims against the live pages #1920, feat(claude-ops,philosophy): surface the two prompt-cache facts this fleet acts on #1922–docs(planning): give the two-knob digest its source, its prior step, and its fences #1924. Campaign record note: EC-4 (EMPIRICAL-CHECKS-RESULTS-2026-08-02) is superseded by this PR's documented conditional-resolution rule; the frozen record stays frozen, the supersession is recorded on the roster.

🤖 Generated with Claude Code

https://claude.ai/code/session_019gaVX25Txd6GXdiu9HEH3X

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[claude]

Claude finished @kyle-sexton's task in 2m 42s —— View job

---

Review: PR #1925

• Gather context (diff, changed files)
• Understand request (code review, CLAUDE.md alignment)
• Review verification-loops-in-skills.md (playbooks)
• Review skill-authoring/SKILL.md changes
• Review criteria.md / official-guidance.md (claude-memory C9)
• Review evals.json, changelogs, plugin.json version bumps
• Post final review

This PR is documentation/reference content (no executable code), so I focused on internal consistency, whether the shipped procedure actually matches its own eval and cross-references, and CLAUDE.md alignment. Security review is out of scope for this lane.

Should fix

C9's step 0 short-circuit contradicts eval 8's expected behavior. criteria.md#L219-L222:

0. First ask whether the commands are stated on another loaded surface — a nested CLAUDE.md, a
   path-scoped rule, or auto memory. If they are, this is a C3 placement question, not a C9
   finding: ... Do not WARN for absence, and go no further in this check

Read literally, "go no further in this check" terminates the whole procedure the moment commands are found reachable elsewhere — before step 1 (find stated commands), step 2 (verify against the manifest), or step 3 (FAIL on a wrong command) ever run.

But evals.json#L89-L99 (eval 8, shipped in this same PR) is built exactly on the case where commands are reachable elsewhere (auto memory) and CLAUDE.md also states a wrong one (pnpm run compile, which package.json doesn't define). The expected output requires step 3's FAIL to still fire for the wrong command — only the absence-WARN (step 4) should be suppressed. As written, step 0's "go no further" reads as gating the entire check, not just step 4, so a model executing the steps in order would plausibly stop at step 0 and never reach the FAIL that eval 8 requires.

This matters here specifically because SKILL.md's determinism contract (SKILL.md#L62-L67) says judgment-tier checks like C9 must apply "fixed criteria" even though wording varies — an ambiguous step ordering undermines that guarantee for the one check that most needs it, since it's the check testing for wrong/missing content rather than surplus.

Suggested fix: scope step 0's short-circuit explicitly to the WARN-for-absence branch, e.g. "...Do not WARN for absence (step 4). Steps 1–3 (verify correctness of anything CLAUDE.md itself states) still apply regardless." Fix this →

Minor nit

Changelog wording could be read as contradicting criteria.md's own boundary statement. claude-memory/CHANGELOG.md#L15-L17 describes the FAIL branch as "C7's wrong-reference class," but criteria.md's own C7 boundary note (criteria.md#L232-L234) explicitly says "A wrong build command is not a C7 finding today, because a command is none of the three things C7 checks." The changelog is presumably using "C7's wrong-reference class" loosely as an analogy (stale/incorrect reference, same shape as C7's currency checks) rather than claiming C7 ownership, but it reads as tension with the criteria file it's summarizing. Worth a tighter phrase, e.g. "the same wrong-reference shape C7 checks for elsewhere, but owned here by C9."

What looks solid

• C9's applicability updates are consistently threaded through SKILL.md (C2-C8 → C2-C9), context/audit.md (adds the CLAUDE.local.md carve-out), criteria.md, and official-guidance.md (sourced quotes match the determinism contract's requirement that criteria trace to official docs).
• Version bumps are correct and match changelog headers: claude-memory 0.5.2→0.5.3, criteria.md 1.3.0→1.4.0, playbooks 0.6.15→0.6.16 — all consistent with CLAUDE.md's "Versioned" rule.
• C9's severity heading [FAIL] for a check with both FAIL and WARN branches correctly follows the existing C6/C7 convention (heading a check at its higher branch) — verified against both.
• The C3/C9 boundary (placement vs. presence-and-correctness) and the C7/C9 boundary (references vs. commands) are each stated once, with an explicit no-double-report rule, matching this repo's general preference for deterministic, non-overlapping checks over prose guidelines.
• verification-loops-in-skills.md is scoped as a plugin-authoring spoke (per plugin design rules: repo-agnostic, no hardcoded paths) and correctly separates vendor-claimed content (marked and dated) from documented harness/platform behavior — appropriate given this file falls under the fresh-docs mandate's contract-surface scope.
• No plugin-form violations spotted: no hardcoded paths, no ../ reach-outs, references are all relative within the plugin tree.

[claude]

Claude finished @kyle-sexton's task in 50s —— View job

---

Security Review — complete

• Gather context / fetch PR diff (git diff origin/main...HEAD)
• Review changed files for security issues
• Post final findings

Scope reviewed: all 11 changed files — two plugin.json version bumps, two CHANGELOG.md additions, evals.json, criteria.md, official-guidance.md, SKILL.md (audit + skill-authoring), context/audit.md, and the new reference/verification-loops-in-skills.md.

Findings: none. This PR contains no code, no GitHub Actions workflow changes, no hooks, no .mcp.json entries, and no settings/permissions changes. Every changed file is prose Markdown, a JSON eval fixture (test prompts/expected-output strings, not executable), or a plugin.json version/description bump. There is no code-execution surface, no secret/token/credential handling, no injection surface (command, SQL, path, template), and no authorization or trust-boundary logic in scope for this diff.

The one file with any "instructional" content that a future agent might act on is plugins/playbooks/skills/skill-authoring/reference/verification-loops-in-skills.md — I checked it for prompt-injection-style directives (e.g., instructions telling an agent to exfiltrate data, run arbitrary commands, or disable safety checks) and found none; it's a documentation reference about skill-creation routes, namespacing, shadowing vs. chaining, and embed-failure diagnosis, consistent with its stated purpose.

No security issues to report for this PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f361dfa129

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

Claude has reviewed this PR 2 times. The lane skips further automatic reviews after 5; deleting this comment resets the count.

[claude]

Claude finished @kyle-sexton's task in 59s —— View job

---

Security Review — complete

• Gather context (gh pr diff / git diff origin/main...HEAD, plus the incremental diff from the latest commit c3dd94d)
• Review changed files for security issues
• Post final findings

Scope reviewed: all 11 files changed in this PR (two plugin.json version bumps, two CHANGELOG.md additions, evals.json, criteria.md, official-guidance.md, SKILL.md for both the audit and skill-authoring skills, context/audit.md, and the new reference/verification-loops-in-skills.md). This invocation re-checked the delta introduced by the latest commit specifically — f361dfa..c3dd94d, which edits only criteria.md (narrowing C9's step-0 carve-out so it suppresses just the absence-WARN, not steps 2-3) and verification-loops-in-skills.md (naming per-plugin owning diagnostics for the non-surfacing case) — both in response to the Codex/human review comments above.

Findings: none.

• No code, no GitHub Actions workflow changes, no hooks, no .mcp.json entries, and no settings/permissions changes anywhere in the diff.
• Every changed file is prose Markdown, a JSON eval fixture (test prompts/expected-output strings, not executable), or a plugin.json version/description bump — no code-execution surface, no secret/token/credential handling, no injection surface (command, SQL, path, template), and no authorization/trust-boundary logic in scope.
• The latest commit's changes are pure wording edits to existing reference docs (tightening a check's scoping language and adding install-aware routing text) — same content class as the rest of the PR, no new surface introduced.
• Checked verification-loops-in-skills.md again for prompt-injection-style directives (instructions telling an agent to exfiltrate data, run arbitrary commands, or disable safety checks) — none found; the new text just names alternate slash-command diagnostics conditioned on plugin installation, which is descriptive routing prose, not an executable instruction to this agent.

No security issues to report for this PR.

[claude]

Claude finished @kyle-sexton's task in 1m 21s —— View job

---

Review: PR #1925 (re-review after push)

• Gather context (diff, changed files, prior review comments)
• Verify the two Codex/prior-review findings against the fix commit
• Check for any other issues introduced by the fix
• Post final review

Commit c3dd94d addresses both P2 findings from the earlier review round (mine and Codex's). Verified each against the actual diff:

1. C9 step 0 short-circuit — confirmed fixed

criteria.md#L219-L223 now reads:

...this is a C3 placement question, not a C9 finding for ABSENCE: do not WARN that CLAUDE.md omits them. The carve-out suppresses only the absence branch — any command CLAUDE.md itself still states goes through steps 2-3 regardless, because a stale stated command misleads whether or not a correct one exists elsewhere

This is scoped correctly now: the short-circuit only kills step 4's absence-WARN, and steps 2–3 (verify + FAIL) explicitly still run. I checked it against eval 8 (evals.json#L89-L99) — pnpm run compile FAILs, the bare "we use pnpm and vitest" is flagged as tool-naming-without-invocation, and the absence-WARN is correctly suppressed since the real commands are reachable via auto memory. All three expectations in the eval are now satisfiable by a literal read of the procedure. The Why note and the C7 boundary section directly below (unchanged) remain internally consistent with this wording.

2. Non-surfacing routing in verification-loops-in-skills.md — confirmed fixed

verification-loops-in-skills.md#L101-L108 now routes by cause rather than prescribing a single diagnostic for every case: description mismatch → /skill-quality:check, budget-drop → /claude-config:audit, listing-consultation habit → /discipline:use-your-skills — each marked "if installed." This resolves the original problem (a playbooks-only consumer previously had no working link) without overclaiming what any one plugin's skill covers.

Scope check on the fix commit

git show c3dd94d --stat confirms the commit touches exactly these two files (5 and 9 lines respectively) — no unrelated changes, no version/changelog bumps needed since the content shipped in this same PR (not a released version). Nothing new introduced by the fix regressed the "solid" items from the first review pass (C2–C9 threading, version bumps, [FAIL] heading convention, C3/C7/C9 boundaries).

Not addressed (by design — was a non-blocking nit)

My earlier minor nit about plugins/claude-memory/CHANGELOG.md's "C7's wrong-reference class" phrasing was flagged as optional wording tightening, not a correctness bug, and wasn't touched by this commit — reasonable to leave as-is; it doesn't affect the shipped criteria file's actual boundary rule, which is unambiguous.

No new findings. Both Codex P2s are resolved and internally consistent with their own eval/cross-references.