Skip to content

docs(playbooks): generalize instance lessons + add lesson-framing guard#159

Merged
phuongnse merged 1 commit into
mainfrom
docs/generalize-practice-playbooks
May 31, 2026
Merged

docs(playbooks): generalize instance lessons + add lesson-framing guard#159
phuongnse merged 1 commit into
mainfrom
docs/generalize-practice-playbooks

Conversation

@phuongnse

@phuongnse phuongnse commented May 31, 2026

Copy link
Copy Markdown
Owner

Summary

Makes the practice docs reusable and adds a mechanism so they stay that way — the recurring problem was incident lessons getting baked into general rules at the moment they were learned, which only fit the originating use case.

Generalize (3 over-fit spots):

  • agent-checklist.md — replace the organization-management incident gap-list with the general principle; drop a register → settings → delete example and an embedded user-quote from the Wrong/Right table.
  • testing.md — reframe "tenant isolation vs async provisioning" as the general rule "keep deterministic request-path tests separate from async-pipeline tests" (dropped the platform-foundation framing and PG-specific diagnosis).

Prevent recurrence (two layers):

  • docs-style.md — new "Keep practice docs general" section + anti-pattern row: principle goes in the playbook, instance specifics go in the use-case/PROGRESS.md/PR retro, concrete instances appear only as labeled examples. Completes the loop on Gate 3's existing "Incident-level detail in rule text?" question.
  • scripts/check-doc-drift.sh — narrow incident/lesson-framing guard over docs/playbooks/*, CLAUDE.md, ARCHITECTURE.md. Deliberately high-precision (flags the recurring "Lesson"-callout class; verified it fires on a probe, ignores the Gate-3 meta-line and legitimate example [...] pointers). Wired into the agent-checklist CI-gates list.

Reference examples meant to be copied (register-org layout, repo-layout config, patterns.md code) are intentionally specific and left as-is.

Linked spec

Process/playbook change — no use-case AC.

Requirements

  • Three over-fit lessons generalized (agent-checklist x2, testing x1)
  • docs-style.md authoring rule + anti-pattern row
  • Narrow drift guard, counterexample-verified (fires on violation, silent on clean tree, no false positive on Gate-3 line / example pointers)
  • ./scripts/check-doc-drift.sh green

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Refined development playbooks for backend completion criteria and self-audit timing.
    • Updated documentation standards to separate reusable principles from incident-specific details.
    • Clarified testing best practices for deterministic vs asynchronous test scenarios.
  • Chores

    • Added automated validation to enforce documentation framing standards.

@coderabbitai

coderabbitai Bot commented May 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f636fa5a-4724-4875-901d-6ef98eb7bdd3

📥 Commits

Reviewing files that changed from the base of the PR and between 3a84e52 and f894b37.

📒 Files selected for processing (4)
  • docs/playbooks/agent-checklist.md
  • docs/playbooks/docs-style.md
  • docs/playbooks/testing.md
  • scripts/check-doc-drift.sh

📝 Walkthrough

Walkthrough

Documentation governance is refined to require playbook sections state reusable general principles rather than incident-specific lessons. A new enforcement guard detects "Lesson" callouts, and existing playbook sections on agent completion criteria and test separation are generalized to follow the new pattern.

Changes

Documentation governance and playbook generalization

Layer / File(s) Summary
Playbook doctrine and guidance
docs/playbooks/docs-style.md
A new rule clarifies that playbook practice docs must state general, reusable principles; incident/lesson specifics belong in use-case files, PROGRESS.md, or PR retros. New "Keep practice docs general" subsection includes a comparison table, a validation test, and a reference to the new doc-drift enforcement guard.
Incident/lesson framing enforcement guard
scripts/check-doc-drift.sh
Adds a new guard that scans docs/playbooks/, CLAUDE.md, and docs/ARCHITECTURE.md for "Lesson"/"Lesson (" callout patterns and fails the check with guidance to generalize and move lesson specifics to use-case/PROGRESS.md/retro materials.
Agent checklist: backend completion and self-audit guidance
docs/playbooks/agent-checklist.md
Replaces a specific "Lesson" example about missed implementation details with generalized guidance on shipping main CRUD/flow endpoints before marking the backend layer complete, and emphasizes validation/edge/cross-cutting behavior coverage (TTL, rollback, cleanup, cancel, isolation) before the first PR push. Adjusts the "Doc drift" bullet placement in the CI-only gates list.
Testing guidance: deterministic vs async-pipeline separation
docs/playbooks/testing.md
Replaces tenant-isolation-vs-async-provisioning guidance with a generalized rule: keep synchronous request-path tests separate from async-pipeline/eventually-consistent tests. Specifies that request-path tests must set up and assert preconditions synchronously, async tests target the coordinator/handler directly, and shared helpers must avoid long polling on background pipelines.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • phuongnse/axis#128: Further refines docs/playbooks/agent-checklist.md's Gate 0 backend sign-off guidance and self-audit timing, building on earlier checklist and pre-push audit additions.
  • phuongnse/axis#48: Updates docs/playbooks/agent-checklist.md and establishes scripts/check-doc-drift.sh enforcement for documentation gates and checklist compliance.
  • phuongnse/axis#30: Introduces the testing playbook that is now refined by this PR's generalized deterministic-vs-async-pipeline guidance.

Poem

🐰 A playbook must teach, not lament the past,
Keep lessons in PRs—let principles last.
No "Lesson" in docs, just rules that ring true,
Enforce with a guard, and the governance shines through.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: generalizing lessons in documentation and adding a guard to prevent lesson-specific framing in practice docs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/generalize-practice-playbooks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Make practice docs reusable and keep them that way:

- Generalize three over-fit spots so they apply to any use case:
  agent-checklist (drop the organization-management incident gap-list and a
  register/settings/delete + embedded-quote example) and testing.md (reframe
  "tenant isolation vs async provisioning" as the general rule "keep
  deterministic request-path tests separate from async-pipeline tests").
- docs-style.md: add "Keep practice docs general" — principle in the playbook,
  instance specifics in the use-case/PROGRESS/retro; instances only as labeled
  examples. Adds the matching anti-pattern row.
- check-doc-drift.sh: narrow incident/lesson-framing guard (flags "Lesson (…)"
  / "**Lesson" callouts in playbooks + CLAUDE.md + ARCHITECTURE.md), wired into
  the CI-only gates list. Closes the loop on Gate 3's existing
  "Incident-level detail in rule text?" question.

Reference examples meant to be copied (register-org layout, repo-layout config,
patterns.md code) are intentionally specific and left as-is.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@phuongnse phuongnse force-pushed the docs/generalize-practice-playbooks branch from 90a13d0 to f894b37 Compare May 31, 2026 05:53
@phuongnse phuongnse merged commit abadb83 into main May 31, 2026
8 checks passed
@coderabbitai coderabbitai Bot mentioned this pull request Jun 17, 2026
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant