chore: CLAUDE.md learnings from PR review (week of 2026-05-22)

[thadeusb]

Summary

Automated analysis of 17 PRs merged 2026-05-08-2026-05-22.
Processed 25 review comments (25 human, 0 bot).
Identified 4 recurring failure modes for CLAUDE.md.

Feature Flags

None - documentation-only change.

New Patterns

CLAUDE.md

• [mistake]: Before adding helper logic, grep for adjacent helpers with the same responsibility and delegate instead of duplicating membership checks or fixture factories. (context: duplication|helpers|tests|shell)

  • PR feat(plan-validate): auto-sync markdown answers to JSON #62 (mikeangstadt): _base_data duplicated the existing make_minimal_data fixture factory.
  • PR FEA-860: Verify user-scoped ClosedLoop plugins #86 (mikeangstadt): mark_plugin_failed duplicated the array-membership loop already in plugin_failed.

• [mistake]: When changing run-loop or self-learning resume/rate-limit logic, preserve persisted state contracts: iteration rows, session IDs, success counts, and terminal statuses. (context: run-loop|resume|state|rate-limit)

  • PR fix(self-learning): CLOSEDLOOP_ITERATION env var as fallback, not override #73 (shafty023): repeated run rows caused later iterations to be scored with the first logged iteration.
  • PR PLN-511: fix(code): fail loop when max iterations reached with no progress #74 (shafty023): resumed loops lost successful iteration counts and session IDs at max-iteration exit.
  • PR PLN-530: Restrict rate-limit detection to "rejected" status #79 (peterulsteen): narrowed rate-limit detection dropped terminal non-rejected rate states.

• [mistake]: Guard boundary data before narrowing filters or status checks; failed reads, malformed entries, and defaulted fields can invert success/failure handling. (context: invariants|filters|error-handling)

  • PR feat(plan-validate): auto-sync markdown answers to JSON #62 (mikeangstadt): filtering openQuestions called .get() on possible non-dict entries.
  • PR FEA-860: Verify user-scoped ClosedLoop plugins #86 (mikeangstadt): failed final plugin-list reads marked every plugin as failed.
  • PR PLN-555: feat(judges): error resilience and agent registry pre-flight validation #87 (mikeangstadt): error CaseScores could be included when error_reason defaulted to None.

Amended Patterns

• CLAUDE.md Documentation & Versioning:
  • Old: Never reference code, APIs, or schema fields that don't exist in the codebase or documented spec. Before writing CHANGELOG "Fixed" or "Replaces" entries, verify the prior implementation exists via grep. Before adding manifest fields, verify they're in the documented schema.
  • New: Never hand-write CHANGELOG entries for plugin changes; when reviewing generated release notes, verify the version heading matches plugin.json and every Fixed/Replaces claim maps to behavior that existed before the PR.
  • PR FEA-859 Add plugin remediation support #85 (mikeangstadt): CHANGELOG.md was manually edited despite repo convention.
  • PR FEA-860: Verify user-scoped ClosedLoop plugins #86 (mikeangstadt): the changelog version label did not match plugin.json.
  • PR PLN-555: feat(judges): error resilience and agent registry pre-flight validation #87 (mikeangstadt): changelog text fabricated a prior implementation.

Deduplicated

• Test assertion/env isolation - already covered by the Testing pattern about sibling assertion coverage and ambient env vars.
• Misleading comments/docstrings - already covered by the Code Quality pattern about auditing adjacent comments and docstrings.
• Test helper duplication alone - already covered by the Testing helper-extraction pattern; the new Code Quality pattern covers the broader production-helper recurrence.
• Security override routing/audit-log comments - only appeared in PR feat(code): add per-user security override mechanism #88, below the 2-PR frequency threshold.

Stats

• PRs analyzed: 17
• Comments processed: 25 (25 human, 0 bot)
• New patterns: 3
• Amended patterns: 1
• Deduplicated: 4

Verification

• Reviewed generated diff.
• Confirmed only CLAUDE.md changed.