New fleet skill proposal: local-strict-review — adversarial pre-PR self-review on a stronger model

[ptr727]

Motivation

From pieter's personal PR-reviewer evaluation (not a fleet doc, kept outside this repo): PR
review-round volume across the fleet's active repos is high — this pass measured mean roundtrips
of 3-6 and mean rework commits of 1.5-4.6 per substantive PR across ProjectTemplate,
Financial-Modeling, and HomeAutomation-Config. The pattern behind it, in pieter's words: a local
coding agent finishes a unit of work, judges it ready, opens the PR; CodeRabbit/Qodo find real
issues the local agent didn't catch; the agent fixes, judges it ready again; another round finds
more. This is a catch-22 — review volume is high because first-pass PR quality is low, and low
first-pass quality partly persists because review-round churn eats the time that would go into
getting it right the first time.

One concrete lever that doesn't require a blanket model swap for all work: run one focused,
adversarial review pass against the staged diff, on a stronger model than whatever wrote the
code, immediately before opening the PR — the same idea as running local lint before push (already
separately in progress fleet-wide), but for logic/design defects instead of style.

Proposed skill: local-strict-review (name negotiable)

Trigger: before opening a PR — i.e. after git add on a completed unit of work, before
gh pr create. Should slot into the same "local checks before push" position as the in-progress
pre-commit lint hook, not replace it.

What it does: dispatches a subagent on a stronger model (Opus, or whatever the fleet's
strongest available model is at the time — this shouldn't hardcode a model ID that goes stale)
against git diff --staged, with read access to full surrounding files (not just the diff hunks —
this evaluation pass found real bugs specifically in the gap between incremental and full-context
review, see below), instructed to find what a strict external reviewer would flag: correctness
bugs, edge cases, unguarded type coercions, TOCTOU/race conditions, platform-specific behavior
differences, cross-file consistency issues. Read-only — reports findings, does not fix.

Starting point for the subagent definition (.claude/agents/strict-review.md shape, to be
adapted to however the hub packages a fleet skill vs. a personal one — see the note on
skill-lifecycle below):

---
name: strict-review
description: Adversarial review of staged changes — flags what a strict reviewer would catch, read-only
model: opus
tools: Read, Grep, Glob, Bash
---

You are reviewing this repo's currently staged changes (`git diff --staged`) as the
strictest reviewer who will ever see this code. Read the diff, then read full
surrounding files where the diff alone doesn't give enough context — cross-file
consistency issues are exactly the kind of thing incremental review misses.

Flag: correctness bugs, edge cases, unguarded type coercions, TOCTOU/race conditions,
platform-specific behavior differences, and anything that contradicts this repo's own
conventions. Do not fix anything — report findings only, file:line, with the concrete
failure scenario for each.

Why "before PR" and not "replace the model for the whole task": cost. A targeted second pass
on the finished diff is a small fixed cost against the alternative of multiple CodeRabbit/Qodo
review rounds burning PR-hosted rate-limit budget (CodeRabbit is capped at 10 reviews/hour
account-wide) and wall-clock on a pushed PR. Switching every task to a stronger model outright
wasn't the fix this evaluation's data pointed to — a dedicated pre-PR review step was more directly
targeted at the actual failure mode (what the local agent checks before declaring "done", not
which model wrote the code).

Evidence this specific shape (full-diff, not incremental) matters: this evaluation pass found
the identical structural pattern twice on ProjectTemplate (#1040→#1041→#1042 and
#1045→#1046→#1047) — a fresh review of the accumulated diff at promotion time caught real bugs
that incremental per-push review during development had missed. The same blind spot applies to a
local agent judging its own work "ready" from having tracked its own incremental changes.

Implementation notes for whoever picks this up

• Should follow the skill-lifecycle skill's source/generated split
  (.agents/skills/ source, .github/skills/ + .claude-plugin/ generated via
  scripts/build_dist.py) rather than being hand-authored only as a loose .claude/agents/*.md
  file, if it's meant to ship fleet-wide.
• Model should probably be configurable/discoverable rather than hardcoded to a specific model ID
  that will go stale — open question for whoever implements.
• Should be advisory, same posture as local lint currently is — not a hard gate that blocks
  git commit/gh pr create, at least initially.

Related follow-on work (not part of this issue — flagging for visibility, may deserve separate issues)

• pr-review-conduct / drive-pr should chain this in explicitly: before opening a PR, run
  (1) local lint (already separately in progress), (2) local-strict-review, and — once
  CodeRabbit graduates from advisory evaluation to standard fleet tooling (see the personal
  evaluation doc's First-Class Support Criteria) — (3) CodeRabbit's own headless CLI
  (coderabbit review --uncommitted --agent) as a third local pre-PR check. Same "catch it before
  it's a PR round" principle for all three, escalating from cheap/mechanical (lint) to structural
  (strict-review) to reviewer-parity (CodeRabbit CLI).
• CodeRabbit/Qodo findings outside resolvable threads get missed by coding agents in practice
  (confirmed, not hypothetical — pieter reads these himself but agents routinely don't): CodeRabbit
  posts real findings as "outside diff range" comments that never become reviewThreads, and Qodo
  posts all of its actual findings as separate PR comments (its formal review body is always
  empty) with its own inline ✓ Resolved/✗ Dismissed self-tracking badges. pr_review.py
  already has dedicated logic for Copilot's equivalent easy-to-miss surface (its collapsed
  "Suppressed comments" section) — CodeRabbit and Qodo need the same treatment once either is
  in standard fleet use, so an agent's own workflow can't silently skip these.

[ptr727]

Confirms this gap firsthand (Financial-Modeling, 2026-08-28): asked Claude Code to "use the local
strict review skill" to review a just-committed local change set (config + Python changes, not yet
in a PR). No such skill currently exists in this repo or the hub — .github/skills/code-review/SKILL.md
is the only review skill carried, and its runbook (.github/copilot-instructions.md's "GitHub
Copilot Review Runbook") is scoped strictly to a GitHub PR diff read via scripts/pr_review.py. It
has no mechanism for reviewing git diff --staged or any local, not-yet-pushed diff.

Worth noting: code-review's own Skill-tool-surfaced description ("Reviews a pull request or
change set... Use this whenever asked to review code, a pull request, a patch, or a proposed
change") reads as if it already covers local/staged review, which is exactly what led to trying
it here instead of recognizing the skill didn't exist yet. Once local-strict-review ships, that
description should probably be tightened to say PR-only, so the two skills' scopes don't overlap
in a user's (or an agent's) mind.

Workaround used: invoked code-review anyway, with an explicit branch-diff target (commit vs.
origin/develop) described in the invocation prompt, since the skill has no way to determine its
own diff source outside a PR context. It ran fine against that manually-specified target and found
two real issues (five stale dollar-amount references left in a JSON config's notes after the
amounts themselves were bumped, and one small code-duplication finding) — so the underlying review
logic clearly works, it's purely the "what diff am I reviewing" input surface that code-review
doesn't have and local-strict-review is proposed to add.

+1 for this proposal. One addition for whoever implements: the model-selection question in the
issue body ("configurable/discoverable rather than hardcoded") should probably also account for
sessions where a stronger model than the one currently running isn't actually available/selected —
this session ran on Sonnet 5, not Opus, with no indication either way whether a stronger model was
reachable.