Skip to content

perf(pr-management-code-review): select, scan and dispose as code, one file per outcome - #1585

Merged
potiuk merged 1 commit into
perf/pr-mgmt-7-stack-reviewfrom
perf/pr-mgmt-8-code-review
Oct 10, 2026
Merged

potiuk merged 1 commit into
perf/pr-mgmt-7-stack-reviewfrom
perf/pr-mgmt-8-code-review

Conversation

@potiuk

@potiuk potiuk commented Oct 10, 2026

Copy link
Copy Markdown
Member

Summary

  • The code-review skill loaded ~38k tokens per run and had the agent compute by hand: the review queue (the five "my reviews" signals and every selector), CODEOWNERS matching, the slop signals and threshold, the security and AI-disclosure scans, licence and artifact checks, the dependency-constraint ledger, the disposition, reviewer suggestions, inline-comment anchoring and the review payload.
  • All of that now runs in pr-management code-review … over reads saved with vetted-op-read --save. The agent reads the diff, writes the findings and judges severity. A run loads ~13.6k tokens, plus only the classification and criteria docs a PR actually needs.
  • Each selector, finding category, verdict, slop outcome, footer variant and skip reason has its own document under classifications/ or criteria/.

Fixes found on the way:

  • "Touching mine" uses GitHub data, not git log --author against a login.
  • CODEOWNERS team ownership checks team membership, not org membership.
  • The three contradictory APPROVE definitions collapse to one: nothing above a nit, real CI green, no unresolved threads or standing change request, no open question.
  • maintain permission gets the maintainer footer.

Type of change

  • Skill change — eval fixtures updated below
  • Python package

Test plan

  • prek run --all-files passes
  • tools/pr-management pytest / ruff / mypy
  • 13 model-graded suites (70 cases) are now pytest cases named after each former case
  • The 15 judgement suites stay model evals: 55/56 on the first run. The one miss, review-handoff/case-3-re-draft-edit, passed on two re-runs of the suite and on main, so it is nondeterministic, not a regression.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5). I reviewed the diff and the test results; commits carry Generated-by:.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv

@potiuk potiuk added family:pr-management pr-management-* skills capability:review Deep per-item code review or contributor mentoring labels Oct 10, 2026
@potiuk
potiuk added this pull request to stack #1586 October 10, 2026 09:23
@github-actions github-actions Bot added family:setup setup-* skills family:tools tools/* family:docs Docs, MISSION.md, READMEs substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:sandbox Tool substrate: agent isolation, egress control, settings audit substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 10, 2026
@potiuk
potiuk force-pushed the perf/pr-mgmt-8-code-review branch from 780dfb6 to 73e30c8 Compare October 10, 2026 09:46
@potiuk
potiuk force-pushed the perf/pr-mgmt-8-code-review branch from 73e30c8 to 63ad744 Compare October 10, 2026 10:18
…e file per outcome

The code-review skill loaded ~38k tokens per run and had the agent
compute, by hand, the review queue (the five "my reviews" signals and
every selector), CODEOWNERS matching, the slop signals and threshold,
the security and AI-disclosure scans, licence and artifact checks, the
dependency-constraint ledger, the disposition, reviewer suggestions,
inline-comment anchoring and the review payload. All of that now runs
in `pr-management code-review …` over reads saved with `vetted-op-read
--save`; the agent reads the diff, writes the findings and judges
severity. A run loads ~13.6k tokens, plus the classification and
criteria docs a PR actually needs.

Fixes found on the way: "touching mine" uses GitHub data, not
`git log --author` against a login; CODEOWNERS team ownership checks
team membership, not org membership; the three contradictory APPROVE
definitions collapse to one (nothing above a nit, real CI green, no
unresolved threads or standing change request, no open question);
`maintain` permission gets the maintainer footer.

Each selector, finding category, verdict, slop outcome, footer variant
and skip reason has its own document under `classifications/` or
`criteria/`. 13 model-graded suites (70 cases) become pytest cases named
after each case; the 15 judgement suites stay model evals.

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the perf/pr-mgmt-8-code-review branch from 63ad744 to 2b05020 Compare October 10, 2026 11:16
@potiuk
potiuk merged commit a7189e1 into main Oct 10, 2026
12 checks passed
@potiuk
potiuk deleted the perf/pr-mgmt-8-code-review branch October 10, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:review Deep per-item code review or contributor mentoring family:docs Docs, MISSION.md, READMEs family:pr-management pr-management-* skills family:setup setup-* skills family:tools tools/* substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:framework-dev Tool substrate: build / validate / eval the framework itself substrate:sandbox Tool substrate: agent isolation, egress control, settings audit

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant