Repository navigation
Onboard dev-lead, pr-review, business-analyst, scrum-master as personas - #1303
Conversation
…onas Formalizes the three existing agents under the persona standard and makes each addressable as @petry-projects/<role> (org team handles — teams created: dev-lead, pr-review, business-analyst; all closed, notifications disabled, 0 members). Closes the manifest half of #1281 / #1282 / #1283. Each persona (all status: draft): - personas/<id>/persona.yml — a compliant manifest documenting the REAL runtime: definition layers point at what exists (dev-lead → prompts/dev-lead/; pr-review → agents/pr-reviewer.md + prompts/; business-analyst → vendored bmad-method analyst v6.8.0 + prompts/incubation/). Triggers reflect reality, advisory by default, every write surface carries its gate_label (§4 rule 2). - prompts/<id>/advisory.md — a role-focused advisory the shared persona runner serves on @-mention (read-only, prints between sentinels, marker-first — the qa-lead contract). dev-lead advises on implementation/scoping (code is still written only by its label-triggered runtime, not this path); pr-review gives review guidance (the full tiered review runs on its existing path); business-analyst advises on framing/research/next-analysis-step. - README.md + seed held-out evals (draft-exempt from the count gate; expand before promotion). pr-review points evals.paths at its EXISTING skill sets (evals/deep-review/, evals/triage/) — the multi-skill case #779 unblocks. Decisions taken (from the onboarding issues): - @-addressability for all three (the initiative's goal). pr-review's legacy @donpetry-bot path is retained (real account, works); this adds the role handle. - business-analyst is the Analyst role (brainstorm/market-research/brief) ONLY; the PRD stays with a separate product-manager persona — the documented Analyst→PM handoff is preserved, not collapsed. - pr-review's canary-rings registration (#710) stays a separate follow-up; draft personas don't require it. Deliberately NOT here (tracked, per-issue): real (non-seed) eval sets; the product-manager persona; pr-review canary registration; and — for the mention paths to go live — enrolling these into the router soak (the framework is live only in the next ring today). MERGE ORDER: after combsmith/.github#779 (schema: evals.paths) and #1302 (validator: reads paths + count gate). pr-review's manifest uses `paths`, which main's validator cannot parse until #1302 lands. Verification - validate-personas.py (with #779 schema + #1302 validator): 4/4 valid, invariants hold. - validate-cases.py: OK, 7 skills / 54 cases, no cross-split id overlap. - markdownlint: clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 31 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (15)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request formalizes three agentic personas (business-analyst, dev-lead, and pr-review) under the Agentic Persona Standard by adding their manifests, README files, evaluation cases, and advisory prompts. The review feedback highlights that both the business-analyst and dev-lead evaluation sets currently fall short of the declared five-case minimum required before promotion to stable, and suggests adding specific test cases to satisfy this constraint. Additionally, the feedback recommends expanding the read-only GitHub CLI commands (gh pr view and gh issue view) in the advisory prompts to explicitly specify the query arguments, ensuring the LLMs receive complete context.
There was a problem hiding this comment.
Pull request overview
This PR formalizes three existing roles (dev-lead, pr-review, business-analyst) under the org’s persona standard by adding persona manifests, headless advisory prompts for @-mentions, and seeded eval case sets (dev/holdout) for new personas.
Changes:
- Adds
personas/<id>/persona.yml+ README fordev-lead,pr-review, andbusiness-analyst. - Introduces headless advisory prompts at
prompts/<id>/advisory.mdfor persona-runner dispatch. - Seeds initial eval fixtures for
dev-leadandbusiness-analystunderevals/<id>/{dev,holdout}/cases.jsonl.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| prompts/pr-review/advisory.md | Headless “review guidance” advisory prompt for pr-review mentions. |
| prompts/dev-lead/advisory.md | Headless implementation advisory prompt for dev-lead mentions. |
| prompts/business-analyst/advisory.md | Headless analysis advisory prompt for business-analyst mentions (BMAD Analyst wrapper). |
| personas/pr-review/README.md | Persona README documenting draft status and addressing. |
| personas/pr-review/persona.yml | Persona manifest defining layers, triggers, evals, and canary mapping for pr-review. |
| personas/dev-lead/README.md | Persona README documenting draft status and addressing. |
| personas/dev-lead/persona.yml | Persona manifest defining layers, triggers, evals, and canary mapping for dev-lead. |
| personas/business-analyst/README.md | Persona README documenting draft status and addressing. |
| personas/business-analyst/persona.yml | Persona manifest defining layers, triggers, evals, and canary mapping for business-analyst. |
| evals/dev-lead/holdout/cases.jsonl | Seed holdout eval cases for dev-lead. |
| evals/dev-lead/dev/cases.jsonl | Seed dev eval cases for dev-lead. |
| evals/business-analyst/holdout/cases.jsonl | Seed holdout eval cases for business-analyst. |
| evals/business-analyst/dev/cases.jsonl | Seed dev eval cases for business-analyst. |
Dev-Lead — review-changes (applied)Changes committed and pushed. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Dev-Lead — review-changes (applied)Changes committed and pushed. |
Dev-Lead — fix-reviews (applied)Changes committed and pushed. |
|
Dev-Lead — review-changes (no-changes)No changes were needed for this PR. |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: dc028f6c37d696bcb1f8e74a8137edf01c88346c
Review mode: triage-approved (single reviewer)
Summary
Onboards dev-lead, pr-review, and business-analyst as draft personas: 15 new files (+480/-0) adding declarative persona manifests, READMEs, headless advisory prompts, and seed eval sets. No workflow, script, auth, or dependency changes. Confirms the triage low-risk assessment; all decision gates pass.
Linked issue analysis
PR body references #1281, #1282, #1283 (onboard each agent under the persona standard). The PR substantively delivers the manifest half of each: persona.yml + README + advisory prompt + seed evals per role, with addressing teams already created as org mutations. Remaining scope (real eval sets, canary registration for pr-review per #710, product-manager persona) is explicitly deferred and tracked, so the issues intentionally remain open.
Findings
No blocking findings.
- All additions are declarative config (persona.yml), documentation, prompts, and static JSONL eval fixtures — no executable workflow or shell changes; thin caller stubs untouched.
- Advisory prompts carry sound guardrails: read-only gh calls only, sentinel-delimited output posted by the workflow (agent holds no write token), recursion-guard marker (#860), and a rule against literal role-handle mentions.
- Write-mode surfaces in dev-lead's manifest document the existing gated runtime and each carries its gate_label per the standard — no new write paths are introduced by this PR.
- All 10 prior bot review threads (trigger/runtime mismatches, evals.paths vs path) are resolved; the current diff uses the schema-supported evals.path and validate-personas passes.
- min_cases: 5 is satisfied per role (3 dev + 2 holdout for dev-lead and business-analyst; 5+10 for pr-review).
- Secret scan note: the run_secret_scanning MCP tool is unavailable in this run; the gitleaks CI check is green and the diff contains no credential material.
CI status
All checks green: Lint, shellcheck/ShellCheck, actionlint, unit/unit-tests, bats, validate-personas, verify-persona-teams, validate-agent-profiles, holdout-guard, prompt-coverage, agent-shield, Agent Security Scan, Secret scan (gitleaks), CodeQL (actions+python), SonarCloud Quality Gate, CodeRabbit, template/stub/caller guards. Skipped checks are ecosystem-conditional audits (npm/pnpm/pip/cargo/go) — expected.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
don-petry
changed the title
|
Added scrum-master (Bob) to this set — now four personas: dev-lead, pr-review, business-analyst, scrum-master. Team |
donpetry-bot
left a comment
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: MEDIUM
Reviewed commit: dc028f6c37d696bcb1f8e74a8137edf01c88346c
Review mode: triage-approved (single reviewer)
Summary
Purely additive PR (480+/0-, 15 files) onboarding the three existing agents (dev-lead, pr-review, business-analyst) under the persona standard: declarative persona.yml manifests, READMEs, headless advisory prompts, and seeded dev/holdout eval cases. No executable code, workflows, or scripts are touched. Confirms the triage tier's low-risk assessment; risk is MEDIUM (agent configuration, not docs) but every gate passes.
Linked issue analysis
PR addresses the manifest half of #1281, #1282, #1283 (onboarding each agent as a compliant persona, part of the org framework rollout). Each issue's manifest deliverable is present: manifest + advisory prompt + eval seed per persona, with triggers documenting the real runtimes (dev-lead's write surfaces carry gate_label, pr-review reuses existing deep-review/triage eval cases, business-analyst scopes to the Analyst role only). Issues remain open for the tracked follow-ups (real eval sets, canary registration #710, product-manager persona), which the PR body explicitly defers.
Findings
- All three manifests are draft status with advisory default mode; the only write-mode surfaces (dev-lead issues/review-fix) document the existing gate_label-protected runtime rather than adding new write paths.
- Advisory prompts enforce read-only gh calls, sentinel-delimited output, and the $AGENT_MARKER recursion guard (#860) — consistent with the qa-lead contract referenced in the PR body.
- Eval JSONL fixtures include simulated vulnerable snippets (SQL string concatenation, pull_request_target patterns) — these are test inputs for the reviewer personas, not live code.
- min_cases: 5 is satisfied for each persona (5/5/15 cases) and only required_before: stable, so draft status is compliant; validate-personas and verify-persona-teams checks are green.
- MCP secret scanning tool unavailable in this run; gitleaks CI check is green.
- Non-blocking: the 17:34 PR comment reports a fourth scrum-master persona ('5/5 manifests validate') that is NOT in head dc028f6 — that push has not landed on this branch. This review covers the three-persona state; a new commit will invalidate the SHA marker and trigger re-review.
- All 10 review threads (gemini, copilot) are resolved; prior bot findings (evals.paths schema issue, trigger/runtime mismatches) were fixed in the fix-reviews commits.
CI status
All required checks green: agent-shield, Agent Security Scan, CodeQL, SonarCloud, gitleaks, shellcheck, actionlint, validate-personas, verify-persona-teams, prompt-coverage, holdout-guard, unit tests, bats. Skipped checks are conditional (dependency-audit ecosystems, dependabot, ci-relay). CodeRabbit rate-limited/dismissed; Codex usage-limited — both non-required.
Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.
* feat(personas): onboard scrum-master (missed by #1303's auto-merge race) #1303 auto-merged on its 3-persona state moments before the scrum-master commit landed, so scrum-master never reached main (the squash even kept the old 3-persona title). The team petry-projects/scrum-master already exists; this lands the files. - personas/scrum-master/persona.yml — draft; framework-agent → vendored bmad-method sprint-planning (v6.8.0) with local_overrides → the org wrapper prompts/bmad/scrum-master.md, plus workflow-prompts → prompts/bmad. Advisory on mention/issues/discussion. Role, not a person (§1.6). - prompts/scrum-master/advisory.md — decomposition/scoping/sequencing advice (read-only/sentinel/marker contract). - README + seed held-out evals (churn-trap cases: too-coarse, missing AC, cyclic blocked_by, and a "don't over-engineer a sound plan" negative). Verification (against LIVE main — #779 schema + #1302 validator both merged): - validate-personas.py: 5/5 valid, invariants hold. - validate-cases.py: OK, 9 skills / 75 cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(bot): address bot feedback [skip ci-relay] * fix(bot): address bot feedback [skip ci-relay] --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: donpetry-bot <281750570+donpetry-bot@users.noreply.github.com>
Formalizes the three existing agents under the persona standard and makes each addressable as
@petry-projects/<role>. Closes the manifest half of #1281 / #1282 / #1283.Teams created (org mutation, done):
dev-lead,pr-review,business-analyst— allclosed, notifications disabled, 0 members (§4.1).Each persona (all
status: draft)prompts/dev-lead/agents/pr-reviewer.md+prompts/prompts/incubation/deep-review+triage(viapaths)Each manifest documents the real runtime — triggers reflect reality, advisory by default, every write surface carries its
gate_label. The advisory prompt is served by the shared persona runner on@-mention (the qa-lead read-only/sentinel/marker contract). dev-lead's advisory gives advice — code is still written only by its label-triggered runtime, not this path.Decisions taken (from the issues)
@-addressability for all three — the initiative's goal. pr-review keeps its legacy@donpetry-botpath (real account, works); this adds the role handle.product-managerpersona — the documented Analyst→PM handoff is preserved, not collapsed.Deliberately NOT here (tracked)
Real (non-seed) eval sets; the
product-managerpersona; pr-review canary registration; and enrolling these mention paths into the router soak (the framework is live only in thenextring today).After combsmith/.github#779 (schema:
evals.paths) and #1302 (validator: readspaths+ count gate). pr-review's manifest usespaths, which main's validator can't parse until #1302 lands.Verification
validate-personas.py(with chore(deps): bump the actions group across 1 directory with 4 updates #779 schema + Enforce the eval count gate; read evals.path OR evals.paths #1302 validator): 4/4 validvalidate-cases.py: OK, 7 skills / 54 cases🤖 Generated with Claude Code