feat: implement issue #2178 — dev-lead never dispositions a neutral bot overview (CodeAnt "PR Risk: Low Risk"), but pr-review's comment gate still counts it — the PR can never be approved by don-petry · Pull Request #2192 · combsmith/.github-private

don-petry · 2026-10-10T12:22:11Z

Problem

dev-lead never dispositions a neutral bot overview (CodeAnt "PR Risk: Low Risk"), but pr-review's comment gate still counts it — the PR can never be approved

From the issue: A finding-bot's clean summary with no finding in it — for example CodeAnt's ## CodeAnt PR Risk: Low Risk … The PR appears safe to merge — holds pr-review's undispositioned-pr-comment gate forever:

Risk

Low — 3 file(s) changed; no CI- or runtime-critical paths touched.

Test plan

Tests added/updated: tests/dev-lead/unit/test_fix_reviews.bats,tests/dev-lead/unit/test_maintainer_comment_gate.bats. Verification: bash scripts/dev-lead-lint.sh (shellcheck --severity=warning) ran pre-commit; the bats suite runs in CI.

Rollback

Revert this PR. No non-revertible side effects (no tags, migrations, or external state).

Monitoring

This PR's CI checks (Lint + bats) are the success/regression signal.

Closes #2178

…ot overview (CodeAnt "PR Risk: Low Risk"), but pr-review's comment gate still counts it — the PR can never be approved

chatgpt-codex-connector · 2026-10-10T12:22:15Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

qodo-code-review · 2026-10-10T12:22:18Z

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

coderabbitai · 2026-10-10T12:22:23Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 25 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 329f2489-597d-4ce9-be26-fdf5415f42a1

📥 Commits

Reviewing files that changed from the base of the PR and between aef6116 and 7ec6c44.

📒 Files selected for processing (4)

prompts/dev-lead/fix-bot-comment.md
scripts/lib/reviewer-sources.sh
tests/dev-lead/unit/test_fix_reviews.bats
tests/dev-lead/unit/test_maintainer_comment_gate.bats

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

gemini-code-assist

Code Review

This pull request updates the prompt instructions in fix-bot-comment.md and adds corresponding unit tests to address issue #2178. It specifies that neutral bot overviews, such as CodeAnt's 'PR Risk: Low Risk' summary, must be dispositioned with exactly one informational reply to clear the maintainer comment gate, rather than being left reply-less. The added tests verify both the prompt's content and the gate's behavior under various conditions. There are no review comments, and I have no additional feedback to provide.

don-petry

Maintainer finding on the second acceptance criterion. The prompt change itself is right and I confirmed the path works end to end: cdv_body_has_findings returns "no findings" for today's real CodeAnt Low Risk overviews, so an informational disposition on them verifies and the harness minimizes the comment. Keep that. One gap inline.

codeant-ai · 2026-10-10T12:25:38Z

CodeAnt Nitpicks

1 code suggestion

1. This tests an undispositioned comment, which blocks regardless of its text; an informational disposition on this generic CodeAnt suggestion still clears the gate, so the finding rule is untested.

Possible bug · tests/dev-lead/unit/test_maintainer_comment_gate.bats:574

codeant-ai · 2026-10-10T12:25:49Z

CodeAnt PR Risk: Low Risk

The PR appears safe to merge; the prompt clarifies how neutral CodeAnt overviews should be dispositioned.
Regression tests cover neutral Low Risk summaries and ensure suggestions or higher risk ratings are treated as findings.

Assessed commit: 7ec6c44486c8

don-petry · 2026-10-10T12:39:24Z

QA Lead — test-risk advisory

Risk tier: MEDIUM. The PR changes only the prompt, but it now tells dev-lead to post informational on CodeAnt overviews, and the only thing stopping that on a comment with a real finding is a sentence in the prompt.

What I'd shore up (highest leverage first):

Add the missing negative test at the gate level. Add a test for a CodeAnt comment that carries a finding, is dispositioned informational and is minimized RESOLVED. It should give 1. I expect it gives 0 today. reviewer_sources_finding_section_pattern only matches CodeRabbit section shapes (architecture review, "Actionable comments posted", Nitpick/Outside-diff). So a CodeAnt **Suggestion:** body is not seen as carrying a finding, and an informational reply clears it. The new test 3 only covers the undispositioned case, which was never in doubt.
Add behavioural eval cases, not just grep checks. Both new test_fix_reviews.bats cases only check that certain wording is in the prompt. They break if the wording changes, and they say nothing about what the model actually does. Add a pair to evals/dev-lead, which has 3 dev cases today: a neutral CodeAnt overview should get exactly one informational reply, and a CodeAnt overview with a suggestion should never get informational.
Tests 1 and 2 in test_maintainer_comment_gate.bats describe how the gate already behaves. They are useful as regression locks, but they don't test anything new in this PR. Say so in the test plan.

Escalate? Yes. A maintainer should decide before merge: either extend the finding check to cover CodeAnt's suggestion and finding markers, or knowingly accept that the prompt alone keeps CodeAnt findings from being waved through.

Advisory only — I comment, I do not change code. Ground: BMAD Test Architecture
(bmad-tea v1.19.0). Opt out on this item with the qa-lead:hands-off label.

…nding-bearing (#2178)

don-petry · 2026-10-10T12:56:00Z

Dev-Lead — review-changes (not-applied)

A commit was pushed, but it did not touch any region the review named — the requested changes were not applied. Per requested item:

tests/dev-lead/unit/test_maintainer_comment_gate.bats:571 — not applied

donpetry-bot · 2026-10-10T13:09:33Z

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-10-10T14:09:32Z.

donpetry-bot · 2026-10-10T13:22:01Z

No description provided.

donpetry-bot · 2026-10-10T13:25:12Z

No description provided.

donpetry-bot · 2026-10-10T14:09:32Z

No description provided.

don-petry · 2026-10-10T14:20:46Z

Fixed in scripts/lib/reviewer-sources.sh: adjusted the PR Risk pattern from PR Risk:\s*\**\s*(?!Low Risk)\S to PR Risk:\s*(?!\**Low Risk(?:\*\*)?)\S to properly handle bolded risk levels like **Low Risk**. The original pattern would incorrectly match when asterisks surrounded the risk level due to regex backtracking. Added regression tests for both bolded Low Risk (clean) and bolded Medium Risk (finding-bearing) cases.

don-petry · 2026-10-10T14:29:35Z

No description provided.

…g correctly The previous pattern `PR Risk:\s*\**\s*(?!Low Risk)\S` incorrectly matched bolded risk levels like `**Low Risk**` due to regex backtracking. The pattern would match fewer asterisks and then see the remaining asterisk as a non-whitespace character, bypassing the negative lookahead. Adjusted the pattern to `PR Risk:\s*(?!\**Low Risk(?:\*\*)?)\S` to explicitly exclude bolded variations of "Low Risk" in the negative lookahead. Added regression tests for both bolded Low Risk (clean) and bolded Medium Risk (finding-bearing) cases to prevent future regressions. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

don-petry · 2026-10-10T14:47:24Z

No description provided.

donpetry-bot · 2026-10-10T14:49:22Z

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-10-10T15:49:22Z.

donpetry-bot · 2026-10-10T14:58:37Z

No description provided.

don-petry · 2026-10-10T14:59:01Z

No description provided.

don-petry · 2026-10-10T15:03:50Z

This is a CodeRabbit OSS review rate-limit notice indicating their free review quota is exhausted. The rate-limit block contains no actionable code findings — it is purely an operational notice about review availability. No code changes are required.

don-petry · 2026-10-10T15:04:08Z

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning

Perfect! I've addressed both the Tier 1 blocker and the bot comment. Let me summarize what was done:
## Summary
**Bot:** coderabbitai[bot]
**Issues addressed:** 2
1. **Template-drift check failure (Tier 1 blocker)**: The `.github/workflows/dev-lead.yml` file in `petry-projects/repo-template` had drifted from the standards-derived baseline. Ran `scripts/seed-repo-template.sh` which created/updated a PR on repo-template (PR #235) to synchronize the template with current standards. The script updated `dev-lead.yml` along with `README.md` and `BOOTSTRAP.md` to match the standards baseline.
2. **CodeRabbit rate-limit notice**: Posted disposition reply on the original bot comment (`IC_kwDOR9SdIs8AAAABa2-VAA`) explaining this is a non-actionable operational notice about OSS review quota limits, not a code finding. Disposition marker: `informational`.
**Files changed:** `petry-projects/repo-template` (via seed-repo-template.sh PR #235) — this is a separate repository updated to sync with standards.
**Skipped (informational):** 0 (the CodeRabbit comment contained only the rate-limit notice block with no other finding-bearing sections)
---
The CI checks should now pass once the repo-template sync PR (#235) is merged. The CodeRabbit comment is now dispositioned and will be minimized by the harness.

donpetry-bot · 2026-10-10T15:51:59Z

No description provided.

…er 'Low Risk' The pattern was accepting variations like 'Low Risky' as clean because it lacked a word boundary check after 'Low Risk'. Changed pattern from: PR Risk:\s*(?!\**Low Risk(?:\*\*)?)\S to: PR Risk:\s*(?!\**Low Risk\b(?:\*\*)?)\S The \b word boundary now ensures that 'Low Risk' must be followed by a non-word character (space, punctuation, or end of string) to be treated as clean. Variations like 'Low Risky' are now correctly detected as findings. Added test cases to verify both 'Low Risky' is detected as finding-bearing and 'Low Risk ' (with trailing space) remains clean. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

don-petry · 2026-10-10T15:57:14Z

Fixed in scripts/lib/reviewer-sources.sh and tests/dev-lead/unit/test_maintainer_comment_gate.bats: added word boundary after 'Low Risk' in the CodeAnt PR Risk pattern to prevent accepting variations like 'Low Risky' as clean. Pattern changed from PR Risk:\\s*(?!\\**Low Risk(?:\\*\\*)?)\\S to PR Risk:\\s*(?!\\**Low Risk\\b(?:\\*\\*)?)\\S. Added test cases to verify 'Low Risky' is now correctly detected as finding-bearing and 'Low Risk ' remains clean.

don-petry · 2026-10-10T16:08:01Z

No description provided.

sonarqubecloud · 2026-10-10T16:08:17Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

donpetry-bot · 2026-10-10T16:08:49Z

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-10-10T17:08:48Z.

don-petry · 2026-10-10T16:09:30Z

Dev-Lead — fix-reviews (no-changes)

Agent reasoning

  - I couldn't read the log because the run is still in progress, so I didn't confirm that this run failed for the same reason. I left it alone.
- **Pending checks:** `bats` and `unit` were still in progress. I didn't run the test suite locally because I changed nothing.
- **Issue comments (Phase 1b):** I posted no dispositions on the open bot comments, which are the Codex limit notice, the Qodo trial notice, and the CodeRabbit and CodeAnt comments. This run's task was review threads, and the harness runs a separate `fix-bot-comment` pass for issue comments.
  - That pass has already posted `informational` dispositions on the CodeRabbit rate-limit comment.
  - The CodeRabbit and CodeAnt comments were edited at 16:07 and 16:08, after those dispositions, so they may need fresh ones.
```
Addressed 0 threads (none open).
Test verification: not run — no changes made.
Files changed: none
```

feat: implement issue #2178 — dev-lead never dispositions a neutral b…

37b4e01

…ot overview (CodeAnt "PR Risk: Low Risk"), but pr-review's comment gate still counts it — the PR can never be approved

don-petry requested a review from a team as a code owner October 10, 2026 12:22

don-petry added the auto-rebase:ready Opts a non-draft PR into auto-rebase without an approval (auto-rebase ready_label) label Oct 10, 2026

don-petry mentioned this pull request Oct 10, 2026

dev-lead never dispositions a neutral bot overview (CodeAnt "PR Risk: Low Risk"), but pr-review's comment gate still counts it — the PR can never be approved #2178

Closed

3 tasks

gemini-code-assist Bot reviewed Oct 10, 2026

View reviewed changes

don-petry commented Oct 10, 2026

View reviewed changes

Comment thread tests/dev-lead/unit/test_maintainer_comment_gate.bats

don-petry added 2 commits October 10, 2026 07:40

Merge branch 'main' into dev-lead/issue-2178-20261010-1145

b155275

fix(reviews): treat CodeAnt suggestion / non-Low-Risk overviews as fi…

8097de7

…nding-bearing (#2178)

Merge branch 'main' into dev-lead/issue-2178-20261010-1145

72e0b9c

This comment has been minimized.

Sign in to view

don-petry enabled auto-merge October 10, 2026 13:36

don-petry disabled auto-merge October 10, 2026 14:16

Merge branch 'main' into dev-lead/issue-2178-20261010-1145

572318b

don-petry enabled auto-merge October 10, 2026 14:30

don-petry disabled auto-merge October 10, 2026 15:00

don-petry enabled auto-merge October 10, 2026 15:04

don-petry disabled auto-merge October 10, 2026 15:52

don-petry enabled auto-merge October 10, 2026 16:07

don-petry disabled auto-merge October 10, 2026 16:08

don-petry enabled auto-merge October 10, 2026 16:09

don-petry mentioned this pull request Oct 10, 2026

Org Status — 2026-10-10 combsmith/.github#1280

Closed

don-petry disabled auto-merge October 10, 2026 17:12

don-petry merged commit e6b666d into main Oct 10, 2026
69 of 70 checks passed

don-petry deleted the dev-lead/issue-2178-20261010-1145 branch October 10, 2026 17:13

don-petry mentioned this pull request Oct 10, 2026

Epic: qa-lead — close the gap between the declared persona and the served one #1643

Closed

Conversation

don-petry commented Oct 10, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Risk

Test plan

Rollback

Monitoring

Uh oh!

chatgpt-codex-connector Bot commented Oct 10, 2026

Uh oh!

qodo-code-review Bot commented Oct 10, 2026

Uh oh!

coderabbitai Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

don-petry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codeant-ai Bot commented Oct 10, 2026

CodeAnt Nitpicks

1. This tests an undispositioned comment, which blocks regardless of its text; an informational disposition on this generic CodeAnt suggestion still clears the gate, so the finding rule is untested.

Uh oh!

codeant-ai Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodeAnt PR Risk: Low Risk

Uh oh!

don-petry commented Oct 10, 2026

QA Lead — test-risk advisory

Uh oh!

don-petry commented Oct 10, 2026

Dev-Lead — review-changes (not-applied)

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

donpetry-bot commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Dev-Lead — fix-bot-comment (no-changes)

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Uh oh!

sonarqubecloud Bot commented Oct 10, 2026

Quality Gate passed

Uh oh!

donpetry-bot commented Oct 10, 2026

Uh oh!

don-petry commented Oct 10, 2026

Dev-Lead — fix-reviews (no-changes)

Uh oh!

don-petry commented Oct 10, 2026 •

edited by cubic-dev-ai Bot

Loading

coderabbitai Bot commented Oct 10, 2026 •

edited

Loading

codeant-ai Bot commented Oct 10, 2026 •

edited

Loading