feat: implement issue #511 — [Fleet Monitor] petry-projects/.github-private — daily-pr-review-health.yml

[don-petry]

Closes #511

Implemented by dev-lead agent. Please review.

Summary by CodeRabbit

• Bug Fixes
  • PR review health checks now gracefully handle Claude API rate-limit errors, deferring analysis with a warning instead of reporting failures.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[coderabbitai]

Warning

Review limit reached

@don-petry, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 36 minutes and 20 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9bb15316-b01c-488d-af5c-ad4913cb4793

📥 Commits

Reviewing files that changed from the base of the PR and between b0e19a5 and 78b3024.

📒 Files selected for processing (3)

• .github/workflows/lint.yml
• scripts/pr_review_health.sh
• tests/test_pr_review_health.bats

📝 Walkthrough

Walkthrough

This PR enhances pr_review_health.sh to gracefully handle Claude API rate-limit and quota errors by deferring analysis instead of surfacing them as actionable failures. It adds a comprehensive Bats test suite that validates rate-limit handling, non-rate-limit failures, and edge cases, and integrates the new tests into the CI lint workflow.

Changes

Claude rate-limit error handling and test coverage

Layer / File(s)	Summary
Script rate-limit error detection scripts/pr_review_health.sh	Script detects Claude rate-limit or quota messages in the output report; on match, it writes an "Analysis deferred" report, sets HAS_FAILURES=false in GITHUB_ENV, cleans up temporary logs, and exits zero to prevent spurious failure notifications.
Test harness setup and stub factories tests/test_pr_review_health.bats	Establishes setup() and teardown() for temp directory and artifact management; provides make_gh_stub_one_failure(), make_gh_stub_no_failures(), and make_claude_stub() to simulate deterministic script behaviors across failure scenarios.
Rate-limit scenario test cases tests/test_pr_review_health.bats	Validates that Claude "rate limit" and "usage limit" messages trigger exit zero, override HAS_FAILURES to false, and emit ::warning:: annotations (deferred analysis).
Non-rate-limit failure and no-failure test cases tests/test_pr_review_health.bats	Asserts that non-rate-limit Claude failures exit one with ::error:: annotation, and that successful runs (no failures) exit zero without invoking Claude.
CI workflow test integration .github/workflows/lint.yml	Lint job extended to run tests/test_pr_review_health.bats alongside existing test files.

Sequence Diagram

sequenceDiagram
  participant Script as pr_review_health.sh
  participant Claude as Claude CLI
  participant Report as pr_review_health_report.md
  participant Env as GITHUB_ENV
  
  Script->>Claude: Invoke with logs
  Claude-->>Report: Output (including rate-limit msg)
  Script->>Report: Scan for rate-limit message
  alt Rate-limit detected
    Script->>Report: Write "Analysis deferred"
    Script->>Env: Set HAS_FAILURES=false
    Script->>Script: Exit 0 (success)
  else No rate-limit
    Script->>Script: Follow original error path
    Script->>Script: Exit 1 or 0 per prior logic
  end

Loading

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• petry-projects/.github-private#175: Updates scripts/pr_review_health.sh to surface Claude CLI invocation errors, overlapping with this PR's error-handling logic in the same script.
• petry-projects/.github-private#197: Removes scripts/pr_review_health.sh entirely, directly conflicting with this PR's updates to the same script.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: implementing issue #511 which addresses the degraded daily-pr-review-health workflow in the GitHub private repo.
Linked Issues check	✅ Passed	The PR addresses issue #511 by adding rate-limit error handling to the pr_review_health script and comprehensive test coverage, directly resolving the workflow's failure condition.
Out of Scope Changes check	✅ Passed	All changes directly support fixing the daily-pr-review-health workflow: error handling for Claude rate limits, test coverage, and lint workflow updates to include new tests.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch dev-lead/issue-511-20260609-2132

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[don-petry]

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

[gemini-code-assist]

Code Review

This pull request adds error-handling logic to scripts/pr_review_health.sh to identify transient Claude rate-limit and quota errors, preventing spurious workflow failures by deferring analysis and overriding the failure status. It also introduces a comprehensive unit test suite in tests/test_pr_review_health.bats using BATS. The review feedback suggests expanding the rate-limit regex pattern to cover more Claude-specific errors and prevent maintenance drift, and recommends refactoring the test suite to use a helper function for stubbing the Claude CLI to reduce code duplication.

[Copilot]

Pull request overview

Adjusts the PR Review Agent daily health check to treat Claude daily usage/rate-limit failures as a non-fatal, transient condition, preventing spurious workflow failures and issue creation while still surfacing an actionable warning/report.

Changes:

• Update scripts/pr_review_health.sh to detect Claude rate-limit/quota output, emit a ::warning::, write a deferred-analysis report, and override HAS_FAILURES=false to skip issue creation.
• Add Bats tests covering Claude rate-limit handling vs non-rate-limit failures, plus the “no failures” early-exit path.
• Extend the lint.yml Bats job to run the new health-check test suite.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
tests/test_pr_review_health.bats	Adds unit/integration-style Bats coverage for Claude invocation failure modes and env/export behavior.
scripts/pr_review_health.sh	Adds rate-limit/quota detection to avoid failing the workflow and to suppress issue creation on transient Claude caps.
.github/workflows/lint.yml	Runs the new Bats test file in CI.

[don-petry]

Dev-Lead — fix-reviews (applied)

Changes committed and pushed.

[don-petry]

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #527
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-06-09T22:23:38Z

[don-petry]

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-06-09T22:23:38Z

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

.github/workflows/lint.yml (1)

36-48: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

CI integration is incomplete: the changed script is not ShellChecked.

This PR modifies scripts/pr_review_health.sh, but the shellcheck step still omits it, so the required shell lint gate does not cover this change.

Suggested fix

-        run: shellcheck --severity=warning scripts/fleet_monitor.sh scripts/fleet_report.sh scripts/token_report.sh scripts/lib/model-pricing.sh scripts/cut-release.sh
+        run: shellcheck --severity=warning \
+          scripts/fleet_monitor.sh \
+          scripts/fleet_report.sh \
+          scripts/token_report.sh \
+          scripts/lib/model-pricing.sh \
+          scripts/cut-release.sh \
+          scripts/pr_review_health.sh

As per coding guidelines, "Lint all Bash scripts with ShellCheck using zero warnings severity" and "scripts/**/*.sh: Lint all Bash scripts with ShellCheck using zero warnings."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/lint.yml around lines 36 - 48, The CI currently does not
ShellCheck the new scripts/pr_review_health.sh so add it to the shellcheck job
in .github/workflows/lint.yml: update the ShellCheck step (e.g., the job named
"Run ShellCheck" or whatever step runs shellcheck) to include
scripts/pr_review_health.sh or, better, change its input pattern to
scripts/**/*.sh so all scripts (including scripts/pr_review_health.sh) are
linted with ShellCheck using the repo's zero-warnings configuration; then ensure
the script fixes any ShellCheck warnings so the job passes.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/pr_review_health.sh`:
- Line 234: Replace the single-bracket test used for the GITHUB_ENV guard with
the bash built-in double-bracket form: change the conditional that reads [ -n
"${GITHUB_ENV:-}" ] && echo "HAS_FAILURES=false" >> "$GITHUB_ENV" to use [[ -n
"${GITHUB_ENV:-}" ]] && echo "HAS_FAILURES=false" >> "$GITHUB_ENV" so the script
uses the safer bash conditional (preserving the existing quoting and redirect
behavior).

In `@tests/test_pr_review_health.bats`:
- Around line 109-118: The test "Claude rate limit: HAS_FAILURES is overridden
to false in GITHUB_ENV" only verifies presence of HAS_FAILURES=false; change it
to assert both HAS_FAILURES=true and HAS_FAILURES=false entries are written and
that the false entry comes after the true entry. After running the script (using
make_gh_stub_one_failure, make_claude_stub and running "$SCRIPT"), verify both
lines exist in "$GITHUB_ENV" and then compare their positions (e.g., via line
numbers) to ensure the HAS_FAILURES=false entry appears after HAS_FAILURES=true,
thereby enforcing the "last value wins" contract.

---

Outside diff comments:
In @.github/workflows/lint.yml:
- Around line 36-48: The CI currently does not ShellCheck the new
scripts/pr_review_health.sh so add it to the shellcheck job in
.github/workflows/lint.yml: update the ShellCheck step (e.g., the job named "Run
ShellCheck" or whatever step runs shellcheck) to include
scripts/pr_review_health.sh or, better, change its input pattern to
scripts/**/*.sh so all scripts (including scripts/pr_review_health.sh) are
linted with ShellCheck using the repo's zero-warnings configuration; then ensure
the script fixes any ShellCheck warnings so the job passes.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: f1485055-03b4-4c0a-8b87-008f95f9a8d9

📥 Commits

Reviewing files that changed from the base of the PR and between 819d1a2 and b0e19a5.

📒 Files selected for processing (3)

• .github/workflows/lint.yml
• scripts/pr_review_health.sh
• tests/test_pr_review_health.bats

[don-petry]

Dev-Lead — fix-reviews (applied)

Changes committed and pushed.

[don-petry]

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.