Skip to content

[Fleet Monitor] petry-projects/.github-private β€” daily-pr-review-health.ymlΒ #511

Closed

Description

@don-petry

🟠 DEGRADED: petry-projects/.github-private / daily-pr-review-health.yml

Metric Value
Failure Rate 50% (1 / 2 runs)
Status DEGRADED
p50 Duration 100s
p95 Duration 100s
Successful Runs 1
Cancelled Runs 0

Workflow: daily-pr-review-health.yml

Last updated by Fleet Monitor run on 2026-06-09.
Threshold: failure rate > 10% over the monitored window.

Activity

  1. added
    health-checkAutomated health check report
    dev-leadFor dev-lead agent pickup
    fleet-trackerTracked by the Actions Fleet Monitor
    and removed
    dev-leadFor dev-lead agent pickup
    on Jun 9, 2026
  2. don-petry commented on Jun 9, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #511 β€” [Fleet Monitor] petry-projects/.github-private β€” daily-pr-review-health.yml

    Root Cause

    The daily-pr-review-health.yml health check fails when the Claude CLI hits its daily usage limit. From the failing run (2026-06-09T08:58:14Z):

    ##[error]Claude invocation failed. CLI output follows:
    You've hit your limit Β· resets 4pm (UTC)
    

    The script calls exit 1 unconditionally on any Claude failure, including transient rate limits that are not indicative of a real PR review agent health problem.

    Scope

    Add graceful rate-limit handling to scripts/pr_review_health.sh so that a transient Claude usage-cap does not mark the health-check workflow as failed. Emit a ::warning:: annotation instead and exit 0.

    Implementation Checklist

    • scripts/pr_review_health.sh β€” detect rate limit in Claude error handler block; emit ::warning::, override HAS_FAILURES=false, exit 0
    • tests/test_pr_review_health.bats β€” new test file covering rate-limit exit-0, non-rate-limit exit-1, and no-failures skip-Claude paths
    • .github/workflows/lint.yml β€” add new test file to bats test list (per AGENTS.md exception rule)

    Tests to Write

    • "Claude rate limit exits 0 and emits ::warning:: annotation" β€” stub claude to output "You've hit your limit", verify exit 0 and ::warning:: in output
    • "Non-rate-limit Claude failure exits 1 and emits ::error:: annotation" β€” stub claude to output generic error, verify exit 1 and ::error:: in output
    • "No failed runs exits 0 without invoking Claude" β€” stub gh to return all-success runs, verify exit 0 and no Claude invocation

    Test Command

    bats tests/test_pr_review_health.bats

    Pattern Reference

    The rate-limit grep pattern follows _rate_limit_pattern() in scripts/engine.sh (the codebase's single source of truth), specifically the hit your limit and resets [0-9]+(am|pm) tokens that match the observed error message.

  3. don-petry commented on Jun 9, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Root Cause

    The failing run (2026-06-09T08:58:14Z) hit the Claude daily usage limit:

    You've hit your limit Β· resets 4pm (UTC)
    

    scripts/pr_review_health.sh called exit 1 unconditionally on any Claude failure, so a transient rate-limit caused the health-check workflow to register as a hard failure β€” inflating the failure rate with noise unrelated to the PR review agent's health.

    Plan Execution

    • scripts/pr_review_health.sh β€” added rate-limit detection after Claude failure; emits ::warning::, overrides HAS_FAILURES=false in $GITHUB_ENV, exits 0
    • tests/test_pr_review_health.bats β€” 5 new tests (3 rate-limit paths, 1 non-rate-limit error, 1 no-failures skip)
    • .github/workflows/lint.yml β€” added tests/test_pr_review_health.bats to bats test list per AGENTS.md exception rule

    Test Results

    1..179
    ...
    ok 175 Claude rate limit exits 0 and emits ::warning:: annotation
    ok 176 Claude rate limit: HAS_FAILURES is overridden to false in GITHUB_ENV
    ok 177 Claude 'usage limit' message also triggers rate-limit handling (exit 0)
    ok 178 non-rate-limit Claude failure exits 1 and emits ::error:: annotation
    ok 179 no failed runs in window exits 0 without invoking Claude
    
    179 tests, 0 failures
    

    Lint: shellcheck --severity=warning on all 40 scripts/**/*.sh β€” passed. Agent profile frontmatter β€” passed.

    Files Changed

    • scripts/pr_review_health.sh: 13-line addition in the Claude error handler β€” detect rate-limit keywords via grep -qiE (pattern mirrors _rate_limit_pattern() in engine.sh), emit ::warning::, override HAS_FAILURES=false, exit 0
    • tests/test_pr_review_health.bats: new test file, 5 tests covering all error-handling paths
    • .github/workflows/lint.yml: added new test file to bats run list

    Notes

    • The HAS_FAILURES=false override prevents the issue-creation step from filing a spurious ticket; a ::warning:: annotation in the Actions log preserves visibility for operators
    • Non-rate-limit Claude errors (e.g. segfault, configuration error) still exit 1 as before β€” only genuine rate-limit/quota messages are handled gracefully
    • Pattern breadth: quota alone is intentionally conservative; combined with the Claude-specific context (only fires after if ! claude ...) false-positive risk is negligible
  4. added a commit that references this issue on Jun 10, 2026
  5. added 6 commits that reference this issue on Jun 12, 2026
  6. 15 remaining items

  7. added 14 commits that reference this issue on Aug 7, 2026
  8. added a commit that references this issue on Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupfleet-trackerTracked by the Actions Fleet Monitorhealth-checkAutomated health check report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions