Skip to content

PR Review Agent — failures detected 2026-09-05 #1675

Closed

Description

@github-actions

Convergence latency (deterministic)

Workflow duration across 152 completed run(s) in the window — computed by pr_review_health.sh, not the model.

Workflow Runs min p50 p95 max
pr-review-trigger.yml 152 1s 31s 5m30s 20m15s

Run-outcome mix by triggering event

pr-review run conclusions over the window, split by the event that triggered each run — computed deterministically by pr-review-outcomes.sh, not the model. A high cancelled share means runs were evicted while PENDING (concurrency admits one running + one pending per group); a cancelled run is non-blocking but historically rendered as a failing check (#1421).

Event success cancelled skipped failure total cancel %
check_suite 4 0 2 0 6 0%
pull_request 0 1 0 0 1 100%
pull_request_review 0 5 4 0 9 55%
repository_dispatch 73 27 0 4 104 25%
workflow_dispatch 32 0 0 0 32 0%
TOTAL 109 33 6 4 152 21%

Sweep hit-rate (deterministic)

How often the pr-review-sweep scheduled backstop actually had to re-review a PR — computed deterministically by pr-review-sweep-metrics.sh, not the model. The scheduled (cron) path re-dispatches only for genuinely un-eventable cases (GitHub-App / cross-repo checks like SonarCloud that emit no workflow_run); the workflow_run fast path handles every eventable case, so a scheduled re-dispatch is always a case the fast path did not cover.

Metric Value
Scheduled sweep ticks 8
Ticks that re-dispatched a review (hits) 8
Sweep hit-rate 100%

Interpretation: a low rate confirms the timer stays exception-only (the event fast path is carrying the load); a rising rate is a signal that an eventable case is leaking onto the timer — a hole in the narrowing worth investigating.


PR Review Agent — Health Report

Workflow: pr-review-trigger.yml | Repo: petry-projects/.github-private
Generated: 2026-09-05


1. Executive Summary

Status: WARNING
Period: 2026-09-04T11:41Z – 2026-09-05T10:26Z (~23 hours)
Result: 4 of 152 runs failed (2.6%) | 33 cancelled (21.7%) | 6 skipped

Key findings:

  • 4 intermittent failures (#72195, #72168, #72150, #72111) spread across the window — not a sustained outage, but unresolved root cause (no log payload was provided for analysis)
  • 33 cancellations concentrated in two tight bursts (~16:55–17:08Z and ~11:50–11:57Z on 2026-09-04), consistent with concurrency-group preemption on rapid push activity — expected behavior, not a defect
  • p95 duration of 5m30s vs p50 of 31s indicates a long tail; occasional runs reach 20m15s, suggesting agent quota or API throttling on heavy PRs

Action required: Retrieve and inspect logs for runs #72195, #72168, #72150, and #72111 to determine whether failures share a common cause before the next release promotion from pr-review/v1-next to pr-review/v1-stable.


2. Failure Breakdown

Note: No log content was returned in the "Failed Run Logs" section of this report. The table below reflects the failure categories that are possible to identify from the run metadata alone. Root-cause classification requires log retrieval.

Failure Category Affected Runs Example Error Message
Unknown / log not provided #72195, #72168, #72150, #72111 (logs not included in analysis payload)
Concurrency preemption (cancellation) #72131, #72130, #72128 + 30 others (expected — not a failure category)

3. Error Patterns

Unable to classify. The ## Failed Run Logs section of the input was empty — no step output, exit codes, or error messages were included for any of the four failed runs. Without log content, the following cannot be determined:

  • Which job step failed
  • Whether the failure is in the trigger stub (pr-review-trigger.yml) or the called reusable (pr-review.yml@pr-review/v1-next)
  • Whether the error is transient (network, rate limit) or deterministic (bad input, permission denial, CLI flag regression)

Temporal note on failures:

Run Timestamp Proximity to cancellation burst
#72111 2026-09-04T17:06:54Z Immediately before the ~17:07 cancellation burst
#72150 2026-09-04T18:27:38Z Isolated
#72168 2026-09-04T23:00:38Z Isolated
#72195 2026-09-05T10:26:35Z Isolated

Run #72111's adjacency to the cancellation burst suggests it may have been a legitimate trigger that failed while concurrent runs were queued — potentially a flaky infra event rather than a code defect. The other three are isolated, which weakens a single-cause hypothesis.


4. Token Scope Analysis

Scopes granted by pr-review-trigger.yml (from workflow source):

Scope Level Purpose
contents: read read Checkout code, read files
pull-requests: write write Post review comments
checks: read read Read check suite results (for check_suite trigger)

Missing scope assessment: Cannot be determined — no gh auth status output or permission-denied errors appear in the provided logs.

Structural observations from the workflow source:

  • secrets: inherit is passed to the reusable, so any token-scoped secrets (e.g. CLAUDE_API_KEY, COPILOT_TOKEN) from the caller environment are forwarded — scope errors in the reusable would not be visible in this trigger file's own logs
  • The check_suite trigger requires only checks: read, which is present
  • The repository_dispatch path (mention-listener flow) relies on the dispatch payload; no scope issues are introduced here

Recommendation: When retrieving the failed run logs, check specifically for 403 responses from the GitHub API and any insufficient_scope messages in the reusable's steps, not in this trigger stub.


5. Recommendations

  1. [HIGH] Retrieve and inspect logs for all four failed runs before advancing the release ring.

    • What: Run gh run view <id> --log-failed for #72195, #72168, #72150, #72111 in petry-projects/.github-private
    • Why: Ring-0 (pr-review/v1-next) is the canary for the fleet promotion to pr-review/v1-stable. Four unexplained failures in 23 hours is a signal that must be cleared before the channel is advanced. The ring strategy (referenced in the workflow header) requires ring-0 to prove healthy.
    • Expected impact: Either confirms failures are transient (no action needed) or surfaces a defect that would otherwise reach the stable fleet
    • Urgency: HIGH
  2. [MEDIUM] Investigate the p95/max duration outliers (5m30s / 20m15s).

    • What: Identify which job steps account for the long tail — likely engine.sh or review-one-pr.sh waiting on Claude/Copilot API responses. Add a timeout-minutes ceiling to the review job in pr-review-trigger.yml if not already present in the reusable.
    • Why: A 20-minute run consuming a runner slot can block queue throughput, especially during burst periods. If the reusable already has a timeout, the trigger stub does not need one — verify in pr-review.yml.
    • Expected impact: Reduces runner utilization and makes hang detection predictable
    • Urgency: MEDIUM
  3. [LOW] Confirm cancellation behavior is intentional and not masking real failures.

    • What: Check whether a concurrency group is defined in pr-review.yml (the reusable) or pr-review-trigger.yml. The two cancellation bursts (33 runs) are consistent with cancel-in-progress: true on rapid synchronize events.
    • Why: If the cancellations are intentional (superseded runs on new pushes), no action is needed. If they are not intentional, older reviews may be silently dropped.
    • Expected impact: Confirms no review coverage gaps during active PR activity
    • Urgency: LOW

6. Health Score

Health: 7/10 — Failure rate is low (2.6%) but root cause is unknown, blocking safe promotion to the stable fleet ring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    automated-reportCreated by automated workflowhealth-checkAutomated health check report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions