[aw] PR Code Quality Reviewer + Test Quality Sentinel — 100% AR (CLI hang-on-exit) #44298

PR Code Quality Reviewer and Test Quality Sentinel — 100% AR cluster (Jul 8)

Both review workflows are failing with 100% action_required in the current window. Based on the identical pattern in Impeccable Skills Reviewer (issue #44243), the root cause is the Copilot CLI hang-on-exit after completing a review.

Evidence

• PR Code Quality Reviewer: 1/1 AR (2026-07-08)
• Test Quality Sentinel: 2/2 AR (2026-07-08)
• Same timing and pattern as Impeccable Skills Reviewer failures ([aw-failures] [aw-fix] Impeccable Skills Reviewer times out at 15m after completing its review — Copilot CLI hangs on exit #44243)
• Fix PR fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254 (fix: reduce post-completion idle watchdog and add cleanup timeouts) is open and directly addresses this class of bug

Impact

• All 4 PR-review agents (Impeccable, PR Code Quality Reviewer, Test Quality Sentinel, Matt Pocock) are producing 0 useful review signals for open PRs
• False-negative AR results obscure whether actual review work completed

Recommended Fix

1. Merge PR fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254 — reduces idle watchdog timeout and adds cleanup timeouts to prevent hang
2. After fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254 merges, validate that all 4 review workflows produce success conclusions
3. If Matt Pocock Skills Reviewer still fails after fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254, evaluate prompt redesign or deprecation

Do not re-file

This issue covers both PR Code Quality Reviewer and Test Quality Sentinel. Track resolution via PR #44254.

Related: #44243, #44234

Fresh evidence (2026-07-10) — still red after the previous sweep

The post-result idle-watchdog SIGTERM recurred on PR Triage Agent §29094180267: agent completed tool calls, then post-result watchdog terminating idle process after 20548ms (SIGTERM) → failureClass=partial_execution, hasOutput=true, all 3 retries exhausted. audit-diff vs the nearest successful run §29076495188 shows no firewall/domain regression (0 anomalies) and the failed run consumed only 9 core API calls vs 246 — the agent was killed mid-work, not blocked by environment drift. This confirms the watchdog class persists into 2026-07-10 and PR #44254 remains insufficient; keep this issue OPEN.

Generated by ⚡ Agent Performance Analyzer - Meta-Orchestrator · 99.4 AIC · ⌖ 13 AIC · ⊞ 10.6K · ◷

• expires on Jul 10, 2026, 5:24 AM UTC-08:00

🔴 Fresh evidence (2026-07-09, 6h failure sweep): merged fix #44254 did NOT resolve this — reopen the fix

Action: the post-result watchdog still SIGTERMs completed Copilot runs — raise/gate it on hasOutput, or treat partial_execution with output as success. PR #44254 merged 2026-07-08T14:05:28Z, but the identical hang-on-exit signature is still killing runs 15+ hours later.

Confirmed watchdog signature (run §29001405167, PR Triage Agent — all 4 attempts identical)

[copilot-harness] attempt N: post-result watchdog armed inactivityTimeout=20000ms
[copilot-harness] attempt N: post-result watchdog terminating idle process after ~20s (SIGTERM)
[copilot-harness] attempt N failed: exitCode=1 failureClass=partial_execution hasOutput=true
[copilot-harness] all 3 retries exhausted — giving up (exitCode=1)  totalDuration=10m 36s

The agent completes its work (hasOutput=true, 184 allowed api.githubcopilot.com calls, non-empty stderr) and is then killed by the 20s post-result inactivity watchdog → exitCode=1. Every completed run is marked failed. stdout=0B on every attempt.

Still-red Copilot workflows in the last 6h (same Execute GitHub Copilot CLI step failure)

Recommended follow-up fix (supersedes "track via #44254")

1. When hasOutput=true and failureClass=partial_execution, do not re-run — treat as success (the result was already produced).
2. Raise the inactivityTimeout well above the 20000ms that PR fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254 left in place, or disarm the post-result watchdog once the final result JSON is captured.
3. Add a regression test asserting a Copilot run that emits output then idles exits 0, not SIGTERM→1.

Keep this issue OPEN. The premise "resolves once #44254 merges" is invalidated by fresh 2026-07-09 evidence.

_{Sweep: §29003066156}

Generated by 🔍 [aw] Failure Investigator (6h) · 224.6 AIC · ⌖ 35.7 AIC · ⊞ 5.2K · ◷

Generated by 🔍 [aw] Failure Investigator (6h) · 205.7 AIC · ⌖ 17.7 AIC · ⊞ 5.2K · ◷

🔴 Fresh evidence (2026-07-12, 6h failure sweep): watchdog hang-on-exit now hitting scheduled daily cron workflows

Ship the already-specified fix now — gate the 20s post-result watchdog on hasOutput (or treat partial_execution + hasOutput=true as success). The bug has spread from PR-triggered reviewers to scheduled daily-report workflows. This is the dominant failure class in the last 6h: 4 of 6 hard-failing runs share the identical signature already documented above.

[copilot-harness] attempt N: post-result watchdog armed inactivityTimeout=20000ms
[copilot-harness] attempt N: post-result watchdog terminating idle process after ~20s (SIGTERM)
[copilot-harness] attempt N failed: exitCode=1 failureClass=partial_execution hasOutput=true isCAPIError400=false isAuthError=false isModelNotSupportedError=false isMCPPolicyError=false
[copilot-harness] all 3 retries exhausted — giving up (exitCode=1)

What changed since the 2026-07-10 update: the failure is no longer confined to PR-review/triage agents — it now also fails scheduled daily cron workflows (Formal Spec Verifier, Security Observability, Issues Report), widening blast radius to the daily automation surface.

Do not re-file — this remains the single tracker for the Copilot post-result watchdog hang-on-exit class. Smoke CI's separate EACCES chroot hosts failure this window is tracked in #44827.

References: §29203936538 · §29198701813 · §29199584830

Generated by 🔍 [aw] Failure Investigator (6h) · 228.9 AIC · ⌖ 34.3 AIC · ⊞ 5.2K · ◷

Fresh evidence (2026-07-13) — watchdog SIGTERM recurred on PR Triage Agent

The post-result idle-watchdog kill persists into 2026-07-13 — PR #44254 remains insufficient. Keep this issue OPEN.

• Run: §29275650494 (PR Triage Agent, 2026-07-13 18:54–18:56 UTC)
• Signature: agent completed a batch of bash/gh api tool calls, then [copilot-harness] attempt 4: post-result watchdog terminating idle process after 20929ms (SIGTERM) → failureClass=partial_execution, hasOutput=true, retriesRemaining=0 → all 3 retries exhausted — giving up (exitCode=1).
• Class match: identical to the 2026-07-10 recurrence already documented here (§29094180267, 20548ms). The agent was killed mid-work after producing output, not blocked by environment drift.
• Secondary noise: post-teardown [WARN] Rootless artifact permission repair failed for /tmp/gh-aw/sandbox/firewall/{logs,audit} — cosmetic, does not change the exit code.

Recommended next step: the fixed idle-watchdog threshold is still killing agents that legitimately pause between tool batches. Either raise the post-result idle threshold (currently ~20s) or reset the watchdog timer on each tool.execution_start/tool.execution_complete event so an actively-iterating agent is never treated as idle. Success criterion: PR Triage Agent produces success on a run with ≥1 post-completion tool batch.

Generated by [aw] Failure Investigator (6h) · analyzed run §29277430211

Generated by 🔍 [aw] Failure Investigator (6h) · 138.9 AIC · ⌖ 38.8 AIC · ⊞ 6.2K · ◷