[Fleet Monitor] petry-projects/.github-private — pr-review.yml

🟠 DEGRADED: `petry-projects/.github-private` / `pr-review.yml`

Metric	Value
Failure Rate	25.2% (33 / 131 runs)
Status	DEGRADED
p50 Duration	11s
p95 Duration	155s
Successful Runs	37
Cancelled Runs	37

Workflow: pr-review.yml

Last updated by Fleet Monitor run on 2026-06-05.
Threshold: failure rate > 10% over the monitored window.

Dev-Lead Implementation Plan

Issue: #402 — [Fleet Monitor] petry-projects/.github-private — pr-review.yml

Scope

Fix prompts/single-review.md to write its verdict to $OUTPUT_FILE using jq (the established pattern), instead of instructing the model to "output to stdout" — which fails silently in agentic mode when --output-format json is active.

Root Cause

When TOKEN_LOG_FILE is set (always in production), engine.sh adds --output-format json to claude --print invocations. In this mode the model's text output lands in the result field of the JSON envelope. The single-review.md prompt tells the model to "Output ONLY valid JSON to stdout", which the model interprets as printing via a Bash echo command. That Bash output goes to the tool result, not result, so result="". _claude_chain_invoke falls back to emitting the raw claude API JSON wrapper (no decision field). extract_verdict_json Style 2 copies this wrapper to $VERDICT_JSON, and jq -r '.decision // ""' returns "" → "invalid decision '' — will retry" → both retries fail → PR flagged for manual review → run exits 1.

Every other agentic prompt (cascade-action.md, deep-review.md) already uses jq > "$OUTPUT_FILE" and includes "IMPORTANT: Do NOT print the JSON to stdout." single-review.md was the only one that did not, causing consistent failures whenever triage cleared a PR for single-pass review.

Implementation Checklist

prompts/single-review.md — add $OUTPUT_FILE to Inputs section; replace "output to stdout" with jq > "$OUTPUT_FILE" pattern (identical to cascade-action.md); add "IMPORTANT: Do NOT print the JSON to stdout" note
tests/test_cli_error_routing.bats — add test case: when $OUTPUT_FILE is absent (model didn't write it) and $VERDICT_JSON.raw contains the claude API JSON wrapper, extract_verdict_json must NOT produce a non-empty decision (validates the bug was present and would have failed)

Tests to Write

extract_verdict_json with only raw = claude API JSON wrapper → decision must be empty (demonstrates pre-fix failure path)
extract_verdict_json with valid review JSON in dest file → decision is approve (demonstrates correct path)

Test Command

bats tests/test_cli_error_routing.bats

Dev-Lead: Implementation Complete

Plan Execution

Root-cause identified: prompts/single-review.md told the model to "Output ONLY valid JSON to stdout" while every other agentic prompt (cascade-action.md, deep-review.md) uses jq > "$OUTPUT_FILE"
Failing tests written (TDD red phase) — 3 prompt-content tests and 2 extract_verdict_json tests
prompts/single-review.md updated: added $OUTPUT_FILE to Inputs, replaced stdout instructions with jq > "$OUTPUT_FILE" pattern, added "IMPORTANT: Do NOT print JSON to stdout"
All 13 tests pass (3 previously-failing tests now green, no regressions)
bash scripts/dev-lead-lint.sh passes (shellcheck + agent frontmatter)

Root Cause

When TOKEN_LOG_FILE is set (always in production), engine.sh passes --output-format json to claude --print. In this mode:

The model's text output lands in the JSON envelope's result field
Bash tool echoes go to tool result, not result
The model followed the prompt's "output to stdout" instruction via a Bash echo, so result = ""
_claude_chain_invoke fell back to emitting the raw claude API JSON wrapper (no decision field)
extract_verdict_json Style 2 copied the wrapper to $VERDICT_JSON, making jq -r '.decision // ""' return "" → "invalid decision '' — will retry" for both attempts → PR flagged for manual review → run exits 1

All other agentic prompts (cascade-action.md, deep-review.md) already use jq > "$OUTPUT_FILE" and include "Do NOT print the JSON to stdout." single-review.md was the sole exception.

Test Results

1..13
ok 1 integration: Copilot 'Invalid command format' routes to per-PR failure (exit 1)
ok 2 integration: 'rate limit exceeded' routes to engine fallback (exit 2)
ok 3 integration: 'overloaded_error' routes to engine fallback (exit 2)
ok 4 integration: '529' routes to engine fallback (exit 2)
ok 5 integration: 'quota exceeded' routes to engine fallback (exit 2)
ok 6 integration: rc=0 with 'rate limit' in stdout does NOT trigger exit 2
ok 7 integration: rc=0 with 'unknown flag' in stdout does NOT trigger exit 1
ok 8 integration: generic error (rc=1, no pattern match) passes through
ok 9 single-review prompt: instructs model to write to OUTPUT_FILE (not stdout)
ok 10 single-review prompt: does NOT instruct model to output to stdout
ok 11 single-review prompt: instructs model NOT to print JSON to stdout
ok 12 extract_verdict_json: dest has review JSON → decision extracted (Style 1)
ok 13 extract_verdict_json: raw=claude-API-wrapper, dest missing → decision empty (bug scenario)

Files Changed

prompts/single-review.md: Added $OUTPUT_FILE to Inputs; replaced "output JSON to stdout" with jq > "$OUTPUT_FILE" pattern matching cascade-action.md; fixed skip-case instructions to write to $OUTPUT_FILE; added "IMPORTANT: Do NOT print the JSON to stdout"
tests/test_cli_error_routing.bats: Added 5 regression tests — 3 checking prompt content (failed before fix, pass after), 2 validating extract_verdict_json behavior (document the bug scenario and the correct path)

Notes

The historical June 6th failures (cluster of ~25 runs failing at "Verify auth scopes") were from a prior version of the workflow code (before the June 7th commit). Those are not reproducible with the current codebase. The June 6th 16:57 failure was an all-engines-exhausted scenario (Claude + Gemini rate-limited, Copilot exit 1) — that is a transient availability issue, not a code bug. The recurrent "invalid decision ''" failure fixed here was the only systematic code defect present in the current codebase.

feat: implement issue #402 — [Fleet Monitor] petry-projects/.github-p…

Navigation Menu

Repository navigation

[Fleet Monitor] petry-projects/.github-private — pr-review.yml #402

🟠 DEGRADED: `petry-projects/.github-private` / `pr-review.yml`

Dev-Lead Implementation Plan

Scope

Root Cause

Implementation Checklist

Tests to Write

Test Command

Dev-Lead: Implementation Complete

Plan Execution

Root Cause

Test Results

Files Changed

Notes

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Fleet Monitor] petry-projects/.github-private — pr-review.yml #402

Description

🟠 DEGRADED: petry-projects/.github-private / pr-review.yml

Activity

don-petry commented on Jun 8, 2026

Dev-Lead Implementation Plan

Scope

Root Cause

Implementation Checklist

Tests to Write

Test Command

don-petry commented on Jun 8, 2026

Dev-Lead: Implementation Complete

Plan Execution

Root Cause

Test Results

Files Changed

Notes

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

🟠 DEGRADED: `petry-projects/.github-private` / `pr-review.yml`