fix: rate-limit detector too broad — misclassifies CLI format errors as rate limits

Parent issue

Problem

In run #715, the Copilot CLI exited with error: Invalid command format (a command syntax error), but is_rate_limited classified it as a rate limit and returned exit code 2, causing the session to abort:

[tier1] usage/rate limit detected — exiting with code 2 for engine fallback
limit message:  (stderr: error: Invalid command format.
Did you mean: copilot -i "suggest -p ..."

The detector inspects stdout/stderr for keyword matches. Its current pattern set is too broad and matches on phrases like Invalid command format or similar output that has nothing to do with API rate limiting. The consequence is catastrophic: a simple CLI misconfiguration aborts the entire batch session.

Acceptance criteria

Audit current patterns: enumerate every regex/string pattern in is_rate_limited (and any callers) and justify each one against actual rate-limit response bodies from Claude, Gemini, and Copilot CLI.
Tighten patterns: remove or narrow patterns that can match non-rate-limit errors. At minimum, Invalid command format (and the Did you mean: suggestion) must NOT trigger a rate-limit exit.
Separate error codes: CLI invocation errors (bad flags, wrong syntax) must exit with code 1 (per-PR non-fatal failure per fix: per-PR isolation and single-review retry (closes #132) #133 semantics), NOT code 2 (rate-limit / engine fallback).
Logging: when is_rate_limited returns false but the command failed, log the actual error category (e.g., [error] CLI format error — treating as per-PR failure) so post-mortem analysis is unambiguous.
No regression: existing rate-limit strings from Claude (overloaded, 529, rate_limit_error) and Copilot (rate limit, quota) still correctly trigger exit code 2.

Expected tests

Unit (bats): is_rate_limited returns false for each of the following inputs:
- error: Invalid command format.
- Did you mean: copilot -i ...
- unknown flag: --foo
- command not found
Unit (bats): is_rate_limited returns true for known rate-limit strings:
- overloaded_error / 529
- rate limit exceeded
- quota exceeded
- usage limit reached
Integration: a simulated CLI format-error (mocked exit 1 + Invalid command format stderr) results in a per-PR failure (exit 1) — not a session abort (exit 2) — and the next candidate PR is processed.

@don-petry

Claude finished @don-petry's task in 8m 57s —— View job • claude/issue-148-20260512-1802 • [Create PR ➔](https://github.com/petry-projects/.github-private/compare/main...claude/issue-148-20260512-1802?quick_pull=1&title=Issue%20%23148%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%23148%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)

Implementation complete

PR

petry-projects/.github-private#149

What changed

File	Change
`scripts/engine.sh`	Narrowed patterns in `is_rate_limited`; added `is_cli_error`
`scripts/review-one-pr.sh`	All 3 rate-limit sites: CLI error → exit 1 first; rate limit → exit 2; single-review got `rc ≠ 0` guard
`tests/test_rate_limit.bats`	Unit tests — all 4 AC false-negatives + all 5 AC true-positives
`tests/test_cli_error_routing.bats`	Integration routing test — Copilot incident + regression guards
·

fix: tighten rate-limit detector, add is_cli_error, route CLI errors …

Navigation Menu

Repository navigation

fix: rate-limit detector too broad — misclassifies CLI format errors as rate limits #148

Parent issue

Problem

Acceptance criteria

Expected tests

Implementation complete

PR

What changed

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

fix: rate-limit detector too broad — misclassifies CLI format errors as rate limits #148

Description

Parent issue

Problem

Acceptance criteria

Expected tests

Activity

don-petry commented on May 12, 2026

Implementation complete

PR

What changed

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions