Skip to content

feat(dev-lead): proactive provider headroom check before engine invocation #206

Closed

Description

@don-petry

Background

Issue #195 introduced intent-based model dispatch and fixed the copilot write fallback. The third proposed fix — proactive usage monitoring before invoking a provider — was deferred from PR #204 because the proposed implementation has design issues.

Problem

The current fallback only reacts to rate-limits after a full model invocation has already been attempted (and charged). By that point the job fails and the PR is left unhandled. There is no way to know proactively that a provider is at capacity and skip to the next one preemptively.

Why Fix 3 was deferred

The original proposal queried the Anthropic API with a 1-token request to read X-RateLimit-Remaining-Tokens / X-RateLimit-Limit-Tokens response headers. This has two problems:

  1. Wrong rate-limit surface: those headers reflect the per-minute token bucket (e.g. 40K TPM on Sonnet), not the daily subscription cap (You've hit your limit · resets 5pm (UTC)) that actually triggered the 2026-05-16 incident. A per-minute bucket at 70% does not predict a daily cap hit.

  2. Auth mismatch: the dev-lead workflows authenticate via CLAUDE_CODE_OAUTH_TOKEN (OAuth), not ANTHROPIC_API_KEY. The proposed curl probe uses the API key — unavailable in CI.

Proposed approach

A correct implementation needs to:

  • Detect daily cap proximity, not per-minute bucket saturation. The daily limit message (hit your limit · resets H:MMpm (UTC)) only appears in model output after the cap is reached, so there is no pre-flight signal from the Anthropic API today.
  • Alternative: track cumulative token spend across runs within the same workflow session using GitHub Actions step summaries or a shared artifact, and compare against a configurable budget (DEV_LEAD_DAILY_BUDGET_TOKENS). Abort and post a rate-limited marker when the session budget is exhausted.
  • Or: implement a simple elapsed-time guard — if a previous run in the same session exited with status=rate-limited, skip the current run immediately and re-queue for the next session window.

Acceptance criteria

  • Before invoking claude/gemini, check whether a rate-limited marker from the current session (same workflow run or same UTC day window) already exists
  • If so, skip the invocation and return exit 2 immediately (same path as a live rate-limit)
  • Configurable via DEV_LEAD_USAGE_THRESHOLD or similar env var
  • Fail-open: a missing/inaccessible check must not block the invocation
  • No additional API calls that themselves consume tokens or require credentials not already present in CI

References

Activity

  1. added 9 commits that reference this issue on Jun 7, 2026
    0c51ebd
    99674f3
    974b936
    beaeea9
    3923cbe
    78bc054
    92c2507
    3a872ad
    8fbe1ea
  2. 48 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions