fix(orb): break to the fallback model on a rate limit in e2e-test-gen

[kai392]

Fixes #8672

Root cause

runWorkersE2eTestGen's per-model retry loop (src/services/ai-e2e-test-gen.ts) had a bare catch { /* retry / fall through */ } with no rate-limit check — isRateLimitError was never imported in the file. So when the default provider 429s, it burned all 3 E2E_TEST_GEN_ATTEMPTS_PER_MODEL retries against the same rate-limited model (half the total 6-call budget) before ever reaching the fallback model.

That directly contradicts the function's own doc comment ("Mirrors runWorkersSlopOpinion's exact shape") — the slop sibling (ai-slop.ts:169-174), planner.ts, issue-plan-draft.ts, and linked-issue-satisfaction-run.ts all break on a 429, and isRateLimitError's doc explicitly says retrying a live 429 has "near-zero chance of success."

Fix

Imported isRateLimitError from ./ai-review (where the other loops get it) and added if (isRateLimitError(error)) break; to the catch — the identical guard the siblings use. A non-rate-limit error still retries/falls through exactly as before.

Tests (both in the existing ai-e2e-test-gen.test.ts, both arms of the new branch)

• A 429 on every call now breaks to the fallback model after one attempt per model: 2 calls (2 models × 1), not 6. Verified to genuinely guard the fix — reverting the one-line change makes this test fail (6 calls).
• A non-429 error still burns all attempts: 6 calls (2 × 3) — the unchanged retry path.

Validation

• ai-e2e-test-gen.test.ts: 47 passed, rebased onto current main
• lcov scoped to the file: the changed catch line and both branches of the new isRateLimitError guard are covered
• npm run typecheck clean; oxlint clean on the changed lines (the one warning at line 257 is pre-existing, present on clean main, outside this diff); git diff --check clean

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.59%. Comparing base (4217721) to head (6abb905).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8718      +/-   ##
==========================================
- Coverage   93.79%   90.59%   -3.20%     
==========================================
  Files         797       97     -700     
  Lines       79477    22572   -56905     
  Branches    24078     3921   -20157     
==========================================
- Hits        74542    20449   -54093     
+ Misses       3563     1945    -1618     
+ Partials     1372      178    -1194     

Flag	Coverage Δ
backend	100.00% <100.00%> (+4.93%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/services/ai-e2e-test-gen.ts	100.00% <100.00%> (ø)

... and 700 files with indirect coverage changes

[loopover-orb]

Tip

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-26 00:46:59 UTC}

2 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
This PR imports isRateLimitError from ai-review and adds a break-on-429 guard to runWorkersE2eTestGen's catch block, matching the identical pattern already used in ai-slop.ts, planner.ts, and other sibling loops. The fix is correctly traced to its source (the missing import/guard), both branches of the new condition are tested with call-count assertions that would fail if reverted, and the change is narrowly scoped to the described bug (#8672) with no unrelated scope creep. This is a clean, well-tested one-line logic fix.

Nits — 2 non-blocking

• The inline comment at ai-e2e-test-gen.ts:220-222 uses a double-hyphen '--' instead of the em-dash style used elsewhere in the file's comments; purely stylistic.
• Consider whether isRateLimitError's detection logic (not shown in this diff) already covers the 'workers_ai_http_429' error message format used in the new test, since that's the exact string the fix depends on matching.

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8672
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 152 registered-repo PR(s), 81 merged, 6 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	kai392; Gittensor profile; 152 PR(s), 6 issue(s).
Improvement	✅ Minor	risk: clean · value: minor

Linked issue satisfaction

Addressed
The diff imports isRateLimitError and adds the exact `if (isRateLimitError(error)) break;` guard to the catch block in ai-e2e-test-gen.ts, mirroring ai-slop.ts, and includes both required tests (rate-limit break after one attempt, non-rate-limit full retry) verifying the two branches.

Review context

• Author: kai392
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: not available
• Official Gittensor activity: 152 PR(s), 6 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Keep the PR focused and include validation evidence before maintainer review.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

LoopOver approves — the gate is satisfied and CI is green.