Repository navigation
retry middleware — RESOURCE_EXHAUSTED retries use exponential backoff without consulting Retry-After header #5270
Description
Finding
The retry middleware in js/plugins/middleware/src/retry.ts includes RESOURCE_EXHAUSTED in its default retry statuses but uses pure exponential backoff without consulting the provider’s Retry-After header.
When a provider returns HTTP 429 with Retry-After: 60, the retry middleware fires at initialDelayMs (default 1000ms) intervals — inside the provider cooldown window. All retries fail until the window expires naturally.
Evidence
DEFAULT_RETRY_STATUSES includes RESOURCE_EXHAUSTED.
Retry delay computation:
let delay = currentDelay;
if (!noJitter) {
delay = delay + 1000 * Math.pow(2, i) * Math.random();
}
await new Promise((resolve) => __setTimeout(resolve, delay));
currentDelay = Math.min(currentDelay * backoffFactor, maxDelayMs);No Retry-After header consulted at any point.
The correct behavior
When RESOURCE_EXHAUSTED is received, check the response headers for Retry-After. If present, use that value as the minimum delay before retrying. Fall back to exponential backoff only when the header is absent.
Related pattern
retry-after-ignored-exponential-backoff
Corpus reference: https://github.com/SirBrenton/pitstop-truth
Activity
MichaelDoyle commentedon May 11, 2026
Hi @SirBrenton - please provide an example of a model provider used by Genkit that uses the
Retry-Afterheader.SirBrenton commentedon May 11, 2026
@MichaelDoyle — Anthropic’s API returns
Retry-Afteron 429 responses. Example:When this fires through the retry middleware with
RESOURCE_EXHAUSTED, the middleware retries at 1s intervals for 60 seconds — burning all attempts inside the cooldown window.Corpus receipt documenting this behavior:
PT-2026-03-21-github-openclaw-venice-models-429-retry-after-unwired
OpenAI also sends
Retry-After. Google’s Gemini API sends it as well — documented in the corpus.MichaelDoyle commentedon May 11, 2026
@pavelgj @ifielker @apascal07 this one is a bit vage/general - but does make sense. Do we have a facility for relaying the back-off time via
generatetoday? If not, could be a nice addition. It would vary plugin by plugin; contrary to the report, the various model providers and SDKs each handle this differently.SirBrenton commentedon May 11, 2026
Agreed, @MichaelDoyle — it varies by plugin.
The Google AI plugin is the clearest case:
js/plugins/google-genai/src/googleai/client.tsalready classifies 429 asRESOURCE_EXHAUSTED. The retry middleware could readerror.headers?.get('retry-after')before computing delay.Happy to sketch a plugin-specific fix if that’s more useful than a middleware-level change.
pavelgj commentedon May 11, 2026
Totally valid feature request. We would have to plumb the "retry after" info through the GenkitError... add a field in there (I think must be a dedicated field) and if the plugin properly sets it, then middleware can respect it.
Option 1: top-level
retryAfterMsfield onGenkitErrorOption 2: convention in existing
detailfieldOption 3: new
responseMetadata(orhttpMetadata) bag onGenkitErrorI'd choose between 1 and 3...
SirBrenton commentedon May 11, 2026
@pavelgj — Option 1 feels cleanest. A dedicated
retryAfterMsfield makes the intent explicit and keeps middleware logic simple.Looking forward to seeing it land.
pavelgj commentedon May 19, 2026
See #5343
SirBrenton commentedon May 19, 2026
@pavelgj this is exactly what the corpus is for.
When #5343 merges, I'll add a receipt documenting the fix. That becomes the canonical reference for Retry-After propagation in framework middleware.
Thank you for the architecture discussion.
Option 1 was the right call.