Skip to content

fix: solve #5513 — back off /api/auth/token mints on a broken session - #5515

Closed
github-actions[bot] wants to merge 1 commit into
mainfrom
triage/issue-5513-28928335295
Closed

github-actions[bot] wants to merge 1 commit into
mainfrom
triage/issue-5513-28928335295

Conversation

@github-actions

@github-actions github-actions Bot commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

Root cause

JwtApiAuthProvider.getJwt() (host-service) mints a JWT by exchanging the session token at /api/auth/token. On failure it simply threw without caching anything, and there was no in-flight coalescing. Because the cloud tRPC client calls getHeaders() → getJwt() on every request (and invalidateCache() on 401), a broken session turned every request into a fresh network mint with no backoff. A wedged multi-workspace client can generate hundreds of POST /api/auth/token calls per session (the issue reports ~600, ~300× the healthy baseline).

That storm is what tripped Vercel's per-IP rate mitigation (x-vercel-mitigated: deny), which then 403'd the login endpoint behind the same blanket rule — a self-locking loop where the human couldn't re-authenticate to clear the broken state.

This PR addresses item #1 from the issue (client-side backoff / circuit breaker). Items #2 (surfacing x-vercel-mitigated: deny as a real UI error) and #3 (server-side firewall scoping of /api/auth/desktop/connect) are out of scope here.

The fix

In JwtApiAuthProvider:

  • Circuit breaker — after a failed mint, refuse to touch the network again until an exponentially growing cooldown elapses (1s → 2s → 4s … capped at 5 min). While the cooldown is active getJwt() rejects immediately with the last error instead of re-hitting /api/auth/token. A success resets the backoff.
  • In-flight coalescing — concurrent callers on a cold cache now share a single mint instead of fanning out N token exchanges.

This collapses a broken-session burst from hundreds of requests to a slow trickle, well under any rate-based mitigation.

Tests

JwtAuthProvider.test.ts:

  • Reproduction/resolution — 50 sequential failing getHeaders() calls now issue ≤3 network fetches (was 50, one per call).
  • Coalescing — 10 concurrent failing calls share a single in-flight fetch (was 10).
  • All 8 pre-existing auth-provider tests still pass; typecheck and lint clean.

Closes #5513


Open in Stage

Summary by cubic

Add a circuit breaker and in-flight coalescing to JwtApiAuthProvider to stop repeated /api/auth/token mints when a session is broken. Addresses #5513 by backing off failed mints to avoid rate-mitigated lockouts.

  • Bug Fixes
    • Circuit breaker with exponential backoff (1s → 2s → 4s … capped at 5 min); during cooldown getJwt() fails fast with the last error; success resets.
    • Coalesces concurrent getJwt() calls so only one network mint runs on a cold cache.
    • Tests: 50 sequential failures perform ≤3 fetches; 10 concurrent failures share 1 fetch; existing tests still pass.

Written for commit c814377. Summary will update on new commits.

Review in cubic

Add a circuit breaker + in-flight coalescing to JwtApiAuthProvider so a
wedged session can't storm /api/auth/token.
@ghost

ghost commented Jul 8, 2026

Copy link
Copy Markdown

Ready to review this PR? Stage has broken it down into 3 individual chapters for you:

Title
1 Define backoff constants and state for circuit breaker
2 Implement circuit breaker and request coalescing
3 Verify backoff and coalescing behavior in tests
Open in Stage

Chapters generated by Stage for commit c814377 on Jul 8, 2026 8:33am UTC.

@Kitenite Kitenite closed this Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant