Skip to content

fix(storage): reduce Azure table retry pressure#10192

Merged
ReubenBond merged 1 commit into
dotnet:mainfrom
ReubenBond:reminders-azure-backoff
Jun 11, 2026
Merged

fix(storage): reduce Azure table retry pressure#10192
ReubenBond merged 1 commit into
dotnet:mainfrom
ReubenBond:reminders-azure-backoff

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Jun 8, 2026

Copy link
Copy Markdown
Member

Reduces Azure Table Storage pressure when many silos experience storage failures by applying bounded jittered backoff to the outer membership and reminder refresh loops, while delegating Azure Table per-operation retries to the Azure SDK exponential retry policy instead of layering an additional Orleans retry wrapper.

This keeps the retry layers separate: Orleans backs off periodic background loops after failures, and the Azure SDK handles per-call retries, server retry hints, and network timeouts.

Azure Table client retry defaults mirror Azure Storage SDK defaults where applicable, while keeping storage max retries at 5. References:

@ReubenBond ReubenBond force-pushed the reminders-azure-backoff branch 4 times, most recently from 558b7b3 to a83ca16 Compare June 11, 2026 02:20
@ReubenBond ReubenBond changed the title fix(reminders): use exponential backoff with jitter for Azure Table retries Reduce Azure table retry pressure Jun 11, 2026
@ReubenBond ReubenBond changed the title Reduce Azure table retry pressure fix(storage): reduce Azure table retry pressure Jun 11, 2026
@ReubenBond ReubenBond force-pushed the reminders-azure-backoff branch 2 times, most recently from 7849b98 to 3020c65 Compare June 11, 2026 02:40
Apply bounded outer-loop backoff for membership storage operations and delegate Azure Table per-operation retries to the Azure SDK exponential retry policy.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant