Skip to content

test(reminders): stabilize Azure Table reminder tests - #10442

Merged
ReubenBond merged 1 commit into
dotnet:mainfrom
ReubenBond:reubenbond-fix-azure-reminder-flakes-9557
Aug 11, 2026
Merged

ReubenBond merged 1 commit into
dotnet:mainfrom
ReubenBond:reubenbond-fix-azure-reminder-flakes-9557

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Aug 11, 2026 •

Copy link
Copy Markdown
Member

Problem

Azure Table reminder tests still had three independent race conditions after the broader reminder-clock work: the restart test waited for one post-reset tick while asserting two, the cross-grain test let independent waits advance one shared fake clock, and failover tests resumed after best-effort liveness stabilization without proving that the shared client could route grain calls through the replacement gateways.

Existing repairs reviewed

Recent merged work addressed adjacent reminder races: #9989, #9997, #10033, #10159, #10296, #10312, #10317, #10400, #10421, and #10430. Current main still skips both #9557 methods and retains the mismatched restart wait and independent cross-grain clock drivers. The latest stale-gateway failure occurred on a head containing #10400; #10430 merged afterward but only changes exact-due reminder recovery and does not affect the failing client GetCounter route.

Solution

Use the single reminder-clock orchestrator and exact counter assertions for the restart and cross-grain scenarios, then re-enable both quarantined tests. Add an explicit grain-reachability phase before and after failover: only expected lifecycle transport failures are retried, each retry re-runs liveness stabilization, and the existing test cancellation deadline remains authoritative.

This preserves the failover coverage without sleeps, test skips, or increased timeouts, and makes failures report the affected grain set and last transport error.

Fixes #9557

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ReubenBond
ReubenBond merged commit b4db395 into dotnet:main Aug 11, 2026
72 checks passed
@ReubenBond
ReubenBond deleted the reubenbond-fix-azure-reminder-flakes-9557 branch August 11, 2026 02:18
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 10, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky Tests: Tester.AzureUtils.TimerTests.ReminderTests_AzureTable - Multiple Test Methods

1 participant