test(reminders): stabilize Azure Table reminder tests - #10442
Merged
ReubenBond merged 1 commit intoAug 11, 2026
Merged
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Azure Table reminder tests still had three independent race conditions after the broader reminder-clock work: the restart test waited for one post-reset tick while asserting two, the cross-grain test let independent waits advance one shared fake clock, and failover tests resumed after best-effort liveness stabilization without proving that the shared client could route grain calls through the replacement gateways.
Existing repairs reviewed
Recent merged work addressed adjacent reminder races: #9989, #9997, #10033, #10159, #10296, #10312, #10317, #10400, #10421, and #10430. Current
mainstill skips both #9557 methods and retains the mismatched restart wait and independent cross-grain clock drivers. The latest stale-gateway failure occurred on a head containing #10400; #10430 merged afterward but only changes exact-due reminder recovery and does not affect the failing clientGetCounterroute.Solution
Use the single reminder-clock orchestrator and exact counter assertions for the restart and cross-grain scenarios, then re-enable both quarantined tests. Add an explicit grain-reachability phase before and after failover: only expected lifecycle transport failures are retried, each retry re-runs liveness stabilization, and the existing test cancellation deadline remains authoritative.
This preserves the failover coverage without sleeps, test skips, or increased timeouts, and makes failures report the affected grain set and last transport error.
Fixes #9557