fix(swarm): wait for job.Runs directly in scheduler skip test#667
Conversation
The launcher recording a spawn and the job's Runs counter incrementing are sequential but distinct steps in the scheduler goroutine (fireIfIdle's Spawn call, then run's job.incRuns()). The test polled only the launcher's recorded count, then asserted Runs immediately with no wait, so a scheduler goroutine preempted between those two steps could be observed with a stale Runs value under CI load. Wait for Runs itself instead of assuming the launcher signal implies it.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThe scheduler test now waits until the job’s ChangesScheduler test synchronization
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
jatmn
left a comment
There was a problem hiding this comment.
Thanks for the contribution. I do not see any actionable issues from my review.
@Vasanthdev2004 LGTM
There was a problem hiding this comment.
Verified the flake fix the diff adds a waitFor on job.Runs == 2 before asserting, instead of treating the launcher recording the spawn as a proxy for the run counter. That's the right fix for the race: the spawn and the subsequent incRuns() are sequential but distinct steps in the scheduler goroutine, so the old proxy read could catch a stale Runs value. Ran the test 50x under -race and it's green. Approving.
gnanam1990
left a comment
There was a problem hiding this comment.
Verdict: Approve
Test-only fix for a real CI flake (job runs=1 skipped=1, want 2/1 on macOS). The added waitFor on job.Runs == 2 is the right fix.
What looks good
- Root cause is correct: in
run(),fireIfIdle()(launcher spawn) andjob.incRuns()are sequential but distinct steps in the scheduler goroutine, solen(l.recorded()) == 2is not a reliable proxy forRuns == 2. - Matches the existing pattern in this test (already waits on
j.Skipped == 1before asserting). - Comment explains the race without being noisy.
- CI green on all three OSes; @Vasanthdev2004 verified 50× under
-race.
Local verification (merged latest main, no conflicts):
make lint— passgo test -race -count=50 ./internal/swarm/... -run TestSchedulerSkipsWhilePreviousRuns— pass (50/50)
Nits
- No linked
issue-approvedparent issue per CONTRIBUTING — acceptable for a maintainer CI flake fix.
Issues
None.
Problem
TestSchedulerSkipsWhilePreviousRuns intermittently failed on CI (observed on macOS) with
job runs=1 skipped=1, want 2/1.fireIfIdle's Spawn call (which the launcher's recorded() count observes) and run's subsequent job.incRuns() are sequential but distinct steps in the same scheduler goroutine. The test polled only the launcher's recorded count via waitFor, then asserted job.Runs immediately afterward with no wait of its own. Under contention, the scheduler goroutine could be preempted between those two steps, so the test's very next read caught a stale Runs value.
Solution
Wait for job.Runs to actually reach 2 before asserting on it, instead of relying on the launcher's recorded count as a proxy for it.
Verified with
go test ./internal/swarm/... -run TestSchedulerSkipsWhilePreviousRuns -count=50 -race, all passing.Summary by CodeRabbit