Conversation
TestBranchCache_Unwind_DropsStaleAboveFloorLazily and
TestBranchCache_Unwind_FloorBoundary built the cache with NewBranchCache(100).
The tail is sharded into branchCacheTailShards buckets, so a nominal capacity of
100 leaves most shards room for a single entry. Both tests use the two keys
{0xa0,0xb0} and {0xa0,0xb1}; when the per-process maphash seed lands them in the
same capacity-1 shard, the second Put evicts the first and the "must survive"
assertion fails. maphash is seeded per process, so the failure is
non-deterministic — it surfaced on a merge-queue race run and dequeued a PR.
Size the tail via DefaultBranchCacheTailCapacity (as the sibling
TestBranchCache_ShardedTailUnwindAcrossShards already does) so per-shard
capacity can't evict the keys under test.
There was a problem hiding this comment.
Pull request overview
This pull request fixes flakiness in BranchCache unwind tests caused by probabilistic shard collisions in the sharded LRU tail when using a small nominal capacity, which can lead to unexpected evictions depending on the per-process hash seed.
Changes:
- Introduces a dedicated helper (
newUnwindTestCache) to construct aBranchCachewithDefaultBranchCacheTailCapacityfor unwind-focused tests. - Updates the two flaky unwind tests to use the helper instead of
NewBranchCache(100), preventing LRU eviction from affecting the assertions.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
On the current branch (after the #22154 merge) the tail is freelru v0.16.0, and
At cap=100 the two test keys are 2 tail entries in one 100-slot shard, so neither can evict the other under any maphash seed — the seed-dependent collision can't occur. Timeline: the fix commit (07-06) predates #22154 (07-07), which replaced the tail with freelru. Before #22154 the tail sharded 256-ways at cap 100 (per-shard cap 1), so the original analysis held then. #22154 is the merge-base here, so on the merged branch the flake is already neutralized. Not a bug — a larger capacity is strictly safe, and keeping it as defensive hygiene (it matches |
|
#22154 already made these two tests deterministic. The fix is a no-op on current main, so there's nothing left to fix. |
Problem
TestBranchCache_Unwind_DropsStaleAboveFloorLazilyandTestBranchCache_Unwind_FloorBoundaryare flaky. They failed on therace-tests / tests-linux (ubuntu-latest, execution-other, parallel)job of a merge-queue run, which failedci-gateand dequeued PR #22264 (an unrelateddb/statechange that happened to share the batch).Root cause
Both tests built the cache with
NewBranchCache(100). The tail is aShardedLRUsplit intobranchCacheTailShardsbuckets, so a nominal capacity of 100 leaves most shards room for a single entry (per-shard cap 1–2). Both tests use the same two keys{0xa0,0xb0}and{0xa0,0xb1}. When the per-processmaphashseed lands those two keys in the same capacity-1 shard, the secondPutevicts the first and therequire.True(ok, "…must survive…")assertion fails.maphashis seeded per process (and mixes the runtime's per-process hash key), so the collision is non-deterministic across runs — which is why the tests usually pass locally and only occasionally fail in CI, and why both fail together (same key pair, same unlucky seed).Verified by forcing a colliding seed in-process: with
cap=100the survivor is evicted; withDefaultBranchCacheTailCapacityit is retained.Fix
Size the tail via
DefaultBranchCacheTailCapacity(per-shard cap ≈ 195), as the siblingTestBranchCache_ShardedTailUnwindAcrossShardsalready does, so LRU eviction can't interfere with the txN/epoch invalidation these tests actually exercise. Capacity does not affect the unwind assertions themselves, so the tested behavior is unchanged.The single-key unwind tests (
AcrossAllTiers,CurrentEpochSurvives,FrozenSurvives) are not affected — one key always fits in its shard — so they're left as-is.Testing
go test ./execution/commitment/ -run TestBranchCache -count=1passes.Not TDD-first: this is a flaky-test fix where the failure is a per-process probabilistic collision, not a deterministic behavior change; the mechanism was reproduced by pinning a colliding seed in-process.