Skip to content

feat(server): cooperative partition reassignment for consumer groups#2734

Merged
spetz merged 3 commits intomasterfrom
collaborative_rebalancing
Feb 14, 2026
Merged

feat(server): cooperative partition reassignment for consumer groups#2734
spetz merged 3 commits intomasterfrom
collaborative_rebalancing

Conversation

@spetz
Copy link
Copy Markdown
Contributor

@spetz spetz commented Feb 12, 2026

Partitions moved instantly on join — the old consumer was
mid-processing, the new one polled the same uncommitted
offset, causing duplicates. Three separate metadata reads
let a left-right swap present inconsistent snapshots.

Cooperative rebalancing: excess partitions marked "pending
revocation" on join, excluded from round-robin. Transfer
completes when committed >= polled offset or timeout.

Atomic resolution: group, member, and partition lookup
under single metadata read guard. last_polled_offsets
moved to PartitionMeta via Arcpapaya::HashMap.

Key mechanics:

  • target_member_id guard against slab slot reuse
  • Leave-scoped offset clearing (not entire group)
  • TOCTOU re-check on shard 0 before completion
  • Release/Acquire ordering on cross-shard atomics
  • Zero-alloc fast path when no pending revocations
  • Two-phase join with separate completion cycles
  • Periodic timeout checker with early-return

Integration tests across Tcp/WebSocket/Quic: reshuffle,
duplicates, auto/manual commit, never-polled transfer,
member leave, target leave, concurrent polls, stale
client, timeout, sequential joins, partition add/delete,
even distribution convergence.

@spetz spetz requested review from hubcio and numinnex February 12, 2026 23:11
@spetz spetz added server iggy-server related change rust Pull requests that update Rust code labels Feb 12, 2026
@spetz spetz force-pushed the collaborative_rebalancing branch from cbdd441 to fd849c2 Compare February 12, 2026 23:36
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 12, 2026

Codecov Report

❌ Patch coverage is 83.54232% with 105 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.75%. Comparing base (153cb37) to head (b0d6c33).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
core/server/src/metadata/reader.rs 64.33% 51 Missing and 5 partials ⚠️
core/server/src/metadata/consumer_group.rs 87.89% 19 Missing and 4 partials ⚠️
core/server/src/configs/defaults.rs 0.00% 15 Missing ⚠️
core/server/src/shard/system/consumer_offsets.rs 89.28% 1 Missing and 5 partials ⚠️
core/server/src/metadata/writer.rs 89.65% 2 Missing and 1 partial ⚠️
core/server/src/metadata/absorb.rs 96.49% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2734      +/-   ##
============================================
+ Coverage     68.58%   68.75%   +0.17%     
  Complexity      585      585              
============================================
  Files           735      736       +1     
  Lines         60370    60926     +556     
  Branches      56783    57339     +556     
============================================
+ Hits          41403    41889     +486     
- Misses        16941    17005      +64     
- Partials       2026     2032       +6     
Flag Coverage Δ
rust 70.28% <83.54%> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
core/server/src/bootstrap.rs 80.68% <100.00%> (+0.04%) ⬆️
core/server/src/configs/displays.rs 93.04% <100.00%> (+0.25%) ⬆️
core/server/src/configs/server.rs 66.66% <ø> (ø)
core/server/src/metadata/consumer_group_member.rs 100.00% <100.00%> (ø)
core/server/src/shard/handlers.rs 78.18% <100.00%> (+1.13%) ⬆️
core/server/src/shard/mod.rs 82.56% <100.00%> (+0.27%) ⬆️
core/server/src/shard/system/consumer_groups.rs 85.00% <100.00%> (+14.45%) ⬆️
core/server/src/shard/system/messages.rs 89.18% <100.00%> (+0.90%) ⬆️
core/server/src/shard/system/partitions.rs 84.54% <100.00%> (+0.06%) ⬆️
core/server/src/shard/system/utils.rs 79.50% <100.00%> (+0.21%) ⬆️
... and 9 more

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spetz spetz force-pushed the collaborative_rebalancing branch 2 times, most recently from bd9a768 to 4346e34 Compare February 13, 2026 09:45
@spetz spetz force-pushed the collaborative_rebalancing branch from 4346e34 to 513f957 Compare February 13, 2026 11:29
@spetz spetz force-pushed the collaborative_rebalancing branch 6 times, most recently from 6d968a5 to f515dc5 Compare February 13, 2026 19:10
@spetz spetz force-pushed the collaborative_rebalancing branch from f515dc5 to b0d6c33 Compare February 13, 2026 19:38
@spetz spetz merged commit 223e8ba into master Feb 14, 2026
62 checks passed
@spetz spetz deleted the collaborative_rebalancing branch February 14, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code server iggy-server related change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants