Repository navigation
Release train for 1.0 #172
Description
- README cleanup/update regarding maintenance
- README regarding “alpha” state
- offset documentation
- javadoc cleanup sweep
- dead code sweep
- todo sweep
- disabled test sweep
- potentially remove streaming interfaces
Issues targeted for 0.5:
In priority order - only items marked high will block the 1.0 release.
Tech debt / refactoring:
- refactor: Poll Context object for API #223
- refactor: Shard object #224
- refactor: Partition state tracking instead of search #218
- refactor: Queue unification #219
Tech debt:
- Refactor: Live tracking of offsets as they change, so we don't need to scan for them #201
- Under some conditions, a shard (by partition or key), can get starved for attention #236
- build: Update Kafka to 3.1.0 #229
- feature: Retry count available in header #197
- build: Crank up Enforcer and turn on ossindex audit #215
Enhancements High:
Features High:
Features Low:
Issues targeted for NEXT:
Fixes:
- vert.x: Run user funtions in a Vertical instead of Java thread pool #205
- Bug: ConsumerOffsetCommitter goes into failure state after broker downtime #203
- Subscribing to two or more topics with KEY ordering, results in messages of the same Key never being processed #184
- NoSuchFieldException when using consumer inherited from KafkaConsumer #195
- Message with null key lead to continuous failure when using KEY ordering #318
Tech debt:
- Major refactor: Package restructure #271
- fixes: #225 Support Java 16+ #289
- major: #200 Shared nothing architecture - Partition Events #270
- Docker naming, fixes and improvements #220
- Feature Request: Unique thread names for PC instances for logging #192
- Refactor test base #290
Enhancements High:
- feature: #242 Explicit retriable exception for cleaner ERROR logging #268
- Let user function throw retry exception, instead of raw errors which pollute log #242
- Allow customization of the ThreadPoolExecutor #78
- Enhanced retry epic #65
- Slow produce with pollAndProduce with many output messages #297
- When unexpected magic bytes occur, improve the feedback in the error shown to the user #327
Features High:
Features Low:
- Feature Request: Pause/Resume PC #193
- Circuit Breaker to stop processing records if HTTP endpoint errors cross a threshold #110
Issues targeted for NEXT NEXT:
Features:
- Ability use different consume and produce types for the key and value #175
- Add support for JDK's CompletableFuture into the API as part of the Reactor.io API, add docs #170
- Health-checks #71
- Enable changing of topic subscription before or after PC has started #187
- Feature: Allow unordered processing of messages without a key in 'KEY' ordering mode #183
- Need ability to seek to the beginning of a partition #191
- Ensure all PC API's are thread safe #186
- Expose way to know if connection to Broker is lost then later regained #185
- MockConsumer does not work #176
- Add a dead letter queue (DQL) implementation #310
- Have processing functions attached to topics instead of a global one for all subscribed topics #372
Tech debt:
- Consider adopting error prone and checker #259
- Adopt Truth Generator #247
- Consider turning on code sorting in the formatter #246
- Parallel join technique documentation #81
Issues targeted for post 1.0:
Features high:
- performance: Async / non blocking producer sends #232
- Improve test suite speed to pre-batch merge levels #226
- build: Update to java 17 #225
- Try refactoring WC type from String to Enum #241
Examples Low:
- How to take a single message, and distribute it to several http end points in parallel, with DLQ for failure #178
- Spring boot example #171
Tech debt:
- Refactor OffsetMapCodecManager.java #233
- Matrix test against multiple AK and JDK versions in GH #103
- Refactor: Consider a shared nothing architecture, to reduce thread complexity #200
- ProducerManager should handle different types of transaction failures appropriately #144
Features low:
Activity
sribee commentedon Jan 4, 2022
Antony Stubbs (@astubbs) - This is a very interesting and useful project for improving the kafka consumer performance and also to reduce the load on the broker. Is there a timeline you could provide for the 1.0 release? thanks in advance.
astubbs commentedon Jan 10, 2022
Yes, will be Q1 22. Once I finish up some work on another project that I'm on, I'm coming back to PC to finish up some things and do the release.
It should be noted, the only changes for 1.0 will be a couple new features, some tidy up, tighten up some of the API and things. There are no known bugs or issues that would be fixed.
astubbs commentedon Feb 4, 2022
Set target issues in priority order. Only high priority items will block 1.0
raphaelauv commentedon Feb 19, 2022
#18 is in high priority , that's cool !
astubbs commentedon Feb 25, 2022
Batching is now drafted complete, and awaiting merge early next week!
I know I know, it's been waiting a while - thanks for the help Benedikt Linse (@1123) !
astubbs commentedon Mar 16, 2022
Merged!
[-]Release 1.0[/-][+]Release train for 1.0[/+]astubbs commentedon May 6, 2022
FYI 0.5 is out
colinhunt commentedon Jul 27, 2023
Hello, just inquiring if we can get a progress update on this. I am investigating using this at my company, but it still being on version 0 is a potential hindrance for adoption. Thanks!
raphaelauv commentedon Jul 27, 2023
scikit-learn have been on v0 for more than 12 years 😁
astubbs commentedon Aug 5, 2026
Tracked in the maintained fork as astubbs#135, which carries a summary of this
issue and its current status there.
The Confluent-hosted version of this project is no longer maintained - its README says so.
Parallel Consumer itself continues in my fork: https://github.com/astubbs/parallel-consumer
Nothing in this repository is closed or changed by this comment.