Skip to content

Support to configure deserializers of KafkaConsumer - #13097

Merged
FrankChen021 merged 1 commit into
apache:masterfrom
eshengit:kafka-custom-serilizer-configurable
Sep 17, 2022
Merged

FrankChen021 merged 1 commit into
apache:masterfrom
eshengit:kafka-custom-serilizer-configurable

Conversation

@eshengit

Copy link
Copy Markdown
Contributor

Fixes #12960.

Description

Allow Kafka Consumer's custom deserializer to be configured after its instantiation.


Key changed/added classes in this PR
  • extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaRecordSupplier.java

This PR has:

  • [ x] been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [ x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • [ x] been tested in a test Druid cluster.

@FrankChen021 FrankChen021 changed the title kafka consumer: custom serializer can't be configured after it's inst… Support to configure deserializers of KafkaConsumer Sep 16, 2022
@FrankChen021

Copy link
Copy Markdown
Member

Thanks for your first contribution @eshengit

@eshengit
eshengit force-pushed the kafka-custom-serilizer-configurable branch from 1fb69ba to fde95dc Compare September 16, 2022 02:16
@eshengit

Copy link
Copy Markdown
Contributor Author

@FrankChen021
Thanks for reviewing! how can I merge once all the check is completed?
should I add you as collaborator of my personal repo, or you add me as druid collaborator?

@FrankChen021

Copy link
Copy Markdown
Member

No worry, we will merge it once the CI finishes.

@FrankChen021

Copy link
Copy Markdown
Member

@eshengit Please fix the failures reported by Travis CI

…antiation (apache#12960)

  * allow kakfa custom serializer to be configured

  * add unit tests
@eshengit
eshengit force-pushed the kafka-custom-serilizer-configurable branch from fde95dc to a46c1d7 Compare September 16, 2022 16:15
@eshengit

Copy link
Copy Markdown
Contributor Author

@FrankChen021 ,

  1. checkestyle error is corrected.
  2. the unit test failure: the module failed indexing-service does not seem to have anything to do with my change which depends on failed module.
    run those failed tests in my IDE didn't show any error.

@FrankChen021
FrankChen021 merged commit da30c80 into apache:master Sep 17, 2022
@kfaraz kfaraz added this to the 25.0 milestone Nov 22, 2022
riovic918data pushed a commit to riovic918data/druid that referenced this pull request Jun 12, 2026
…antiation (apache#12960) (apache#13097)

* allow kakfa custom serializer to be configured

  * add unit tests

Co-authored-by: ellen shen <ellenshen@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kafka consumer: custom serializer can't be configured after it's instantiation

4 participants