Repository navigation
Quieter streaming supervisors. - #13392
Conversation
Eliminates two common sources of noise with Kafka supervisors that have large numbers of tasks and partitions: 1) Log the report at DEBUG rather than INFO level at each run cycle. It can get quite large, and can be retrieved via API when needed. 2) Use log4j2.xml to quiet down the org.apache.kafka.clients.consumer.internals package. Avoids a log message per-partition per-minute as part of seeking to the latest offset in the reporting thread. In the tasks, where this sort of logging might be more useful, we have another log message with the same information: "Seeking partition[%s] to[%s]".
|
Code coverage checks are failing because only one line changed, and it only gets logged at debug level. I suggest we merge the patch anyway, ignoring the coverage checks, once other tests are passing. I restarted the other failure, which I think was a flake. |
Yeah, makes sense. I had triggered the phase 2 tests. I think some are still running. We can merge once they finish. |
Thanks. It looks like they're done now. I'll merge this. |
Follow-up to apache#13392, which added this to a single log4j2.xml.
Follow-up to #13392, which added this to a single log4j2.xml.
Eliminates two common sources of noise with Kafka supervisors that have large numbers of tasks and partitions: 1) Log the report at DEBUG rather than INFO level at each run cycle. It can get quite large, and can be retrieved via API when needed. 2) Use log4j2.xml to quiet down the org.apache.kafka.clients.consumer.internals package. Avoids a log message per-partition per-minute as part of seeking to the latest offset in the reporting thread. In the tasks, where this sort of logging might be more useful, we have another log message with the same information: "Seeking partition[%s] to[%s]".
Follow-up to apache#13392, which added this to a single log4j2.xml.
3 participants
Eliminates two common sources of noise with Kafka supervisors that have large numbers of tasks and partitions:
Log the report at DEBUG rather than INFO level at each run cycle.
It can get quite large, and can be retrieved via API when needed.
Use log4j2.xml to quiet down the org.apache.kafka.clients.consumer.internals
package. Avoids a log message per-partition per-minute as part of seeking
to the latest offset in the reporting thread. In the tasks, where this
sort of logging might be more useful, we have another log message with
the same information: "Seeking partition[%s] to[%s]".