confluentinc#205: vert.x: Run user funtions in a Vertical instead of Java thread pool #249

Mirror of confluentinc/parallel-consumer#205
Upstream title (verbatim, 2023-07-07): vert.x: Run user funtions in a Vertical instead of Java thread pool
Opened by astubbs, 2022-03-01 · no upstream discussion
Closed upstream 2023-07-07 as completed by an administrative sweep. Not a resolution.
Summarised, not copied. Discussion belongs here - the Confluent-hosted repository is no longer maintained.

Summary

In the vert.x module, user functions are expected to be non-blocking - but that is enforced only by
convention, and a blocking user function stalls the single worker thread that runs them. Running
user functions inside a vert.x verticle instead of a plain Java thread pool would make blocking
structurally impossible, bring the module closer to the core module's execution model, and pick up
vert.x's own thread-execution monitoring (the blocked-thread checker).

Upstream is candid that the case is not clear-cut: the advantage of a verticle over a thread pool is
debatable, since there is little use for an actor model in this part of the system.

Fork status

Not implemented - confirmed by reading the code. The only mention of Verticle in the vert.x
module is a javadoc extension point on VertxParallelEoSStreamProcessor ("Extension point for
running after Vertx Verticles finish"). User functions are not deployed as verticles.

The observability half of the argument is the part that has aged best: vert.x's blocked-thread
checker would turn "a user function blocked and throughput quietly collapsed" into a logged warning,
which is a real diagnostic gain independent of the actor-model question.

Prior work

Upstream PR #204 ("vert.x: Run user
functions on a Vert.x vertical, instead of a Java Thread pool", branch
confluentinc/improvements/vertx-vertical, head 02ab32894) was closed unmerged in the 2023-06-15
sweep. Commit still reachable.

Should be weighed against the fork's wider position on the vert.x module before investing.

_{Mirrored because the upstream issue was closed by an administrative sweep on 2023-07-07, not by a fix or a decision: all 28 issues closed that day carried the single comment "Closing Issue" and were marked completed. A separate sweep three weeks earlier (2023-06-15) closed 35 unmerged PRs with "Closing - Stale.", several of which implemented these issues. Mapping lives in src/docs/development/upstream-map.yaml.}