Repository navigation
bench(h2o): window Top-N sweep over a declared-sorted input - #24732
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #24732 +/- ##
==========================================
- Coverage 81.60% 81.60% -0.01%
==========================================
Files 1123 1123
Lines 408898 408898
Branches 408898 408898
==========================================
- Hits 333670 333664 -6
- Misses 55625 55626 +1
- Partials 19603 19608 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kosiew
left a comment
There was a problem hiding this comment.
@SubhamSinghal, thanks for adding the sorted-input H2O Window Top-N benchmarks. The overall setup looks good to me. I have one non-blocking suggestion to make sure the benchmark continues to exercise the intended declared-ordering setup as it evolves.
| # operator replaces the heap one for declared-sorted input. That is the point of | ||
| # the subgroup — the comparison is this file's numbers across two branches, so | ||
| # the assertion must not be the thing that differs between them. | ||
| expect_plan PartitionedTopKExec |
There was a problem hiding this comment.
Could we also add a plan-level check that the WITH ORDER declaration actually reaches DataSourceExec, for example by checking the source output_ordering? The current PartitionedTopKExec assertion does not verify this because that operator only requires key distribution and can still appear with an unordered source. Having a separate ordering check would help catch accidental loss of the declared-order setup, especially if an ordering-dependent Top-N implementation uses this benchmark in the future.
|
🚀 |
Which issue does this PR close?
Related to #6899 — adds benchmark coverage for the WindowTopN operators.
Rationale for this change
The existing
h2o --subgroup windowTop-N sweep (q13–q29) registersxwith no declared ordering, sooutput_ordering()isNone. Any plan that depends on the input being sorted is unreachable from those queries, however the data happens to be laid out on disk. This adds awindow_sortedsubgroup that publishes the same sweep over aWITH ORDERtable, so ordering-dependent plans can be measured.What changes are included in this PR?
A
window_sortedh2o subgroup: 16 queries covering ROW_NUMBER / RANK / DENSE_RANK × 100 / 1K / 10K / 100K partitions, plus heavy-ties variants.Two
loaddirectives: the existingload_window_${SIZE}_${FORMAT}.sqlcreatesx, then a new script writes a sorted copy viaCOPY (... ORDER BY pk, ob DESC)and registers itWITH ORDER (pk ASC, ob DESC). Reusing the existing loader keeps both the--sizeand--formataxes working with no duplication.loadis untimed, so the sort stays out of the measurement.WITH ORDERcan only name columns, so the partition key (id3 % N) and the tie expression are materialized aspkandob.Asserts guard the three ways this could silently measure nothing: both config flags took effect, the sorted copy holds every source row, and
obhas the cardinality the query name claims.expect_plan PartitionedTopKExecis deliberately the shared substring, so the same file validates the heap operator and any streaming variant that replaces it.No Rust changes; benchmark files only.
Are these changes tested?
benchmark_runner h2o --subgroup window_sortedruns all 16 queries green. Row counts are exact: 2 per partition for the distinct-ORDER-BY shapes, and 1,010,812 (RANK) / 2,020,722 (DENSE_RANK) for the tie shapes — the top-1 and top-2 of 10 distinct values over 10M rows.Are there any user-facing changes?
No.