Repository navigation
Add HashVectorGrouper based on MemoryOpenHashTable. - #9314
Conversation
Additional supporting changes: 1) Modifies VectorGrouper interface to use Memory instead of ByteBuffers. 2) Modifies BufferArrayGrouper to match the new VectorGrouper interface. 3) Removes "implements VectorGrouper" from BufferHashGrouper.
|
This pull request introduces 1 alert when merging 672fa2d into 54bc4dc - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 2ce825f into 54bc4dc - view on LGTM.com new alerts:
|
|
Please check the LGTM warning. |
|
I just looked at it. It's complaining that |
|
This pull request introduces 1 alert when merging 78bc768 into 2e1dbe5 - view on LGTM.com new alerts:
|
jihoonson
left a comment
There was a problem hiding this comment.
The latest change is enough to prevent the overflow.
* Add HashVectorGrouper based on MemoryOpenHashTable. Additional supporting changes: 1) Modifies VectorGrouper interface to use Memory instead of ByteBuffers. 2) Modifies BufferArrayGrouper to match the new VectorGrouper interface. 3) Removes "implements VectorGrouper" from BufferHashGrouper. * Fix comment. * Fix another comment. * Remove unused stuff. * Include hoisted bounds checks. * Checks against too-large keySpaces.
This causes vectorized hash-based groupBys to use MemoryOpenHashTable
instead of ByteBufferHashTable (see #9308).
Additional supporting changes:
Benchmarks:
The largest improvements are on queries 10 and 11 (~50%) and queries 15–18 (3–5x). Queries 12–14 look very slightly faster or maybe unchanged.
Queries 10 & 11 are groupBys on two strings and would use hash-based grouping.
Queries 12–14 are groupBys on one string and would use array-based grouping. It makes sense that they haven't changed much, since they wouldn't use hash-based grouping. The changes to the array-based grouping code were more minimal.
Queries 15–18 are groupBys on one long column and would use hash-based grouping.