Repository navigation
[SPARK-47384][BUILD] Upgrade RoaringBitmap to 1.0.5 - #45507
panbingkun wants to merge 2 commits into
Conversation
|
Let it run through GA first, and then I will add the benchmark( |
| Num Maps: 50000 Fetch partitions:1500 1767 1830 55 0.0 1767363076.0 0.4X | ||
| Num Maps: 50000 Fetch partitions:500 714 716 2 0.0 713899011.0 1.0X | ||
| Num Maps: 50000 Fetch partitions:1000 1602 1647 59 0.0 1602358288.0 0.4X | ||
| Num Maps: 50000 Fetch partitions:1500 2517 2538 22 0.0 2517027078.0 0.3X |
There was a problem hiding this comment.
Although we care only the relative ratio, it seems that this release is a little slower than before?
There was a problem hiding this comment.
I have noticed this as well. It may be caused by some fluctuations. Wait a little bit and I will run it a few more times to see the performance data.
There was a problem hiding this comment.
New result seems to be better. Thank you for re-run. Let's merge this.
================================================================================================
MapStatuses Convert Benchmark
================================================================================================
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
MapStatuses Convert: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Num Maps: 50000 Fetch partitions:500 719 731 18 0.0 719070889.0 1.0X
Num Maps: 50000 Fetch partitions:1000 1456 1468 11 0.0 1455501002.0 0.5X
Num Maps: 50000 Fetch partitions:1500 2207 2223 18 0.0 2207061425.0 0.3X
|
Just for the record, new the benchmark( |
|
Merged to master for Apache Spark 4.0.0. |
2 participants
What changes were proposed in this pull request?
The pr aims to upgrade
RoaringBitmapfrom1.0.1to1.0.5.Why are the changes needed?
Release notes: https://github.com/RoaringBitmap/RoaringBitmap/releases/tag/1.0.5
This version includes some bugs fixed, eg:
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GA.
Was this patch authored or co-authored using generative AI tooling?
No.