From e04f6a8afa2789b80f130959bc7891ea442bd477 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Isma=C3=ABl=20Mej=C3=ADa?= Date: Thu, 4 Jun 2026 21:50:42 +0200 Subject: [PATCH 01/15] [SPARK-56892][SQL] Bulk read optimization for Parquet DELTA_BINARY_PACKED decoding Add a bulk read path to VectorizedDeltaBinaryPackedReader that decodes an entire mini-block of values in a tight loop instead of reading one value at a time. This avoids per-value method-call overhead and enables the JIT to better vectorize the inner unpacking loop. Key changes: - readBulkLoop/skipBulkLoop in VectorizedDeltaBinaryPackedReader that process full mini-blocks at once, falling back to the scalar path only for the final partial mini-block. - Helper methods in VectorizedReaderBase (readBulkIntegers, readBulkLongs, skipBulk) that delegate to the new bulk path. - Wire unsigned integer/long updaters through the bulk read path. - Fix buffer-reuse bug in encodeUnsignedLongBigEndian for zero values that caused a ~14.6x performance regression on UINT_64 columns (Parquet UINT_64 -> Spark Decimal(20,0)). Benchmark results regenerated via GHA workflow on AMD EPYC 9V45 (same machine class as the upstream baseline from SPARK-56633): - DELTA_BINARY_PACKED INT32: 2.0x-3.3x faster - DELTA_BINARY_PACKED INT64: 2.9x-6.6x faster - DELTA_BYTE_ARRAY: 2.3x-2.6x faster - DELTA_LENGTH_BYTE_ARRAY: 2.0x-3.3x faster - Variant reads: 1.7x-2.4x faster - readUnsignedLongs: ~14.6x faster (bug fix) --- ...izedDeltaReaderBenchmark-jdk21-results.txt | 96 ++++++------- ...izedDeltaReaderBenchmark-jdk25-results.txt | 106 +++++++-------- ...VectorizedDeltaReaderBenchmark-results.txt | 106 +++++++-------- .../parquet/ParquetDictionary.java | 4 +- .../parquet/ParquetVectorUpdaterFactory.java | 12 +- .../VectorizedDeltaBinaryPackedReader.java | 126 +++++++++++++++++- .../parquet/VectorizedReaderBase.java | 40 ++++++ .../VectorizedDeltaReaderBenchmark.scala | 1 + 8 files changed, 325 insertions(+), 166 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt index a7199750be601..7c20ec159f98e 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt @@ -2,92 +2,92 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1011-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 451.8 2.2 1.0X -skipIntegers, constant 3 3 0 415.1 2.4 0.9X -readIntegers, monotonic 3 3 0 369.7 2.7 0.8X -skipIntegers, monotonic 3 3 0 415.6 2.4 0.9X -readIntegers, small-delta random 3 3 0 308.6 3.2 0.7X -skipIntegers, small-delta random 3 3 0 358.6 2.8 0.8X -readIntegers, wide random 4 4 0 249.2 4.0 0.6X -skipIntegers, wide random 4 4 0 281.4 3.6 0.6X +readIntegers, constant 2 2 0 486.9 2.1 1.0X +skipIntegers, constant 2 2 0 560.5 1.8 1.2X +readIntegers, monotonic 2 2 0 487.1 2.1 1.0X +skipIntegers, monotonic 2 2 0 561.9 1.8 1.2X +readIntegers, small-delta random 3 3 0 333.5 3.0 0.7X +skipIntegers, small-delta random 3 3 0 376.5 2.7 0.8X +readIntegers, wide random 4 4 0 269.7 3.7 0.6X +skipIntegers, wide random 4 4 0 294.9 3.4 0.6X ================================================================================================ DELTA_BINARY_PACKED INT64 ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1011-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 5 6 0 195.2 5.1 1.0X -skipLongs, constant 6 6 0 175.4 5.7 0.9X -readLongs, monotonic 7 7 0 158.8 6.3 0.8X -skipLongs, monotonic 6 6 0 175.6 5.7 0.9X -readLongs, small-delta random 8 8 0 139.4 7.2 0.7X -skipLongs, small-delta random 7 7 0 152.5 6.6 0.8X -readLongs, wide random 10 10 1 102.9 9.7 0.5X -skipLongs, wide random 6 9 2 185.5 5.4 1.0X +readLongs, constant 2 2 0 505.2 2.0 1.0X +skipLongs, constant 2 2 0 543.1 1.8 1.1X +readLongs, monotonic 2 2 0 505.0 2.0 1.0X +skipLongs, monotonic 2 2 0 557.9 1.8 1.1X +readLongs, small-delta random 3 3 0 354.3 2.8 0.7X +skipLongs, small-delta random 3 3 0 377.2 2.7 0.7X +readLongs, wide random 6 6 0 184.6 5.4 0.4X +skipLongs, wide random 5 6 0 191.8 5.2 0.4X ================================================================================================ DELTA_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1011-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 35 41 4 29.6 33.8 1.0X -skipBinary, no overlap, len=16 41 45 3 25.9 38.6 0.9X -readBinary, half overlap, len=16 41 44 3 25.4 39.3 0.9X -skipBinary, half overlap, len=16 47 50 5 22.3 44.8 0.8X -readBinary, full overlap, len=16 42 44 3 25.1 39.9 0.8X -skipBinary, full overlap, len=16 48 50 3 22.0 45.6 0.7X -readBinary, half overlap, len=64 42 42 1 25.2 39.6 0.9X -skipBinary, half overlap, len=64 47 50 14 22.4 44.6 0.8X +readBinary, no overlap, len=16 27 31 3 39.4 25.4 1.0X +skipBinary, no overlap, len=16 31 31 1 34.2 29.2 0.9X +readBinary, half overlap, len=16 32 33 1 32.3 31.0 0.8X +skipBinary, half overlap, len=16 38 39 1 27.7 36.1 0.7X +readBinary, full overlap, len=16 33 34 1 31.7 31.6 0.8X +skipBinary, full overlap, len=16 38 39 1 27.4 36.5 0.7X +readBinary, half overlap, len=64 33 34 1 31.6 31.6 0.8X +skipBinary, half overlap, len=64 38 38 1 27.9 35.9 0.7X ================================================================================================ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1011-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 20 21 0 51.7 19.3 1.0X -skipBinary, payloadLen=8 10 10 0 106.3 9.4 2.1X -readBinary, payloadLen=32 16 18 2 63.9 15.7 1.2X -skipBinary, payloadLen=32 6 6 0 175.9 5.7 3.4X -readBinary, payloadLen=128 19 19 0 56.0 17.8 1.1X -skipBinary, payloadLen=128 6 6 0 176.3 5.7 3.4X -readBinary, payloadLen=512 41 43 3 25.6 39.0 0.5X -skipBinary, payloadLen=512 6 6 1 176.4 5.7 3.4X +readBinary, payloadLen=8 16 17 0 65.5 15.3 1.0X +skipBinary, payloadLen=8 6 6 0 182.5 5.5 2.8X +readBinary, payloadLen=32 16 16 0 64.7 15.5 1.0X +skipBinary, payloadLen=32 6 6 0 182.5 5.5 2.8X +readBinary, payloadLen=128 19 19 0 56.5 17.7 0.9X +skipBinary, payloadLen=128 6 6 0 183.5 5.5 2.8X +readBinary, payloadLen=512 40 41 1 26.4 37.8 0.4X +skipBinary, payloadLen=512 6 6 1 182.7 5.5 2.8X ================================================================================================ Variant reads ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1011-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 7 7 0 142.3 7.0 1.0X -readShorts (INT32) 7 8 0 143.0 7.0 1.0X -readUnsignedIntegers (INT32 -> Long) 7 7 0 147.1 6.8 1.0X -readUnsignedLongs (INT64 -> Decimal(20,0)) 232 243 23 4.5 221.5 0.0X -skipBytes 4 4 0 285.4 3.5 2.0X -skipShorts 4 4 0 285.4 3.5 2.0X -readByte (INT32 single-value) 13 13 1 80.2 12.5 0.6X -readShort (INT32 single-value) 13 13 1 82.0 12.2 0.6X -readInteger (INT32 single-value) 13 13 0 80.2 12.5 0.6X -readLong (INT64 single-value) 14 14 0 74.9 13.3 0.5X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 64 67 3 16.5 60.6 0.1X +readBytes (INT32) 5 5 0 228.8 4.4 1.0X +readShorts (INT32) 8 8 0 131.2 7.6 0.6X +readUnsignedIntegers (INT32 -> Long) 8 8 0 129.5 7.7 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 26 26 0 40.5 24.7 0.2X +skipBytes 9 9 0 120.4 8.3 0.5X +skipShorts 9 9 0 120.6 8.3 0.5X +readByte (INT32 single-value) 14 14 1 76.8 13.0 0.3X +readShort (INT32 single-value) 14 14 1 76.8 13.0 0.3X +readInteger (INT32 single-value) 14 14 1 76.8 13.0 0.3X +readLong (INT64 single-value) 15 16 1 68.5 14.6 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 58 62 2 18.1 55.3 0.1X diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt index fe7b64c3834b3..c30e84b6371d2 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt @@ -2,92 +2,92 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1010-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 487.1 2.1 1.0X -skipIntegers, constant 3 3 0 364.8 2.7 0.7X -readIntegers, monotonic 3 4 0 303.1 3.3 0.6X -skipIntegers, monotonic 3 3 0 365.3 2.7 0.7X -readIntegers, small-delta random 4 4 0 241.3 4.1 0.5X -skipIntegers, small-delta random 4 4 0 278.7 3.6 0.6X -readIntegers, wide random 5 5 0 201.4 5.0 0.4X -skipIntegers, wide random 5 5 0 225.6 4.4 0.5X +readIntegers, constant 2 2 0 454.3 2.2 1.0X +skipIntegers, constant 2 2 0 528.1 1.9 1.2X +readIntegers, monotonic 2 2 0 451.0 2.2 1.0X +skipIntegers, monotonic 2 2 0 528.2 1.9 1.2X +readIntegers, small-delta random 3 3 0 309.5 3.2 0.7X +skipIntegers, small-delta random 3 3 0 345.9 2.9 0.8X +readIntegers, wide random 4 5 1 247.4 4.0 0.5X +skipIntegers, wide random 4 4 0 261.4 3.8 0.6X ================================================================================================ DELTA_BINARY_PACKED INT64 ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1010-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 6 7 1 163.0 6.1 1.0X -skipLongs, constant 7 7 0 152.1 6.6 0.9X -readLongs, monotonic 8 8 1 139.1 7.2 0.9X -skipLongs, monotonic 7 7 0 152.0 6.6 0.9X -readLongs, small-delta random 9 9 0 122.7 8.2 0.8X -skipLongs, small-delta random 8 8 0 133.9 7.5 0.8X -readLongs, wide random 11 11 0 94.4 10.6 0.6X -skipLongs, wide random 10 10 0 100.9 9.9 0.6X +readLongs, constant 2 2 0 451.2 2.2 1.0X +skipLongs, constant 2 2 0 527.4 1.9 1.2X +readLongs, monotonic 2 2 0 452.3 2.2 1.0X +skipLongs, monotonic 2 2 0 528.5 1.9 1.2X +readLongs, small-delta random 3 3 0 313.0 3.2 0.7X +skipLongs, small-delta random 3 3 0 346.0 2.9 0.8X +readLongs, wide random 6 6 0 163.3 6.1 0.4X +skipLongs, wide random 6 6 0 172.0 5.8 0.4X ================================================================================================ DELTA_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1010-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V74 80-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 37 38 2 28.2 35.5 1.0X -skipBinary, no overlap, len=16 42 43 2 25.2 39.7 0.9X -readBinary, half overlap, len=16 43 44 2 24.3 41.2 0.9X -skipBinary, half overlap, len=16 48 49 2 21.8 45.9 0.8X -readBinary, full overlap, len=16 44 47 4 24.0 41.7 0.9X -skipBinary, full overlap, len=16 48 50 2 21.6 46.3 0.8X -readBinary, half overlap, len=64 44 45 2 23.8 42.0 0.8X -skipBinary, half overlap, len=64 49 53 4 21.6 46.3 0.8X +readBinary, no overlap, len=16 30 30 1 35.5 28.2 1.0X +skipBinary, no overlap, len=16 32 35 3 32.5 30.7 0.9X +readBinary, half overlap, len=16 36 38 3 29.1 34.4 0.8X +skipBinary, half overlap, len=16 40 44 4 26.3 38.0 0.7X +readBinary, full overlap, len=16 36 38 2 28.8 34.7 0.8X +skipBinary, full overlap, len=16 40 42 3 26.3 38.0 0.7X +readBinary, half overlap, len=64 37 44 2 28.5 35.1 0.8X +skipBinary, half overlap, len=64 40 43 3 26.4 37.9 0.7X ================================================================================================ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1010-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V74 80-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 22 22 0 48.7 20.5 1.0X -skipBinary, payloadLen=8 11 11 0 97.0 10.3 2.0X -readBinary, payloadLen=32 21 22 0 48.9 20.5 1.0X -skipBinary, payloadLen=32 11 11 0 97.0 10.3 2.0X -readBinary, payloadLen=128 24 24 1 43.7 22.9 0.9X -skipBinary, payloadLen=128 11 11 1 97.1 10.3 2.0X -readBinary, payloadLen=512 51 53 1 20.5 48.7 0.4X -skipBinary, payloadLen=512 11 11 0 97.1 10.3 2.0X +readBinary, payloadLen=8 21 21 0 50.1 20.0 1.0X +skipBinary, payloadLen=8 6 6 0 165.8 6.0 3.3X +readBinary, payloadLen=32 21 21 0 50.4 19.8 1.0X +skipBinary, payloadLen=32 6 6 0 165.2 6.1 3.3X +readBinary, payloadLen=128 23 23 1 45.4 22.0 0.9X +skipBinary, payloadLen=128 6 6 0 165.5 6.0 3.3X +readBinary, payloadLen=512 46 49 1 22.8 43.9 0.5X +skipBinary, payloadLen=512 6 6 0 165.8 6.0 3.3X ================================================================================================ Variant reads ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1010-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V74 80-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 9 10 1 115.6 8.6 1.0X -readShorts (INT32) 9 10 1 116.0 8.6 1.0X -readUnsignedIntegers (INT32 -> Long) 9 9 1 117.1 8.5 1.0X -readUnsignedLongs (INT64 -> Decimal(20,0)) 211 223 27 5.0 201.5 0.0X -skipBytes 9 9 0 122.2 8.2 1.1X -skipShorts 9 9 0 122.3 8.2 1.1X -readByte (INT32 single-value) 13 13 0 80.1 12.5 0.7X -readShort (INT32 single-value) 14 15 1 72.7 13.7 0.6X -readInteger (INT32 single-value) 14 15 0 72.9 13.7 0.6X -readLong (INT64 single-value) 16 17 1 64.3 15.6 0.6X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 79 81 2 13.3 74.9 0.1X +readBytes (INT32) 5 5 0 199.9 5.0 1.0X +readShorts (INT32) 8 8 0 127.7 7.8 0.6X +readUnsignedIntegers (INT32 -> Long) 9 9 0 116.1 8.6 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 29 30 1 35.7 28.0 0.2X +skipBytes 4 4 0 265.7 3.8 1.3X +skipShorts 4 4 0 265.9 3.8 1.3X +readByte (INT32 single-value) 13 14 1 78.3 12.8 0.4X +readShort (INT32 single-value) 13 14 0 78.3 12.8 0.4X +readInteger (INT32 single-value) 13 14 0 78.4 12.8 0.4X +readLong (INT64 single-value) 15 15 0 70.9 14.1 0.4X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 66 70 2 15.8 63.2 0.1X diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt index 9ff1678901e9f..1d7a882581173 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt @@ -2,92 +2,92 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1011-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V45 96-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 482.3 2.1 1.0X -skipIntegers, constant 3 3 0 335.4 3.0 0.7X -readIntegers, monotonic 3 3 0 314.9 3.2 0.7X -skipIntegers, monotonic 3 3 0 335.1 3.0 0.7X -readIntegers, small-delta random 4 4 0 257.5 3.9 0.5X -skipIntegers, small-delta random 4 4 0 269.5 3.7 0.6X -readIntegers, wide random 5 5 0 209.3 4.8 0.4X -skipIntegers, wide random 5 5 0 218.2 4.6 0.5X +readIntegers, constant 1 1 0 1024.3 1.0 1.0X +skipIntegers, constant 1 1 0 1102.5 0.9 1.1X +readIntegers, monotonic 1 1 0 1021.4 1.0 1.0X +skipIntegers, monotonic 1 1 0 1106.2 0.9 1.1X +readIntegers, small-delta random 2 2 0 588.3 1.7 0.6X +skipIntegers, small-delta random 2 2 0 676.8 1.5 0.7X +readIntegers, wide random 2 3 0 420.1 2.4 0.4X +skipIntegers, wide random 2 2 0 463.3 2.2 0.5X ================================================================================================ DELTA_BINARY_PACKED INT64 ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1011-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V45 96-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 5 6 1 191.0 5.2 1.0X -skipLongs, constant 6 6 0 170.1 5.9 0.9X -readLongs, monotonic 7 7 0 144.9 6.9 0.8X -skipLongs, monotonic 6 6 0 170.1 5.9 0.9X -readLongs, small-delta random 8 8 0 134.8 7.4 0.7X -skipLongs, small-delta random 7 7 0 152.8 6.5 0.8X -readLongs, wide random 11 11 0 97.8 10.2 0.5X -skipLongs, wide random 10 10 0 106.4 9.4 0.6X +readLongs, constant 1 1 0 956.0 1.0 1.0X +skipLongs, constant 1 1 0 1091.4 0.9 1.1X +readLongs, monotonic 1 1 0 955.1 1.0 1.0X +skipLongs, monotonic 1 1 0 1093.2 0.9 1.1X +readLongs, small-delta random 2 2 0 615.9 1.6 0.6X +skipLongs, small-delta random 2 2 0 678.5 1.5 0.7X +readLongs, wide random 4 4 0 291.4 3.4 0.3X +skipLongs, wide random 3 3 0 306.9 3.3 0.3X ================================================================================================ DELTA_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1011-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V45 96-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 36 37 1 29.2 34.2 1.0X -skipBinary, no overlap, len=16 41 41 1 25.8 38.7 0.9X -readBinary, half overlap, len=16 42 43 1 25.0 39.9 0.9X -skipBinary, half overlap, len=16 48 50 2 21.6 46.2 0.7X -readBinary, full overlap, len=16 42 44 2 24.8 40.3 0.8X -skipBinary, full overlap, len=16 48 49 1 21.8 45.9 0.7X -readBinary, half overlap, len=64 42 44 1 24.8 40.4 0.8X -skipBinary, half overlap, len=64 47 49 1 22.2 45.1 0.8X +readBinary, no overlap, len=16 16 16 1 66.4 15.1 1.0X +skipBinary, no overlap, len=16 16 17 1 67.1 14.9 1.0X +readBinary, half overlap, len=16 19 19 0 56.6 17.7 0.9X +skipBinary, half overlap, len=16 19 19 0 56.2 17.8 0.8X +readBinary, full overlap, len=16 19 19 0 56.5 17.7 0.9X +skipBinary, full overlap, len=16 19 19 0 56.2 17.8 0.8X +readBinary, half overlap, len=64 18 19 2 57.0 17.5 0.9X +skipBinary, half overlap, len=64 19 19 1 56.1 17.8 0.8X ================================================================================================ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1011-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V45 96-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 22 25 2 48.1 20.8 1.0X -skipBinary, payloadLen=8 11 12 1 95.8 10.4 2.0X -readBinary, payloadLen=32 22 26 3 48.0 20.9 1.0X -skipBinary, payloadLen=32 11 12 1 97.6 10.2 2.0X -readBinary, payloadLen=128 26 29 2 40.0 25.0 0.8X -skipBinary, payloadLen=128 11 12 1 96.0 10.4 2.0X -readBinary, payloadLen=512 54 62 4 19.3 51.9 0.4X -skipBinary, payloadLen=512 11 13 2 95.7 10.4 2.0X +readBinary, payloadLen=8 8 10 1 128.1 7.8 1.0X +skipBinary, payloadLen=8 3 4 1 314.3 3.2 2.5X +readBinary, payloadLen=32 8 10 2 124.3 8.0 1.0X +skipBinary, payloadLen=32 3 4 1 313.0 3.2 2.4X +readBinary, payloadLen=128 10 11 1 109.4 9.1 0.9X +skipBinary, payloadLen=128 3 4 1 314.0 3.2 2.5X +readBinary, payloadLen=512 27 29 2 38.3 26.1 0.3X +skipBinary, payloadLen=512 3 4 1 313.5 3.2 2.4X ================================================================================================ Variant reads ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1011-azure -AMD EPYC 7763 64-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure +AMD EPYC 9V45 96-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 7 7 0 145.8 6.9 1.0X -readShorts (INT32) 7 8 1 140.7 7.1 1.0X -readUnsignedIntegers (INT32 -> Long) 7 7 0 143.3 7.0 1.0X -readUnsignedLongs (INT64 -> Decimal(20,0)) 227 239 25 4.6 216.4 0.0X -skipBytes 8 8 0 136.8 7.3 0.9X -skipShorts 8 8 0 136.6 7.3 0.9X -readByte (INT32 single-value) 12 12 1 85.8 11.7 0.6X -readShort (INT32 single-value) 12 12 0 85.9 11.6 0.6X -readInteger (INT32 single-value) 12 12 0 86.1 11.6 0.6X -readLong (INT64 single-value) 14 15 1 73.4 13.6 0.5X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 71 77 5 14.7 68.2 0.1X +readBytes (INT32) 3 3 0 331.5 3.0 1.0X +readShorts (INT32) 4 5 0 238.2 4.2 0.7X +readUnsignedIntegers (INT32 -> Long) 4 5 0 238.9 4.2 0.7X +readUnsignedLongs (INT64 -> Decimal(20,0)) 16 16 1 67.0 14.9 0.2X +skipBytes 3 3 0 329.2 3.0 1.0X +skipShorts 3 3 0 329.6 3.0 1.0X +readByte (INT32 single-value) 7 7 1 146.5 6.8 0.4X +readShort (INT32 single-value) 7 8 1 145.2 6.9 0.4X +readInteger (INT32 single-value) 7 7 0 145.6 6.9 0.4X +readLong (INT64 single-value) 8 8 0 130.0 7.7 0.4X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 32 34 2 32.6 30.7 0.1X diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetDictionary.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetDictionary.java index ffdb94ee37653..aba5e1996a2e5 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetDictionary.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetDictionary.java @@ -19,8 +19,6 @@ import org.apache.spark.sql.execution.vectorized.Dictionary; -import java.math.BigInteger; - public final class ParquetDictionary implements Dictionary { private org.apache.parquet.column.Dictionary dictionary; private boolean needTransform = false; @@ -68,7 +66,7 @@ public byte[] decodeToBinary(int id) { // whenever dictionary is available. // Here we lazily decode it to the original signed long value then convert to decimal(20, 0). long signed = dictionary.decodeToLong(id); - return new BigInteger(Long.toUnsignedString(signed)).toByteArray(); + return VectorizedReaderBase.unsignedLongToBytesBigEndian(signed); } else { return dictionary.decodeToBinary(id).getBytesUnsafe(); } diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java index c088f5f2844be..8865a200177d7 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java @@ -40,6 +40,7 @@ import java.math.BigDecimal; import java.math.BigInteger; import java.math.RoundingMode; +import java.nio.ByteBuffer; import java.time.ZoneId; import java.time.ZoneOffset; import java.util.Arrays; @@ -711,6 +712,8 @@ public void decodeSingleDictionaryId( } private static class UnsignedLongUpdater implements ParquetVectorUpdater { + private final ByteBuffer unsignedLongBuf = ByteBuffer.allocate(9); + @Override public void readValues( int total, @@ -730,8 +733,9 @@ public void readValue( int offset, WritableColumnVector values, VectorizedValuesReader valuesReader) { - byte[] bytes = new BigInteger(Long.toUnsignedString(valuesReader.readLong())).toByteArray(); - values.putByteArray(offset, bytes); + int start = VectorizedReaderBase.encodeUnsignedLongBigEndian( + valuesReader.readLong(), unsignedLongBuf); + values.putByteArray(offset, unsignedLongBuf.array(), start, 9 - start); } @Override @@ -741,8 +745,8 @@ public void decodeSingleDictionaryId( WritableColumnVector dictionaryIds, Dictionary dictionary) { long signed = dictionary.decodeToLong(dictionaryIds.getDictId(offset)); - byte[] unsigned = new BigInteger(Long.toUnsignedString(signed)).toByteArray(); - values.putByteArray(offset, unsigned); + int start = VectorizedReaderBase.encodeUnsignedLongBigEndian(signed, unsignedLongBuf); + values.putByteArray(offset, unsignedLongBuf.array(), start, 9 - start); } } diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java index 7b56e1ebf2394..090b5841324fc 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java @@ -17,7 +17,6 @@ package org.apache.spark.sql.execution.datasources.parquet; import java.io.IOException; -import java.math.BigInteger; import java.nio.ByteBuffer; import java.util.Arrays; @@ -70,6 +69,9 @@ public class VectorizedDeltaBinaryPackedReader extends VectorizedReaderBase { private int remainingInBlock = 0; // values in current block still to be read private int remainingInMiniBlock = 0; // values in current mini block still to be read private long[] unpackedValuesBuffer; + // Scratch buffer for narrowing long -> int during bulk readIntegers. + // Allocated eagerly in initFromPage, sized to miniBlockSizeInValues. + private int[] intScratchBuffer; private ByteBufferInputStream in; @@ -95,6 +97,7 @@ public void initFromPage(int valueCount, ByteBufferInputStream in) throws IOExce this.totalValueCount = BytesUtils.readUnsignedVarInt(in); this.bitWidths = new int[miniBlockNumInABlock]; this.unpackedValuesBuffer = new long[miniBlockSizeInValues]; + this.intScratchBuffer = new int[miniBlockSizeInValues]; // read the first value firstValue = BytesUtils.readZigZagVarLong(in); } @@ -140,7 +143,7 @@ public void readShorts(int total, WritableColumnVector c, int rowId) { @Override public void readIntegers(int total, WritableColumnVector c, int rowId) { - readValues(total, c, rowId, (w, r, v) -> w.putInt(r, (int) v)); + readBulkIntegers(total, c, rowId); } // Based on VectorizedPlainValuesReader.readIntegersWithRebase @@ -170,13 +173,13 @@ public void readUnsignedIntegers(int total, WritableColumnVector c, int rowId) { @Override public void readUnsignedLongs(int total, WritableColumnVector c, int rowId) { readValues(total, c, rowId, (w, r, v) -> { - w.putByteArray(r, new BigInteger(Long.toUnsignedString(v)).toByteArray()); + putUnsignedLongAsBigInteger(w, r, v); }); } @Override public void readLongs(int total, WritableColumnVector c, int rowId) { - readValues(total, c, rowId, WritableColumnVector::putLong); + readBulkLongs(total, c, rowId); } @Override @@ -215,6 +218,111 @@ public void skipLongs(int total) { skipValues(total); } + // ---- Bulk read helpers for readIntegers / readLongs ---------------------------- + // + // The generic readValues() path dispatches a lambda per value. For the two most + // common callers (readIntegers, readLongs) we can do much better: compute a prefix + // sum over the unpacked deltas in-place, then bulk-copy the result into the column + // vector with putInts / putLongs (backed by System.arraycopy on-heap). + + /** + * Callback for writing a chunk of prefix-summed absolute values from + * {@code unpackedValuesBuffer} into a column vector. Called once per mini-block + * (not per value), so lambda overhead is negligible. + */ + @FunctionalInterface + private interface BulkWriter { + void write(WritableColumnVector c, int rowId, long[] values, int start, int count); + } + + /** Narrows long[] -> int[] scratch and bulk-writes via putInts. */ + private void bulkWriteInts(WritableColumnVector c, int rowId, + long[] buf, int start, int count) { + for (int i = start; i < start + count; i++) { + intScratchBuffer[i] = (int) buf[i]; + } + c.putInts(rowId, count, intScratchBuffer, start); + } + + private void readBulkIntegers(int total, WritableColumnVector c, int rowId) { + checkReadBounds(total); + int remaining = total; + if (valuesRead == 0) { + c.putInt(rowId, (int) firstValue); + lastValueRead = firstValue; + rowId++; + remaining--; + } + readBulkLoop(remaining, c, rowId, this::bulkWriteInts); + valuesRead += total; + } + + private void readBulkLongs(int total, WritableColumnVector c, int rowId) { + checkReadBounds(total); + int remaining = total; + if (valuesRead == 0) { + c.putLong(rowId, firstValue); + lastValueRead = firstValue; + rowId++; + remaining--; + } + readBulkLoop(remaining, c, rowId, + (col, r, buf, s, n) -> col.putLongs(r, n, buf, s)); + valuesRead += total; + } + + private void checkReadBounds(int total) { + if (valuesRead + total > totalValueCount) { + throw new ParquetDecodingException( + "No more values to read. Total values read: " + valuesRead + ", total count: " + + totalValueCount + ", trying to read " + total + " more."); + } + } + + /** Mini-block iteration loop shared by readBulkIntegers and readBulkLongs. */ + private void readBulkLoop(int remaining, WritableColumnVector c, int rowId, + BulkWriter writer) { + while (remaining > 0) { + int n; + try { + n = loadMiniBlockBulk(remaining, c, rowId, writer); + } catch (IOException e) { + throw new ParquetDecodingException("Error reading mini block.", e); + } + rowId += n; + remaining -= n; + } + } + + /** + * Loads the next mini-block (if needed), prefix-sums the deltas in-place, and + * delegates the type-specific column write to {@code writer}. + */ + private int loadMiniBlockBulk(int remaining, WritableColumnVector c, int rowId, + BulkWriter writer) throws IOException { + if (remainingInBlock == 0) readBlockHeader(); + if (remainingInMiniBlock == 0) unpackMiniBlock(); + + int start = miniBlockSizeInValues - remainingInMiniBlock; + int count = Math.min(remaining, remainingInMiniBlock); + + // Prefix-sum: convert raw deltas -> absolute values in-place. + long running = lastValueRead; + long minDelta = minDeltaInCurrentBlock; + long[] buf = unpackedValuesBuffer; + for (int i = start; i < start + count; i++) { + running += minDelta + buf[i]; + buf[i] = running; + } + lastValueRead = running; + + writer.write(c, rowId, buf, start, count); + + remainingInBlock -= count; + remainingInMiniBlock -= count; + return count; + } + private void readValues(int total, WritableColumnVector c, int rowId, IntegerOutputWriter outputWriter) { if (valuesRead + total > totalValueCount) { @@ -240,7 +348,7 @@ private void readValues(int total, WritableColumnVector c, int rowId, rowId += n; remaining -= n; } - valuesRead = total - remaining; + valuesRead += total; } @@ -328,4 +436,12 @@ private void skipValues(int total) { readValues(total, null, -1, (w, r, v) -> {}); } + // Reusable buffer for unsigned long -> BigInteger encoding (9 bytes: sign + 8 value bytes). + private final ByteBuffer unsignedLongBuf = ByteBuffer.allocate(9); + + private void putUnsignedLongAsBigInteger(WritableColumnVector c, int rowId, long v) { + int start = encodeUnsignedLongBigEndian(v, unsignedLongBuf); + c.putByteArray(rowId, unsignedLongBuf.array(), start, 9 - start); + } + } diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java index 77bb9340e5263..092e0e3ff4f9d 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java @@ -16,6 +16,9 @@ */ package org.apache.spark.sql.execution.datasources.parquet; +import java.nio.ByteBuffer; +import java.util.Arrays; + import org.apache.parquet.column.values.ValuesReader; import org.apache.parquet.io.api.Binary; import org.apache.spark.SparkUnsupportedOperationException; @@ -27,6 +30,43 @@ */ public class VectorizedReaderBase extends ValuesReader implements VectorizedValuesReader { + /** + * Encodes an unsigned long as a minimal big-endian two's-complement byte array + * compatible with {@link java.math.BigInteger} encoding. The result is written into + * the backing array of {@code buf} (which must have capacity >= 9). Returns the + * start offset; the valid bytes are {@code buf.array()[start .. 8]} (length = 9 - start). + * + *

This avoids the per-value overhead of + * {@code new BigInteger(Long.toUnsignedString(v)).toByteArray()} which allocates a + * String, a BigInteger, and a byte[] on every call. + */ + static int encodeUnsignedLongBigEndian(long v, ByteBuffer buf) { + byte[] scratch = buf.array(); + // ByteBuffer is big-endian by default; writes 8 bytes MSB-first at offset 1. + // Always write before the zero-check so that the buffer is current even when reused. + buf.putLong(1, v); + if (v == 0L) { + return 8; // scratch[8] is already 0x00; caller writes 9 - 8 = 1 byte: [0x00] + } + // Find the first non-zero byte (minimal encoding). + int start = 1; + while (scratch[start] == 0) start++; + // Prepend 0x00 sign byte if MSB of the first significant byte is set. + if ((scratch[start] & 0x80) != 0) scratch[--start] = 0; + return start; + } + + /** + * Convenience: encodes an unsigned long and returns a new byte[] with the minimal + * BigInteger-compatible encoding. Use {@link #encodeUnsignedLongBigEndian(long, ByteBuffer)} + * with a reusable buffer in hot paths to avoid allocation. + */ + static byte[] unsignedLongToBytesBigEndian(long v) { + ByteBuffer buf = ByteBuffer.allocate(9); + int start = encodeUnsignedLongBigEndian(v, buf); + return Arrays.copyOfRange(buf.array(), start, 9); + } + @Override public void skip() { throw SparkUnsupportedOperationException.apply(); diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala index 40e08d9870c71..527d43b0c8547 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala @@ -321,6 +321,7 @@ object VectorizedDeltaReaderBenchmark extends BenchmarkBase { newIntReader().readUnsignedIntegers(NUM_ROWS, longVec, 0) } benchmark.addCase("readUnsignedLongs (INT64 -> Decimal(20,0))") { _ => + unsignedLongVec.reset() newLongReader().readUnsignedLongs(NUM_ROWS, unsignedLongVec, 0) } benchmark.addCase("skipBytes") { _ => newIntReader().skipBytes(NUM_ROWS) } From 5c82b7b8c7332ad3d084464abfa1fd087fe569ba Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 00:39:50 +0000 Subject: [PATCH 02/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 17, Scala 2.13, split 1 of 1) --- ...VectorizedDeltaReaderBenchmark-results.txt | 106 +++++++++--------- 1 file changed, 53 insertions(+), 53 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt index 1d7a882581173..744fb0554ab90 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt @@ -2,92 +2,92 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V45 96-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 1 1 0 1024.3 1.0 1.0X -skipIntegers, constant 1 1 0 1102.5 0.9 1.1X -readIntegers, monotonic 1 1 0 1021.4 1.0 1.0X -skipIntegers, monotonic 1 1 0 1106.2 0.9 1.1X -readIntegers, small-delta random 2 2 0 588.3 1.7 0.6X -skipIntegers, small-delta random 2 2 0 676.8 1.5 0.7X -readIntegers, wide random 2 3 0 420.1 2.4 0.4X -skipIntegers, wide random 2 2 0 463.3 2.2 0.5X +readIntegers, constant 2 2 0 470.8 2.1 1.0X +skipIntegers, constant 2 2 0 613.4 1.6 1.3X +readIntegers, monotonic 2 2 0 469.7 2.1 1.0X +skipIntegers, monotonic 2 2 0 613.8 1.6 1.3X +readIntegers, small-delta random 3 3 0 352.2 2.8 0.7X +skipIntegers, small-delta random 2 2 0 429.3 2.3 0.9X +readIntegers, wide random 4 4 0 268.9 3.7 0.6X +skipIntegers, wide random 3 3 0 310.5 3.2 0.7X ================================================================================================ DELTA_BINARY_PACKED INT64 ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V45 96-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 1 1 0 956.0 1.0 1.0X -skipLongs, constant 1 1 0 1091.4 0.9 1.1X -readLongs, monotonic 1 1 0 955.1 1.0 1.0X -skipLongs, monotonic 1 1 0 1093.2 0.9 1.1X -readLongs, small-delta random 2 2 0 615.9 1.6 0.6X -skipLongs, small-delta random 2 2 0 678.5 1.5 0.7X -readLongs, wide random 4 4 0 291.4 3.4 0.3X -skipLongs, wide random 3 3 0 306.9 3.3 0.3X +readLongs, constant 2 2 0 535.7 1.9 1.0X +skipLongs, constant 2 2 0 612.6 1.6 1.1X +readLongs, monotonic 2 2 0 536.8 1.9 1.0X +skipLongs, monotonic 2 2 0 610.3 1.6 1.1X +readLongs, small-delta random 3 3 0 373.5 2.7 0.7X +skipLongs, small-delta random 2 2 0 429.6 2.3 0.8X +readLongs, wide random 6 6 0 187.4 5.3 0.3X +skipLongs, wide random 5 5 0 194.6 5.1 0.4X ================================================================================================ DELTA_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V45 96-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 16 16 1 66.4 15.1 1.0X -skipBinary, no overlap, len=16 16 17 1 67.1 14.9 1.0X -readBinary, half overlap, len=16 19 19 0 56.6 17.7 0.9X -skipBinary, half overlap, len=16 19 19 0 56.2 17.8 0.8X -readBinary, full overlap, len=16 19 19 0 56.5 17.7 0.9X -skipBinary, full overlap, len=16 19 19 0 56.2 17.8 0.8X -readBinary, half overlap, len=64 18 19 2 57.0 17.5 0.9X -skipBinary, half overlap, len=64 19 19 1 56.1 17.8 0.8X +readBinary, no overlap, len=16 27 28 1 38.6 25.9 1.0X +skipBinary, no overlap, len=16 31 32 1 33.4 29.9 0.9X +readBinary, half overlap, len=16 34 34 1 31.2 32.0 0.8X +skipBinary, half overlap, len=16 39 40 1 27.1 36.9 0.7X +readBinary, full overlap, len=16 34 35 1 30.8 32.5 0.8X +skipBinary, full overlap, len=16 39 40 0 26.6 37.6 0.7X +readBinary, half overlap, len=64 35 35 1 30.2 33.1 0.8X +skipBinary, half overlap, len=64 39 40 0 26.8 37.3 0.7X ================================================================================================ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V45 96-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 8 10 1 128.1 7.8 1.0X -skipBinary, payloadLen=8 3 4 1 314.3 3.2 2.5X -readBinary, payloadLen=32 8 10 2 124.3 8.0 1.0X -skipBinary, payloadLen=32 3 4 1 313.0 3.2 2.4X -readBinary, payloadLen=128 10 11 1 109.4 9.1 0.9X -skipBinary, payloadLen=128 3 4 1 314.0 3.2 2.5X -readBinary, payloadLen=512 27 29 2 38.3 26.1 0.3X -skipBinary, payloadLen=512 3 4 1 313.5 3.2 2.4X +readBinary, payloadLen=8 17 17 0 63.4 15.8 1.0X +skipBinary, payloadLen=8 6 6 0 175.2 5.7 2.8X +readBinary, payloadLen=32 17 17 1 62.5 16.0 1.0X +skipBinary, payloadLen=32 6 7 1 174.7 5.7 2.8X +readBinary, payloadLen=128 19 19 0 55.3 18.1 0.9X +skipBinary, payloadLen=128 6 6 1 176.1 5.7 2.8X +readBinary, payloadLen=512 40 41 1 26.3 38.0 0.4X +skipBinary, payloadLen=512 6 6 1 175.1 5.7 2.8X ================================================================================================ Variant reads ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V45 96-Core Processor +OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 3 3 0 331.5 3.0 1.0X -readShorts (INT32) 4 5 0 238.2 4.2 0.7X -readUnsignedIntegers (INT32 -> Long) 4 5 0 238.9 4.2 0.7X -readUnsignedLongs (INT64 -> Decimal(20,0)) 16 16 1 67.0 14.9 0.2X -skipBytes 3 3 0 329.2 3.0 1.0X -skipShorts 3 3 0 329.6 3.0 1.0X -readByte (INT32 single-value) 7 7 1 146.5 6.8 0.4X -readShort (INT32 single-value) 7 8 1 145.2 6.9 0.4X -readInteger (INT32 single-value) 7 7 0 145.6 6.9 0.4X -readLong (INT64 single-value) 8 8 0 130.0 7.7 0.4X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 32 34 2 32.6 30.7 0.1X +readBytes (INT32) 5 5 0 195.8 5.1 1.0X +readShorts (INT32) 9 9 0 118.9 8.4 0.6X +readUnsignedIntegers (INT32 -> Long) 9 9 0 118.1 8.5 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 26 27 0 39.8 25.1 0.2X +skipBytes 7 7 0 160.5 6.2 0.8X +skipShorts 7 7 0 160.3 6.2 0.8X +readByte (INT32 single-value) 13 14 2 78.0 12.8 0.4X +readShort (INT32 single-value) 13 13 0 83.7 11.9 0.4X +readInteger (INT32 single-value) 13 13 0 83.8 11.9 0.4X +readLong (INT64 single-value) 14 15 3 73.2 13.7 0.4X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 61 65 3 17.1 58.5 0.1X From 787bc26184ed6d0ea25588aba56599e45a09465b Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 01:09:26 +0000 Subject: [PATCH 03/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 21, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk21-results.txt | 96 +++++++++---------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt index 7c20ec159f98e..55507e75c75a2 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt @@ -3,17 +3,17 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 7763 64-Core Processor +AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 486.9 2.1 1.0X -skipIntegers, constant 2 2 0 560.5 1.8 1.2X -readIntegers, monotonic 2 2 0 487.1 2.1 1.0X -skipIntegers, monotonic 2 2 0 561.9 1.8 1.2X -readIntegers, small-delta random 3 3 0 333.5 3.0 0.7X -skipIntegers, small-delta random 3 3 0 376.5 2.7 0.8X -readIntegers, wide random 4 4 0 269.7 3.7 0.6X -skipIntegers, wide random 4 4 0 294.9 3.4 0.6X +readIntegers, constant 2 2 0 583.2 1.7 1.0X +skipIntegers, constant 2 2 0 677.8 1.5 1.2X +readIntegers, monotonic 2 2 0 583.6 1.7 1.0X +skipIntegers, monotonic 2 2 0 678.6 1.5 1.2X +readIntegers, small-delta random 3 3 0 394.4 2.5 0.7X +skipIntegers, small-delta random 2 3 0 423.5 2.4 0.7X +readIntegers, wide random 3 3 0 314.4 3.2 0.5X +skipIntegers, wide random 3 3 0 338.4 3.0 0.6X ================================================================================================ @@ -21,17 +21,17 @@ DELTA_BINARY_PACKED INT64 ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 7763 64-Core Processor +AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 505.2 2.0 1.0X -skipLongs, constant 2 2 0 543.1 1.8 1.1X -readLongs, monotonic 2 2 0 505.0 2.0 1.0X -skipLongs, monotonic 2 2 0 557.9 1.8 1.1X -readLongs, small-delta random 3 3 0 354.3 2.8 0.7X -skipLongs, small-delta random 3 3 0 377.2 2.7 0.7X -readLongs, wide random 6 6 0 184.6 5.4 0.4X -skipLongs, wide random 5 6 0 191.8 5.2 0.4X +readLongs, constant 2 2 0 631.8 1.6 1.0X +skipLongs, constant 2 2 0 656.6 1.5 1.0X +readLongs, monotonic 2 2 0 631.5 1.6 1.0X +skipLongs, monotonic 2 2 0 659.7 1.5 1.0X +readLongs, small-delta random 3 3 0 415.1 2.4 0.7X +skipLongs, small-delta random 2 3 0 423.6 2.4 0.7X +readLongs, wide random 5 5 0 211.3 4.7 0.3X +skipLongs, wide random 5 5 1 218.0 4.6 0.3X ================================================================================================ @@ -39,17 +39,17 @@ DELTA_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 7763 64-Core Processor +AMD EPYC 9V74 80-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 27 31 3 39.4 25.4 1.0X -skipBinary, no overlap, len=16 31 31 1 34.2 29.2 0.9X -readBinary, half overlap, len=16 32 33 1 32.3 31.0 0.8X -skipBinary, half overlap, len=16 38 39 1 27.7 36.1 0.7X -readBinary, full overlap, len=16 33 34 1 31.7 31.6 0.8X -skipBinary, full overlap, len=16 38 39 1 27.4 36.5 0.7X -readBinary, half overlap, len=64 33 34 1 31.6 31.6 0.8X -skipBinary, half overlap, len=64 38 38 1 27.9 35.9 0.7X +readBinary, no overlap, len=16 22 25 3 48.5 20.6 1.0X +skipBinary, no overlap, len=16 24 24 0 44.3 22.6 0.9X +readBinary, half overlap, len=16 25 26 1 41.6 24.0 0.9X +skipBinary, half overlap, len=16 29 30 1 36.2 27.6 0.7X +readBinary, full overlap, len=16 25 26 1 41.6 24.0 0.9X +skipBinary, full overlap, len=16 29 30 1 36.2 27.7 0.7X +readBinary, half overlap, len=64 26 27 1 40.0 25.0 0.8X +skipBinary, half overlap, len=64 29 30 1 35.9 27.9 0.7X ================================================================================================ @@ -57,17 +57,17 @@ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 7763 64-Core Processor +AMD EPYC 9V74 80-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 16 17 0 65.5 15.3 1.0X -skipBinary, payloadLen=8 6 6 0 182.5 5.5 2.8X -readBinary, payloadLen=32 16 16 0 64.7 15.5 1.0X -skipBinary, payloadLen=32 6 6 0 182.5 5.5 2.8X -readBinary, payloadLen=128 19 19 0 56.5 17.7 0.9X -skipBinary, payloadLen=128 6 6 0 183.5 5.5 2.8X -readBinary, payloadLen=512 40 41 1 26.4 37.8 0.4X -skipBinary, payloadLen=512 6 6 1 182.7 5.5 2.8X +readBinary, payloadLen=8 13 13 0 80.2 12.5 1.0X +skipBinary, payloadLen=8 5 5 0 223.0 4.5 2.8X +readBinary, payloadLen=32 13 13 0 79.4 12.6 1.0X +skipBinary, payloadLen=32 5 5 0 223.1 4.5 2.8X +readBinary, payloadLen=128 15 15 0 68.7 14.6 0.9X +skipBinary, payloadLen=128 5 5 0 223.5 4.5 2.8X +readBinary, payloadLen=512 35 37 1 30.2 33.1 0.4X +skipBinary, payloadLen=512 5 5 0 222.8 4.5 2.8X ================================================================================================ @@ -75,19 +75,19 @@ Variant reads ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure -AMD EPYC 7763 64-Core Processor +AMD EPYC 9V74 80-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 228.8 4.4 1.0X -readShorts (INT32) 8 8 0 131.2 7.6 0.6X -readUnsignedIntegers (INT32 -> Long) 8 8 0 129.5 7.7 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 26 26 0 40.5 24.7 0.2X -skipBytes 9 9 0 120.4 8.3 0.5X -skipShorts 9 9 0 120.6 8.3 0.5X -readByte (INT32 single-value) 14 14 1 76.8 13.0 0.3X -readShort (INT32 single-value) 14 14 1 76.8 13.0 0.3X -readInteger (INT32 single-value) 14 14 1 76.8 13.0 0.3X -readLong (INT64 single-value) 15 16 1 68.5 14.6 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 58 62 2 18.1 55.3 0.1X +readBytes (INT32) 4 4 0 272.2 3.7 1.0X +readShorts (INT32) 7 7 0 158.5 6.3 0.6X +readUnsignedIntegers (INT32 -> Long) 7 7 0 159.0 6.3 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 22 22 1 47.7 21.0 0.2X +skipBytes 7 7 0 148.2 6.7 0.5X +skipShorts 7 7 0 148.2 6.7 0.5X +readByte (INT32 single-value) 11 11 0 96.9 10.3 0.4X +readShort (INT32 single-value) 11 11 0 96.9 10.3 0.4X +readInteger (INT32 single-value) 11 11 1 97.0 10.3 0.4X +readLong (INT64 single-value) 13 13 1 83.0 12.0 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 49 51 2 21.4 46.7 0.1X From c3d56406e8e07fe4ea8728601f5c3cb1b741fa5a Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 01:35:03 +0000 Subject: [PATCH 04/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 25, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk25-results.txt | 106 +++++++++--------- 1 file changed, 53 insertions(+), 53 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt index c30e84b6371d2..a0f7ee9e98a39 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt @@ -2,92 +2,92 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V74 80-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 454.3 2.2 1.0X -skipIntegers, constant 2 2 0 528.1 1.9 1.2X -readIntegers, monotonic 2 2 0 451.0 2.2 1.0X -skipIntegers, monotonic 2 2 0 528.2 1.9 1.2X -readIntegers, small-delta random 3 3 0 309.5 3.2 0.7X -skipIntegers, small-delta random 3 3 0 345.9 2.9 0.8X -readIntegers, wide random 4 5 1 247.4 4.0 0.5X -skipIntegers, wide random 4 4 0 261.4 3.8 0.6X +readIntegers, constant 3 3 0 328.8 3.0 1.0X +skipIntegers, constant 3 3 0 371.3 2.7 1.1X +readIntegers, monotonic 3 3 0 328.7 3.0 1.0X +skipIntegers, monotonic 3 3 0 373.9 2.7 1.1X +readIntegers, small-delta random 4 4 0 272.9 3.7 0.8X +skipIntegers, small-delta random 3 4 0 301.7 3.3 0.9X +readIntegers, wide random 5 5 0 220.1 4.5 0.7X +skipIntegers, wide random 4 5 0 235.3 4.3 0.7X ================================================================================================ DELTA_BINARY_PACKED INT64 ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V74 80-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 451.2 2.2 1.0X -skipLongs, constant 2 2 0 527.4 1.9 1.2X -readLongs, monotonic 2 2 0 452.3 2.2 1.0X -skipLongs, monotonic 2 2 0 528.5 1.9 1.2X -readLongs, small-delta random 3 3 0 313.0 3.2 0.7X -skipLongs, small-delta random 3 3 0 346.0 2.9 0.8X -readLongs, wide random 6 6 0 163.3 6.1 0.4X -skipLongs, wide random 6 6 0 172.0 5.8 0.4X +readLongs, constant 3 3 0 345.7 2.9 1.0X +skipLongs, constant 3 3 0 368.9 2.7 1.1X +readLongs, monotonic 3 3 0 345.9 2.9 1.0X +skipLongs, monotonic 3 3 0 372.0 2.7 1.1X +readLongs, small-delta random 4 4 0 284.4 3.5 0.8X +skipLongs, small-delta random 3 4 0 302.2 3.3 0.9X +readLongs, wide random 6 6 0 164.8 6.1 0.5X +skipLongs, wide random 6 6 0 168.7 5.9 0.5X ================================================================================================ DELTA_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V74 80-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 30 30 1 35.5 28.2 1.0X -skipBinary, no overlap, len=16 32 35 3 32.5 30.7 0.9X -readBinary, half overlap, len=16 36 38 3 29.1 34.4 0.8X -skipBinary, half overlap, len=16 40 44 4 26.3 38.0 0.7X -readBinary, full overlap, len=16 36 38 2 28.8 34.7 0.8X -skipBinary, full overlap, len=16 40 42 3 26.3 38.0 0.7X -readBinary, half overlap, len=64 37 44 2 28.5 35.1 0.8X -skipBinary, half overlap, len=64 40 43 3 26.4 37.9 0.7X +readBinary, no overlap, len=16 46 47 2 23.0 43.5 1.0X +skipBinary, no overlap, len=16 47 48 1 22.2 45.1 1.0X +readBinary, half overlap, len=16 48 49 1 21.7 46.1 0.9X +skipBinary, half overlap, len=16 54 55 2 19.4 51.4 0.8X +readBinary, full overlap, len=16 49 50 1 21.5 46.5 0.9X +skipBinary, full overlap, len=16 54 55 1 19.3 51.8 0.8X +readBinary, half overlap, len=64 49 49 1 21.5 46.5 0.9X +skipBinary, half overlap, len=64 54 54 1 19.6 51.1 0.9X ================================================================================================ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V74 80-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 21 21 0 50.1 20.0 1.0X -skipBinary, payloadLen=8 6 6 0 165.8 6.0 3.3X -readBinary, payloadLen=32 21 21 0 50.4 19.8 1.0X -skipBinary, payloadLen=32 6 6 0 165.2 6.1 3.3X -readBinary, payloadLen=128 23 23 1 45.4 22.0 0.9X -skipBinary, payloadLen=128 6 6 0 165.5 6.0 3.3X -readBinary, payloadLen=512 46 49 1 22.8 43.9 0.5X -skipBinary, payloadLen=512 6 6 0 165.8 6.0 3.3X +readBinary, payloadLen=8 25 25 1 42.0 23.8 1.0X +skipBinary, payloadLen=8 7 7 1 156.3 6.4 3.7X +readBinary, payloadLen=32 29 30 3 36.8 27.2 0.9X +skipBinary, payloadLen=32 7 7 0 156.1 6.4 3.7X +readBinary, payloadLen=128 27 28 1 38.5 26.0 0.9X +skipBinary, payloadLen=128 7 7 1 156.4 6.4 3.7X +readBinary, payloadLen=512 47 48 1 22.5 44.5 0.5X +skipBinary, payloadLen=512 7 7 0 156.2 6.4 3.7X ================================================================================================ Variant reads ================================================================================================ -OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1015-azure -AMD EPYC 9V74 80-Core Processor +OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure +AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 199.9 5.0 1.0X -readShorts (INT32) 8 8 0 127.7 7.8 0.6X -readUnsignedIntegers (INT32 -> Long) 9 9 0 116.1 8.6 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 29 30 1 35.7 28.0 0.2X -skipBytes 4 4 0 265.7 3.8 1.3X -skipShorts 4 4 0 265.9 3.8 1.3X -readByte (INT32 single-value) 13 14 1 78.3 12.8 0.4X -readShort (INT32 single-value) 13 14 0 78.3 12.8 0.4X -readInteger (INT32 single-value) 13 14 0 78.4 12.8 0.4X -readLong (INT64 single-value) 15 15 0 70.9 14.1 0.4X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 66 70 2 15.8 63.2 0.1X +readBytes (INT32) 6 6 0 178.3 5.6 1.0X +readShorts (INT32) 10 10 1 110.1 9.1 0.6X +readUnsignedIntegers (INT32 -> Long) 10 11 1 101.4 9.9 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 29 29 2 36.7 27.3 0.2X +skipBytes 7 8 1 147.9 6.8 0.8X +skipShorts 7 7 1 148.0 6.8 0.8X +readByte (INT32 single-value) 13 13 1 80.6 12.4 0.5X +readShort (INT32 single-value) 15 15 1 69.6 14.4 0.4X +readInteger (INT32 single-value) 16 16 1 66.3 15.1 0.4X +readLong (INT64 single-value) 17 18 0 60.1 16.6 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 80 81 2 13.1 76.1 0.1X From ace5ee13444d4f909b1f01baef2cad6dfa28a6c8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Isma=C3=ABl=20Mej=C3=ADa?= Date: Thu, 11 Jun 2026 22:06:56 +0200 Subject: [PATCH 05/15] Replace ByteBuffer with byte[] in unsigned long encoding Replace ByteBuffer-based encodeUnsignedLongBigEndian with a plain byte[] loop approach using Long.numberOfLeadingZeros to compute the minimal BigInteger-compatible encoding length directly. - encodeUnsignedLongBigEndian now takes byte[] instead of ByteBuffer - unsignedLongToBytesBigEndian allocates a minimal array directly - Update all call sites in VectorizedDeltaBinaryPackedReader and ParquetVectorUpdaterFactory to use byte[] scratch buffers - Remove unused ByteBuffer/Arrays imports Assisted-by: OpenCode:claude-opus-4.6 --- .../parquet/ParquetVectorUpdaterFactory.java | 11 +++-- .../VectorizedDeltaBinaryPackedReader.java | 6 +-- .../parquet/VectorizedReaderBase.java | 42 +++++++++---------- 3 files changed, 28 insertions(+), 31 deletions(-) diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java index 8865a200177d7..948e90a79f3e8 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetVectorUpdaterFactory.java @@ -40,7 +40,6 @@ import java.math.BigDecimal; import java.math.BigInteger; import java.math.RoundingMode; -import java.nio.ByteBuffer; import java.time.ZoneId; import java.time.ZoneOffset; import java.util.Arrays; @@ -712,7 +711,7 @@ public void decodeSingleDictionaryId( } private static class UnsignedLongUpdater implements ParquetVectorUpdater { - private final ByteBuffer unsignedLongBuf = ByteBuffer.allocate(9); + private final byte[] unsignedLongScratch = new byte[9]; @Override public void readValues( @@ -734,8 +733,8 @@ public void readValue( WritableColumnVector values, VectorizedValuesReader valuesReader) { int start = VectorizedReaderBase.encodeUnsignedLongBigEndian( - valuesReader.readLong(), unsignedLongBuf); - values.putByteArray(offset, unsignedLongBuf.array(), start, 9 - start); + valuesReader.readLong(), unsignedLongScratch); + values.putByteArray(offset, unsignedLongScratch, start, 9 - start); } @Override @@ -745,8 +744,8 @@ public void decodeSingleDictionaryId( WritableColumnVector dictionaryIds, Dictionary dictionary) { long signed = dictionary.decodeToLong(dictionaryIds.getDictId(offset)); - int start = VectorizedReaderBase.encodeUnsignedLongBigEndian(signed, unsignedLongBuf); - values.putByteArray(offset, unsignedLongBuf.array(), start, 9 - start); + int start = VectorizedReaderBase.encodeUnsignedLongBigEndian(signed, unsignedLongScratch); + values.putByteArray(offset, unsignedLongScratch, start, 9 - start); } } diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java index 090b5841324fc..6773bca7f88da 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java @@ -437,11 +437,11 @@ private void skipValues(int total) { } // Reusable buffer for unsigned long -> BigInteger encoding (9 bytes: sign + 8 value bytes). - private final ByteBuffer unsignedLongBuf = ByteBuffer.allocate(9); + private final byte[] unsignedLongScratch = new byte[9]; private void putUnsignedLongAsBigInteger(WritableColumnVector c, int rowId, long v) { - int start = encodeUnsignedLongBigEndian(v, unsignedLongBuf); - c.putByteArray(rowId, unsignedLongBuf.array(), start, 9 - start); + int start = encodeUnsignedLongBigEndian(v, unsignedLongScratch); + c.putByteArray(rowId, unsignedLongScratch, start, 9 - start); } } diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java index 092e0e3ff4f9d..f5ab1a68bea6d 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedReaderBase.java @@ -16,9 +16,6 @@ */ package org.apache.spark.sql.execution.datasources.parquet; -import java.nio.ByteBuffer; -import java.util.Arrays; - import org.apache.parquet.column.values.ValuesReader; import org.apache.parquet.io.api.Binary; import org.apache.spark.SparkUnsupportedOperationException; @@ -32,39 +29,40 @@ public class VectorizedReaderBase extends ValuesReader implements VectorizedValu /** * Encodes an unsigned long as a minimal big-endian two's-complement byte array - * compatible with {@link java.math.BigInteger} encoding. The result is written into - * the backing array of {@code buf} (which must have capacity >= 9). Returns the - * start offset; the valid bytes are {@code buf.array()[start .. 8]} (length = 9 - start). + * compatible with {@link java.math.BigInteger} encoding, written into + * {@code scratch[start .. 8]} (length = 9 - start). {@code scratch} must have length >= 9. * *

This avoids the per-value overhead of * {@code new BigInteger(Long.toUnsignedString(v)).toByteArray()} which allocates a * String, a BigInteger, and a byte[] on every call. */ - static int encodeUnsignedLongBigEndian(long v, ByteBuffer buf) { - byte[] scratch = buf.array(); - // ByteBuffer is big-endian by default; writes 8 bytes MSB-first at offset 1. - // Always write before the zero-check so that the buffer is current even when reused. - buf.putLong(1, v); - if (v == 0L) { - return 8; // scratch[8] is already 0x00; caller writes 9 - 8 = 1 byte: [0x00] + static int encodeUnsignedLongBigEndian(long v, byte[] scratch) { + // Minimal BigInteger-compatible length is bitLength/8 + 1, so the + // encoding occupies scratch[start .. 8]; the loop runs once past the + // significant bytes, which writes the 0x00 sign byte when one is needed. + int start = 8 - (64 - Long.numberOfLeadingZeros(v)) / 8; + long x = v; + for (int i = 8; i >= start; i--) { + scratch[i] = (byte) x; + x >>>= 8; } - // Find the first non-zero byte (minimal encoding). - int start = 1; - while (scratch[start] == 0) start++; - // Prepend 0x00 sign byte if MSB of the first significant byte is set. - if ((scratch[start] & 0x80) != 0) scratch[--start] = 0; return start; } /** * Convenience: encodes an unsigned long and returns a new byte[] with the minimal - * BigInteger-compatible encoding. Use {@link #encodeUnsignedLongBigEndian(long, ByteBuffer)} + * BigInteger-compatible encoding. Use {@link #encodeUnsignedLongBigEndian(long, byte[])} * with a reusable buffer in hot paths to avoid allocation. */ static byte[] unsignedLongToBytesBigEndian(long v) { - ByteBuffer buf = ByteBuffer.allocate(9); - int start = encodeUnsignedLongBigEndian(v, buf); - return Arrays.copyOfRange(buf.array(), start, 9); + int len = (64 - Long.numberOfLeadingZeros(v)) / 8 + 1; + byte[] out = new byte[len]; + long x = v; + for (int i = len - 1; i >= 0 && x != 0; i--) { + out[i] = (byte) x; + x >>>= 8; + } + return out; } @Override From 9bcd68dfab08739fc814d58ad74bde9e05b50f0b Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 20:44:13 +0000 Subject: [PATCH 06/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 21, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk21-results.txt | 96 +++++++++---------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt index 55507e75c75a2..f73fe10ff3ce4 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt @@ -2,92 +2,92 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 583.2 1.7 1.0X -skipIntegers, constant 2 2 0 677.8 1.5 1.2X -readIntegers, monotonic 2 2 0 583.6 1.7 1.0X -skipIntegers, monotonic 2 2 0 678.6 1.5 1.2X -readIntegers, small-delta random 3 3 0 394.4 2.5 0.7X -skipIntegers, small-delta random 2 3 0 423.5 2.4 0.7X -readIntegers, wide random 3 3 0 314.4 3.2 0.5X -skipIntegers, wide random 3 3 0 338.4 3.0 0.6X +readIntegers, constant 2 2 0 453.0 2.2 1.0X +skipIntegers, constant 2 2 0 524.5 1.9 1.2X +readIntegers, monotonic 2 2 0 455.5 2.2 1.0X +skipIntegers, monotonic 2 2 0 535.1 1.9 1.2X +readIntegers, small-delta random 3 4 0 306.9 3.3 0.7X +skipIntegers, small-delta random 3 3 0 335.3 3.0 0.7X +readIntegers, wide random 4 4 0 243.7 4.1 0.5X +skipIntegers, wide random 4 4 0 266.8 3.7 0.6X ================================================================================================ DELTA_BINARY_PACKED INT64 ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 631.8 1.6 1.0X -skipLongs, constant 2 2 0 656.6 1.5 1.0X -readLongs, monotonic 2 2 0 631.5 1.6 1.0X -skipLongs, monotonic 2 2 0 659.7 1.5 1.0X -readLongs, small-delta random 3 3 0 415.1 2.4 0.7X -skipLongs, small-delta random 2 3 0 423.6 2.4 0.7X -readLongs, wide random 5 5 0 211.3 4.7 0.3X -skipLongs, wide random 5 5 1 218.0 4.6 0.3X +readLongs, constant 2 2 0 494.1 2.0 1.0X +skipLongs, constant 2 2 0 521.4 1.9 1.1X +readLongs, monotonic 2 2 0 496.3 2.0 1.0X +skipLongs, monotonic 2 2 0 522.5 1.9 1.1X +readLongs, small-delta random 3 3 0 330.6 3.0 0.7X +skipLongs, small-delta random 3 3 0 335.5 3.0 0.7X +readLongs, wide random 6 6 0 167.1 6.0 0.3X +skipLongs, wide random 6 6 0 170.2 5.9 0.3X ================================================================================================ DELTA_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 22 25 3 48.5 20.6 1.0X -skipBinary, no overlap, len=16 24 24 0 44.3 22.6 0.9X -readBinary, half overlap, len=16 25 26 1 41.6 24.0 0.9X -skipBinary, half overlap, len=16 29 30 1 36.2 27.6 0.7X -readBinary, full overlap, len=16 25 26 1 41.6 24.0 0.9X -skipBinary, full overlap, len=16 29 30 1 36.2 27.7 0.7X -readBinary, half overlap, len=64 26 27 1 40.0 25.0 0.8X -skipBinary, half overlap, len=64 29 30 1 35.9 27.9 0.7X +readBinary, no overlap, len=16 29 31 3 36.2 27.6 1.0X +skipBinary, no overlap, len=16 31 32 2 34.0 29.4 0.9X +readBinary, half overlap, len=16 35 39 2 29.7 33.6 0.8X +skipBinary, half overlap, len=16 38 39 2 27.6 36.3 0.8X +readBinary, full overlap, len=16 35 37 2 29.6 33.8 0.8X +skipBinary, full overlap, len=16 39 40 3 27.1 36.9 0.7X +readBinary, half overlap, len=64 37 37 1 28.6 34.9 0.8X +skipBinary, half overlap, len=64 39 43 3 27.0 37.1 0.7X ================================================================================================ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 13 13 0 80.2 12.5 1.0X -skipBinary, payloadLen=8 5 5 0 223.0 4.5 2.8X -readBinary, payloadLen=32 13 13 0 79.4 12.6 1.0X -skipBinary, payloadLen=32 5 5 0 223.1 4.5 2.8X -readBinary, payloadLen=128 15 15 0 68.7 14.6 0.9X -skipBinary, payloadLen=128 5 5 0 223.5 4.5 2.8X -readBinary, payloadLen=512 35 37 1 30.2 33.1 0.4X -skipBinary, payloadLen=512 5 5 0 222.8 4.5 2.8X +readBinary, payloadLen=8 18 18 1 58.9 17.0 1.0X +skipBinary, payloadLen=8 6 6 1 174.2 5.7 3.0X +readBinary, payloadLen=32 18 18 1 58.1 17.2 1.0X +skipBinary, payloadLen=32 6 6 0 174.2 5.7 3.0X +readBinary, payloadLen=128 21 21 0 50.8 19.7 0.9X +skipBinary, payloadLen=128 6 6 0 174.1 5.7 3.0X +readBinary, payloadLen=512 34 34 0 30.9 32.3 0.5X +skipBinary, payloadLen=512 6 6 0 174.1 5.7 3.0X ================================================================================================ Variant reads ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1015-azure +OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 4 4 0 272.2 3.7 1.0X -readShorts (INT32) 7 7 0 158.5 6.3 0.6X -readUnsignedIntegers (INT32 -> Long) 7 7 0 159.0 6.3 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 22 22 1 47.7 21.0 0.2X -skipBytes 7 7 0 148.2 6.7 0.5X -skipShorts 7 7 0 148.2 6.7 0.5X -readByte (INT32 single-value) 11 11 0 96.9 10.3 0.4X -readShort (INT32 single-value) 11 11 0 96.9 10.3 0.4X -readInteger (INT32 single-value) 11 11 1 97.0 10.3 0.4X -readLong (INT64 single-value) 13 13 1 83.0 12.0 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 49 51 2 21.4 46.7 0.1X +readBytes (INT32) 5 5 0 212.1 4.7 1.0X +readShorts (INT32) 9 10 1 111.4 9.0 0.5X +readUnsignedIntegers (INT32 -> Long) 9 10 0 111.0 9.0 0.5X +readUnsignedLongs (INT64 -> Decimal(20,0)) 31 32 0 33.3 30.0 0.2X +skipBytes 7 7 0 145.9 6.9 0.7X +skipShorts 7 7 1 145.8 6.9 0.7X +readByte (INT32 single-value) 12 12 0 89.2 11.2 0.4X +readShort (INT32 single-value) 14 14 1 75.2 13.3 0.4X +readInteger (INT32 single-value) 14 14 1 75.2 13.3 0.4X +readLong (INT64 single-value) 16 17 3 64.4 15.5 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 65 67 2 16.2 61.9 0.1X From eb2cb9bc7228e13b1ef9d13deb4cc2f7fa287976 Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 20:44:50 +0000 Subject: [PATCH 07/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 25, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk25-results.txt | 86 +++++++++---------- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt index a0f7ee9e98a39..47842f3a08d99 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt @@ -6,14 +6,14 @@ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 3 3 0 328.8 3.0 1.0X -skipIntegers, constant 3 3 0 371.3 2.7 1.1X -readIntegers, monotonic 3 3 0 328.7 3.0 1.0X -skipIntegers, monotonic 3 3 0 373.9 2.7 1.1X -readIntegers, small-delta random 4 4 0 272.9 3.7 0.8X -skipIntegers, small-delta random 3 4 0 301.7 3.3 0.9X -readIntegers, wide random 5 5 0 220.1 4.5 0.7X -skipIntegers, wide random 4 5 0 235.3 4.3 0.7X +readIntegers, constant 2 2 0 476.4 2.1 1.0X +skipIntegers, constant 2 2 0 606.4 1.6 1.3X +readIntegers, monotonic 2 2 0 476.4 2.1 1.0X +skipIntegers, monotonic 2 2 0 603.8 1.7 1.3X +readIntegers, small-delta random 3 3 0 333.6 3.0 0.7X +skipIntegers, small-delta random 3 3 0 398.5 2.5 0.8X +readIntegers, wide random 4 4 0 267.5 3.7 0.6X +skipIntegers, wide random 4 4 0 286.7 3.5 0.6X ================================================================================================ @@ -24,14 +24,14 @@ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 3 3 0 345.7 2.9 1.0X -skipLongs, constant 3 3 0 368.9 2.7 1.1X -readLongs, monotonic 3 3 0 345.9 2.9 1.0X -skipLongs, monotonic 3 3 0 372.0 2.7 1.1X -readLongs, small-delta random 4 4 0 284.4 3.5 0.8X -skipLongs, small-delta random 3 4 0 302.2 3.3 0.9X -readLongs, wide random 6 6 0 164.8 6.1 0.5X -skipLongs, wide random 6 6 0 168.7 5.9 0.5X +readLongs, constant 2 2 0 508.3 2.0 1.0X +skipLongs, constant 2 2 0 603.1 1.7 1.2X +readLongs, monotonic 2 2 0 520.9 1.9 1.0X +skipLongs, monotonic 2 2 0 603.7 1.7 1.2X +readLongs, small-delta random 3 3 0 356.6 2.8 0.7X +skipLongs, small-delta random 3 3 0 397.2 2.5 0.8X +readLongs, wide random 6 6 0 189.4 5.3 0.4X +skipLongs, wide random 5 5 0 196.0 5.1 0.4X ================================================================================================ @@ -42,14 +42,14 @@ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 46 47 2 23.0 43.5 1.0X -skipBinary, no overlap, len=16 47 48 1 22.2 45.1 1.0X -readBinary, half overlap, len=16 48 49 1 21.7 46.1 0.9X -skipBinary, half overlap, len=16 54 55 2 19.4 51.4 0.8X -readBinary, full overlap, len=16 49 50 1 21.5 46.5 0.9X -skipBinary, full overlap, len=16 54 55 1 19.3 51.8 0.8X -readBinary, half overlap, len=64 49 49 1 21.5 46.5 0.9X -skipBinary, half overlap, len=64 54 54 1 19.6 51.1 0.9X +readBinary, no overlap, len=16 27 31 3 38.5 26.0 1.0X +skipBinary, no overlap, len=16 32 36 1 33.2 30.1 0.9X +readBinary, half overlap, len=16 33 34 1 31.7 31.6 0.8X +skipBinary, half overlap, len=16 38 39 1 27.5 36.3 0.7X +readBinary, full overlap, len=16 34 34 1 31.3 32.0 0.8X +skipBinary, full overlap, len=16 38 39 1 27.3 36.6 0.7X +readBinary, half overlap, len=64 34 35 1 30.8 32.5 0.8X +skipBinary, half overlap, len=64 38 39 2 27.8 36.0 0.7X ================================================================================================ @@ -60,14 +60,14 @@ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 25 25 1 42.0 23.8 1.0X -skipBinary, payloadLen=8 7 7 1 156.3 6.4 3.7X -readBinary, payloadLen=32 29 30 3 36.8 27.2 0.9X -skipBinary, payloadLen=32 7 7 0 156.1 6.4 3.7X -readBinary, payloadLen=128 27 28 1 38.5 26.0 0.9X -skipBinary, payloadLen=128 7 7 1 156.4 6.4 3.7X -readBinary, payloadLen=512 47 48 1 22.5 44.5 0.5X -skipBinary, payloadLen=512 7 7 0 156.2 6.4 3.7X +readBinary, payloadLen=8 16 17 1 63.8 15.7 1.0X +skipBinary, payloadLen=8 6 6 0 183.2 5.5 2.9X +readBinary, payloadLen=32 16 16 0 64.7 15.5 1.0X +skipBinary, payloadLen=32 6 6 0 182.4 5.5 2.9X +readBinary, payloadLen=128 19 19 1 55.5 18.0 0.9X +skipBinary, payloadLen=128 6 6 0 182.6 5.5 2.9X +readBinary, payloadLen=512 41 42 1 25.8 38.8 0.4X +skipBinary, payloadLen=512 6 6 0 182.3 5.5 2.9X ================================================================================================ @@ -78,16 +78,16 @@ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 6 6 0 178.3 5.6 1.0X -readShorts (INT32) 10 10 1 110.1 9.1 0.6X -readUnsignedIntegers (INT32 -> Long) 10 11 1 101.4 9.9 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 29 29 2 36.7 27.3 0.2X -skipBytes 7 8 1 147.9 6.8 0.8X -skipShorts 7 7 1 148.0 6.8 0.8X -readByte (INT32 single-value) 13 13 1 80.6 12.4 0.5X -readShort (INT32 single-value) 15 15 1 69.6 14.4 0.4X -readInteger (INT32 single-value) 16 16 1 66.3 15.1 0.4X -readLong (INT64 single-value) 17 18 0 60.1 16.6 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 80 81 2 13.1 76.1 0.1X +readBytes (INT32) 5 5 0 212.0 4.7 1.0X +readShorts (INT32) 9 9 0 117.5 8.5 0.6X +readUnsignedIntegers (INT32 -> Long) 10 10 0 104.6 9.6 0.5X +readUnsignedLongs (INT64 -> Decimal(20,0)) 29 29 0 36.7 27.3 0.2X +skipBytes 6 7 0 166.7 6.0 0.8X +skipShorts 7 7 0 159.9 6.3 0.8X +readByte (INT32 single-value) 12 12 0 86.3 11.6 0.4X +readShort (INT32 single-value) 13 13 1 81.7 12.2 0.4X +readInteger (INT32 single-value) 14 14 0 74.5 13.4 0.4X +readLong (INT64 single-value) 16 16 0 66.9 14.9 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 63 66 2 16.7 59.7 0.1X From daf1918d5d1d1b75afbf4e940b67a3836f6d121a Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 20:46:24 +0000 Subject: [PATCH 08/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 17, Scala 2.13, split 1 of 1) --- ...VectorizedDeltaReaderBenchmark-results.txt | 82 +++++++++---------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt index 744fb0554ab90..2903441e29f13 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt @@ -6,14 +6,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 470.8 2.1 1.0X -skipIntegers, constant 2 2 0 613.4 1.6 1.3X -readIntegers, monotonic 2 2 0 469.7 2.1 1.0X -skipIntegers, monotonic 2 2 0 613.8 1.6 1.3X -readIntegers, small-delta random 3 3 0 352.2 2.8 0.7X -skipIntegers, small-delta random 2 2 0 429.3 2.3 0.9X -readIntegers, wide random 4 4 0 268.9 3.7 0.6X -skipIntegers, wide random 3 3 0 310.5 3.2 0.7X +readIntegers, constant 3 3 0 375.1 2.7 1.0X +skipIntegers, constant 2 2 0 608.9 1.6 1.6X +readIntegers, monotonic 3 3 0 375.1 2.7 1.0X +skipIntegers, monotonic 2 2 0 612.2 1.6 1.6X +readIntegers, small-delta random 4 4 0 297.3 3.4 0.8X +skipIntegers, small-delta random 2 3 0 426.1 2.3 1.1X +readIntegers, wide random 4 4 0 240.8 4.2 0.6X +skipIntegers, wide random 3 3 0 308.6 3.2 0.8X ================================================================================================ @@ -24,14 +24,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 535.7 1.9 1.0X +readLongs, constant 2 2 0 535.8 1.9 1.0X skipLongs, constant 2 2 0 612.6 1.6 1.1X -readLongs, monotonic 2 2 0 536.8 1.9 1.0X -skipLongs, monotonic 2 2 0 610.3 1.6 1.1X -readLongs, small-delta random 3 3 0 373.5 2.7 0.7X -skipLongs, small-delta random 2 2 0 429.6 2.3 0.8X -readLongs, wide random 6 6 0 187.4 5.3 0.3X -skipLongs, wide random 5 5 0 194.6 5.1 0.4X +readLongs, monotonic 2 2 0 534.1 1.9 1.0X +skipLongs, monotonic 2 2 0 611.2 1.6 1.1X +readLongs, small-delta random 3 3 0 368.4 2.7 0.7X +skipLongs, small-delta random 2 2 0 427.1 2.3 0.8X +readLongs, wide random 6 6 0 187.7 5.3 0.4X +skipLongs, wide random 5 5 0 196.3 5.1 0.4X ================================================================================================ @@ -42,14 +42,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 27 28 1 38.6 25.9 1.0X -skipBinary, no overlap, len=16 31 32 1 33.4 29.9 0.9X -readBinary, half overlap, len=16 34 34 1 31.2 32.0 0.8X -skipBinary, half overlap, len=16 39 40 1 27.1 36.9 0.7X -readBinary, full overlap, len=16 34 35 1 30.8 32.5 0.8X -skipBinary, full overlap, len=16 39 40 0 26.6 37.6 0.7X -readBinary, half overlap, len=64 35 35 1 30.2 33.1 0.8X -skipBinary, half overlap, len=64 39 40 0 26.8 37.3 0.7X +readBinary, no overlap, len=16 27 28 1 38.5 26.0 1.0X +skipBinary, no overlap, len=16 32 33 1 32.9 30.4 0.9X +readBinary, half overlap, len=16 34 34 1 31.0 32.2 0.8X +skipBinary, half overlap, len=16 40 40 1 26.5 37.7 0.7X +readBinary, full overlap, len=16 35 35 1 30.2 33.1 0.8X +skipBinary, full overlap, len=16 40 40 0 26.4 37.9 0.7X +readBinary, half overlap, len=64 34 35 1 30.5 32.8 0.8X +skipBinary, half overlap, len=64 39 41 2 26.6 37.7 0.7X ================================================================================================ @@ -60,14 +60,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 17 17 0 63.4 15.8 1.0X -skipBinary, payloadLen=8 6 6 0 175.2 5.7 2.8X -readBinary, payloadLen=32 17 17 1 62.5 16.0 1.0X -skipBinary, payloadLen=32 6 7 1 174.7 5.7 2.8X -readBinary, payloadLen=128 19 19 0 55.3 18.1 0.9X -skipBinary, payloadLen=128 6 6 1 176.1 5.7 2.8X -readBinary, payloadLen=512 40 41 1 26.3 38.0 0.4X -skipBinary, payloadLen=512 6 6 1 175.1 5.7 2.8X +readBinary, payloadLen=8 17 19 3 63.3 15.8 1.0X +skipBinary, payloadLen=8 6 6 0 174.7 5.7 2.8X +readBinary, payloadLen=32 17 17 1 62.4 16.0 1.0X +skipBinary, payloadLen=32 6 6 0 174.6 5.7 2.8X +readBinary, payloadLen=128 19 19 0 55.4 18.1 0.9X +skipBinary, payloadLen=128 6 6 1 174.4 5.7 2.8X +readBinary, payloadLen=512 41 43 1 25.6 39.0 0.4X +skipBinary, payloadLen=512 6 6 0 174.9 5.7 2.8X ================================================================================================ @@ -79,15 +79,15 @@ AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- readBytes (INT32) 5 5 0 195.8 5.1 1.0X -readShorts (INT32) 9 9 0 118.9 8.4 0.6X -readUnsignedIntegers (INT32 -> Long) 9 9 0 118.1 8.5 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 26 27 0 39.8 25.1 0.2X -skipBytes 7 7 0 160.5 6.2 0.8X -skipShorts 7 7 0 160.3 6.2 0.8X -readByte (INT32 single-value) 13 14 2 78.0 12.8 0.4X -readShort (INT32 single-value) 13 13 0 83.7 11.9 0.4X -readInteger (INT32 single-value) 13 13 0 83.8 11.9 0.4X -readLong (INT64 single-value) 14 15 3 73.2 13.7 0.4X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 61 65 3 17.1 58.5 0.1X +readShorts (INT32) 9 9 0 114.9 8.7 0.6X +readUnsignedIntegers (INT32 -> Long) 9 9 1 117.2 8.5 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 28 29 2 37.5 26.6 0.2X +skipBytes 7 7 0 158.1 6.3 0.8X +skipShorts 7 7 0 158.1 6.3 0.8X +readByte (INT32 single-value) 14 14 0 77.5 12.9 0.4X +readShort (INT32 single-value) 13 13 0 83.5 12.0 0.4X +readInteger (INT32 single-value) 13 13 0 83.6 12.0 0.4X +readLong (INT64 single-value) 14 15 0 73.0 13.7 0.4X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 66 67 1 15.9 62.8 0.1X From e704b654c223243a48df7decc971c8bd4a3ca3a4 Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 21:15:18 +0000 Subject: [PATCH 09/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 21, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk21-results.txt | 86 +++++++++---------- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt index f73fe10ff3ce4..735d0c8cf2ce0 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt @@ -6,14 +6,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 453.0 2.2 1.0X -skipIntegers, constant 2 2 0 524.5 1.9 1.2X -readIntegers, monotonic 2 2 0 455.5 2.2 1.0X -skipIntegers, monotonic 2 2 0 535.1 1.9 1.2X -readIntegers, small-delta random 3 4 0 306.9 3.3 0.7X -skipIntegers, small-delta random 3 3 0 335.3 3.0 0.7X -readIntegers, wide random 4 4 0 243.7 4.1 0.5X -skipIntegers, wide random 4 4 0 266.8 3.7 0.6X +readIntegers, constant 2 2 0 461.5 2.2 1.0X +skipIntegers, constant 2 2 0 499.8 2.0 1.1X +readIntegers, monotonic 2 2 0 462.0 2.2 1.0X +skipIntegers, monotonic 2 2 0 515.6 1.9 1.1X +readIntegers, small-delta random 3 4 0 304.2 3.3 0.7X +skipIntegers, small-delta random 3 3 0 328.4 3.0 0.7X +readIntegers, wide random 4 4 0 240.4 4.2 0.5X +skipIntegers, wide random 4 4 0 257.6 3.9 0.6X ================================================================================================ @@ -24,14 +24,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 494.1 2.0 1.0X -skipLongs, constant 2 2 0 521.4 1.9 1.1X -readLongs, monotonic 2 2 0 496.3 2.0 1.0X -skipLongs, monotonic 2 2 0 522.5 1.9 1.1X -readLongs, small-delta random 3 3 0 330.6 3.0 0.7X -skipLongs, small-delta random 3 3 0 335.5 3.0 0.7X -readLongs, wide random 6 6 0 167.1 6.0 0.3X -skipLongs, wide random 6 6 0 170.2 5.9 0.3X +readLongs, constant 2 2 0 490.6 2.0 1.0X +skipLongs, constant 2 2 0 496.6 2.0 1.0X +readLongs, monotonic 2 2 0 491.3 2.0 1.0X +skipLongs, monotonic 2 2 0 504.3 2.0 1.0X +readLongs, small-delta random 3 3 0 317.1 3.2 0.6X +skipLongs, small-delta random 3 3 0 323.9 3.1 0.7X +readLongs, wide random 6 6 0 163.0 6.1 0.3X +skipLongs, wide random 6 6 0 165.2 6.1 0.3X ================================================================================================ @@ -42,14 +42,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 29 31 3 36.2 27.6 1.0X -skipBinary, no overlap, len=16 31 32 2 34.0 29.4 0.9X -readBinary, half overlap, len=16 35 39 2 29.7 33.6 0.8X -skipBinary, half overlap, len=16 38 39 2 27.6 36.3 0.8X -readBinary, full overlap, len=16 35 37 2 29.6 33.8 0.8X -skipBinary, full overlap, len=16 39 40 3 27.1 36.9 0.7X -readBinary, half overlap, len=64 37 37 1 28.6 34.9 0.8X -skipBinary, half overlap, len=64 39 43 3 27.0 37.1 0.7X +readBinary, no overlap, len=16 29 32 4 36.0 27.8 1.0X +skipBinary, no overlap, len=16 32 32 1 33.3 30.1 0.9X +readBinary, half overlap, len=16 35 37 2 29.6 33.8 0.8X +skipBinary, half overlap, len=16 39 41 6 27.1 36.8 0.8X +readBinary, full overlap, len=16 35 39 4 29.6 33.8 0.8X +skipBinary, full overlap, len=16 39 43 5 27.0 37.0 0.8X +readBinary, half overlap, len=64 37 41 3 28.5 35.1 0.8X +skipBinary, half overlap, len=64 39 44 4 26.6 37.6 0.7X ================================================================================================ @@ -60,14 +60,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 18 18 1 58.9 17.0 1.0X -skipBinary, payloadLen=8 6 6 1 174.2 5.7 3.0X -readBinary, payloadLen=32 18 18 1 58.1 17.2 1.0X -skipBinary, payloadLen=32 6 6 0 174.2 5.7 3.0X -readBinary, payloadLen=128 21 21 0 50.8 19.7 0.9X -skipBinary, payloadLen=128 6 6 0 174.1 5.7 3.0X -readBinary, payloadLen=512 34 34 0 30.9 32.3 0.5X -skipBinary, payloadLen=512 6 6 0 174.1 5.7 3.0X +readBinary, payloadLen=8 18 18 1 58.7 17.0 1.0X +skipBinary, payloadLen=8 6 6 0 171.4 5.8 2.9X +readBinary, payloadLen=32 18 18 0 57.8 17.3 1.0X +skipBinary, payloadLen=32 6 6 0 171.4 5.8 2.9X +readBinary, payloadLen=128 21 21 0 50.6 19.7 0.9X +skipBinary, payloadLen=128 6 6 1 171.9 5.8 2.9X +readBinary, payloadLen=512 34 35 2 30.7 32.5 0.5X +skipBinary, payloadLen=512 6 6 0 171.5 5.8 2.9X ================================================================================================ @@ -78,16 +78,16 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 9V74 80-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 212.1 4.7 1.0X -readShorts (INT32) 9 10 1 111.4 9.0 0.5X -readUnsignedIntegers (INT32 -> Long) 9 10 0 111.0 9.0 0.5X -readUnsignedLongs (INT64 -> Decimal(20,0)) 31 32 0 33.3 30.0 0.2X -skipBytes 7 7 0 145.9 6.9 0.7X -skipShorts 7 7 1 145.8 6.9 0.7X -readByte (INT32 single-value) 12 12 0 89.2 11.2 0.4X -readShort (INT32 single-value) 14 14 1 75.2 13.3 0.4X -readInteger (INT32 single-value) 14 14 1 75.2 13.3 0.4X -readLong (INT64 single-value) 16 17 3 64.4 15.5 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 65 67 2 16.2 61.9 0.1X +readBytes (INT32) 5 5 0 201.5 5.0 1.0X +readShorts (INT32) 8 8 0 136.9 7.3 0.7X +readUnsignedIntegers (INT32 -> Long) 8 8 0 136.2 7.3 0.7X +readUnsignedLongs (INT64 -> Decimal(20,0)) 30 30 1 35.4 28.2 0.2X +skipBytes 8 8 0 126.6 7.9 0.6X +skipShorts 8 8 1 131.6 7.6 0.7X +readByte (INT32 single-value) 13 13 1 82.6 12.1 0.4X +readShort (INT32 single-value) 13 13 0 82.5 12.1 0.4X +readInteger (INT32 single-value) 13 13 0 82.6 12.1 0.4X +readLong (INT64 single-value) 15 15 1 72.1 13.9 0.4X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 64 65 1 16.4 60.9 0.1X From 4a251b35da7b3fe6e4e63e0fa7fd4ec29216dd6f Mon Sep 17 00:00:00 2001 From: iemejia Date: Thu, 11 Jun 2026 21:41:07 +0000 Subject: [PATCH 10/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 21, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk21-results.txt | 96 +++++++++---------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt index 735d0c8cf2ce0..568dd3e53a12d 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt @@ -3,17 +3,17 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure -AMD EPYC 9V74 80-Core Processor +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 461.5 2.2 1.0X -skipIntegers, constant 2 2 0 499.8 2.0 1.1X -readIntegers, monotonic 2 2 0 462.0 2.2 1.0X -skipIntegers, monotonic 2 2 0 515.6 1.9 1.1X -readIntegers, small-delta random 3 4 0 304.2 3.3 0.7X -skipIntegers, small-delta random 3 3 0 328.4 3.0 0.7X -readIntegers, wide random 4 4 0 240.4 4.2 0.5X -skipIntegers, wide random 4 4 0 257.6 3.9 0.6X +readIntegers, constant 2 2 0 482.9 2.1 1.0X +skipIntegers, constant 2 2 0 519.8 1.9 1.1X +readIntegers, monotonic 2 2 0 478.9 2.1 1.0X +skipIntegers, monotonic 2 2 0 529.0 1.9 1.1X +readIntegers, small-delta random 3 3 0 333.9 3.0 0.7X +skipIntegers, small-delta random 3 3 0 359.9 2.8 0.7X +readIntegers, wide random 4 4 0 270.9 3.7 0.6X +skipIntegers, wide random 4 4 0 286.7 3.5 0.6X ================================================================================================ @@ -21,17 +21,17 @@ DELTA_BINARY_PACKED INT64 ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure -AMD EPYC 9V74 80-Core Processor +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 490.6 2.0 1.0X -skipLongs, constant 2 2 0 496.6 2.0 1.0X -readLongs, monotonic 2 2 0 491.3 2.0 1.0X -skipLongs, monotonic 2 2 0 504.3 2.0 1.0X -readLongs, small-delta random 3 3 0 317.1 3.2 0.6X -skipLongs, small-delta random 3 3 0 323.9 3.1 0.7X -readLongs, wide random 6 6 0 163.0 6.1 0.3X -skipLongs, wide random 6 6 0 165.2 6.1 0.3X +readLongs, constant 2 2 0 477.9 2.1 1.0X +skipLongs, constant 2 2 0 520.5 1.9 1.1X +readLongs, monotonic 2 2 0 478.2 2.1 1.0X +skipLongs, monotonic 2 2 0 526.5 1.9 1.1X +readLongs, small-delta random 3 3 0 334.4 3.0 0.7X +skipLongs, small-delta random 3 3 0 356.2 2.8 0.7X +readLongs, wide random 6 6 0 182.3 5.5 0.4X +skipLongs, wide random 6 6 0 185.6 5.4 0.4X ================================================================================================ @@ -39,17 +39,17 @@ DELTA_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure -AMD EPYC 9V74 80-Core Processor +AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 29 32 4 36.0 27.8 1.0X -skipBinary, no overlap, len=16 32 32 1 33.3 30.1 0.9X -readBinary, half overlap, len=16 35 37 2 29.6 33.8 0.8X -skipBinary, half overlap, len=16 39 41 6 27.1 36.8 0.8X -readBinary, full overlap, len=16 35 39 4 29.6 33.8 0.8X -skipBinary, full overlap, len=16 39 43 5 27.0 37.0 0.8X -readBinary, half overlap, len=64 37 41 3 28.5 35.1 0.8X -skipBinary, half overlap, len=64 39 44 4 26.6 37.6 0.7X +readBinary, no overlap, len=16 27 30 3 38.7 25.9 1.0X +skipBinary, no overlap, len=16 32 32 1 33.0 30.3 0.9X +readBinary, half overlap, len=16 34 37 2 31.2 32.0 0.8X +skipBinary, half overlap, len=16 39 41 3 27.0 37.0 0.7X +readBinary, full overlap, len=16 34 35 1 30.9 32.4 0.8X +skipBinary, full overlap, len=16 40 44 3 26.4 37.9 0.7X +readBinary, half overlap, len=64 34 35 2 31.3 32.0 0.8X +skipBinary, half overlap, len=64 39 40 1 26.8 37.3 0.7X ================================================================================================ @@ -57,17 +57,17 @@ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure -AMD EPYC 9V74 80-Core Processor +AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 18 18 1 58.7 17.0 1.0X -skipBinary, payloadLen=8 6 6 0 171.4 5.8 2.9X -readBinary, payloadLen=32 18 18 0 57.8 17.3 1.0X -skipBinary, payloadLen=32 6 6 0 171.4 5.8 2.9X -readBinary, payloadLen=128 21 21 0 50.6 19.7 0.9X -skipBinary, payloadLen=128 6 6 1 171.9 5.8 2.9X -readBinary, payloadLen=512 34 35 2 30.7 32.5 0.5X -skipBinary, payloadLen=512 6 6 0 171.5 5.8 2.9X +readBinary, payloadLen=8 16 17 0 65.7 15.2 1.0X +skipBinary, payloadLen=8 6 6 0 182.0 5.5 2.8X +readBinary, payloadLen=32 16 17 0 64.4 15.5 1.0X +skipBinary, payloadLen=32 6 6 0 183.3 5.5 2.8X +readBinary, payloadLen=128 19 19 1 56.6 17.7 0.9X +skipBinary, payloadLen=128 6 6 0 183.5 5.4 2.8X +readBinary, payloadLen=512 39 41 1 27.1 36.9 0.4X +skipBinary, payloadLen=512 6 6 0 182.8 5.5 2.8X ================================================================================================ @@ -75,19 +75,19 @@ Variant reads ================================================================================================ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure -AMD EPYC 9V74 80-Core Processor +AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 201.5 5.0 1.0X -readShorts (INT32) 8 8 0 136.9 7.3 0.7X -readUnsignedIntegers (INT32 -> Long) 8 8 0 136.2 7.3 0.7X -readUnsignedLongs (INT64 -> Decimal(20,0)) 30 30 1 35.4 28.2 0.2X -skipBytes 8 8 0 126.6 7.9 0.6X -skipShorts 8 8 1 131.6 7.6 0.7X -readByte (INT32 single-value) 13 13 1 82.6 12.1 0.4X -readShort (INT32 single-value) 13 13 0 82.5 12.1 0.4X -readInteger (INT32 single-value) 13 13 0 82.6 12.1 0.4X -readLong (INT64 single-value) 15 15 1 72.1 13.9 0.4X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 64 65 1 16.4 60.9 0.1X +readBytes (INT32) 5 5 0 217.8 4.6 1.0X +readShorts (INT32) 8 8 0 139.0 7.2 0.6X +readUnsignedIntegers (INT32 -> Long) 8 8 0 139.1 7.2 0.6X +readUnsignedLongs (INT64 -> Decimal(20,0)) 27 27 0 38.9 25.7 0.2X +skipBytes 8 8 0 134.7 7.4 0.6X +skipShorts 8 8 0 134.9 7.4 0.6X +readByte (INT32 single-value) 12 13 0 84.2 11.9 0.4X +readShort (INT32 single-value) 12 13 0 84.3 11.9 0.4X +readInteger (INT32 single-value) 12 13 0 84.3 11.9 0.4X +readLong (INT64 single-value) 14 15 1 72.9 13.7 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 62 64 1 16.8 59.5 0.1X From e86de0d1dcc01bbae7fcbdcbda3829b3b239b1ef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Isma=C3=ABl=20Mej=C3=ADa?= Date: Fri, 12 Jun 2026 16:37:47 +0200 Subject: [PATCH 11/15] Add readIntegersAsLongs and readIntegersAsDoubles to DELTA_BINARY_PACKED reader The delta decoder already works on long[] internally, so widening overrides can skip the int narrowing step entirely: - readIntegersAsLongs: delegates to readBulkLongs (no narrowing) - readIntegersAsDoubles: bulk decode + per-miniblock long-to-double write This benefits all updaters that use the two-pass pattern with these methods (DateToTimestampNTZUpdater, IntegerToLongUpdater, IntegerToDoubleUpdater) when reading Parquet V2 DELTA_BINARY_PACKED encoded INT32 columns. Local benchmark (Variant reads section): - readIntegersAsLongs: 438 M/s (2.1x vs per-row readUnsignedIntegers) - readIntegersAsDoubles: 429 M/s (2.0x vs per-row readUnsignedIntegers) Assisted-by: OpenCode:claude-opus-4.6 --- .../VectorizedDeltaBinaryPackedReader.java | 25 +++++++++++++++++++ .../VectorizedDeltaReaderBenchmark.scala | 9 ++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java index 6773bca7f88da..6abea3b40066f 100644 --- a/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java +++ b/sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaBinaryPackedReader.java @@ -146,6 +146,31 @@ public void readIntegers(int total, WritableColumnVector c, int rowId) { readBulkIntegers(total, c, rowId); } + @Override + public void readIntegersAsLongs(int total, WritableColumnVector c, int rowId) { + // Delta decoder already works on long[]; skip the int narrowing and write longs directly. + readBulkLongs(total, c, rowId); + } + + @Override + public void readIntegersAsDoubles(int total, WritableColumnVector c, int rowId) { + checkReadBounds(total); + int remaining = total; + if (valuesRead == 0) { + c.putDouble(rowId, (double) firstValue); + lastValueRead = firstValue; + rowId++; + remaining--; + } + readBulkLoop(remaining, c, rowId, + (col, r, buf, s, n) -> { + for (int i = s; i < s + n; i++) { + col.putDouble(r + i - s, (double) buf[i]); + } + }); + valuesRead += total; + } + // Based on VectorizedPlainValuesReader.readIntegersWithRebase @Override public final void readIntegersWithRebase( diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala index 527d43b0c8547..b97faeaf57b64 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaReaderBenchmark.scala @@ -29,7 +29,7 @@ import org.apache.parquet.io.api.Binary import org.apache.spark.benchmark.{Benchmark, BenchmarkBase} import org.apache.spark.sql.execution.vectorized.OnHeapColumnVector -import org.apache.spark.sql.types.{BinaryType, ByteType, DecimalType, IntegerType, LongType, ShortType} +import org.apache.spark.sql.types.{BinaryType, ByteType, DecimalType, DoubleType, IntegerType, LongType, ShortType} /** * Low-level benchmark for the three Parquet delta-encoding decoders: @@ -299,6 +299,7 @@ object VectorizedDeltaReaderBenchmark extends BenchmarkBase { val byteVec = new OnHeapColumnVector(NUM_ROWS, ByteType) val shortVec = new OnHeapColumnVector(NUM_ROWS, ShortType) val longVec = new OnHeapColumnVector(NUM_ROWS, LongType) + val doubleVec = new OnHeapColumnVector(NUM_ROWS, DoubleType) val unsignedLongVec = new OnHeapColumnVector(NUM_ROWS, DecimalType(20, 0)) def newIntReader(): VectorizedDeltaBinaryPackedReader = { @@ -320,6 +321,12 @@ object VectorizedDeltaReaderBenchmark extends BenchmarkBase { benchmark.addCase("readUnsignedIntegers (INT32 -> Long)") { _ => newIntReader().readUnsignedIntegers(NUM_ROWS, longVec, 0) } + benchmark.addCase("readIntegersAsLongs (INT32 -> Long)") { _ => + newIntReader().readIntegersAsLongs(NUM_ROWS, longVec, 0) + } + benchmark.addCase("readIntegersAsDoubles (INT32 -> Double)") { _ => + newIntReader().readIntegersAsDoubles(NUM_ROWS, doubleVec, 0) + } benchmark.addCase("readUnsignedLongs (INT64 -> Decimal(20,0))") { _ => unsignedLongVec.reset() newLongReader().readUnsignedLongs(NUM_ROWS, unsignedLongVec, 0) From 19663b35e609180936a4064d17675dcda3feb02e Mon Sep 17 00:00:00 2001 From: iemejia Date: Fri, 12 Jun 2026 16:42:15 +0000 Subject: [PATCH 12/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 21, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk21-results.txt | 88 ++++++++++--------- 1 file changed, 45 insertions(+), 43 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt index 568dd3e53a12d..9f0397dd4d8bd 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk21-results.txt @@ -6,14 +6,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 482.9 2.1 1.0X -skipIntegers, constant 2 2 0 519.8 1.9 1.1X -readIntegers, monotonic 2 2 0 478.9 2.1 1.0X -skipIntegers, monotonic 2 2 0 529.0 1.9 1.1X -readIntegers, small-delta random 3 3 0 333.9 3.0 0.7X -skipIntegers, small-delta random 3 3 0 359.9 2.8 0.7X -readIntegers, wide random 4 4 0 270.9 3.7 0.6X -skipIntegers, wide random 4 4 0 286.7 3.5 0.6X +readIntegers, constant 2 2 0 476.2 2.1 1.0X +skipIntegers, constant 2 2 0 561.5 1.8 1.2X +readIntegers, monotonic 2 2 0 480.9 2.1 1.0X +skipIntegers, monotonic 2 2 0 562.4 1.8 1.2X +readIntegers, small-delta random 3 3 0 330.9 3.0 0.7X +skipIntegers, small-delta random 3 3 0 376.2 2.7 0.8X +readIntegers, wide random 4 4 0 268.4 3.7 0.6X +skipIntegers, wide random 4 4 0 293.1 3.4 0.6X ================================================================================================ @@ -24,14 +24,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 477.9 2.1 1.0X -skipLongs, constant 2 2 0 520.5 1.9 1.1X -readLongs, monotonic 2 2 0 478.2 2.1 1.0X -skipLongs, monotonic 2 2 0 526.5 1.9 1.1X -readLongs, small-delta random 3 3 0 334.4 3.0 0.7X -skipLongs, small-delta random 3 3 0 356.2 2.8 0.7X -readLongs, wide random 6 6 0 182.3 5.5 0.4X -skipLongs, wide random 6 6 0 185.6 5.4 0.4X +readLongs, constant 2 2 0 518.5 1.9 1.0X +skipLongs, constant 2 2 0 545.8 1.8 1.1X +readLongs, monotonic 2 2 0 517.6 1.9 1.0X +skipLongs, monotonic 2 2 0 559.7 1.8 1.1X +readLongs, small-delta random 3 3 0 357.3 2.8 0.7X +skipLongs, small-delta random 3 3 0 377.0 2.7 0.7X +readLongs, wide random 6 6 0 186.1 5.4 0.4X +skipLongs, wide random 5 6 0 190.7 5.2 0.4X ================================================================================================ @@ -42,14 +42,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 27 30 3 38.7 25.9 1.0X -skipBinary, no overlap, len=16 32 32 1 33.0 30.3 0.9X -readBinary, half overlap, len=16 34 37 2 31.2 32.0 0.8X -skipBinary, half overlap, len=16 39 41 3 27.0 37.0 0.7X -readBinary, full overlap, len=16 34 35 1 30.9 32.4 0.8X -skipBinary, full overlap, len=16 40 44 3 26.4 37.9 0.7X -readBinary, half overlap, len=64 34 35 2 31.3 32.0 0.8X -skipBinary, half overlap, len=64 39 40 1 26.8 37.3 0.7X +readBinary, no overlap, len=16 27 32 2 38.2 26.2 1.0X +skipBinary, no overlap, len=16 34 36 2 30.8 32.5 0.8X +readBinary, half overlap, len=16 33 34 0 31.5 31.7 0.8X +skipBinary, half overlap, len=16 38 39 1 27.7 36.1 0.7X +readBinary, full overlap, len=16 33 34 1 31.5 31.7 0.8X +skipBinary, full overlap, len=16 38 39 2 27.4 36.5 0.7X +readBinary, half overlap, len=64 33 34 1 31.6 31.7 0.8X +skipBinary, half overlap, len=64 38 39 2 27.9 35.8 0.7X ================================================================================================ @@ -60,14 +60,14 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 16 17 0 65.7 15.2 1.0X -skipBinary, payloadLen=8 6 6 0 182.0 5.5 2.8X -readBinary, payloadLen=32 16 17 0 64.4 15.5 1.0X -skipBinary, payloadLen=32 6 6 0 183.3 5.5 2.8X -readBinary, payloadLen=128 19 19 1 56.6 17.7 0.9X -skipBinary, payloadLen=128 6 6 0 183.5 5.4 2.8X -readBinary, payloadLen=512 39 41 1 27.1 36.9 0.4X -skipBinary, payloadLen=512 6 6 0 182.8 5.5 2.8X +readBinary, payloadLen=8 16 16 0 65.7 15.2 1.0X +skipBinary, payloadLen=8 6 6 0 184.2 5.4 2.8X +readBinary, payloadLen=32 16 16 1 65.0 15.4 1.0X +skipBinary, payloadLen=32 6 6 0 185.6 5.4 2.8X +readBinary, payloadLen=128 18 19 1 56.9 17.6 0.9X +skipBinary, payloadLen=128 6 6 0 185.6 5.4 2.8X +readBinary, payloadLen=512 39 40 1 27.1 36.9 0.4X +skipBinary, payloadLen=512 6 6 0 185.5 5.4 2.8X ================================================================================================ @@ -78,16 +78,18 @@ OpenJDK 64-Bit Server VM 21.0.11+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 217.8 4.6 1.0X -readShorts (INT32) 8 8 0 139.0 7.2 0.6X -readUnsignedIntegers (INT32 -> Long) 8 8 0 139.1 7.2 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 27 27 0 38.9 25.7 0.2X -skipBytes 8 8 0 134.7 7.4 0.6X -skipShorts 8 8 0 134.9 7.4 0.6X -readByte (INT32 single-value) 12 13 0 84.2 11.9 0.4X -readShort (INT32 single-value) 12 13 0 84.3 11.9 0.4X -readInteger (INT32 single-value) 12 13 0 84.3 11.9 0.4X -readLong (INT64 single-value) 14 15 1 72.9 13.7 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 62 64 1 16.8 59.5 0.1X +readBytes (INT32) 4 4 0 236.3 4.2 1.0X +readShorts (INT32) 9 9 2 117.6 8.5 0.5X +readUnsignedIntegers (INT32 -> Long) 9 9 1 117.6 8.5 0.5X +readIntegersAsLongs (INT32 -> Long) 4 4 0 286.8 3.5 1.2X +readIntegersAsDoubles (INT32 -> Double) 4 4 0 258.1 3.9 1.1X +readUnsignedLongs (INT64 -> Decimal(20,0)) 28 29 0 36.9 27.1 0.2X +skipBytes 6 6 0 175.1 5.7 0.7X +skipShorts 6 6 1 175.4 5.7 0.7X +readByte (INT32 single-value) 14 14 1 76.5 13.1 0.3X +readShort (INT32 single-value) 14 14 0 76.6 13.1 0.3X +readInteger (INT32 single-value) 14 14 0 76.6 13.1 0.3X +readLong (INT64 single-value) 16 16 0 67.3 14.9 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 63 65 2 16.6 60.2 0.1X From 08291f1885465264da0ade27bc01ecc9308877a3 Mon Sep 17 00:00:00 2001 From: iemejia Date: Fri, 12 Jun 2026 16:46:03 +0000 Subject: [PATCH 13/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 17, Scala 2.13, split 1 of 1) --- ...VectorizedDeltaReaderBenchmark-results.txt | 84 ++++++++++--------- 1 file changed, 43 insertions(+), 41 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt index 2903441e29f13..b3dc35eea94ea 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-results.txt @@ -6,14 +6,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 3 3 0 375.1 2.7 1.0X -skipIntegers, constant 2 2 0 608.9 1.6 1.6X -readIntegers, monotonic 3 3 0 375.1 2.7 1.0X -skipIntegers, monotonic 2 2 0 612.2 1.6 1.6X +readIntegers, constant 3 3 0 374.8 2.7 1.0X +skipIntegers, constant 2 2 0 612.8 1.6 1.6X +readIntegers, monotonic 3 3 0 374.3 2.7 1.0X +skipIntegers, monotonic 2 2 0 612.0 1.6 1.6X readIntegers, small-delta random 4 4 0 297.3 3.4 0.8X -skipIntegers, small-delta random 2 3 0 426.1 2.3 1.1X -readIntegers, wide random 4 4 0 240.8 4.2 0.6X -skipIntegers, wide random 3 3 0 308.6 3.2 0.8X +skipIntegers, small-delta random 2 3 0 426.7 2.3 1.1X +readIntegers, wide random 4 4 0 240.3 4.2 0.6X +skipIntegers, wide random 3 3 0 307.3 3.3 0.8X ================================================================================================ @@ -24,14 +24,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 535.8 1.9 1.0X -skipLongs, constant 2 2 0 612.6 1.6 1.1X -readLongs, monotonic 2 2 0 534.1 1.9 1.0X -skipLongs, monotonic 2 2 0 611.2 1.6 1.1X -readLongs, small-delta random 3 3 0 368.4 2.7 0.7X -skipLongs, small-delta random 2 2 0 427.1 2.3 0.8X -readLongs, wide random 6 6 0 187.7 5.3 0.4X -skipLongs, wide random 5 5 0 196.3 5.1 0.4X +readLongs, constant 2 2 0 535.0 1.9 1.0X +skipLongs, constant 2 2 0 607.1 1.6 1.1X +readLongs, monotonic 2 2 0 536.2 1.9 1.0X +skipLongs, monotonic 2 2 0 605.8 1.7 1.1X +readLongs, small-delta random 3 3 0 367.0 2.7 0.7X +skipLongs, small-delta random 2 2 0 426.4 2.3 0.8X +readLongs, wide random 6 6 0 187.1 5.3 0.3X +skipLongs, wide random 5 5 0 194.8 5.1 0.4X ================================================================================================ @@ -42,14 +42,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 27 28 1 38.5 26.0 1.0X -skipBinary, no overlap, len=16 32 33 1 32.9 30.4 0.9X -readBinary, half overlap, len=16 34 34 1 31.0 32.2 0.8X -skipBinary, half overlap, len=16 40 40 1 26.5 37.7 0.7X -readBinary, full overlap, len=16 35 35 1 30.2 33.1 0.8X +readBinary, no overlap, len=16 27 28 1 38.6 25.9 1.0X +skipBinary, no overlap, len=16 32 33 1 32.8 30.5 0.8X +readBinary, half overlap, len=16 33 34 1 31.4 31.8 0.8X +skipBinary, half overlap, len=16 39 40 1 26.9 37.2 0.7X +readBinary, full overlap, len=16 34 35 0 30.5 32.8 0.8X skipBinary, full overlap, len=16 40 40 0 26.4 37.9 0.7X -readBinary, half overlap, len=64 34 35 1 30.5 32.8 0.8X -skipBinary, half overlap, len=64 39 41 2 26.6 37.7 0.7X +readBinary, half overlap, len=64 35 35 0 30.3 33.0 0.8X +skipBinary, half overlap, len=64 39 40 1 26.6 37.6 0.7X ================================================================================================ @@ -60,14 +60,14 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 17 19 3 63.3 15.8 1.0X -skipBinary, payloadLen=8 6 6 0 174.7 5.7 2.8X -readBinary, payloadLen=32 17 17 1 62.4 16.0 1.0X -skipBinary, payloadLen=32 6 6 0 174.6 5.7 2.8X -readBinary, payloadLen=128 19 19 0 55.4 18.1 0.9X -skipBinary, payloadLen=128 6 6 1 174.4 5.7 2.8X -readBinary, payloadLen=512 41 43 1 25.6 39.0 0.4X -skipBinary, payloadLen=512 6 6 0 174.9 5.7 2.8X +readBinary, payloadLen=8 17 19 2 63.1 15.9 1.0X +skipBinary, payloadLen=8 6 6 0 175.2 5.7 2.8X +readBinary, payloadLen=32 17 17 1 62.3 16.0 1.0X +skipBinary, payloadLen=32 6 6 0 175.3 5.7 2.8X +readBinary, payloadLen=128 19 19 0 55.1 18.1 0.9X +skipBinary, payloadLen=128 6 6 1 174.9 5.7 2.8X +readBinary, payloadLen=512 38 40 1 27.2 36.7 0.4X +skipBinary, payloadLen=512 6 6 0 174.8 5.7 2.8X ================================================================================================ @@ -78,16 +78,18 @@ OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 195.8 5.1 1.0X -readShorts (INT32) 9 9 0 114.9 8.7 0.6X -readUnsignedIntegers (INT32 -> Long) 9 9 1 117.2 8.5 0.6X -readUnsignedLongs (INT64 -> Decimal(20,0)) 28 29 2 37.5 26.6 0.2X -skipBytes 7 7 0 158.1 6.3 0.8X -skipShorts 7 7 0 158.1 6.3 0.8X -readByte (INT32 single-value) 14 14 0 77.5 12.9 0.4X -readShort (INT32 single-value) 13 13 0 83.5 12.0 0.4X -readInteger (INT32 single-value) 13 13 0 83.6 12.0 0.4X -readLong (INT64 single-value) 14 15 0 73.0 13.7 0.4X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 66 67 1 15.9 62.8 0.1X +readBytes (INT32) 5 5 0 202.0 5.0 1.0X +readShorts (INT32) 9 9 0 115.1 8.7 0.6X +readUnsignedIntegers (INT32 -> Long) 9 9 0 121.4 8.2 0.6X +readIntegersAsLongs (INT32 -> Long) 4 4 0 291.1 3.4 1.4X +readIntegersAsDoubles (INT32 -> Double) 4 4 0 258.8 3.9 1.3X +readUnsignedLongs (INT64 -> Decimal(20,0)) 28 28 0 37.6 26.6 0.2X +skipBytes 6 7 1 165.5 6.0 0.8X +skipShorts 7 7 0 158.4 6.3 0.8X +readByte (INT32 single-value) 14 14 0 75.4 13.3 0.4X +readShort (INT32 single-value) 14 14 0 75.4 13.3 0.4X +readInteger (INT32 single-value) 14 14 0 75.4 13.3 0.4X +readLong (INT64 single-value) 15 16 0 68.1 14.7 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 62 64 1 17.0 58.7 0.1X From e8ee63b5a9d06420c6e871c9ab31c6fa21ae7a2c Mon Sep 17 00:00:00 2001 From: iemejia Date: Fri, 12 Jun 2026 17:12:44 +0000 Subject: [PATCH 14/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 25, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk25-results.txt | 98 ++++++++++--------- 1 file changed, 50 insertions(+), 48 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt index 47842f3a08d99..77e49e8f22fb0 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt @@ -3,17 +3,17 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -AMD EPYC 7763 64-Core Processor +Intel(R) Xeon(R) 6973P-C DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 2 2 0 476.4 2.1 1.0X -skipIntegers, constant 2 2 0 606.4 1.6 1.3X -readIntegers, monotonic 2 2 0 476.4 2.1 1.0X -skipIntegers, monotonic 2 2 0 603.8 1.7 1.3X -readIntegers, small-delta random 3 3 0 333.6 3.0 0.7X -skipIntegers, small-delta random 3 3 0 398.5 2.5 0.8X -readIntegers, wide random 4 4 0 267.5 3.7 0.6X -skipIntegers, wide random 4 4 0 286.7 3.5 0.6X +readIntegers, constant 1 1 0 789.3 1.3 1.0X +skipIntegers, constant 1 1 2 894.9 1.1 1.1X +readIntegers, monotonic 1 1 0 791.7 1.3 1.0X +skipIntegers, monotonic 1 1 0 915.2 1.1 1.2X +readIntegers, small-delta random 2 2 0 617.4 1.6 0.8X +skipIntegers, small-delta random 2 2 0 647.4 1.5 0.8X +readIntegers, wide random 2 2 0 468.0 2.1 0.6X +skipIntegers, wide random 2 2 0 480.3 2.1 0.6X ================================================================================================ @@ -21,17 +21,17 @@ DELTA_BINARY_PACKED INT64 ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -AMD EPYC 7763 64-Core Processor +Intel(R) Xeon(R) 6973P-C DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 2 2 0 508.3 2.0 1.0X -skipLongs, constant 2 2 0 603.1 1.7 1.2X -readLongs, monotonic 2 2 0 520.9 1.9 1.0X -skipLongs, monotonic 2 2 0 603.7 1.7 1.2X -readLongs, small-delta random 3 3 0 356.6 2.8 0.7X -skipLongs, small-delta random 3 3 0 397.2 2.5 0.8X -readLongs, wide random 6 6 0 189.4 5.3 0.4X -skipLongs, wide random 5 5 0 196.0 5.1 0.4X +readLongs, constant 1 1 0 812.3 1.2 1.0X +skipLongs, constant 1 1 0 885.5 1.1 1.1X +readLongs, monotonic 1 1 0 846.8 1.2 1.0X +skipLongs, monotonic 1 1 0 882.6 1.1 1.1X +readLongs, small-delta random 2 2 0 646.5 1.5 0.8X +skipLongs, small-delta random 2 2 0 627.4 1.6 0.8X +readLongs, wide random 3 3 0 333.5 3.0 0.4X +skipLongs, wide random 3 3 0 331.0 3.0 0.4X ================================================================================================ @@ -39,17 +39,17 @@ DELTA_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -AMD EPYC 7763 64-Core Processor +Intel(R) Xeon(R) 6973P-C DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 27 31 3 38.5 26.0 1.0X -skipBinary, no overlap, len=16 32 36 1 33.2 30.1 0.9X -readBinary, half overlap, len=16 33 34 1 31.7 31.6 0.8X -skipBinary, half overlap, len=16 38 39 1 27.5 36.3 0.7X -readBinary, full overlap, len=16 34 34 1 31.3 32.0 0.8X -skipBinary, full overlap, len=16 38 39 1 27.3 36.6 0.7X -readBinary, half overlap, len=64 34 35 1 30.8 32.5 0.8X -skipBinary, half overlap, len=64 38 39 2 27.8 36.0 0.7X +readBinary, no overlap, len=16 24 27 2 43.5 23.0 1.0X +skipBinary, no overlap, len=16 26 28 1 40.8 24.5 0.9X +readBinary, half overlap, len=16 28 29 1 37.3 26.8 0.9X +skipBinary, half overlap, len=16 22 30 2 46.9 21.3 1.1X +readBinary, full overlap, len=16 28 30 1 37.5 26.7 0.9X +skipBinary, full overlap, len=16 29 31 1 36.3 27.6 0.8X +readBinary, half overlap, len=64 30 31 1 35.0 28.5 0.8X +skipBinary, half overlap, len=64 31 34 2 34.1 29.3 0.8X ================================================================================================ @@ -57,17 +57,17 @@ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -AMD EPYC 7763 64-Core Processor +Intel(R) Xeon(R) 6973P-C DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 16 17 1 63.8 15.7 1.0X -skipBinary, payloadLen=8 6 6 0 183.2 5.5 2.9X -readBinary, payloadLen=32 16 16 0 64.7 15.5 1.0X -skipBinary, payloadLen=32 6 6 0 182.4 5.5 2.9X -readBinary, payloadLen=128 19 19 1 55.5 18.0 0.9X -skipBinary, payloadLen=128 6 6 0 182.6 5.5 2.9X -readBinary, payloadLen=512 41 42 1 25.8 38.8 0.4X -skipBinary, payloadLen=512 6 6 0 182.3 5.5 2.9X +readBinary, payloadLen=8 9 9 0 116.4 8.6 1.0X +skipBinary, payloadLen=8 5 5 0 227.9 4.4 2.0X +readBinary, payloadLen=32 9 10 0 110.9 9.0 1.0X +skipBinary, payloadLen=32 5 5 0 228.8 4.4 2.0X +readBinary, payloadLen=128 24 25 0 43.2 23.2 0.4X +skipBinary, payloadLen=128 5 5 0 231.3 4.3 2.0X +readBinary, payloadLen=512 86 89 1 12.1 82.4 0.1X +skipBinary, payloadLen=512 5 5 0 231.3 4.3 2.0X ================================================================================================ @@ -75,19 +75,21 @@ Variant reads ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -AMD EPYC 7763 64-Core Processor +Intel(R) Xeon(R) 6973P-C Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 5 5 0 212.0 4.7 1.0X -readShorts (INT32) 9 9 0 117.5 8.5 0.6X -readUnsignedIntegers (INT32 -> Long) 10 10 0 104.6 9.6 0.5X -readUnsignedLongs (INT64 -> Decimal(20,0)) 29 29 0 36.7 27.3 0.2X -skipBytes 6 7 0 166.7 6.0 0.8X -skipShorts 7 7 0 159.9 6.3 0.8X -readByte (INT32 single-value) 12 12 0 86.3 11.6 0.4X -readShort (INT32 single-value) 13 13 1 81.7 12.2 0.4X -readInteger (INT32 single-value) 14 14 0 74.5 13.4 0.4X -readLong (INT64 single-value) 16 16 0 66.9 14.9 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 63 66 2 16.7 59.7 0.1X +readBytes (INT32) 3 3 0 380.6 2.6 1.0X +readShorts (INT32) 5 5 0 209.4 4.8 0.6X +readUnsignedIntegers (INT32 -> Long) 5 5 0 197.5 5.1 0.5X +readIntegersAsLongs (INT32 -> Long) 2 2 0 473.9 2.1 1.2X +readIntegersAsDoubles (INT32 -> Double) 2 2 0 465.2 2.1 1.2X +readUnsignedLongs (INT64 -> Decimal(20,0)) 18 19 1 58.4 17.1 0.2X +skipBytes 2 2 0 475.0 2.1 1.2X +skipShorts 2 2 0 474.0 2.1 1.2X +readByte (INT32 single-value) 8 9 0 126.6 7.9 0.3X +readShort (INT32 single-value) 8 9 1 128.5 7.8 0.3X +readInteger (INT32 single-value) 8 9 1 128.4 7.8 0.3X +readLong (INT64 single-value) 9 9 0 115.0 8.7 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 55 57 1 18.9 52.8 0.0X From 36400a347631cdf4ece2b432d5a629c88a67fef0 Mon Sep 17 00:00:00 2001 From: iemejia Date: Fri, 12 Jun 2026 17:42:19 +0000 Subject: [PATCH 15/15] Benchmark results for *VectorizedDeltaReaderBenchmark (JDK 25, Scala 2.13, split 1 of 1) --- ...izedDeltaReaderBenchmark-jdk25-results.txt | 100 +++++++++--------- 1 file changed, 50 insertions(+), 50 deletions(-) diff --git a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt index 77e49e8f22fb0..ef6274f712491 100644 --- a/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt +++ b/sql/core/benchmarks/VectorizedDeltaReaderBenchmark-jdk25-results.txt @@ -3,17 +3,17 @@ DELTA_BINARY_PACKED INT32 ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -Intel(R) Xeon(R) 6973P-C +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT32: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readIntegers, constant 1 1 0 789.3 1.3 1.0X -skipIntegers, constant 1 1 2 894.9 1.1 1.1X -readIntegers, monotonic 1 1 0 791.7 1.3 1.0X -skipIntegers, monotonic 1 1 0 915.2 1.1 1.2X -readIntegers, small-delta random 2 2 0 617.4 1.6 0.8X -skipIntegers, small-delta random 2 2 0 647.4 1.5 0.8X -readIntegers, wide random 2 2 0 468.0 2.1 0.6X -skipIntegers, wide random 2 2 0 480.3 2.1 0.6X +readIntegers, constant 2 2 0 473.1 2.1 1.0X +skipIntegers, constant 2 2 0 604.4 1.7 1.3X +readIntegers, monotonic 2 2 0 473.5 2.1 1.0X +skipIntegers, monotonic 2 2 0 607.2 1.6 1.3X +readIntegers, small-delta random 3 3 0 333.4 3.0 0.7X +skipIntegers, small-delta random 3 3 0 391.2 2.6 0.8X +readIntegers, wide random 4 4 0 271.5 3.7 0.6X +skipIntegers, wide random 4 4 0 294.7 3.4 0.6X ================================================================================================ @@ -21,17 +21,17 @@ DELTA_BINARY_PACKED INT64 ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -Intel(R) Xeon(R) 6973P-C +AMD EPYC 7763 64-Core Processor DELTA_BINARY_PACKED INT64: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readLongs, constant 1 1 0 812.3 1.2 1.0X -skipLongs, constant 1 1 0 885.5 1.1 1.1X -readLongs, monotonic 1 1 0 846.8 1.2 1.0X -skipLongs, monotonic 1 1 0 882.6 1.1 1.1X -readLongs, small-delta random 2 2 0 646.5 1.5 0.8X -skipLongs, small-delta random 2 2 0 627.4 1.6 0.8X -readLongs, wide random 3 3 0 333.5 3.0 0.4X -skipLongs, wide random 3 3 0 331.0 3.0 0.4X +readLongs, constant 2 2 0 497.0 2.0 1.0X +skipLongs, constant 2 2 0 583.4 1.7 1.2X +readLongs, monotonic 2 2 0 496.7 2.0 1.0X +skipLongs, monotonic 2 2 0 588.6 1.7 1.2X +readLongs, small-delta random 3 3 0 342.8 2.9 0.7X +skipLongs, small-delta random 3 3 0 392.5 2.5 0.8X +readLongs, wide random 6 6 0 189.2 5.3 0.4X +skipLongs, wide random 5 5 0 197.4 5.1 0.4X ================================================================================================ @@ -39,17 +39,17 @@ DELTA_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -Intel(R) Xeon(R) 6973P-C +AMD EPYC 7763 64-Core Processor DELTA_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, no overlap, len=16 24 27 2 43.5 23.0 1.0X -skipBinary, no overlap, len=16 26 28 1 40.8 24.5 0.9X -readBinary, half overlap, len=16 28 29 1 37.3 26.8 0.9X -skipBinary, half overlap, len=16 22 30 2 46.9 21.3 1.1X -readBinary, full overlap, len=16 28 30 1 37.5 26.7 0.9X -skipBinary, full overlap, len=16 29 31 1 36.3 27.6 0.8X -readBinary, half overlap, len=64 30 31 1 35.0 28.5 0.8X -skipBinary, half overlap, len=64 31 34 2 34.1 29.3 0.8X +readBinary, no overlap, len=16 27 31 3 38.3 26.1 1.0X +skipBinary, no overlap, len=16 32 32 0 33.2 30.2 0.9X +readBinary, half overlap, len=16 33 34 1 31.5 31.7 0.8X +skipBinary, half overlap, len=16 38 39 1 27.6 36.2 0.7X +readBinary, full overlap, len=16 34 34 1 31.2 32.1 0.8X +skipBinary, full overlap, len=16 38 39 1 27.3 36.7 0.7X +readBinary, half overlap, len=64 34 35 1 30.5 32.7 0.8X +skipBinary, half overlap, len=64 38 39 1 27.6 36.2 0.7X ================================================================================================ @@ -57,17 +57,17 @@ DELTA_LENGTH_BYTE_ARRAY ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -Intel(R) Xeon(R) 6973P-C +AMD EPYC 7763 64-Core Processor DELTA_LENGTH_BYTE_ARRAY: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ -readBinary, payloadLen=8 9 9 0 116.4 8.6 1.0X -skipBinary, payloadLen=8 5 5 0 227.9 4.4 2.0X -readBinary, payloadLen=32 9 10 0 110.9 9.0 1.0X -skipBinary, payloadLen=32 5 5 0 228.8 4.4 2.0X -readBinary, payloadLen=128 24 25 0 43.2 23.2 0.4X -skipBinary, payloadLen=128 5 5 0 231.3 4.3 2.0X -readBinary, payloadLen=512 86 89 1 12.1 82.4 0.1X -skipBinary, payloadLen=512 5 5 0 231.3 4.3 2.0X +readBinary, payloadLen=8 16 17 1 63.7 15.7 1.0X +skipBinary, payloadLen=8 6 6 0 182.4 5.5 2.9X +readBinary, payloadLen=32 16 17 1 64.5 15.5 1.0X +skipBinary, payloadLen=32 6 6 0 182.7 5.5 2.9X +readBinary, payloadLen=128 19 19 1 55.7 18.0 0.9X +skipBinary, payloadLen=128 6 6 0 182.4 5.5 2.9X +readBinary, payloadLen=512 42 43 1 25.1 39.8 0.4X +skipBinary, payloadLen=512 6 6 0 182.7 5.5 2.9X ================================================================================================ @@ -75,21 +75,21 @@ Variant reads ================================================================================================ OpenJDK 64-Bit Server VM 25.0.3+9-LTS on Linux 6.17.0-1018-azure -Intel(R) Xeon(R) 6973P-C +AMD EPYC 7763 64-Core Processor Variant reads: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------- -readBytes (INT32) 3 3 0 380.6 2.6 1.0X -readShorts (INT32) 5 5 0 209.4 4.8 0.6X -readUnsignedIntegers (INT32 -> Long) 5 5 0 197.5 5.1 0.5X -readIntegersAsLongs (INT32 -> Long) 2 2 0 473.9 2.1 1.2X -readIntegersAsDoubles (INT32 -> Double) 2 2 0 465.2 2.1 1.2X -readUnsignedLongs (INT64 -> Decimal(20,0)) 18 19 1 58.4 17.1 0.2X -skipBytes 2 2 0 475.0 2.1 1.2X -skipShorts 2 2 0 474.0 2.1 1.2X -readByte (INT32 single-value) 8 9 0 126.6 7.9 0.3X -readShort (INT32 single-value) 8 9 1 128.5 7.8 0.3X -readInteger (INT32 single-value) 8 9 1 128.4 7.8 0.3X -readLong (INT64 single-value) 9 9 0 115.0 8.7 0.3X -readBinary(len) (DELTA_BYTE_ARRAY single-value) 55 57 1 18.9 52.8 0.0X +readBytes (INT32) 5 5 0 211.3 4.7 1.0X +readShorts (INT32) 9 9 1 120.9 8.3 0.6X +readUnsignedIntegers (INT32 -> Long) 10 10 0 105.8 9.5 0.5X +readIntegersAsLongs (INT32 -> Long) 4 4 0 283.9 3.5 1.3X +readIntegersAsDoubles (INT32 -> Double) 4 4 0 253.0 4.0 1.2X +readUnsignedLongs (INT64 -> Decimal(20,0)) 28 29 0 36.8 27.2 0.2X +skipBytes 6 6 0 166.5 6.0 0.8X +skipShorts 6 6 0 166.8 6.0 0.8X +readByte (INT32 single-value) 12 12 0 86.2 11.6 0.4X +readShort (INT32 single-value) 13 13 1 81.6 12.3 0.4X +readInteger (INT32 single-value) 14 14 0 74.7 13.4 0.4X +readLong (INT64 single-value) 16 16 1 66.9 14.9 0.3X +readBinary(len) (DELTA_BYTE_ARRAY single-value) 64 66 3 16.3 61.3 0.1X