Use `Arc<[Buffer]>` instead of raw `Vec<Buffer>` in `GenericByteViewArray` for faster `slice` by ShiKaiWi · Pull Request #6427 · apache/arrow-rs

ShiKaiWi · 2024-09-20T15:56:38Z

Which issue does this PR close?

Rationale for this change

In the GenericByteViewArray, the buffers field is a raw vector, leading to heap allocation when some methods are called, e.g. clone, slice. Using Arc<[Buffer]> instead of the raw Vec<Buffer> can avoid such heap allocation.

And the newly-add benchmark cases about slice shows the improvement:

gc view types all       time:   [687.92 µs 694.54 µs 706.74 µs]
                        change: [+2.9773% +4.3614% +6.3877%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

gc view types slice half
                        time:   [322.61 µs 325.05 µs 327.29 µs]
                        change: [-0.7292% +0.5853% +1.7277%] (p = 0.37 > 0.05)
                        No change in performance detected.
Found 16 outliers among 100 measurements (16.00%)
  14 (14.00%) high mild
  2 (2.00%) high severe

view types slice        time:   [146.78 ns 147.03 ns 147.24 ns]
                        change: [-15.424% -15.106% -14.772%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 23 outliers among 100 measurements (23.00%)
  12 (12.00%) low severe
  4 (4.00%) low mild
  2 (2.00%) high mild
  5 (5.00%) high severe

What changes are included in this PR?

Use Arc<[Buffer]> instead of the raw Vec<Buffer> as the type of buffers field of GenericByteViewArray.

Are there any user-facing changes?

The signature of the method GenericByteViewArray::new_unchecked is changed from:

pub unsafe fn new_unchecked(
        views: ScalarBuffer<u128>,
        buffers: Vec<[Buffer]>,
        nulls: Option<NullBuffer>,
    ) -> Self;

to

pub unsafe fn new_unchecked(
        views: ScalarBuffer<u128>,
        buffers: impl Into<Arc<[Buffer]>>,
        nulls: Option<NullBuffer>,
    ) -> Self;

However, any usage of this method before this PR should still work without any modification.

…rray` for faster `slice`

tustvold · 2024-09-20T21:25:22Z

Unfortunately the use of impl is still a breaking change as it could impact type inference, e.g if collecting an interator into the argument

tustvold · 2024-09-20T21:26:34Z


    /// Deconstruct this array into its constituent parts
-    pub fn into_parts(self) -> (ScalarBuffer<u128>, Vec<Buffer>, Option<NullBuffer>) {
+    pub fn into_parts(self) -> (ScalarBuffer<u128>, Arc<[Buffer]>, Option<NullBuffer>) {


This is also a breaking change

alamb · 2024-09-20T22:33:47Z

FWIW the reason "breaking change" is important is that it restricts when we can merge this PR:

https://github.com/apache/arrow-rs/blob/master/CONTRIBUTING.md#breaking-changes

findepi · 2024-10-12T19:59:37Z

    data_type: DataType,
    views: ScalarBuffer<u128>,
-    buffers: Vec<Buffer>,
+    buffers: Arc<[Buffer]>,


What's the rationale for Arc<[Buffer]> vs Vec<Arc>?

Cloning an Arc is relatively cheap (no allocation), cloning a Vec isn't.

i get it. However, if i understand correctly, Arc<[Buffer]> means the buffers can be passed around and shared only when they are within single slice, which can be limiting. For example, Can i merge two arrays, combining their Arc<Buffer> s without moving or cloning the buffers?

Can i merge two arrays, combining their Arc s without moving or cloning the buffers?

No -- you would have to create a new Vec<Buffer> (or some other way to get Arc<[Buffer]>)

So while there are some cases where new allocations are required, slicing / cloning is faster

you would have to create a new Vec<Buffer>

but that would prevent buffer sharing between two arrays, right?

slicing / cloning is faster

cloning yes

slicing -- i didn't see it

I think the observation is that during StringViewArray::slice, the slice actually happens on the views -- the buffers (that the views can point at) must be copied

Here is the clone of buffers: https://docs.rs/arrow-array/53.1.0/src/arrow_array/array/byte_view_array.rs.html#385

tustvold · 2024-11-22T14:38:45Z

Coming back to this now, I think we could merge this although I am curious if there is something funny going on here, which is why this is showing up in benchmarks. Someone popped up on discord the other day reporting a StringViewArray with ~10k buffers, this would suggest something is likely off somewhere.

I wonder if kernels are blindly concatenating identical buffers together, instead of using something like Buffer::ptr_eq to avoid a new entry for the exact same buffer allocation?

alamb · 2024-11-22T15:58:11Z

I wonder if kernels are blindly concatenating identical buffers together, instead of using something like Buffer::ptr_eq to avoid a new entry for the exact same buffer allocation?

What was happening in DataFusion was we had a Filter --> Coalesce chain and thus basically calling concat a few thousand different input RecordBatch each with a few rows.

However, to your point, it may well be the case that the input RecordBatches shared the same underlying buffer so maybe the same buffer was being appended multiple times

@XiangpengHao do you remember if you looked for this ? (related to "Section 3.5: Buffer size tuning
" in https://www.influxdata.com/blog/faster-queries-with-stringview-part-two-influxdb/)

XiangpengHao · 2024-11-22T16:32:02Z

I wonder if kernels are blindly concatenating identical buffers together, instead of using something like Buffer::ptr_eq to avoid a new entry for the exact same buffer allocation?

I think so:

arrow-rs/arrow-data/src/transform/mod.rs

Lines 630 to 637 in def94a8

    
           let variadic_data_buffers = match &data_type { 
        
               DataType::BinaryView | DataType::Utf8View => arrays 
        
                   .iter() 
        
                   .flat_map(|x| x.buffers().iter().skip(1)) 
        
                   .map(Buffer::clone) 
        
                   .collect(), 
        
               _ => vec![], 
        
           };

Someone popped up on discord the other day reporting a StringViewArray with ~10k buffers, this would suggest something is likely off somewhere.

I have experienced this when loading string view from Parquet. If the parquet data has 10k buffers of string data, the string view will just hold them. Typically we should run a filter and then gc it. This is handled in DF but if used out side DF users might need to do something similar like this: https://github.com/apache/datafusion/blob/c0ca4b4e449e07c3bcd6f3593fa31dd31ed5e0c5/datafusion/physical-plan/src/coalesce/mod.rs#L201-L221

In other words, if StringViewArray is constructed by us, it's very unlikely to have 10k buffers as we will exponentially grow the buffers ultil 2MB; so 10k buffers means ~20GB data

alamb · 2024-11-23T13:08:25Z

In other words, if StringViewArray is constructed by us, it's very unlikely to have 10k buffers as we will exponentially grow the buffers ultil 2MB; so 10k buffers means ~20GB data

Good call -- @onursatici has tracked down a related (perhaps the same) issue:

StringView: Using the Interleave kernel (and potentially others) results in many repeated buffers in variadic_buffers #6780

alamb · 2024-11-23T13:09:12Z

All the discussion above not withstanding, I think the fact that this PR improves slice speed (by avoiding allocations) for GenericByteViewArray means it is still worth merging

tustvold · 2024-11-26T10:31:32Z

            data_type: T::DATA_TYPE,
            views,
-            buffers,
+            buffers: buffers.into(),


I think we should take impl Into<Arc<[Buffer]>> in this method as well

tustvold · 2024-11-26T10:34:02Z

means it is still worth merging

Agreed

What do people think about extracting a newtype ViewBuffers or similar, much like we have Fields. This would allow us to change the internal representation down the line, whilst also potentially supporting member functions, etc...

alamb · 2024-11-26T11:18:21Z

means it is still worth merging

Agreed

What do people think about extracting a newtype ViewBuffers or similar, much like we have Fields. This would allow us to change the internal representation down the line, whilst also potentially supporting member functions, etc...

I think it would be a great idea. @ShiKaiWi is that something you would be willing to try?

ShiKaiWi · 2024-12-10T08:19:53Z

means it is still worth merging

Agreed
What do people think about extracting a newtype ViewBuffers or similar, much like we have Fields. This would allow us to change the internal representation down the line, whilst also potentially supporting member functions, etc...

I think it would be a great idea. @ShiKaiWi is that something you would be willing to try?

Sorry for the late reply (I was a bit busy with work in the last few months). I'd love to try it on this weekend.

alamb · 2024-12-10T19:04:11Z

Sorry for the late reply (I was a bit busy with work in the last few months). I'd love to try it on this weekend.

Thank @ShiKaiWi -- I don't think there is any huge rush to get this feature in, so it would be great if you could do so ❤️

alamb · 2024-12-16T18:33:43Z

Converting to draft as I think there is some new planned work here.

ctsk · 2025-06-22T20:04:02Z

May I pick this up? It seems like it would be useful for DF :)

alamb · 2025-06-23T15:01:25Z

May I pick this up? It seems like it would be useful for DF :)

Please do -- I agree I have seen this come up in some profiles

The other thing Arc<[Buffer]> might help with is reduce the cost of clone() ing StringViewArrays too

Jefffrey · 2025-12-25T08:53:23Z

Closing this PR as stale. Looks like it might be superseded by #9016 anyway.

…aper cloning (#10708) # Which issue does this PR close? - Related to #10692. # Rationale for this change This change follows directly from the existing ByteView design history: - #6408 traced slow ByteView slicing to allocating and cloning the backing-buffer list. The discussion specifically identified `take` as another operation that would benefit from sharing this list. - #6427 distinguished two related but separate concerns: very large buffer lists may indicate missing GC or deduplication, but avoiding allocation when an operation retains the complete list is independently worthwhile. - #6808 demonstrated why eager deduplication in a general-purpose path is a different tradeoff: it added substantial overhead to common operations, and review favored treating ByteView compaction more holistically. - #9016 subsequently changed `GenericByteViewArray` to store the collection as `Arc<[Buffer]>`, making it possible to share the complete list without allocation. The downstream impact is concrete. [apache/datafusion#16206](apache/datafusion#16206) describes hash joins where concatenating build-side batches produces ByteView payload columns with many backing buffers, and constructing join output repeatedly calls `take`. In that pattern, cloning and later dropping every buffer handle can become a significant part of execution time. Arrow issue #10692 provides a compact multi-stage `take` and `BatchCoalescer` reproduction of the same ownership-metadata amplification. `take_byte_view` and `filter_byte_view` do not yet use this shared representation. They rebuild the collection with `data_buffers().to_vec()`, allocating a new collection and cloning every `Buffer`. This makes the ownership bookkeeping for selection O(number of backing buffers), despite retaining exactly the same complete buffer list. Selection already copies the chosen views while leaving their payloads zero-copy. The cost of retaining the unchanged backing-buffer collection should therefore not grow with the number of entries in that collection. This PR completes that narrow part of the earlier design while leaving buffer canonicalization as a separate problem. # What changes are included in this PR? - Change `GenericByteViewArray::data_buffers()` to return `&Arc<[Buffer]>`, allowing callers to inspect the buffers as before or clone the collection's `Arc` in O(1). - Change `GenericByteViewArray::new_unchecked` to accept `Arc<[Buffer]>` directly, making shared versus newly constructed buffer-list ownership explicit at each call site. - Use `Arc::clone` in the ByteView `take` and `filter` kernels instead of rebuilding the collection. - Replace three additional `data_buffers().to_vec()` call sites found by the stricter constructor signature with shared `Arc` clones. - Verify for both StringView and BinaryView that selection results share the input buffer collection. This PR only removes repeated ownership-metadata cloning. It retains the same complete collection of backing buffers as before. It does not prune or deduplicate buffer entries, remap views, run GC, or copy string/binary payloads. The broader buffer-fragmentation problem described in #10692 remains separate. # Are these changes tested? ```shell cargo fmt --all -- --check cargo check --workspace --all-targets cargo test -p arrow-array -p arrow-select -p arrow-row -p arrow-ipc cargo test -p parquet --test arrow_reader invalid_utf8 cargo clippy -p arrow-array -p arrow-select -p arrow-row -p arrow-ipc -p parquet --all-targets --all-features -- -D warnings cargo doc -p arrow-array --no-deps ``` A temporary local Criterion benchmark was used to validate the asymptotic behavior. It takes 8,192 views distributed across a varying number of buffer entries. The entries share one immutable payload allocation, isolating collection-ownership cost from payload size. `main` and this PR were built in separate Cargo target directories on an Intel Xeon Platinum 8474C: | Buffer entries | `main` | This PR | Speedup | |---:|---:|---:|---:| | 1 | 5.4900 us | 5.4619 us | 1.01x | | 16 | 5.7342 us | 5.4577 us | 1.05x | | 256 | 9.2806 us | 5.4518 us | 1.70x | | 4,096 | 65.774 us | 5.4584 us | 12.05x | Existing normal-size benchmarks did not regress: | Benchmark | `main` | This PR | |---|---:|---:| | `take stringview 512` | 504.73 ns | 439.40 ns | | `take stringview 1024` | 872.36 ns | 773.48 ns | | `filter context mixed string view (kept 1/2)` | 73.406 us | 72.083 us | # Are there any user-facing changes? There are two public signature changes. `data_buffers()` now returns `&Arc<[Buffer]>` instead of `&[Buffer]`; slice methods remain available through deref, while direct `for` iteration can use `.iter()`. `new_unchecked` now requires `Arc<[Buffer]>`; callers constructing a `Vec<Buffer>` can convert it with `.into()`. ByteView selection results now share the immutable backing-buffer collection instead of allocating an equivalent collection. Logical values, null handling, buffer indexes, payload lifetimes, and GC behavior are unchanged. # AI assistance I used OpenAI Codex to help inspect the relevant implementation history, draft the patch and tests, and prepare the benchmark harness and PR text. I reviewed the implementation and benchmark methodology and ran the checks above locally.

…aper cloning (apache#10708) # Which issue does this PR close? - Related to apache#10692. # Rationale for this change This change follows directly from the existing ByteView design history: - apache#6408 traced slow ByteView slicing to allocating and cloning the backing-buffer list. The discussion specifically identified `take` as another operation that would benefit from sharing this list. - apache#6427 distinguished two related but separate concerns: very large buffer lists may indicate missing GC or deduplication, but avoiding allocation when an operation retains the complete list is independently worthwhile. - apache#6808 demonstrated why eager deduplication in a general-purpose path is a different tradeoff: it added substantial overhead to common operations, and review favored treating ByteView compaction more holistically. - apache#9016 subsequently changed `GenericByteViewArray` to store the collection as `Arc<[Buffer]>`, making it possible to share the complete list without allocation. The downstream impact is concrete. [apache/datafusion#16206](apache/datafusion#16206) describes hash joins where concatenating build-side batches produces ByteView payload columns with many backing buffers, and constructing join output repeatedly calls `take`. In that pattern, cloning and later dropping every buffer handle can become a significant part of execution time. Arrow issue apache#10692 provides a compact multi-stage `take` and `BatchCoalescer` reproduction of the same ownership-metadata amplification. `take_byte_view` and `filter_byte_view` do not yet use this shared representation. They rebuild the collection with `data_buffers().to_vec()`, allocating a new collection and cloning every `Buffer`. This makes the ownership bookkeeping for selection O(number of backing buffers), despite retaining exactly the same complete buffer list. Selection already copies the chosen views while leaving their payloads zero-copy. The cost of retaining the unchanged backing-buffer collection should therefore not grow with the number of entries in that collection. This PR completes that narrow part of the earlier design while leaving buffer canonicalization as a separate problem. # What changes are included in this PR? - Change `GenericByteViewArray::data_buffers()` to return `&Arc<[Buffer]>`, allowing callers to inspect the buffers as before or clone the collection's `Arc` in O(1). - Change `GenericByteViewArray::new_unchecked` to accept `Arc<[Buffer]>` directly, making shared versus newly constructed buffer-list ownership explicit at each call site. - Use `Arc::clone` in the ByteView `take` and `filter` kernels instead of rebuilding the collection. - Replace three additional `data_buffers().to_vec()` call sites found by the stricter constructor signature with shared `Arc` clones. - Verify for both StringView and BinaryView that selection results share the input buffer collection. This PR only removes repeated ownership-metadata cloning. It retains the same complete collection of backing buffers as before. It does not prune or deduplicate buffer entries, remap views, run GC, or copy string/binary payloads. The broader buffer-fragmentation problem described in apache#10692 remains separate. # Are these changes tested? ```shell cargo fmt --all -- --check cargo check --workspace --all-targets cargo test -p arrow-array -p arrow-select -p arrow-row -p arrow-ipc cargo test -p parquet --test arrow_reader invalid_utf8 cargo clippy -p arrow-array -p arrow-select -p arrow-row -p arrow-ipc -p parquet --all-targets --all-features -- -D warnings cargo doc -p arrow-array --no-deps ``` A temporary local Criterion benchmark was used to validate the asymptotic behavior. It takes 8,192 views distributed across a varying number of buffer entries. The entries share one immutable payload allocation, isolating collection-ownership cost from payload size. `main` and this PR were built in separate Cargo target directories on an Intel Xeon Platinum 8474C: | Buffer entries | `main` | This PR | Speedup | |---:|---:|---:|---:| | 1 | 5.4900 us | 5.4619 us | 1.01x | | 16 | 5.7342 us | 5.4577 us | 1.05x | | 256 | 9.2806 us | 5.4518 us | 1.70x | | 4,096 | 65.774 us | 5.4584 us | 12.05x | Existing normal-size benchmarks did not regress: | Benchmark | `main` | This PR | |---|---:|---:| | `take stringview 512` | 504.73 ns | 439.40 ns | | `take stringview 1024` | 872.36 ns | 773.48 ns | | `filter context mixed string view (kept 1/2)` | 73.406 us | 72.083 us | # Are there any user-facing changes? There are two public signature changes. `data_buffers()` now returns `&Arc<[Buffer]>` instead of `&[Buffer]`; slice methods remain available through deref, while direct `for` iteration can use `.iter()`. `new_unchecked` now requires `Arc<[Buffer]>`; callers constructing a `Vec<Buffer>` can convert it with `.into()`. ByteView selection results now share the immutable backing-buffer collection instead of allocating an equivalent collection. Logical values, null handling, buffer indexes, payload lifetimes, and GC behavior are unchanged. # AI assistance I used OpenAI Codex to help inspect the relevant implementation history, draft the patch and tests, and prepare the benchmark harness and PR text. I reviewed the implementation and benchmark methodology and ran the checks above locally.

Use Arc<[Buffer]> instead of raw Vec<Buffer> in `GenericByteViewA…

18e3113

…rray` for faster `slice`

github-actions Bot added the arrow Changes to the arrow crate label Sep 20, 2024

add benchmark case about view array slice

33eae4b

ShiKaiWi marked this pull request as ready for review September 20, 2024 16:38

ShiKaiWi changed the title ~~Use Arc<[Buffer]> instead of raw Vec<Buffer> in `GenericByteViewA…~~ Use Arc<[Buffer]> instead of raw Vec<Buffer> in GenericByteViewArray for faster slice Sep 20, 2024

ShiKaiWi changed the title ~~Use Arc<[Buffer]> instead of raw Vec<Buffer> in GenericByteViewArray for faster slice~~ Use Arc<[Buffer]> instead of raw Vec<Buffer> in GenericByteViewArray for faster slice Sep 20, 2024

tustvold added the api-change Breaking API change, needs to wait for a major version release label Sep 20, 2024

tustvold added the next-major-release the PR has API changes and it waiting on the next major version label Sep 20, 2024

tustvold reviewed Sep 20, 2024

View reviewed changes

findepi reviewed Oct 12, 2024

View reviewed changes

alamb mentioned this pull request Nov 23, 2024

StringView: Using the Interleave kernel (and potentially others) results in many repeated buffers in variadic_buffers #6780

Closed

tustvold reviewed Nov 26, 2024

View reviewed changes

alamb marked this pull request as draft December 16, 2024 18:33

Dandandan mentioned this pull request May 30, 2025

Excessive Arc-clone in HashJoinStream with StringView on build-side apache/datafusion#16206

Open

ctsk mentioned this pull request Jun 25, 2025

Continued: Use Arc<[Buffer]> instead of raw Vec<Buffer> in GenericByteViewArray for faster slice #7773

Closed

Jefffrey closed this Dec 25, 2025

YimingQiao mentioned this pull request Aug 16, 2026

Expose &Arc[Buffer] in GenericByteViewArray::data_buffers for cheaper cloning #10708

Merged

Conversation

ShiKaiWi commented Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Uh oh!

tustvold commented Sep 20, 2024

Uh oh!

tustvold Sep 20, 2024

Choose a reason for hiding this comment

Uh oh!

alamb commented Sep 20, 2024

Uh oh!

findepi Oct 12, 2024

Choose a reason for hiding this comment

Uh oh!

Dandandan Oct 13, 2024

Choose a reason for hiding this comment

Uh oh!

findepi Oct 13, 2024

Choose a reason for hiding this comment

Uh oh!

alamb Oct 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

findepi Oct 15, 2024

Choose a reason for hiding this comment

Uh oh!

alamb Oct 15, 2024

Choose a reason for hiding this comment

Uh oh!

tustvold commented Nov 22, 2024

Uh oh!

alamb commented Nov 22, 2024

Uh oh!

XiangpengHao commented Nov 22, 2024

Uh oh!

alamb commented Nov 23, 2024

Uh oh!

alamb commented Nov 23, 2024

Uh oh!

tustvold Nov 26, 2024

Choose a reason for hiding this comment

Uh oh!

tustvold commented Nov 26, 2024

Uh oh!

alamb commented Nov 26, 2024

Uh oh!

ShiKaiWi commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Dec 10, 2024

Uh oh!

alamb commented Dec 16, 2024

Uh oh!

ctsk commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Jun 23, 2025

Uh oh!

Jefffrey commented Dec 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

ShiKaiWi commented Sep 20, 2024 •

edited

Loading

alamb Oct 15, 2024 •

edited

Loading

ShiKaiWi commented Dec 10, 2024 •

edited

Loading

ctsk commented Jun 22, 2025 •

edited

Loading