[ObjectStore] Non-Contiguous Write Payloads #81

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Currently ObjectStore::put accepts a Bytes object, which is a contiguous memory region. This is simple and efficient where payload sizes are known ahead of time, and buffers can therefore be sized appropriately up-front, however, it may force additional copies in situations where this is not known.

Describe the solution you'd like

I am not sure, the HTTP-based stores are fairly coupled to HTTP requests containing a Bytes body, and whilst reqwest does have the ability to wrap a byte stream, this would break retries, AWS content signing and possibly some other things.

Describe alternatives you've considered

With apache/arrow-rs#5500 it becomes possible to do chunked writes with no additional copies, this potentially reduces the need for this functionality as users can just allocate data in 5 MiB chunks.

Additional context

apache/arrow-rs#5500 adds a put_part API with a similar requirement