You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: rust/arrow/README.md
+20-6Lines changed: 20 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -116,12 +116,12 @@ Generally, `unsafe` should only be used when a `safe` counterpart is not availab
116
116
#### cache-line aligned memory management
117
117
118
118
The arrow format recommends storing buffers aligned with cache lines, and this crate adopts this behavior.
119
-
However, Rust's global allocator does not allocates memory aligned with cache-lines. As such, many of the low-level operations related to memory management require `unsafe`.
119
+
However, Rust's global allocator does not allocate memory aligned with cache-lines. As such, many of the low-level operations related to memory management require `unsafe`.
120
120
121
121
#### Interpreting bytes
122
122
123
123
The arrow format is specified in bytes (`u8`), which can be logically represented as certain types
124
-
depending on the DataType.
124
+
depending on the `DataType`.
125
125
For many operations, such as access, representation, numerical computation and string manipulation,
126
126
it is often necessary to interpret bytes as other physical types (e.g. `i32`).
127
127
@@ -131,7 +131,7 @@ Usage of `unsafe` for the purpose of interpreting bytes in their corresponding t
131
131
132
132
The arrow format declares an ABI for zero-copy from and to libraries that implement the specification
133
133
(foreign interfaces). In Rust, receiving and sending pointers via FFI requires usage of `unsafe` due to
134
-
the impossibility of the compiler to derive the invariants (such as lifetime, null pointers, and pointer alignment), from the source code alone as they are part of the FFI contract.
134
+
the impossibility of the compiler to derive the invariants (such as lifetime, null pointers, and pointer alignment) from the source code alone as they are part of the FFI contract.
135
135
136
136
#### IPC
137
137
@@ -150,17 +150,31 @@ This requires accessing the values buffer e.g. `array.buffers()[0]`, picking the
150
150
`[i * size_of<i32>(), (i + 1) * size_of<i32>()]`, and then transmuting it to `i32`. In safe Rust,
151
151
this operation requires boundary checks that are detrimental to performance.
152
152
153
-
Usage of `unsafe` for performance reasons is justified only when the performance difference of a publicly available API is estatistically significantly larger than 10%, as demonstrated by a `bench`.
153
+
Usage of `unsafe` for performance reasons is justified only when all other alternatives have been exhausted and the performance benefits are sufficiently large (e.g. >~10%).
154
154
155
155
### Considerations when introducing `unsafe`
156
156
157
157
Usage of `unsafe` in this crate *must*:
158
158
159
159
* not expose a public API as `safe` when there are necessary invariants for that API to be defined behavior.
160
-
* have code documentation for why `safe` is not used / possible (e.g. `// 30% performance degradation if the safe counterpart is used, see bench X`)
161
-
* have code documentation about which invariant the user needs to enforce to ensure no undefined behavior (e.g. `// this buffer must be constructed according to the arrow specification`)
160
+
* have code documentation for why `safe` is not used / possible
161
+
* have code documentation about which invariant the user needs to enforce to ensure [soundness](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library), or which
162
+
* invariant is being preserved.
162
163
* if applicable, use `debug_assert`s to relevant invariants (e.g. bound checks)
163
164
165
+
Example of code documentation:
166
+
167
+
```rust
168
+
// JUSTIFICATION
169
+
// Benefit
170
+
// Describe the benefit of using unsafe. E.g.
171
+
// "30% performance degradation if the safe counterpart is used, see bench X."
172
+
// Soundness
173
+
// Describe why the code remains sound (according to the definition of rust's unsafe code guidelines). E.g.
174
+
// "We bounded check these values at initialization and the array is immutable."
175
+
let...=unsafe { ... };
176
+
```
177
+
164
178
# Publishing to crates.io
165
179
166
180
An Arrow committer can publish this crate after an official project release has
0 commit comments