Many std iterators do not implement Send/Sync

[huonw]

Many iterators use * internally, for performance. This means they don't automatically implement Send or Sync, but many/most can.

Based on the regexp, \bIterator for, the following (non-compiler/non-test) files have Iterator implementations. We should look through them to find raw pointers etc. and ensure that they implement Send/Sync if there are such pointers. We can do it progressively, comment below if you feel like taking some files:

• liballoc/boxed.rs
• libcollections @edwardw Send/Sync audit for libcollections #22715
  • /binary_heap.rs
  • /bit.rs
  • /btree/map.rs
  • /btree/node.rs
  • /btree/set.rs
  • /enum_set.rs
  • /linked_list.rs
  • /slice.rs
  • /str.rs
  • /vec.rs
  • /vec_deque.rs
  • /vec_map.rs
• libcore/char.rs @Nashenas88 Added test for Sync/Send on iterators within char. #26041
• libcore/iter.rs @Nashenas88 Adding Iterator Sync/Send tests for libcore/iter.rs #26126
• libcore/option.rs
• libcore/result.rs
• libcore/slice.rs
• libcore/str/mod.rs
• libfmt_macros/lib.rs
• librand/lib.rs
• libserialize/json.rs
• libstd/ascii.rs
• libstd/collections/hash/map.rs @GuillaumeGomez Send sync implementation on iterators #27615
• libstd/collections/hash/set.rs @GuillaumeGomez Send sync implementation on iterators #27615
• libstd/collections/hash/table.rs @GuillaumeGomez Send sync implementation on iterators #27615
• libstd/env.rs
• libstd/fs.rs
• libstd/io/mod.rs
• libstd/net/mod.rs
• libstd/net/tcp.rs
• libstd/path.rs
• libstd/sync/mpsc/mod.rs
• libstd/sync/mpsc/select.rs
• libstd/sys/common/net.rs
• libstd/sys/common/wtf8.rs
• libstd/sys/unix/fs2.rs
• libstd/sys/unix/os.rs
• libstd/sys/windows/fs2.rs
• libstd/sys/windows/os.rs
• libunicode/u_str.rs

The &mut self of Iterator means that almost all iterators will be safe to implement Send and Sync.

[edwardw]

I guess libcollections is mine.

[munds]

I would like to take on some files, but slightly unclear on what to do. For instance, in liballoc/boxed.rs:

#[stable(feature = "rust1", since = "1.0.0")]
impl<I: Iterator + ?Sized> Iterator for Box<I> {
    type Item = I::Item;
    fn next(&mut self) -> Option<I::Item> { (**self).next() }
    fn size_hint(&self) -> (usize, Option<usize>) { (**self).size_hint() }
}

Where would I implement Send/Sync? I'm looking at #22715 for examples/guidance but not getting much out of it. Once I can do that single file, it would be my pleasure to work on the libcore files as well.

[huonw]

Thanks @mund.

That impl is saying that if a Box contains an iterator, then the whole pointer can be treated as an iterator, i.e. just a proxy implementation to allow Boxs to be thrown around more freely. Box already implements Send and Sync so this case is handled.

The "interesting" ones (for this bug) are when there's a struct that's defined purely to be an iterator, e.g. the Iter struct in libcore/slice.rs. In particular, that struct contains raw pointers (*const's), and so does not automatically implement Send or Sync.

However, it is threadsafe (with the appropriate bounds on the T type) so implementing them manually is correct!

For that specific type:

unsafe impl<'a, T: Sync> Sync for Iter<'a, T> {}
unsafe impl<'a, T: Sync> Send for Iter<'a, T> {}

The appropriate bounds to use depends on the element type that the iterator yields, iterators (for the most part) can be regarded as having the same send/sync-ness as the element type, meaning if the element type of an iterator Foo<T> is:

• &T, the appropriate bounds are probably T: Sync for both Sync and Send,
• &mut T, the appropriate bounds are probably T: Sync for Sync and T: Send for Send
• T, the appropriate bounds are probably T: Sync for Sync and T: Send for Send (same as &mut T)

In summary, it is likely only the iterator structs/enums that contain raw pointers that need to be considered for this bug.