Skip to content

ContiguousMemmap: unbuffered cached file handles - #347

Merged
jrapin merged 6 commits into
mainfrom
sdascoli/contiguous-unbuffered
Nov 2, 2026
Merged

jrapin merged 6 commits into
mainfrom
sdascoli/contiguous-unbuffered

Conversation

@sdascoli

Copy link
Copy Markdown
Contributor

ContiguousMemmap.__array__ caches one open file handle per cache data file for the lifetime of the process (thread-local, reset after fork). Those handles were opened with default buffering, so each one allocates a BufferedReader buffer of st_blksize bytes. On our /checkpoint filesystem st_blksize is 4 MiB. Since a typical read is smaller than that (e.g. a 40 s fMRI window: 40 × 20484 × 4 B ≈ 3.3 MB), every read goes through the buffer and makes it resident.

A process reading from thousands of cache files therefore holds GBs of read buffers. This is especially costly with DataLoader workers: each worker reopens every file it touches, so each one ends up with its own full set.

Measured (neuralbind TRIBE, Algonauts Friends s01–06, 36k segments, one epoch, anonymous RSS of the workers):

workers before after
4 39.7 GB 18.6 GB
10 97.2 GB 46.0 GB
16 147.9 GB 73.3 GB

The loader also got faster (≈46–50 → 33 ms/batch), because each read now fetches only the requested bytes instead of filling a 4 MiB buffer after every seek. The remaining ~4.6 GB per worker is copy-on-write of the parent process's heap.

Change

  • Open cached handles with buffering=0 (raw FileIO); readinto then reads straight into the destination array.
  • Loop until the span is filled, because a raw readinto can return fewer bytes than requested (Linux caps a single read at ~2 GiB; network filesystems may return partial reads). Previously BufferedReader did this looping.
  • Test: cached handles are raw FileIO, and reads complete through a file object that returns at most 7 bytes per call. The test fails on main.

sdascoli and others added 2 commits October 26, 2026 11:10
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 26, 2026
sdascoli and others added 4 commits October 26, 2026 11:17
@jrapin
jrapin merged commit be9590d into main Nov 2, 2026
7 checks passed
@jrapin
jrapin deleted the sdascoli/contiguous-unbuffered branch November 2, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants