Repository navigation
ContiguousMemmap: unbuffered cached file handles - #347
Merged
Merged
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ContiguousMemmap.__array__caches one open file handle per cache data file for the lifetime of the process (thread-local, reset after fork). Those handles were opened with default buffering, so each one allocates aBufferedReaderbuffer ofst_blksizebytes. On our/checkpointfilesystemst_blksizeis 4 MiB. Since a typical read is smaller than that (e.g. a 40 s fMRI window: 40 × 20484 × 4 B ≈ 3.3 MB), every read goes through the buffer and makes it resident.A process reading from thousands of cache files therefore holds GBs of read buffers. This is especially costly with
DataLoaderworkers: each worker reopens every file it touches, so each one ends up with its own full set.Measured (neuralbind TRIBE, Algonauts Friends s01–06, 36k segments, one epoch, anonymous RSS of the workers):
The loader also got faster (≈46–50 → 33 ms/batch), because each read now fetches only the requested bytes instead of filling a 4 MiB buffer after every
seek. The remaining ~4.6 GB per worker is copy-on-write of the parent process's heap.Change
buffering=0(rawFileIO);readintothen reads straight into the destination array.readintocan return fewer bytes than requested (Linux caps a singlereadat ~2 GiB; network filesystems may return partial reads). PreviouslyBufferedReaderdid this looping.FileIO, and reads complete through a file object that returns at most 7 bytes per call. The test fails onmain.