Repository navigation
Description
Hi,
macOS sometimes allocates one more block than a file's size needs, so the last extent has an unused tail. When Linux updates that file's inode for any reason, even just the access time when reading it, the driver rewrites the dstream's alloced_size as the size rounded up to blocks. After that, fsck_apfs finds the volume corrupt.
Setup
- Ubuntu 26.04 VM on an M5 Mac (arm64, 4 KiB pages), kernel 7.0.0-34-generic.
- Driver at master (0318712) plus Keep the name from disk when preparing the first hard link #132. v0.3.22 has the same code.
- 1 GiB GPT image made on macOS 27.0.1 with
diskutil eraseDisk APFS Test GPT diskN.
Steps
# macOS: write a file, then extend it in a later session
head -c 65537 /dev/urandom > /Volumes/Test/A
diskutil eject diskN
diskutil image attach test.img # (waiting 30 seconds before the next step works too)
python3 -c "import os; fd = os.open('/Volumes/Test/A', os.O_WRONLY); os.pwrite(fd, os.urandom(61543), 65536)"
stat -f '%z %b' /Volumes/Test/A # 127079 264: 135168 bytes allocated for 131072 of data
diskutil eject diskN # fsck_apfs -n on the image says it's OK at this point
# Linux: only read the file
mount -t apfs -o readwrite /dev/loop0p1 /mnt
cat /mnt/A > /dev/null
umount /mnt
# macOS
fsck_apfs -n /dev/rdiskN
Result
** The volume /dev/rdisk17s1 ... was found to be corrupt and needs to be repaired.
error: alloced_size (131072) of dstream (id 3904) does not match calculated size (135168)
warning: mismatch between extentref entry reference count (1) and calculated fsroot entry reference count (0) for extent (21033 + 1)
warning: apfs_fs_alloc_count is not valid (expected 18951, actual 18952)
warning: overallocation detected on Main device: (0x5229+1) bitmap address (0x318)
error: orphan/invalid physical extent (21018)
error: missing/invalid physical extent (21018 + 15) with refcnt 1
error: found file extent overlap (id 3904) with next extent / end of dstream at logical address 69632+61440
apfsck reports Data stream: wrong allocated space.
- With
-o readwrite,noatimethe samecatleaves the volume clean, so it's the access time update. touch,chmodor a 4 KiB write into the file does the same damage, even on anoatimemount. Setting a small xattr didn't.- On macOS, the extra block comes and goes with timing. Mounting again right after the first step never left a tail, while ejecting and attaching again (or waiting about 30 seconds) did every time.
What I think happens
apfs_update_inode() rewrites the dstream xfield with alloced_size = apfs_alloced_size() (inode.c), which is the size rounded up to the block size. Its comment already wonders about this case: "TODO: is this always correct? Or could the extents have an unused tail?". Here they do: filefrag shows logical blocks 0-16 and 17-32, so 33 blocks, while the data ends in block 31. Block 32 (physical 21033) is the one fsck_apfs complains about. Keeping the alloced_size read from disk unless the extents actually change would probably avoid it.
I found it with a randomized test that runs operations on Linux and macOS in turn: a long run hit it, and I narrowed it down from there. I'm happy to try anything on more images.
Activity
eafer commentedon Oct 31, 2026
Well that's just terrible. I guess nobody is using the driver for writing all that much, or else this would have been noticed before. Thanks for the report again.
Truncate preallocated tail extents on inode read
eafer commentedon Nov 6, 2026
I don't think it's a good idea for me to try to handle the overallocations, I would just make a bigger mess. Instead I pushed a patch to truncate any preallocated extents when the inode is first read. I think that should solve the issue, but feedback is of course appreciated.
logicalangel commentedon Nov 8, 2026
Thanks, that fixed it! I tried 949677e, alone and with #132, using the steps from the report, and also with hard links, clones, a snapshot and a nearly full volume. apfsck and
fsck_apfsare clean in all of them, and the random test that first hit this passes now too.Two small things I noticed, though neither seems to do any damage:
ds_alloced_sizeisn't reset for inodes without a dstream, so an empty file can pick up an old value from the slab and go through the trim on a plain lookup.i_blockskeeps the old value after the trim, until the inode is read again.I'll close this one. Thanks again!
Fix two bugs in the previous patch
eafer commentedon Nov 9, 2026
Thanks for the review. Hopefully this is done now.