entry: fix fscache stale directory listing in parallel checkout pathspec#917
Draft
tyrielv wants to merge 2 commits into
Draft
entry: fix fscache stale directory listing in parallel checkout pathspec#917tyrielv wants to merge 2 commits into
tyrielv wants to merge 2 commits into
Conversation
When checkout.workers > 1 and core.fscache is enabled on Windows, 'git checkout <tree> -- <pathspec>' fails with 'cannot create directory: Directory not empty' when restoring files into directories that do not yet exist on disk. Root cause: create_directories() creates parent directories via mkdir(), but the Windows fscache (which caches directory listings) is not invalidated. Subsequent has_dirs_only_path() calls for the same parent directory return stale ENOENT from the cached listing. The recovery path then tries to unlink+recreate the directory, which fails because mkdir() already populated it with child directories. With workers=1, write_entry() calls flush_fscache() after each file, keeping the cache in sync. With workers>1, enqueue_checkout() defers the write (and the flush), leaving the cache stale for the next entry. Add a test that reproduces this deterministically: create two files sharing a nested parent directory, delete them in a second commit, then restore both via 'git checkout <tree> -- <pathspec>' with workers>1. Bug: https://dev.azure.com/microsoft/OS/_workitems/edit/62260193 Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
tyrielv
force-pushed
the
tyrielv/fscache-parallel-checkout-fix
branch
from
c4db787 to
a3c1aff
Compare
When checkout.workers > 1 and core.fscache is enabled on Windows, the fscache caches directory listings that become stale when create_directories() creates new parent directories via mkdir() or when write_pc_item() writes new files. Subsequent lstat() calls through the fscache return ENOENT for these just-created filesystem entries, causing two failure modes: 1. create_directories(): has_dirs_only_path() reports a just-created directory as non-existent, triggering the unlink+mkdir recovery path which fails with 'Directory not empty' because the directory already has children from earlier mkdir() calls. 2. write_pc_item(): after writing and closing a file, lstat() cannot see it through the stale parent directory listing cache, failing with 'unable to stat just-written file'. With workers=1 these do not occur because write_entry() calls flush_fscache() after each file, keeping the cache in sync. With workers>1, enqueue_checkout() defers the write (and the flush), leaving the cache stale for subsequent entries. Fix both by adding flush_fscache() calls: - In create_directories() after each successful mkdir() - In write_pc_item() before lstat() of the just-written file On non-Windows platforms flush_fscache() is a no-op, so there is no behavioral change. Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
tyrielv
force-pushed
the
tyrielv/fscache-parallel-checkout-fix
branch
from
c4db787 to
8b819f9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
git checkout <tree> -- <pathspec>withcheckout.workers > 1andcore.fscache=true(Windows) fails with:\
fatal: cannot create directory at '...': Directory not empty
\\
when restoring files into directories that do not yet exist on disk.
Root Cause
create_directories()creates parent directories viamkdir(), but the Windows fscache (which caches directory listings) is not invalidated. Subsequenthas_dirs_only_path()calls for the same parent directory return stale ENOENT from the cached listing. The recovery path then tries to unlink+recreate the directory, which fails becausemkdir()already populated it with child directories.With
workers=1,write_entry()callsflush_fscache()after each file, keeping the cache in sync. Withworkers>1,enqueue_checkout()defers the write (and the flush), leaving the cache stale for the next entry.This PR
Adds a regression test (currently expected-failure) that deterministically reproduces the bug. Fix commit incoming.
Bug: https://dev.azure.com/microsoft/OS/_workitems/edit/62260193