Skip to content

copy_directories: unbounded find from repo root for includeDirs patterns (127x slowdown from one slash pattern) #189

Description

@rejifald

Summary

copy_directories resolves every includeDirs pattern with an unbounded find from the repo root. For a pattern containing a slash this is unconditional; for a basename pattern it happens whenever the shallow probe misses. In a repo whose root contains a large node_modules and nested worktrees, a single extra includeDirs entry takes git gtr copy from 0.87s to 1:49.98 — a 127× regression from one line of config.

This is the cost of the slash support added for #99. It is distinct from #175 (which fixed the copy side of copy_directories); this is the pattern resolution side, before any copying happens.

Environment

  • OS: macOS 15.6 (Darwin 24.6.0), APFS
  • gtr version: 2.8.0 (Homebrew)
  • Git version: 2.55.0
  • Shell: zsh
  • Repo shape: JavaScript monorepo — 3.1 GB root node_modules (~1,550 packages), plus ~40 nested worktrees under a subdirectory of the repo root, each with its own node_modules

Reproduction

# .gtrconfig
[copy]
    includeDirs = node_modules
    include = .env.local
$ time git gtr copy my-branch -n
[OK] [dry-run] Would copy directory node_modules
git gtr copy my-branch -n  0.22s user 0.35s system 66% cpu 0.866 total

Now add one directory whose path contains a slash:

[copy]
    includeDirs = node_modules
    includeDirs = build/generated
    include = .env.local
$ time git gtr copy my-branch -n
[OK] [dry-run] Would copy directory node_modules
[OK] [dry-run] Would copy directory build/generated
git gtr copy my-branch -n  1.81s user 40.80s system 38% cpu 1:49.98 total

Nothing else changed between the two runs. On a cold page cache the first run of this exceeded 120s before I killed it.

Root cause

lib/copy.sh, in copy_directories (v2.8.0):

case "$pattern" in
  */*) find_results=$(find . -type d -path "./$pattern" 2>/dev/null || true) ;;      # line 471
  *)   find_results=$(find . -maxdepth 1 -type d -name "$pattern" 2>/dev/null || true)
       if [ -z "$find_results" ]; then
         find_results=$(find . -type d -name "$pattern" 2>/dev/null || true)         # line 474
       fi ;;
esac

Two unbounded walks:

  • Line 471 — every slash pattern walks the entire tree, descending into node_modules and into every nested worktree's node_modules. The comment above it notes -maxdepth 1 was added "to avoid scanning entire repo (e.g. node_modules)" for the basename case, but the slash branch has no equivalent bound.
  • Line 474 — the basename fallback has the same problem, and fires silently whenever the directory isn't at depth 1 (e.g. includeDirs = vendor in a repo where vendor/ is nested).

Neither prunes node_modules, .git, or nested worktrees.

It also runs per pattern per target, because copy_directories is called inside the target loop in lib/commands/copy.sh:

for target_id in "${targets[@]}"; do
  ...
  [ -n "$dir_includes" ] && copy_directories "$src_path" "$dst_path" "$dir_includes" "$dir_excludes"
done

So git gtr copy -a multiplies the full-tree walk by the number of worktrees.

Suggested fix

Most includeDirs values are literal paths, not globs, so the common case needs no find at all:

case "$pattern" in
  *[*?[]*) ;;                                   # has glob metacharacters — needs find
  *) if [ -d "$pattern" ]; then                 # literal path — O(1)
       find_results="./$pattern"
     fi
     ;;
esac

For genuinely globbed patterns, bounding the depth to the pattern's component count (-maxdepth $(count_slashes + 1)) keeps it proportional to the pattern rather than the repo. Pruning node_modules and .git during the walk would help the remaining cases:

find . \( -name node_modules -o -name .git \) -prune -o -type d -path "./$pattern" -print

Hoisting the resolution out of the per-target loop would also remove the -a multiplier.

Workaround

Keep slash-containing patterns out of includeDirs and provision those paths another way.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions