Skip to content

fix(pi-fff): dedup concurrent aux finders and bound grep time (#746) - #750

Merged
dmtrKovalenko merged 3 commits into
mainfrom
triage-bot/issue-746
Aug 6, 2026
Merged

dmtrKovalenko merged 3 commits into
mainfrom
triage-bot/issue-746

Conversation

@gustav-fff

Copy link
Copy Markdown
Collaborator

Closes #746

Root cause

AuxFinderPool.acquire() pushes into entries only after waitForScan() resolves (packages/pi-fff/src/aux-finders.ts:84), so two concurrent acquire("/Users/<user>") both observe empty entries and each spawn a separate full-home scan — no dedup, unlike the main finder's finderPromise. Separately, picker.grep() (packages/pi-fff/src/index.ts:708) was called with no timeBudgetMs; the native call is synchronous so an AbortSignal cannot interrupt an in-flight search, and the fuzzy fallback repeated the full traversal after an exhausted exact pass.

Fix

Coalesce in-flight AuxFinderPool creations by root through a pending map so concurrent acquires share one finder/scan. Pass a finite GREP_TIME_BUDGET_MS (10s) to both native grep calls (engine returns partial results + nextCursor on timeout). Skip the automatic fuzzy fallback when the exact pass left a nextCursor (corpus not exhausted), avoiding a repeated broad-root walk.

Steps to reproduce

On pre-fix origin/main:

cd packages/pi-fff
cat > test/repro.test.ts <<'TS'
import { describe, expect, mock, test } from "bun:test";
const created: any[] = [];
const mod = { FileFinder: { create: (o: any) => {
  const f = { isDestroyed: false, basePath: o.basePath,
    waitForScan: () => new Promise((r) => setTimeout(r, 50)),
    destroy() { f.isDestroyed = true; } };
  created.push(f); return { ok: true, value: f };
} } };
mock.module("@ff-labs/fff-node", () => mod);
mock.module("@ff-labs/fff-bun", () => mod);
const { AuxFinderPool } = await import("../src/aux-finders");
describe("#746", () => { test("dedup", async () => {
  const pool = new AuxFinderPool({ enableFsRootScanning: false });
  const [a, b] = await Promise.all([pool.acquire("/Users/x"), pool.acquire("/Users/x")]);
  expect(created.length).toBe(1);
  expect(a.finder).toBe(b.finder);
}); });
TS
bun test test/repro.test.ts

Expected: created.length === 1, one shared finder.
Actual (pre-fix):

Expected: 1
Received: 2
(fail) #746 > dedup

Two separate finders created for the same root → two concurrent full-home scans, matching the reported tens-of-minutes hang.

How verified

cd packages/pi-fff && bun test test/
# 47 pass, 0 fail (includes new test/aux-dedup.test.ts covering the race)

The regression test two concurrent acquires for same root share one finder fails on pre-fix main and passes with this change.

Automated triage via Gustav. Honk-Honk 🪿

Concurrent AuxFinderPool.acquire() calls for the same root each started a
full scan because entries was populated only after waitForScan() resolved.
Coalesce in-flight creations by root via a pending map. Also pass a finite
timeBudgetMs to native grep (sync call, uninterruptible by AbortSignal) and
skip the fuzzy fallback when the exact pass left a nextCursor.

Refs #746
Comment thread packages/pi-fff/src/index.ts Outdated
Comment thread packages/pi-fff/src/index.ts Outdated
Co-authored-by: Dmitriy Kovalenko <dmitriy@iusevimbtw.com>
Comment thread packages/pi-fff/src/index.ts Outdated
Co-authored-by: Dmitriy Kovalenko <dmitriy@iusevimbtw.com>
@dmtrKovalenko
dmtrKovalenko merged commit 695724e into main Aug 6, 2026
52 checks passed
abhijit-s pushed a commit to abhijit-s/fff that referenced this pull request Aug 6, 2026
Upstream: empty-dir scan (dmtrKovalenko#735), gitignore fix (dmtrKovalenko#744), pi-fff aux dedup (dmtrKovalenko#750).

Conflict resolution:
- file_picker.rs: keep fork's ignore_globs/user_gi filter atop upstream's sort
- background_watcher.rs: merge fork's per-root ignore_globs/user_gi into
  upstream's walk_collect_files rewrite of index_new_directory (dmtrKovalenko#735); adopt
  upstream's clean Linux watch registration, dropping the fork's per-new-dir
  self-heal retry to avoid a picker->debouncer lock-order inversion. Primary
  self-heal (create_debouncer + coverage recheck) preserved.
abhijit-s pushed a commit to abhijit-s/fff that referenced this pull request Aug 10, 2026
First stable release since 0.17.2. Rolls up the fork's Linux APT delivery
pipeline, the worker bootstrap thundering-herd fix, the watcher lock-order
hardening, the macOS git-status staleness fix, and three upstream syncs —
empty-dir scan (dmtrKovalenko#735), gitignore fix (dmtrKovalenko#744), pi-fff dedup (dmtrKovalenko#750), and
rescan throttling for giant folders (dmtrKovalenko#751).
wpenistone pushed a commit to wpenistone/fffork that referenced this pull request Oct 13, 2026
…valenko#746) (dmtrKovalenko#750)

* fix(pi-fff): dedup concurrent aux finders and bound grep time (dmtrKovalenko#746)

Concurrent AuxFinderPool.acquire() calls for the same root each started a
full scan because entries was populated only after waitForScan() resolved.
Coalesce in-flight creations by root via a pending map. Also pass a finite
timeBudgetMs to native grep (sync call, uninterruptible by AbortSignal) and
skip the fuzzy fallback when the exact pass left a nextCursor.

Refs dmtrKovalenko#746

* Apply suggestions from code review

Co-authored-by: Dmitriy Kovalenko <dmitriy@iusevimbtw.com>

* fix typo

Co-authored-by: Dmitriy Kovalenko <dmitriy@iusevimbtw.com>

---------

Co-authored-by: gustav-fff <286169375+gustav-fff@users.noreply.github.com>
Co-authored-by: Dmitriy Kovalenko <dmitriy@iusevimbtw.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: broad absolute home-path ffgrep can appear to hang without returning a result

2 participants