Skip to content

fix(server): update fff to stop runaway watcher rescans - #14543

Open
realhasanshoaib wants to merge 2 commits into
pingdotgg:mainfrom
realhasanshoaib:fix/fff-watcher-rescan-throttle
Open

realhasanshoaib wants to merge 2 commits into
pingdotgg:mainfrom
realhasanshoaib:fix/fff-watcher-rescan-throttle

Conversation

@realhasanshoaib

@realhasanshoaib realhasanshoaib commented Oct 1, 2026 •

Copy link
Copy Markdown

Problem

When a project is a big non-git folder that keeps changing, the backend stays at 60–190% CPU even when T3 Code is idle. The file-search library, fff 0.9.4, starts a full rescan of the whole folder after almost any burst of file changes, and nothing limits how often that happens. A burst can be more than 1024 changed paths, an FSEvents rescan flag, or any .gitignore edit. Full details, stack samples and a repro harness are in #14536.

This fixes watcher rescan throttling only. A separate upstream cost remains: binary detection reads changed files to the end (dmtrKovalenko/fff#895).

Change

fff fixed the rescan loop upstream in dmtrKovalenko/fff#751. That change adds a throttle: watcher rescans wait at least 30 seconds, and 5 minutes for non-git roots with 1M or more files. The first release with it is 0.10.3. This PR moves T3 to the current release, 0.11.0.

File Why
apps/server/package.json, pnpm-lock.yaml @ff-labs/fff-node 0.9.4 → 0.11.0. The lockfile diff only touches fff packages.
patches/@ff-labs__fff-node@0.11.0.patch, pnpm-workspace.yaml Moves T3's existing patch to the new version. 0.11.0 ships separate index.cjs and index.js builds. As before, the patch sends require to the ESM build, so the server and its tests share one module instance. It also keeps the app.asar.unpacked native-binary lookup, now in dist/index.js.
scripts/build-desktop-artifact.ts and its test The Windows native-load probe pointed at dist/src/index.js, which no longer exists. It now points at dist/index.js.

The fff API that T3 calls is unchanged between these versions; 0.11.0 only adds watch(). So there are no server code changes.

Scope and approval

This fixes one problem: runaway rescans in the pinned native dependency. Bug report and evidence: #14536. The PR doesn't touch product behavior, and it's complementary to #11500, which keeps a partial index on scan timeout.

Verification

Problem and fix, measured. I ran a standalone harness with the exact FileFinder.create options from WorkspaceSearchIndex.ts on a real 1.65M-file ~/code folder with live agent activity. It also writes and deletes 1500 files under node_modules every 20 s. Each version ran for about 4 minutes on an Apple M4. The harness is in #14536.

fff-node full scans CPU used average load
0.9.4 7 in 240 s 191 s 80% of a core
0.11.0 4 in 291 s 137 s 47% of a core (−41%)

Readiness refresh (2026-10-02): merged current main 54084ae1e into the existing PR branch, preserving the current Expo patches when resolving the lockfile/workspace conflicts. The PR still changes only the seven files above. The update is a normal fast-forward from the existing head; no history rewrite is needed.

Checks on refreshed head dcff125e3, run on a separate Apple M4 test host with Node 24.13.1 / pnpm 11.10.0:

  • Frozen-lockfile installation: passed.
  • Workspace and desktop-artifact tests: 137 passed across 5 files (62 workspace + 75 artifact tests).
  • Server and scripts tsc --noEmit: passed; existing Effect suggestions only.
  • Targeted lint for the artifact script and test: 0 warnings, 0 errors.
  • One scoped Codex CLI review (GPT-6.1-Sol, high reasoning): no actionable regressions. The reviewer also checked native loading, shared require/import identity, index warmup and search. After main advanced, the same scoped change was retained and the focused checks were re-run.

Packaged Electron native-load evidence: an isolated minimal macOS arm64 fixture used Electron 44.4.2 and the exact patched fff-node package/dependency closure from this lockfile. It ran from an actual app.asar with native .node / .dylib files unpacked. The fixture asserted packaged-app mode, shared require/import identity, an .asar.unpacked native-library path, successful index creation and the expected file-search result.

{"electron":"44.4.2","node":"24.21.0","isPackaged":true,"binary":"<fixture>.app/Contents/Resources/app.asar.unpacked/node_modules/@ff-labs/fff-bin-darwin-arm64/libfff_c.dylib","indexedFiles":1,"result":"needle.ts"}

Limits: this was a minimal packaged Electron native-load smoke test, not a complete T3 installer build. Packaged Windows/Linux execution and full desktop workflows were not tested. The existing Windows probe unit tests passed. The benchmark above consists of sequential runs on a changing live workspace, not a controlled claim about all projects or total app memory.

Readiness refresh performed with Codex (GPT-6); scoped CLI review used GPT-6.1-Sol at high reasoning.

Built with Claude Opus 5.5 (1M context) in T3 Code through the Claude Code harness, at medium reasoning effort.

🤖 Generated with Claude Code

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Oct 1, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 58a8f4c

Macroscope's review found this PR approvable — This is a focused fff-node dependency bug fix that throttles runaway watcher rescans while preserving the existing search integration and native packaging behavior. The remaining changes are lockfile, patch-layout, and packaging-probe updates rather than new product capability or broad infrastructure changes.

No code changes detected at c09c710. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

fff-node 0.9.4 starts a full workspace rescan on any large file-event
batch, overflow, FSEvents rescan flag, or ignore-file change, with no
rate limit. On large non-git roots under active development this keeps
the backend at 60-190% CPU. fff 0.10.3 added a rescan throttle; 0.11.0
is the current release.

0.11.0 ships separate CJS and ESM builds, so the patch now routes
`require` to the ESM build (one module instance for the server and its
tests) and carries the asar.unpacked binary lookup over to dist/index.js.

Refs pingdotgg#14536

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@realhasanshoaib
realhasanshoaib force-pushed the fix/fff-watcher-rescan-throttle branch from 58a8f4c to c09c710 Compare October 1, 2026 06:24
@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: pingdotgg/t3code/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: ae89284a-d8d7-4e69-9134-e14a4f7c32ef

📥 Commits

Reviewing files that changed from the base of the PR and between c09c710 and dcff125.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (1)
  • pnpm-workspace.yaml

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The server dependency and workspace patch move to fff-node 0.11.0. The patch updates its require export target and ASAR binary-path resolution. The Windows native-load probe and its test use the package’s updated entry point.

Changes

fff-node dependency and packaging

Layer / File(s) Summary
Upgrade fff-node and update its patch
apps/server/package.json, pnpm-workspace.yaml, patches/@ff-labs__fff-node@0.9.4.patch, patches/@ff-labs__fff-node@0.11.0.patch
The dependency and patch mapping move to version 0.11.0, and the previous version’s patch is removed. The new patch points the require export to dist/index.js and uses an existing .asar.unpacked path when available, otherwise retaining the original binary path.
Update the Windows native-load probe
scripts/build-desktop-artifact.ts, scripts/build-desktop-artifact.test.ts
The Windows probe and its test now use dist/index.js instead of dist/src/index.js.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to dcff1

The dependency mapping and Windows probe follow the updated entry path. Although the 0.11.0 archive was unavailable for independent inspection, no actionable mismatch or merge blocker is established.

Architecture Summary

Architecture risk: 🔵 Low · up to dcff1

The change affects 4 systems.

Changed systems: patches, scripts, apps/server, pnpm-workspace.yaml

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — patches (service) was modified; 2 changed files map to changed impact.
  • observed — scripts (service) was modified; 2 changed files map to changed impact.
  • observed — apps/server (service) was modified; 1 changed file maps to changed impact.
  • observed — pnpm-workspace.yaml (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in apps/server/package.json: The @ff-labs/fff-node dependency version changes from 0.9.4 to 0.11.0.
  • observed — Modified behavior in patches/@ff-labs__fff-node@0.11.0.patch: The package’s require export changes from dist/index.cjs to dist/index.js.
  • observed — Modified behavior in patches/@ff-labs__fff-node@0.11.0.patch: dist/index.js adds the sep path import needed to split and rejoin paths using the platform separator.
  • observed — Modified behavior in patches/@ff-labs__fff-node@0.11.0.patch: Adds resolveUnpackedAsarPath: it finds the last path segment ending in .asar, returns the original path if none exists, and otherwise uses the corresponding .asar.unpacked path only when that path exists.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the dependency update and its purpose: stopping runaway watcher rescans in the server.
Description check ✅ Passed The description includes complete Problem, Change, Scope and approval, and Verification sections. It explains the issue, implementation, scope, test results, benchmark limits, and untested areas.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 2, 2026 — with ChatGPT Codex Connector

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:L 100-499 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants