Dev: generated route.js is rewritten with identical content on every compile, thrashing the webpack watcher #3451

Activity

1. josuediazflores commented on Aug 11, 2026

   

   josuediazflores

   on Aug 11, 2026

   Contributor

   More actions

   I independently reproduced this on a minimal app. The mechanism holds up. Posting the control experiment the report is missing, plus a few corrections, since I think both change how this should be triaged.

   The plugin is the cause, and here is the webpack-vs-webpack control

   The report's only control is Turbopack, which changes two variables at once, so it does not by itself rule out "Next runs separate client/server/edge compilers and 3 compilations per edit is just baseline." It is not baseline. Same app, same 994 modules, webpack in every row:

   Setup	Compilations per edit
   withWorkflow, workflow route warmed	3
   Same tree, withWorkflow removed	1
   withWorkflow + generated dir added to watchOptions.ignored	1

   Hiding the generated dir from webpack's watcher alone collapses 3 back to 1, which isolates the write as the trigger.

   (To be clear, watchOptions.ignored is a diagnostic here, not a proposed fix. Shipping it would hide genuine regenerations from the dev server and serve stale workflow code.)

   Reproduction precondition that is missing from the steps

   The generated routes have to already be in webpack's module graph. On a fresh next dev where you only ever request /, the plugin still rewrites the files and bumps mtimes, but you get 1 compilation per edit and it looks indistinguishable from baseline. The cascade only appears after a workflow route has been hit. Anyone following the steps as written will likely land on "cannot reproduce."

   Also, the curl in the repro is causally inert. The edit bumps the mtime on its own; a request with no edit never does. And there is no settle step before the second shasum/stat, so the rebuild queue may not have drained.

   Corrections

   1. It is not only the mtime. writeGeneratedFile does temp-write plus rename over the target, so the inode changes too and a transient route.js.<uuid>.tmp sibling appears inside the watched app dir on every write. Worth stating because it rules out any mtime-preserving fix. The write has to be skipped outright.
   2. "On every dev compile" is not quite the trigger. Webpack never writes these files. The plugin's own watcher does, out of band, on a watched file event. There is no self-sustaining loop, which is consistent with the bounded count of 3 rather than a runaway.
   3. The Turbopack figures did not reproduce. I measured 2 compilations per edit, not 1, and 741ms cold for the workflow route, not 0.13s. The damping is real but it is roughly 10x, not elimination. The stated mechanism is also not quite right: Turbopack does not hash instead of trusting mtime at detection time. Its watcher marks the read task dirty and the cascade dies at the first turbo-tasks cell whose value compares equal, which is not guaranteed for all value types.
   4. The 1.37s cold compile did reproduce, almost exactly.
   5. The 2.2s figure, the 2 client invalidations, the 4 HMR pushes and the 4 CSS module rebuilds are the parts I could not corroborate. No HMR client was attached in my runs, so I have no reading on the last three either way.

   Status on main (5.0.0-beta.40)

   Half of this is already fixed, and the half that is not is still there.

   Fixed: step/route.js is retired in favour of flow/route.js, and the specific repro in this issue no longer fires, because classifyRebuild returns {kind:'none'} for an unrelated file and the builder returns before any write. Editing an unrelated file on a v5 app gave me 1 compilation and zero rewrites, with every generated file identical in both sha and mtime.

   Not fixed: a no-op touch of a workflow-relevant file on main still rewrites flow/route.js and manifest.json byte-identically, three cycles running, and still costs 3 compilations per edit.

   So the title and the repro are stale, but the defect is not. Might be worth re-scoping this issue around flow/route.js and manifest.json, and around "any no-op modification of a workflow-relevant file" rather than "every dev compile."

   Prior art

   This exact fix was already proposed in #2809, closed 2026-07-08, second bullet under Suggested fixes, verbatim: "Independently: skip the write when generated file content is unchanged (compare before writeFile), which would break the cycle even for unknown watched paths."

   Only the first bullet shipped (the distDir addition to the watcher ignore list, #2813). The write-if-changed half was never addressed and the single reply on the thread does not mention it, so this reads as unfinished rather than declined. That makes #3451 the remaining half of a fix that is already half-landed.

   One detail in #2809 is wrong for dev and should not be carried forward: it blames an unconditional writeFile of config.json, but writeFunctionsConfig early-returns when NODE_ENV === 'development'.

   Scope note on the fix

   Guarding writeGeneratedFile alone is not sufficient. manifest.json bypasses that helper entirely and is a bare writeFile, and it is rewritten identically on every hot rebuild. webhook/[token]/route.js and the generated .gitignore are also unguarded, though both are startup-only and lower priority.

   Separately, and independent of any compare guard: esbuild already skips writing byte-identical output, but only within a single BuildContext, keyed on latestHashes. fullRebuild disposes the contexts and constructs fresh ones, which clears that memory. main already reuses the context on the hot path, and I confirmed __step_registrations.js is never rewritten there, but fullRebuild still disposes. Not disposing would get part of this for free.

   Finally, worth saying in the thread so nobody is sent on a pointless upgrade: 4.1.0 to 4.1.5 does not help. The only functional delta in builder-eager.js across that range is the distDir addition, writeGeneratedFile is unchanged, and classifyRebuild only exists on the 5.0.0-beta line. Since npm latest still resolves to the 4.x line, a backport looks warranted, and 4.x has neither classifyRebuild nor context reuse.
2. josuediazflores commented on Aug 11, 2026

   

   josuediazflores

   on Aug 11, 2026

   Contributor

   More actions

   Correcting my own comment above. I re-measured on current main and one of my numbers does not generalize the way I implied. I also built the fix, which turned up a landmine worth knowing about before anyone implements it.

   The "3 compilations" number is version-specific

   My table was measured on the reporter's stack (v4, Next 15.5.20), and it does hold there. I re-derived it from the raw dev-server logs, counting only webpack's own ✓ Compiled … (N modules) lines and discarding the plugin's separate ✓ Compiled workflows … lines, in case those had inflated the count. They had not:

   v4 / Next 15.5.20	webpack compilations per edit
   withWorkflow	3, 3, 3
   plugin removed	1, 1, 1
   watchOptions.ignored covering the generated dir	1, 1, 1

   But on current main with Next 16.2.11 (workbench/nextjs-webpack), the steady-state cost is 2 rounds, not 3. Tapping compiler.hooks.watchRun and reading modifiedFiles directly, a no-op touch of a workflow file gives:

   round 1  nodejs  <- ["workflows/1_simple.ts"]                       # the edit
   round 2  nodejs  <- ["app/.well-known/workflow/v1/flow/route.js"]   # the identical rewrite
   

   So the mechanism is exactly as described, but on current versions it costs one extra round per rebuild, not two. Anyone reproducing this on Next 16 should expect 2 and 1, not 3 and 1. I should have scoped the original number to the stack it came from.

   The more important correction: what a write-if-changed guard can and cannot fix

   The extra round only exists when the rebuild produces byte-identical output. When an edit genuinely changes the bundle, round 2 is legitimate work and no content-comparison guard can remove it. That is a real limit on the proposed fix, and neither the original report nor my comment said so.

   It matters less than it sounds for the reporter's case, and more than it sounds for main:

   • On v4, most edits qualify. fullRebuild() fires on any watched file event, so editing an unrelated file rebuilds and emits identical bytes. In a ~520-file app, that is the common case, and the guard removes the spurious rounds for all of it.
   • On main, far fewer qualify. classifyRebuild already returns {kind:'none'} for unrelated files, so the redundant rebuild does not happen at all. What remains is no-op saves of workflow-relevant files, which editors that save on focus loss produce constantly, but it is a narrower win.

   A landmine for whoever implements this

   I have a working patch, and the first version of it silently broke Nitro/Nuxt dev.

   The Nitro dev handler cache-busts its dynamic import of steps.mjs using the mtime of workflows.mjs (packages/nitro/src/index.ts:496, and the v2 branch at :520). A step-body-only edit leaves workflows.mjs byte-identical, so guarding that write freezes the version key and the handler keeps serving the previous step code until restart. Measured across three builds: workflows.mjs held sha 973409a2… with a frozen mtime while steps.mjs went 0c834319… → f2fe550d….

   The unconditional rewrite was accidentally load-bearing. grep -rn "mtimeMs" packages/ says Nitro is the only such consumer, but it is worth checking again before merging. Opting Nitro out costs nothing, since it generates into <buildDir>/workflow, which no dev server watches.

   Two smaller things from building it:

   • Guarding writeGeneratedFile alone leaves manifest.json unguarded, and guarding manifest.json does nothing on its own, because entry discovery runs concurrently and its key order varies between identical builds. Two consecutive next build --webpack runs produced deep-equal manifests with different key order in steps and workflows. It needs sorting to become comparable.
   • Compare bytes rather than utf8-decoded strings. Decoding is lossy, so byte-different files can compare equal.

   With those in place, two consecutive builds leave every generated file identical in sha, mtime, and inode, and the steady-state no-op rebuild drops from 2 rounds to 1.

   Still unverified

   The 2.2s figure, the 2 client invalidations, the 4 HMR pushes, and the 4 CSS module rebuilds. No HMR client was attached in any of my runs, so I have no reading on the last three either way. The 603 KB is app-specific. The 1.37s cold compile did reproduce almost exactly.
3. VaguelySerious

   closed this as completedin #3454on Aug 19, 2026