Repository navigation
Warm the Start server graph on isolates that persist - #1679
Merged
Merged
Conversation
Timing the import apart from the request's own work in production: the loadEntries import is p50 3.1s, the request's work is p50 33ms, and a warm isolate answers in 9-104ms. The same bundle in local workerd serves the same path in ~3ms, so this is a cold-isolate cost on production metal, not slow code. Nearly every page request pays it: /mcp dispatches before fetchHandler, so MCP traffic creates and occupies isolates without ever loading the graph, and worker.dispatch ran 1,666 requests across 1,608 isolates. #1628 warmed on every isolate's first fetch and was reverted because it pulled the SSR graph into every isolate including single-shot MCP ones, taking memory kills from ~500-900 to 3.7k-9.4k per 5min. Wait for evidence the isolate persists instead: warm only after it has served a couple of fetches. START_GRAPH_WARM=false disables without a deploy.
Contributor
Cloudflare previewTorn down — the PR is closed. |
RhysSullivan
added a commit
that referenced
this pull request
Aug 18, 2026
RhysSullivan
added a commit
that referenced
this pull request
Aug 19, 2026
Isolates ARE reused - those serving a page request had already served a median of 19 spans across 3 paths - they are just never warm for Start, because /mcp returns before fetchHandler. Two import-based warmups (#1679, #1681) moved nothing: loadEntries awaits three specifiers plus a separately cached manifest, so warming a subset leaves the request paying for the rest. Drive the real handler instead; it populates whatever those caches are and cannot drift with Start's internals.
RhysSullivan
added a commit
that referenced
this pull request
Aug 19, 2026
* Revert "Fix lint: do not stringify the unknown warmup error (#1688)" This reverts commit 1c57887. * Revert "Log whether the warmup runs and whether warm isolates still pay for Start (#1687)" This reverts commit 1b2509c. * Revert "Warm the Start graph by replaying a real request per isolate (#1686)" This reverts commit 46625d2. * Reapply "Warm the Start server graph on isolate first fetch (#1681)" (#1682) This reverts commit aff1f39. * Revert "Warm the Start server graph on isolate first fetch (#1681)" This reverts commit 29a0aea. * Reapply "Warm the Start server graph on isolates that persist (#1679)" (#1680) This reverts commit 332955f. * Revert "Warm the Start server graph on isolates that persist (#1679)" This reverts commit ca9617e.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The measurement
Timing Start's
loadEntriesimport apart from the request's own work, on a cold isolate in production:loadEntriesimport of the server graphThe same production bundle in local workerd serves the same path in ~3ms. So the graph is not slow code — it is a cold-isolate cost on production metal (4.08MB gzipped, 203 chunks).
And nearly every page request is cold:
/mcpdispatches beforefetchHandler, so MCP traffic (~14.4k req/hr) creates and occupies isolates without ever loading the graph. Page requests land on those —worker.dispatchran 1,666 requests across 1,608 isolates, andwasWarmwas false on every sampled request.Ruled out along the way: OOM churn (0
exceededMemoryin 24h), CPU (67-357ms against 3.1s), the Shiki grammars (lazily imported;start-*.jsnever references the highlighter chunk), andsmoke-render(3.15MB but dynamic).Why this is not just a re-land of #1628
#1628 warmed on every isolate's first fetch and was reverted for good reason: it pulled the SSR graph into every isolate including MCP-only ones, and memory-limit kills went from ~500-900 to 3.7k-9.4k per 5min.
The memory blowup was made of single-shot isolates that never lived to use the graph. So warming now waits for evidence the isolate persists — it warms only after the isolate has served a couple of fetches. An isolate that has handled several requests is one that sticks around, which is exactly the kind a later page request can land on. Ephemeral isolates are never warmed.
Baseline memory kills are currently zero, so there is headroom, and
START_GRAPH_WARM=falseturns it off without a deploy (START_GRAPH_WARM_AFTERtunes the threshold).Verification
format,lint(0 errors),typecheck(44/44),apps/cloudvitest 280 passed.Post-merge I will watch
exceededMemoryand page-request latency, and flip the switch if kills move.