Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
753c981
Add opt-in QuickJS WASM VM engine (WORKFLOW_VM=quickjs) with full eve…
TooTallNate Jul 22, 2026
adb3d0e
QuickJS engine: AbortController, setAttributes, terminal drain, turbo…
TooTallNate Jul 22, 2026
6be76e2
QuickJS engine: hook.getConflict support, cross-run writable forwardi…
TooTallNate Jul 22, 2026
9d2530e
QuickJS engine: stream framing round-trip, bound step proxies, webhoo…
TooTallNate Jul 22, 2026
c57127d
Apply biome fixes to QuickJS engine files
TooTallNate Jul 22, 2026
ff400af
Address review feedback: anchor source-map strip to end-of-input, use…
TooTallNate Jul 22, 2026
5d4c578
CI: include generated QuickJS source assets in shared e2e build artif…
TooTallNate Jul 22, 2026
24017cc
Fix same-token hook ordering and conflicted-hook disposal in the Quic…
TooTallNate Jul 23, 2026
ad80ba8
CI: run both VM engines across all frameworks and worlds; label jobs …
TooTallNate Jul 23, 2026
2bd67b9
Fix stack overflow stripping inline source maps from webpack dev bund…
TooTallNate Jul 23, 2026
2a082dd
e2e: poll step listings until analytics rows include attempt (optiona…
TooTallNate Jul 23, 2026
0518e1c
e2e: use --withData to force storage-backed step listings for attempt…
TooTallNate Jul 23, 2026
6336b47
Sort imports in QuickJS serialization files (biome organizeImports)
TooTallNate Jul 31, 2026
2f3434d
QuickJS engine: resolve the run's full payload-key capability so seal…
TooTallNate Jul 31, 2026
1ff0ed9
Address review: crypto/process parity, loud Intl guards, lazy engine …
TooTallNate Jul 31, 2026
fe96bca
QuickJS engine: implement resilient resumeHook (hookInput materializa…
TooTallNate Jul 31, 2026
4506623
QuickJS engine: inline step execution via live-VM continuation loop +…
TooTallNate Jul 22, 2026
b232778
Address review: exclusive inline step claims, self-write requeue, in-…
TooTallNate Jul 31, 2026
6aea4c5
Add world.snapshots storage interface with local, postgres, and verce…
TooTallNate Jul 22, 2026
968cfa1
Address review: validate runId before filesystem paths in snapshots s…
TooTallNate Jul 31, 2026
ad89c02
Add threshold-based VM-memory snapshotting to the QuickJS engine (WOR…
TooTallNate Jul 22, 2026
c139c2a
Address review: snapshot correctness — total event ceiling, position-…
TooTallNate Jul 31, 2026
6af2c2c
Make QuickJS the default workflow VM engine (WORKFLOW_VM=node opts ba…
TooTallNate Jul 31, 2026
8f88b5e
Merge origin/main (post-#3048) into quickjs-vm-perf; address review t…
TooTallNate Aug 4, 2026
d2a115b
Merge branch 'quickjs-vm-perf' into quickjs-vm-snapshots
TooTallNate Aug 4, 2026
e294ed1
Merge quickjs-vm-snapshots (post-main sync) into quickjs-vm-threshold…
TooTallNate Aug 4, 2026
74bd053
Merge branch 'quickjs-vm-threshold-snapshots' into quickjs-vm-default
TooTallNate Aug 4, 2026
0584471
Merge remote-tracking branch 'origin/main' into quickjs-vm-snapshots
TooTallNate Aug 10, 2026
a5ee711
Merge branch 'quickjs-vm-snapshots' into quickjs-vm-threshold-snapshots
TooTallNate Aug 10, 2026
d41872a
Merge branch 'quickjs-vm-threshold-snapshots' into quickjs-vm-default
TooTallNate Aug 10, 2026
374dbaa
Address review: atomic snapshot envelope, optional interface, idempot…
TooTallNate Aug 10, 2026
f624888
Merge branch 'quickjs-vm-snapshots' into quickjs-vm-threshold-snapshots
TooTallNate Aug 10, 2026
eee5695
Address review: prod-world metadata round-trip, engine-build pin, clo…
TooTallNate Aug 10, 2026
c671016
Merge branch 'quickjs-vm-threshold-snapshots' into quickjs-vm-default
TooTallNate Aug 10, 2026
a8bff02
core: quickjs engine divergence-detection and write-fencing parity
TooTallNate Aug 11, 2026
8a78801
Merge quickjs-divergence-parity into quickjs-vm-default
TooTallNate Aug 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/quickjs-default-engine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': minor
---

The QuickJS WASM VM is now the default workflow engine. Set `WORKFLOW_VM=node` to opt back into the `node:vm` engine. Existing runs keep executing on the engine stamped in their `executionContext` at start.
5 changes: 5 additions & 0 deletions .changeset/quickjs-divergence-parity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@workflow/core': patch
---

QuickJS engine: divergence-detection and write-fencing parity with the node:vm engine. Replays now arbitrate the event log at each fixed point (orphaned events, stepName/token/resumeAt mismatches) and escalate `ReplayDivergenceError` through the existing recovery machinery instead of silently delivering wrong payloads or surfacing corruption as `USER_ERROR`; all replay-context event writes now carry the optimistic-concurrency precondition snapshot (closing the documented KNOWN GAP), with `run_failed` left unfenced to match the node engine.
6 changes: 6 additions & 0 deletions .changeset/quickjs-threshold-snapshots.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@workflow/core': minor
'workflow': minor
---

Add threshold-based VM-memory snapshotting to the QuickJS engine via `WORKFLOW_SNAPSHOT_THRESHOLD` (or per-run `executionContext.snapshotThreshold`). Once the configured number of events has been processed since the last snapshot, suspensions persist a compressed (and encrypted, when configured) VM snapshot through `world.snapshots`; resumptions restore the VM and replay only the delta events, with automatic fallback to full replay on any load/restore failure. `0` (default) disables snapshotting.
8 changes: 8 additions & 0 deletions .changeset/world-snapshots-interface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@workflow/world': minor
'@workflow/world-local': minor
'@workflow/world-postgres': minor
'@workflow/world-vercel': minor
---

Add an OPTIONAL `snapshots` storage interface (`save`/`load`/`delete` + `SnapshotMetadata`, plus `encodeSnapshotEnvelope`/`decodeSnapshotEnvelope` helpers that pack metadata and bytes into one atomically-storable blob) for VM-memory snapshotting, with implementations in world-local (single envelope file per run), world-postgres (`workflow_snapshots` table storing the envelope), and world-vercel (workflow-server `/v2/runs/:runId/snapshot` endpoints with W3C trace-context injection; delete is 404-idempotent). Inert until the runtime opts in; community World implementations that don't provide `snapshots` are unaffected (the runtime feature-detects and falls back to full replay).
47 changes: 29 additions & 18 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,9 @@ jobs:
strategy:
fail-fast: false
matrix:
# Workflow VM engines: node:vm (default) and the opt-in QuickJS
# WASM engine (WORKFLOW_VM=quickjs).
vm: [node, quickjs]
# Workflow VM engines: QuickJS WASM (the default — WORKFLOW_VM left
# unset) and the explicit node:vm opt-in (WORKFLOW_VM=node).
vm: [quickjs, node]
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
Expand All @@ -278,7 +278,8 @@ jobs:
run: pnpm test
working-directory: workbench/vitest
env:
WORKFLOW_VM: ${{ matrix.vm }}
# quickjs is the engine default — leave WORKFLOW_VM unset for it.
WORKFLOW_VM: ${{ matrix.vm == 'node' && 'node' || '' }}

e2e-package-build:
name: Build Shared E2E Packages
Expand Down Expand Up @@ -331,12 +332,14 @@ jobs:
strategy:
fail-fast: false
matrix:
# Workflow VM engines: node:vm (default) and the opt-in QuickJS
# WASM engine. The env var is set on the e2e test runner, which is
# the client that starts runs against the deployed app — start()
# stamps executionContext.workflowVm so the deployed handler
# executes each run on the requested engine.
vm: [node, quickjs]
# Workflow VM engines: QuickJS WASM (the default — WORKFLOW_VM left
# unset so the default-selection path is exercised end to end) and
# the explicit node:vm opt-in (WORKFLOW_VM=node). The env var is set
# on the e2e test runner, which is the client that starts runs
# against the deployed app — start() stamps
# executionContext.workflowVm so the deployed handler executes each
# run on the requested engine.
vm: [quickjs, node]
app:
- name: "example"
project-id: "prj_xWq20Dd860HHAfzMjK2Mb6TPVxMa"
Expand Down Expand Up @@ -438,13 +441,16 @@ jobs:
run: echo "ms=$(($(date +%s) * 1000))" >> "$GITHUB_OUTPUT"

- name: Run E2E Tests
run: pnpm run test:e2e --reporter=verbose --reporter=json --reporter=./packages/core/e2e/github-reporter.ts "--outputFile=e2e-vercel-prod-$APP_NAME-$WORKFLOW_VM.json"
run: pnpm run test:e2e --reporter=verbose --reporter=json --reporter=./packages/core/e2e/github-reporter.ts "--outputFile=e2e-vercel-prod-$APP_NAME-$MATRIX_VM.json"
env:
NODE_OPTIONS: "--enable-source-maps"
DEPLOYMENT_URL: ${{ steps.waitForDeployment.outputs.deployment-url || steps.prodDeployment.outputs.deployment-url }}
VERCEL_DEPLOYMENT_ID: ${{ steps.waitForDeployment.outputs.deployment-id || steps.prodDeployment.outputs.deployment-id }}
APP_NAME: ${{ matrix.app.name }}
WORKFLOW_VM: ${{ matrix.vm }}
# quickjs is the engine default — leave WORKFLOW_VM unset for it
# (MATRIX_VM carries the label for file/job naming).
WORKFLOW_VM: ${{ matrix.vm == 'node' && 'node' || '' }}
MATRIX_VM: ${{ matrix.vm }}
# changeset-release PRs test main's production deployment, so they
# must be treated as a production run everywhere downstream.
WORKFLOW_VERCEL_ENV: ${{ (github.ref == 'refs/heads/main' || startsWith(github.head_ref, 'changeset-release/')) && 'production' || 'preview' }}
Expand Down Expand Up @@ -483,8 +489,8 @@ jobs:
if: always()
env:
APP_NAME: ${{ matrix.app.name }}
WORKFLOW_VM: ${{ matrix.vm }}
run: node .github/scripts/aggregate-e2e-results.js . --job-name "E2E Vercel Prod ($APP_NAME - $WORKFLOW_VM)" >> $GITHUB_STEP_SUMMARY || true
MATRIX_VM: ${{ matrix.vm }}
run: node .github/scripts/aggregate-e2e-results.js . --job-name "E2E Vercel Prod ($APP_NAME - $MATRIX_VM)" >> $GITHUB_STEP_SUMMARY || true

- name: Upload E2E results
if: always()
Expand Down Expand Up @@ -882,6 +888,7 @@ jobs:
WORKFLOW_DEV_HMR_LOGS: "1"
NEXT_CANARY: ${{ matrix.app.canary && '1' || '' }}
WORKFLOW_VM: ${{ matrix.app.vm || '' }}
WORKFLOW_SNAPSHOT_THRESHOLD: ${{ matrix.app.snapshotThreshold || '' }}

- name: Generate E2E summary
if: always()
Expand Down Expand Up @@ -971,6 +978,7 @@ jobs:
DEPLOYMENT_URL: "http://localhost:${{ matrix.app.name == 'sveltekit' && '4173' || (matrix.app.name == 'astro' && '4321' || '3000') }}"
NEXT_CANARY: ${{ matrix.app.canary && '1' || '' }}
WORKFLOW_VM: ${{ matrix.app.vm || '' }}
WORKFLOW_SNAPSHOT_THRESHOLD: ${{ matrix.app.snapshotThreshold || '' }}

- name: Generate E2E summary
if: always()
Expand Down Expand Up @@ -1080,6 +1088,7 @@ jobs:
DEPLOYMENT_URL: "http://localhost:${{ matrix.app.name == 'sveltekit' && '4173' || (matrix.app.name == 'astro' && '4321' || '3000') }}"
NEXT_CANARY: ${{ matrix.app.canary && '1' || '' }}
WORKFLOW_VM: ${{ matrix.app.vm || '' }}
WORKFLOW_SNAPSHOT_THRESHOLD: ${{ matrix.app.snapshotThreshold || '' }}

- name: Generate E2E summary
if: always()
Expand All @@ -1103,9 +1112,9 @@ jobs:
strategy:
fail-fast: false
matrix:
# Workflow VM engines: node:vm (default) and the opt-in QuickJS
# WASM engine (WORKFLOW_VM=quickjs).
vm: [node, quickjs]
# Workflow VM engines: QuickJS WASM (the default — WORKFLOW_VM left
# unset) and the explicit node:vm opt-in (WORKFLOW_VM=node).
vm: [quickjs, node]
env:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
Expand Down Expand Up @@ -1216,7 +1225,9 @@ jobs:
DEV_TEST_CONFIG: '{"generatedStepRegistrationPath":"app/.well-known/workflow/v1/flow/__step_registrations.js","generatedWorkflowPath":"app/.well-known/workflow/v1/flow/route.js","apiFilePath":"app/api/chat/route.ts","apiFileImportPath":"../../..","port":3000,"testWorkflowFile":"96_many_steps.ts"}'
DEV_SERVER_LOG_PATH: "${{ github.workspace }}/nextjs-server.log"
WORKFLOW_DEV_HMR_LOGS: "1"
WORKFLOW_VM: ${{ matrix.vm }}
# quickjs is the engine default — leave WORKFLOW_VM unset for it
# (MATRIX_VM carries the label for reporting).
WORKFLOW_VM: ${{ matrix.vm == 'node' && 'node' || '' }}
MATRIX_VM: ${{ matrix.vm }}

- name: Print Next.js server logs
Expand Down
20 changes: 16 additions & 4 deletions docs/content/docs/v5/configuration/runtime-tuning.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -165,19 +165,31 @@ For example, a workflow can run a 10-minute inline step even with `WORKFLOW_REPL

### `WORKFLOW_VM`

- Default: `node`
- Default: `quickjs`
- Values: `node` or `quickjs`
- Selects the sandboxed VM engine that executes workflow functions (`"use workflow"`). Step functions are unaffected — they always run with full Node.js access.
- `node` (default) runs workflow code in a [`node:vm`](https://nodejs.org/api/vm.html) context.
- `quickjs` (experimental) runs workflow code in a [QuickJS](https://github.com/quickjs-ng/quickjs) VM compiled to WebAssembly (via [`quickjs-wasi`](https://github.com/vercel-labs/quickjs-wasi)). Both engines implement the same event-replay execution model (seeded PRNG, deterministic clock, and correlation-ID sequences are identical), but the **global surface is not identical** — see the differences below before switching an existing deployment. The QuickJS engine is intended for platforms that do not implement `node:vm`, and is the foundation for future VM-memory snapshotting.
- Global-surface differences under `quickjs` (workflow functions only — step functions always have full Node.js):
- `quickjs` (default) runs workflow code in a [QuickJS](https://github.com/quickjs-ng/quickjs) VM compiled to WebAssembly (via [`quickjs-wasi`](https://github.com/vercel-labs/quickjs-wasi)). It works on platforms that do not implement `node:vm` (e.g. edge runtimes), and is the foundation for VM-memory snapshotting.
Comment thread
TooTallNate marked this conversation as resolved.
Resolved
- `node` runs workflow code in a [`node:vm`](https://nodejs.org/api/vm.html) context instead. Both engines implement the same event-replay execution model (seeded PRNG, deterministic clock, and correlation-ID sequences are identical), but the **global surface is not identical** — review the differences below before flipping an existing deployment either direction.
- Global-surface differences under `quickjs` relative to `node` (workflow functions only — step functions always have full Node.js):
- `crypto.getRandomValues()` and `crypto.randomUUID()` are provided and deterministic (seeded like the node engine's). All `crypto.subtle.*` methods throw with guidance to move to a step function — including `digest`, which the node engine supports.
- `Intl` is not available (QuickJS has no ICU). The `Intl.*` constructors throw, and `toLocaleString`-family methods (including `localeCompare`) throw when called **with an explicit locale** — calling them without arguments keeps the engine default. Perform locale-sensitive formatting in a step function.
- `WebAssembly` and `Atomics` are not available.
- `process` exposes only a frozen copy of `env`, matching the node engine.
- The engine choice is stamped into the run's `executionContext` when the run starts, so a run keeps executing on the engine it started on even if the deployment's `WORKFLOW_VM` changes. Runs without a stamped engine use the handler's `WORKFLOW_VM` value.
- Unknown values throw at startup.

### `WORKFLOW_SNAPSHOT_THRESHOLD`

- Default: `0` (disabled)
- Values: non-negative integer
- Only used by the QuickJS engine (the default; see `WORKFLOW_VM`).
- When set above `0`, the runtime persists a **VM-memory snapshot** at a suspension once at least this many events have been processed since the last snapshot. Subsequent invocations restore the VM from the snapshot and replay only the events recorded since — instead of re-executing the workflow from the top against the full event log.
- Short-lived runs below the threshold never pay the snapshot cost; long-running or unbounded runs stop scaling their resume cost with total event-log length. `1` snapshots at every qualifying suspension.
- Snapshots are an optimization, not a source of truth: the event log remains authoritative, and a missing, corrupt, or incompatible snapshot automatically falls back to a full replay.
- Snapshot bytes are compressed (zstd, gzip fallback) and encrypted with the run's encryption key (when configured) before being handed to the World's `snapshots` storage. They are deleted when the run reaches a terminal state.
- Like `WORKFLOW_VM`, the policy is stamped into the run's `executionContext` at start, so a run keeps the snapshot policy it started with.
- Invalid values throw at startup.

## Compression and tracing

### `WORKFLOW_DISABLE_COMPRESSION`
Expand Down
18 changes: 18 additions & 0 deletions packages/core/scripts/build-quickjs-assets.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,16 @@ const files = {
structuredCloneSo: require_.resolve('quickjs-wasi/structured-clone.so'),
};

// The installed quickjs-wasi version identifies the exact WASM build the
// base64 blobs below came from. Persisted into VM snapshot metadata and
// compared on restore: the QJSS heap-image header is identical across
// builds, so without this pin a snapshot taken by one build would restore
// under another and execute as undefined behavior in the interpreter
// (real hazard during a rolling deploy that bumps quickjs-wasi).
const quickjsWasiVersion = JSON.parse(
readFileSync(require_.resolve('quickjs-wasi/package.json'), 'utf8')
).version;

let output = `/**
* Auto-generated by scripts/build-quickjs-assets.js
* Do not edit manually.
Expand Down Expand Up @@ -74,6 +84,14 @@ for (const [name, filePath] of Object.entries(files)) {

output += `export { quickjsWasm };\n\n`;

output += `/**
* The quickjs-wasi package version these embedded binaries came from.
* Persisted into VM snapshot metadata (engineVersion) and compared on
* restore — a heap image only restores under the exact WASM build that
* produced it (the QJSS header does not distinguish builds).
*/
export const quickjsWasiVersion = ${JSON.stringify(quickjsWasiVersion)};\n\n`;

output += `export const quickjsExtensions: ExtensionDescriptor[] = [
{ name: 'encoding', wasm: encodingSo },
{ name: 'headers', wasm: headersSo },
Expand Down
6 changes: 6 additions & 0 deletions packages/core/src/runtime-trace-mode.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ import { workflowEntrypoint } from './runtime.js';
import { dehydrateWorkflowArguments } from './serialization.js';
import { getNextTraceCarrier, getWorkflowTraceMode } from './telemetry.js';

// These tests exercise the node:vm engine's replay loop internals against
// mock worlds. Pin the engine explicitly — QuickJS is the default, and
// dispatching there would instantiate a WASM VM per entrypoint call (and
// bypass the node replay loop these tests assert on).
process.env.WORKFLOW_VM = 'node';

vi.mock('@vercel/functions', () => ({
waitUntil: vi.fn((p: Promise<unknown>) => {
p.catch(() => {});
Expand Down
6 changes: 6 additions & 0 deletions packages/core/src/runtime.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ import {
hydrateRunError,
} from './serialization.js';

// These tests exercise the node:vm engine's replay loop internals against
// mock worlds. Pin the engine explicitly — QuickJS is the default, and
// dispatching there would instantiate a WASM VM per entrypoint call (and
// bypass the node replay loop these tests assert on).
process.env.WORKFLOW_VM = 'node';
Comment thread
TooTallNate marked this conversation as resolved.
Resolved

// Capture every promise handed to `waitUntil` so tests can assert that
// progress-critical sends are never registered on a detached, unconsumed
// promise (which would reject → unhandled rejection → process exit 128, and
Expand Down
6 changes: 6 additions & 0 deletions packages/core/src/runtime/precondition-guard-replay.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,12 @@ import {
} from './constants.js';
import { setWorld } from './world.js';

// These tests exercise the node:vm engine's replay loop internals against
// mock worlds. Pin the engine explicitly — QuickJS is the default, and
// dispatching there would instantiate a WASM VM per entrypoint call (and
// bypass the node replay loop these tests assert on).
process.env.WORKFLOW_VM = 'node';

vi.mock('@vercel/functions', () => ({
waitUntil: vi.fn(),
}));
Expand Down
Loading
Loading