Repository navigation
[core] Fix setAttributes() resolution bypassing delivery barriers - #4861
Conversation
…ing (#4770) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pranay Prakash <1797812+pranaygp@users.noreply.github.com> Co-Authored-By: Pranay Prakash <1797812+pranaygp@users.noreply.github.com>
🦋 Changeset detectedLatest commit: c005102 The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 113014ms → this run 103375ms (Δ -9639ms, -9%) ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor. |
🧪 E2E Test Results✅ All tests passed
|
| Passed | Failed | Skipped | Total | |
|---|---|---|---|---|
| ✅ ▲ Vercel Production | 3982 | 0 | 878 | 4860 |
| ✅ 💻 Local Development | 4669 | 0 | 551 | 5220 |
| ✅ 📦 Local Production | 4669 | 0 | 551 | 5220 |
| ✅ 🐘 Local Postgres | 4669 | 0 | 551 | 5220 |
| ✅ 🪟 Windows | 348 | 0 | 12 | 360 |
| ✅ 🌐 Cross-language Conformance | 68 | 0 | 87 | 155 |
| ✅ dynamic-runs | 0 | 0 | 0 | 0 |
| ✅ vercel-http-transport | 897 | 0 | 183 | 1080 |
| ✅ vercel-multi-region | 27 | 0 | 0 | 27 |
| ✅ vercel-ws-transport | 607 | 0 | 113 | 720 |
| Total | 19936 | 0 | 2926 | 22862 |
Details by Category
✅ ▲ Vercel Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-node | 145 | 0 | 35 |
| ✅ astro-quickjs | 145 | 0 | 35 |
| ✅ example-node | 145 | 0 | 35 |
| ✅ example-quickjs | 145 | 0 | 35 |
| ✅ express-node | 145 | 0 | 35 |
| ✅ express-quickjs | 145 | 0 | 35 |
| ✅ fastify-node | 145 | 0 | 35 |
| ✅ fastify-quickjs | 145 | 0 | 35 |
| ✅ hono-node | 145 | 0 | 35 |
| ✅ hono-quickjs | 145 | 0 | 35 |
| ✅ nest-node | 145 | 0 | 35 |
| ✅ nest-quickjs | 145 | 0 | 35 |
| ✅ nextjs-turbopack-node | 172 | 0 | 8 |
| ✅ nextjs-turbopack-quickjs | 172 | 0 | 8 |
| ✅ nextjs-webpack-node | 172 | 0 | 8 |
| ✅ nextjs-webpack-quickjs | 172 | 0 | 8 |
| ✅ nitro-node | 145 | 0 | 35 |
| ✅ nitro-quickjs | 145 | 0 | 35 |
| ✅ nuxt-node | 145 | 0 | 35 |
| ✅ nuxt-quickjs | 145 | 0 | 35 |
| ✅ python-node | 66 | 0 | 114 |
| ✅ sveltekit-node | 164 | 0 | 16 |
| ✅ sveltekit-quickjs | 164 | 0 | 16 |
| ✅ tanstack-start-node | 145 | 0 | 35 |
| ✅ tanstack-start-quickjs | 145 | 0 | 35 |
| ✅ vite-node | 145 | 0 | 35 |
| ✅ vite-quickjs | 145 | 0 | 35 |
✅ 💻 Local Development
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 151 | 0 | 29 |
| ✅ astro-stable-quickjs | 151 | 0 | 29 |
| ✅ express-stable-node | 151 | 0 | 29 |
| ✅ express-stable-quickjs | 151 | 0 | 29 |
| ✅ fastify-stable-node | 151 | 0 | 29 |
| ✅ fastify-stable-quickjs | 151 | 0 | 29 |
| ✅ hono-stable-node | 151 | 0 | 29 |
| ✅ hono-stable-quickjs | 151 | 0 | 29 |
| ✅ nest-stable-node | 151 | 0 | 29 |
| ✅ nest-stable-quickjs | 151 | 0 | 29 |
| ✅ nextjs-turbopack-canary-node | 179 | 0 | 1 |
| ✅ nextjs-turbopack-canary-quickjs | 179 | 0 | 1 |
| ✅ nextjs-turbopack-quickjs-snapshot | 179 | 0 | 1 |
| ✅ nextjs-turbopack-stable-node | 179 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 179 | 0 | 1 |
| ✅ nextjs-webpack-canary-node | 179 | 0 | 1 |
| ✅ nextjs-webpack-canary-quickjs | 179 | 0 | 1 |
| ✅ nextjs-webpack-stable-node | 179 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 179 | 0 | 1 |
| ✅ nitro-stable-node | 151 | 0 | 29 |
| ✅ nitro-stable-quickjs | 151 | 0 | 29 |
| ✅ nuxt-stable-node | 151 | 0 | 29 |
| ✅ nuxt-stable-quickjs | 151 | 0 | 29 |
| ✅ sveltekit-stable-node | 170 | 0 | 10 |
| ✅ sveltekit-stable-quickjs | 170 | 0 | 10 |
| ✅ tanstack-start-node | 151 | 0 | 29 |
| ✅ tanstack-start-quickjs | 151 | 0 | 29 |
| ✅ vite-stable-node | 151 | 0 | 29 |
| ✅ vite-stable-quickjs | 151 | 0 | 29 |
✅ 📦 Local Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 151 | 0 | 29 |
| ✅ astro-stable-quickjs | 151 | 0 | 29 |
| ✅ express-stable-node | 151 | 0 | 29 |
| ✅ express-stable-quickjs | 151 | 0 | 29 |
| ✅ fastify-stable-node | 151 | 0 | 29 |
| ✅ fastify-stable-quickjs | 151 | 0 | 29 |
| ✅ hono-stable-node | 151 | 0 | 29 |
| ✅ hono-stable-quickjs | 151 | 0 | 29 |
| ✅ nest-stable-node | 151 | 0 | 29 |
| ✅ nest-stable-quickjs | 151 | 0 | 29 |
| ✅ nextjs-turbopack-canary-node | 179 | 0 | 1 |
| ✅ nextjs-turbopack-canary-quickjs | 179 | 0 | 1 |
| ✅ nextjs-turbopack-quickjs-snapshot | 179 | 0 | 1 |
| ✅ nextjs-turbopack-stable-node | 179 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 179 | 0 | 1 |
| ✅ nextjs-webpack-canary-node | 179 | 0 | 1 |
| ✅ nextjs-webpack-canary-quickjs | 179 | 0 | 1 |
| ✅ nextjs-webpack-stable-node | 179 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 179 | 0 | 1 |
| ✅ nitro-stable-node | 151 | 0 | 29 |
| ✅ nitro-stable-quickjs | 151 | 0 | 29 |
| ✅ nuxt-stable-node | 151 | 0 | 29 |
| ✅ nuxt-stable-quickjs | 151 | 0 | 29 |
| ✅ sveltekit-stable-node | 170 | 0 | 10 |
| ✅ sveltekit-stable-quickjs | 170 | 0 | 10 |
| ✅ tanstack-start-node | 151 | 0 | 29 |
| ✅ tanstack-start-quickjs | 151 | 0 | 29 |
| ✅ vite-stable-node | 151 | 0 | 29 |
| ✅ vite-stable-quickjs | 151 | 0 | 29 |
✅ 🐘 Local Postgres
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 151 | 0 | 29 |
| ✅ astro-stable-quickjs | 151 | 0 | 29 |
| ✅ express-stable-node | 151 | 0 | 29 |
| ✅ express-stable-quickjs | 151 | 0 | 29 |
| ✅ fastify-stable-node | 151 | 0 | 29 |
| ✅ fastify-stable-quickjs | 151 | 0 | 29 |
| ✅ hono-stable-node | 151 | 0 | 29 |
| ✅ hono-stable-quickjs | 151 | 0 | 29 |
| ✅ nest-stable-node | 151 | 0 | 29 |
| ✅ nest-stable-quickjs | 151 | 0 | 29 |
| ✅ nextjs-turbopack-canary-node | 179 | 0 | 1 |
| ✅ nextjs-turbopack-canary-quickjs | 179 | 0 | 1 |
| ✅ nextjs-turbopack-quickjs-snapshot | 179 | 0 | 1 |
| ✅ nextjs-turbopack-stable-node | 179 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 179 | 0 | 1 |
| ✅ nextjs-webpack-canary-node | 179 | 0 | 1 |
| ✅ nextjs-webpack-canary-quickjs | 179 | 0 | 1 |
| ✅ nextjs-webpack-stable-node | 179 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 179 | 0 | 1 |
| ✅ nitro-stable-node | 151 | 0 | 29 |
| ✅ nitro-stable-quickjs | 151 | 0 | 29 |
| ✅ nuxt-stable-node | 151 | 0 | 29 |
| ✅ nuxt-stable-quickjs | 151 | 0 | 29 |
| ✅ sveltekit-stable-node | 170 | 0 | 10 |
| ✅ sveltekit-stable-quickjs | 170 | 0 | 10 |
| ✅ tanstack-start-node | 151 | 0 | 29 |
| ✅ tanstack-start-quickjs | 151 | 0 | 29 |
| ✅ vite-stable-node | 151 | 0 | 29 |
| ✅ vite-stable-quickjs | 151 | 0 | 29 |
✅ 🪟 Windows
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-node | 174 | 0 | 6 |
| ✅ nextjs-turbopack-quickjs | 174 | 0 | 6 |
✅ 🌐 Cross-language Conformance
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ python | 68 | 0 | 87 |
✅ dynamic-runs
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-vercel | 0 | 0 | 0 |
| ✅ example-vercel | 0 | 0 | 0 |
| ✅ express-vercel | 0 | 0 | 0 |
| ✅ fastify-vercel | 0 | 0 | 0 |
| ✅ hono-vercel | 0 | 0 | 0 |
| ✅ nest-vercel | 0 | 0 | 0 |
| ✅ nextjs-turbopack-vercel | 0 | 0 | 0 |
| ✅ nextjs-webpack-vercel | 0 | 0 | 0 |
| ✅ nitro-vercel | 0 | 0 | 0 |
| ✅ nuxt-vercel | 0 | 0 | 0 |
| ✅ sveltekit-vercel | 0 | 0 | 0 |
| ✅ tanstack-start-vercel | 0 | 0 | 0 |
| ✅ vite-vercel | 0 | 0 | 0 |
✅ vercel-http-transport
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ example | 145 | 0 | 35 |
| ✅ express | 145 | 0 | 35 |
| ✅ hono | 145 | 0 | 35 |
| ✅ nextjs-turbopack | 172 | 0 | 8 |
| ✅ nitro | 145 | 0 | 35 |
| ✅ vite | 145 | 0 | 35 |
✅ vercel-multi-region
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack | 27 | 0 | 0 |
✅ vercel-ws-transport
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ example | 145 | 0 | 35 |
| ✅ express | 145 | 0 | 35 |
| ✅ nextjs-turbopack | 172 | 0 | 8 |
| ✅ vite | 145 | 0 | 35 |
Sim WorldSimulated world deterministic testing for races. Traces 🟢 world-sim scenario book — 0 fail of 42 total
Full trace: |
About these numbersSizes are gzip; parentheses show the change against
|
…4770) Register the attr_set acknowledgement as an 'attribute' delivery barrier at its log index and resolve setAttributes() only after every earlier delivery it defers behind has been handed to the workflow. Every other kind defers behind an earlier attribute too. The clock is left untouched, as before. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Pranay Prakash <1797812+pranaygp@users.noreply.github.com> Co-Authored-By: Pranay Prakash <1797812+pranaygp@users.noreply.github.com>
VaguelySerious
left a comment
There was a problem hiding this comment.
AI review: no blocking issues
| // The ordering is captured here, at consumption time, for the same | ||
| // reason as in step.ts and sleep.ts. | ||
| const eventIndex = ctx.eventsConsumer.eventIndex; | ||
| const barrier = registerDeliveryBarrier(ctx, eventIndex, 'attribute', { |
There was a problem hiding this comment.
AI Review: Note
This overlaps with #4848, which also closes #4770 and adds a per-run marker so runs started before the upgrade keep the old ordering. This PR follows the #4215 precedent (hook registration outcomes were moved onto the barriers without gating), which I think is reasonable here: I could not build a history where the old ordering was deterministic and the new one picks a different winner. Re-recording three extra shapes on the base commit and on this branch (unawaited setAttributes() alongside a pending step, setAttributes() in a Promise.all with a sleep and an earlier step, and setAttributes() while a hook payload sits buffered and unclaimed) gave identical outputs and resolution order in both draw modes, with cold and warm cache replays stable. One of the two PRs should be closed before merge.
| // Attribute writes have never moved the workflow clock. | ||
| deliveredAt: null, | ||
| }); | ||
| const earlierDelivered = awaitEarlierDeliveries( |
There was a problem hiding this comment.
AI Review: Note
With log-order draws on (the default), every setAttributes() resolution now pays the quiesce turn in awaitEarlierDeliveries, even when nothing earlier is in flight. I measured a 30-iteration sequential setAttributes() loop: replay went from ~37ms to ~40ms, so about 0.1ms per write. Fine for progress-style attribute updates; noting it in case someone writes attributes in a tight loop.
|
No backport to This is a genuine correctness fix (REPLAY_DIVERGENCE / branches receiving each other's step results), but it targets functionality that does not exist on To override, re-run the Backport to stable workflow manually via |
Catches the docs up with SDK changes that shipped in the last day without (complete) docs updates. | Change | Source PR | Docs edit | | --- | --- | --- | | QuickJS runs with snapshotting are exempt from the default per-run event limit on the Local and Vercel Worlds, and `MAX_EVENTS_EXCEEDED` / `REPLAY_TIMEOUT` now have their own error pages | #4870 | The error-code table in Errors and retries, the log-growth note in Event sourcing, and the event-limit row in What's new now mention the QuickJS exemption and link to the new error pages. #4870 updated the errors index, Runtime tuning, Worlds, and Vercel World pages but not these | I reviewed the other PRs from the last day and they need no further docs: removing the experimental resilient step dispatch (#4818) updated its own docs and the variable no longer appears outside changelogs, the `setAttributes()` delivery-order fix (#4861) and terminal-inert replay (#4819) are bug fixes, reading step creation from `step_started` (#4866) is reader-side forward compatibility with no user-facing change yet, and the devalue upgrade (#4856), dependency bumps, and Version Packages PRs aren't documented behavior. The Workflows pricing and limits page in `vercel/front` gets the matching event-limit note in vercel/front#91076. ## Docs Preview | Page | Preview | | --- | --- | | Errors and retries | [Error codes](https://workflow-docs-git-workflow-docs-cleanup-wed-14-oct.vercel.sh/docs/foundations/errors-and-retries) | | Event sourcing | [How fast a log grows](https://workflow-docs-git-workflow-docs-cleanup-wed-14-oct.vercel.sh/docs/how-it-works/event-sourcing#how-fast-a-log-grows) | | What's new | [Breaking changes: Application code](https://workflow-docs-git-workflow-docs-cleanup-wed-14-oct.vercel.sh/docs/whats-new#application-code) | 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com> Co-authored-by: Pranay Prakash <1797812+pranaygp@users.noreply.github.com>
Allow operators to select older runs whose saved decisions depend on the attribute delivery behavior before vercel#4861. Keep that choice for each replay session and retain ordered delivery for other runs. Cover old and newer histories with both draw-order settings, checking the inputs sent to a new step after replay. Signed-off-by: Muhammadjon Marufov <marufov254@gmail.com>
Closes #4770.
Problem
When
setAttributes()resolves, itsattr_setacknowledgement settled straight offpromiseQueuewithout registering a delivery barrier. Every other delivery that resumes a branch (step results, waits, hook payloads and, since #4215, hook registration outcomes) registers one and waits for earlier deliveries. A branch awaitingsetAttributes()could therefore resume ahead of a sibling's step result that comes earlier in the log, and draw that sibling's next correlation ID. Depending on what the sibling draws next, the run either fails withREPLAY_DIVERGENCE, or both branches call the same step and silently receive each other's results.Fix
attributeDeliveryKind. It defers behind every earlier kind, and every kind defers behind an earlierattribute. Like a step, it skips unclaimed buffered hook payloads, because resolving it never depends on a hook read.attribute-dispatcher.tsregisters the barrier and capturesawaitEarlierDeliverieswhen it consumes the event, then resolves from a continuation that is detached from the queue tail, following the same pattern assleep.ts.setAttributes()resolution still does not move the workflow clock.registerDeliveryBarriernow acceptsdeliveredAt: nullto opt out explicitly.Tests
set-attributes-delivery-order.test.tsdrives the two shapes from the issue against an in-memory event log and replays the committed log in fresh VMs with cold and warm caches. It covers bothWORKFLOW_LOG_ORDER_DRAWSmodes and several hop counts. Onmainall 16 cases fail (CI confirmed this on the first commit). With the fix they all pass.workflow@5.2.1/@workflow/vitest@5.0.4, 32 runs per workflow). Stock 5.2.1 failsattrSwapin 25–32 of 32 runs andattrVsHookin 4–24 of 32 runs. With this build of@workflow/coreswapped in, all four tests pass in both draw modes.Note on in-flight runs
Unlike #4848, this does not pin the old ordering for runs started before the upgrade. The old ordering was not a deterministic function of the log for the reported shapes. Even so, a history that recorded
setAttributes()winning aPromise.raceagainst an earlier-in-log delivery could replay differently once these runs continue on the new SDK. #4848 handles that case with a per-run marker. Maintainers should decide whether that gating is worth the extra surface.🤖 Generated with Claude Code