Repository navigation
[core] Re-queue the workflow when a step's terminal write conflicts - #4824
Closed
VaguelySerious wants to merge 1 commit into
Closed
VaguelySerious wants to merge 1 commit into
VaguelySerious wants to merge 1 commit into
Conversation
The world retries step_completed/step_failed POSTs in-process when a response is lost. If the first write landed, the retry gets a 409, and the step handler treated that as "another invocation finished the step" and returned without queuing the workflow continuation. Nothing else wakes the workflow in that case, so the run stays `running` forever with its last event a step_completed. Fall through to the continuation enqueue on a 409 from step_completed and every step_failed write, matching the step_started conflict path and the max-deliveries path. A duplicate wake only costs one replay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
🦋 Changeset detectedLatest commit: 21c2b7d The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (1 failed)sveltekit (1 failed):
🌍 Community Worlds (105 failed)redis (20 failed):
turso (85 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
karthikscale3
approved these changes
Oct 9, 2026
karthikscale3
left a comment
karthikscale3
left a comment
Contributor
There was a problem hiding this comment.
Reviewed commit 21c2b7d. The change correctly publishes the workflow continuation after terminal step-write conflicts while preserving redelivery on other write or publication failures.
Validated 43 focused tests, including four additional regression checks covering lost-response retry, publication failure/redelivery, FatalError, and missing-startedAt. All four additional checks fail against the parent and pass against this commit.
Approving the code change. The remaining failed/pending CI checks should be resolved before merging. Missing source events and recovery of already-orphaned runs remain separate work.
Merged
2 of 3 tasks
Member
Author
|
Superseeded by #5028 |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On stable, a run can get stuck in
runningwith its last event astep_completed, and nothing ever wakes it.How it happens
step_completed. The write lands, but the response never reaches the client before the 30s transport timeout.step_completed, and the backend answers 409 because the step is already completed.EntityConflictErrorthe step handler logged "step has already finished" and returned without queuing the workflow continuation. It assumed another invocation had finished the step and would wake the workflow. Here the conflicting write was its own, so no one queued the continuation. The step message is acked, so nothing redelivers it.This was seen in production on
workflow@4.8.x: the step invocation ran for 30.5s, the retriedstep_completedgot a 409, and no further invocation or event followed for the run.Fix
On a 409 from
step_completedor anystep_failedwrite, fall through and queue the workflow continuation. This matches what the handler already does for a 409 onstep_started, and what #4422 did for the max-deliveriesstep_failedpath. A duplicate wake only costs one replay.step_retryingis unchanged. It is not retried in-process, so a 409 there cannot come from its own write.Tests
Updated the existing 409 unit tests in
step-handler.test.tsto assert that the continuation is queued, coveringstep_completed, max-retriesstep_failed, the pre-execution guardstep_failed, and the missing-functionstep_failed. All four fail on the previous handler.🤖 Generated with Claude Code