Repository navigation
Inline step execution optimization on Vercel doesn't make use of full function timeout #1927
Description
After #1338, we inline-execute steps for higher performance, but the flow still suspends every ~2 minutes, even if I set my function timeouts to 13 min (800s).
This should be configurable or set to a better heuristic
Activity
shtefcs commentedon May 11, 2026
Hey, just reporting from our setup since we're running into this same 120s boundary daily.
We're on the latest beta tarball (the one you said would become 5.0.0-beta.5) for a multi-agent chat product. Coding and browser agent sessions routinely run 5-15 minutes, and we're seeing the worker cycle every 2 minutes on the dot. Cross-invocation streaming itself works correctly via WorkflowChatTransport reconnecting on each cycle, so functionally we're OK, but every cycle is wasted overhead and an extra failure point for the client.
We saw
WORKFLOW_V2_TIMEOUT_MSin the runtime source. Two questions before we set it:Is bumping it to something like 700000 (just under our Vercel function
maxDuration=800) safe to do today, or are there gotchas you've seen? Our function has 800s and is in the iad1 region same as the workflow backend.Is there an ETA for the auto-derive heuristic? If it's days away we'd rather wait than carry an env var. If it's weeks, we'll set the env var now.
For context on what "works fine across cycles" looks like: one of our coding sessions yesterday spanned 4 worker invocations over 8 minutes, 53 messages persisted to DB, UI never froze. So the architecture is solid for our workload, it's just the cycle frequency that's higher than necessary for chats with a long maxDuration.
Happy to share a runId if useful.
VaguelySerious commentedon May 23, 2026
@shtefcs You can override
WORKFLOW_V2_TIMEOUT_MSif you want. The general concerns are as follows:If your invocation runs for any time
timeElapsed <= WORKFLOW_V2_TIMEOUT_MSand then starts to execute a step withexpectedStepDuration >= maxFunctionDuration - timeElapsed, then your step will always time out and needs to be re-run in a fresh invocation.This is to say, if your max function duration is 15 minutes, and you have a step which runs 10 minutes on average somewhere in the flow, then setting
WORKFLOW_V2_TIMEOUT_MSto >= 5 minutes will likely lead to performance degradation, since the long-running step will get interrupted on average, but setting it to e.g. ~3 minutes will have a slight performance benefit. 2 minutes was chosen as a heuristic for the average use case. Feel free to play with the number.shtefcs commentedon Jul 4, 2026
Sharing some real-world data since we just hit this hard.
We run durable coding agents where a single run does 40 to 50 steps. Our function maxDuration is 800s. With WORKFLOW_V2_TIMEOUT_MS unset (so the 120s default), one build cycled through 17 separate /flow invocations on a roughly 2 minute heartbeat. We measured about 53 minutes of billed function time for around 48 steps of actual work, and the per-step latency itself was healthy (about 13s). So roughly 93 percent of the wall clock went to cross-invocation dispatch and idle, not real work.
Setting WORKFLOW_V2_TIMEOUT_MS closer to our function budget fixes most of it. We landed on 430000 using the rule of thumb from this thread (maxDuration minus the longest single step, which for us is a ~363s doStreamStep). That keeps a safe buffer while cutting the re-invocation count a lot.
Two things that would help:
Derive the default from the function maxDuration instead of a fixed 120s. For anyone running functions longer than 2 minutes, which is common for AI agent workloads on Fluid compute, the 120s default is silently close to the worst case. Even deriving it as maxDuration minus a conservative step estimate would remove this footgun for most people.
The current tradeoff is awkward. Raise it and long steps risk always timing out, lower it and you pay constant re-invocation overhead. An option to let a single in-progress step extend the invocation past the threshold, instead of forcing a re-run, would let us set the value aggressively without the timeout risk.
Is the auto-derive heuristic still on the roadmap, or is the current recommendation to just set this manually per deployment?
shtefcs commentedon Sep 3, 2026
Following up on my July comment with production evidence, now that #3360 has shipped — because the tiering fixed the default but the failure mode described in @VaguelySerious' May reply is still reachable, and we hit it hard.
Setup:
workflow@5.0.0-beta.46, Vercel World, flowmaxDuration: 800. Our sandbox exec has a client-side bound of 380s. We still carryWORKFLOW_V2_TIMEOUT_MS=430000from before #3360.The arithmetic that bites, restating your rule with our numbers:
A step admitted near the 430s boundary cannot finish inside the function, so it is killed every time.
Observed (one run,
wrun_41M1GZDG…, ~2.3h):Vercel Runtime Timeout Error: Task timed out after 800 secondson every/.well-known/workflow/v1/flowinvocation[workflow-sdk] Re-executing inline steps owned by this queue message — a previous delivery crashed mid-body and this redelivery is recovering themafter eachThe ownership machinery behaved exactly as documented — this is the crash-recovery path working. The problem is upstream of it: the step should not have been admitted.
Two observations that may be useful:
1. The tiering is safer than the override, so the override is now a footgun in the other direction. For our 800s function #3360 derives 300s, and 300 + 380 = 680s fits. Our 430000 — chosen in July using your rule with the wrong
longestSingleStep— overrides a safe value with an unsafe one. We are removing it. It might be worth a line inruntime-tuning.mdxnoting that an override set before #3360 should be re-evaluated, since it silently wins over the derived value.2. The re-request from my July comment now has evidence behind it. Even at the derived 300s, our longest legitimate step (a 600s production build) still exceeds the remaining budget: 300 + 600 = 900 > 800. No single number for
WORKFLOW_V2_TIMEOUT_MScan be correct whenexpectedStepDurationvaries by call site — ours is 380s / 480s / 600s depending on the step.So the useful primitive is admission-time, not configuration-time. Something like:
now + expectedStepDuration > runtimeDeadline, queueing it for a fresh invocation instead; orgetRuntimeDeadline()already landed in the World interface with #3360, so the runtime now has the information needed for the first option. What it lacks is any expectation of step duration — which could be an optional hint on the step ("use step"with a declared budget), defaulting to today's behaviour when absent.What we did meanwhile, in case it is useful to others hitting this: rather than re-tuning the constant, we clamp each sandbox command to
min(requested, deadline − now − reserve)using the samegetDeadline()the World reads. A command admitted early keeps its full budget; one admitted late gets a shorter one and fails into our existing "still running in the background, output at <file>" recovery — so the step completes and journals instead of the function being killed mid-body. That removes the re-execution entirely, but it only works because our long steps happen to be interruptible. It is containment on our side of the boundary, not a fix for the admission decision.Happy to share run IDs or the full log extract if that helps.
VaguelySerious commentedon Oct 6, 2026
Fixed by #3360