Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/shared-hook-history.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@workflow/core": patch
"@workflow/world": patch
"@workflow/world-vercel": patch
---

Avoid redundant history reads when independent hooks register concurrently. Keep existing single-event requests parallel and reconcile their committed events together before continuing, with the normal history reload when the responses do not cover a complete consecutive suffix.
2 changes: 1 addition & 1 deletion docs/content/docs/v5/how-it-works/event-sourcing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ Both save a reload, so the rule for which writes send them is not about the even
| `run_created` | `start()` | Nothing | No log exists yet. |
| `run_started` | Replay, first write of a delivery | `eventCount` when a log is loaded; no cursor | The `run_started` response already carries the full log as a preload. |
| `step_created`, `wait_created`, `hook_disposed`, `attr_set` | Suspension handler | `eventCount` | The handler keeps writing from this log and the replay resumes from it. |
| `hook_created` | Suspension handler | `eventCount` and `sinceCursor` (only when the suspension creates exactly one hook) | The delta is how a hook whose payload already arrived resolves without another invocation. Two creates diffing against one cursor would give two deltas of which only the first could be taken. |
| `hook_created` | Suspension handler | `eventCount` and `sinceCursor` for a lone hook; independent concurrent Node.js registrations in a hooks-only suspension defer both | A lone create returns a delta. Concurrent registrations reconcile their committed events together and reload when the responses do not cover a consecutive history suffix. |
| `hook_received` | Replay resuming a hook | `eventCount` | The replay continues from the log right after. |
| `hook_received` | Webhook or `resumeHook()` | Nothing | Out-of-band. It enqueues a delivery that loads the log. |
| `wait_completed` | Replay completing an elapsed wait | `eventCount` | The replay continues from the log right after. |
Expand Down
12 changes: 11 additions & 1 deletion docs/content/worlds/v5/building-a-world.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,16 @@ The version a World stamps comes from `mintedSpecVersion()`: 8 by default or the

Not every write carries `eventCount`, and a World is free to skip the report read for event types whose writer never holds a log to merge it into (the step executor's `step_started`, `step_retrying`, `step_completed`, and `step_failed`, and the run-terminal `run_completed`, `run_failed`, and `run_cancelled`). Which writes send `eventCount` or `sinceCursor`, and why, is tabulated in [Events returned on a write](/docs/how-it-works/event-sourcing#events-returned-on-a-write). When a write carries both, the `sinceCursor` delta is a superset of the report, so return the delta and skip the report.

Independent concurrent hook registrations in a hooks-only suspension deliberately omit `eventCount` and `sinceCursor`, even though the worker holds a loaded log. The Node.js runtime waits for every registration to settle and verifies that their returned events fill every position immediately after that log. A gap, incomplete response, or missing committed-event cursor keeps the normal history reload. Mixed suspensions, forced claims, and same-token lifecycle operations keep their usual catch-up parameters.

### Optional: returning a committed-event cursor

An `events.create()` result may include `eventCursor`: a World-owned cursor that its own ascending `events.list()` accepts as an exclusive position immediately after the returned `event`. It is distinct from a history-page `cursor`, and does not assert that the caller has loaded earlier events.

Returning `eventCursor` opts into using the returned `event` as replay input. That event **must equal the event returned by `events.list()` for the same position and `resolveData` option**, including its stored `createdAt`, normalized `tokenRetentionUntil`, resolved metadata bytes, and any `hook_conflict` details such as `conflictingRunId` or `forceRefusedReason`. Return the persisted event rather than reconstructing it from the creation request.

The runtime uses the cursor only after proving that the committed responses extend its loaded prefix without a hole. This is optional: a World that omits `eventCursor` remains supported and retains the reload path. The runtime never interprets a World's cursor encoding.

### Optional: rejecting a stale write

A replay writes events derived from the event log it loaded, so a write made from an event log that no longer matches the store can commit events no correct replay would produce. Rejecting one means answering `events.create()` with a `PreconditionFailedError` when the run's log already holds more events than the caller's `eventCount` says it had loaded.
Expand All @@ -246,7 +256,7 @@ Implement rejection when your store allocates positions outside the commit and c
Two rules make this safe:

- **Only reject on evidence.** If your record of a run's events is incomplete, expired, or cannot answer the question, accept the creation. A rejection must always indicate a real discrepancy because the runtime responds by discarding a replay.
- **Accept a creation that carries no `eventCount`.** It came from a caller with no loaded log that could be stale, such as a queued step body or out-of-band writer, rather than a caller claiming the log is empty.
- **Accept a creation that carries no `eventCount`.** The caller is not claiming the log is empty. It may have no loaded log, as with a queued step body or out-of-band writer, or may deliberately defer catch-up until concurrent hook registrations settle. Without a count there is no evidence for count-based rejection; independent ownership checks and write fences still apply.

A rejection may include the events the caller was missing as `{ events, cursor }` on the error's `details`. Include them only when you can prove that the set fully accounts for the discrepancy, isn't truncated, and contains only events from the run being written. The runtime merges them directly into the replay's event log, so incorrect data is worse than no delta. Otherwise, omit them, and the runtime performs a full reload.

Expand Down
117 changes: 117 additions & 0 deletions packages/core/src/runtime.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3915,6 +3915,123 @@ describe('workflowEntrypoint hook conflict beside a pre-claimed inline step', ()
});
});

describe('workflowEntrypoint concurrent hook continuation', () => {
afterEach(() => {
setWorld(undefined);
vi.clearAllMocks();
vi.unstubAllEnvs();
});

it.each([
false,
true,
])('continues after concurrent registrations, interleaved writer=%s', async (interleaved) => {
vi.stubEnv('WORKFLOW_RETAINED_VM', '1');
const runId = `wrun_hook_reconciliation_${interleaved}`;
const workflowRun = {
runId,
workflowName: 'workflow',
status: 'running',
specVersion: 6,
deploymentId: 'test-deployment',
input: await dehydrateWorkflowArguments([], runId, undefined, []),
createdAt: new Date(),
updatedAt: new Date(),
startedAt: new Date(),
} satisfies WorkflowRun;
const durableEvents: Event[] = [];
const record = (data: any): Event => {
const event = {
...data,
runId,
eventId: slotToEventId(durableEvents.length + 1),
createdAt: new Date(),
} as Event;
durableEvents.push(event);
return event;
};
record({
eventType: 'run_created',
specVersion: 6,
eventData: {
workflowName: 'workflow',
deploymentId: 'test-deployment',
input: workflowRun.input,
},
});
let hooks = 0;
const create = vi.fn(async (_runId: string, data: any, _params?: any) => {
if (data.eventType === 'hook_created' && hooks++ === 1 && interleaved) {
// A concurrent delivery started after this invocation read its prefix.
record({ eventType: 'run_started', specVersion: 6 });
}
const event = record(data);
if (data.eventType === 'run_started')
return {
event,
run: workflowRun,
events: [...durableEvents],
cursor: `eid:${event.eventId}`,
hasMore: false,
};
if (data.eventType === 'run_completed')
workflowRun.status = 'completed' as any;
return { event, eventCursor: `eid:${event.eventId}` };
});
const list = vi.fn(async ({ pagination }: any) => ({
data: durableEvents.filter(
(event) =>
!pagination?.cursor || event.eventId > pagination.cursor.slice(4)
),
cursor: `eid:${durableEvents.at(-1)!.eventId}`,
hasMore: false,
}));
const queue = vi.fn(async () => ({ messageId: null }));
let handler!: (message: unknown, metadata: unknown) => Promise<unknown>;
setWorld({
specVersion: SPEC_VERSION_CURRENT,
createQueueHandler: vi.fn((_prefix: string, callback: typeof handler) => {
handler = callback;
return async () => new Response(null, { status: 204 });
}),
events: { create, list },
runs: { get: vi.fn(async () => workflowRun) },
queue,
getEncryptionKeyForRun: vi.fn(async () => undefined),
} as any);
const entry = workflowEntrypoint(`
const createHook = globalThis[Symbol.for('WORKFLOW_CREATE_HOOK')];
async function workflow() {
const first = createHook({ token: 'customer:first' });
const second = createHook({ token: 'customer:second' });
const owners = await Promise.all([first.getConflict(), second.getConflict()]);
return owners.every(owner => owner === null);
}
globalThis.__private_workflows = new Map([['workflow', workflow]]);`);
await entry(new Request('https://example.test'));
await handler(
{ runId, requestedAt: new Date() },
{
requestId: 'request',
attempt: 1,
queueName: '__wkf_workflow_workflow',
messageId: 'msg_hooks',
}
);
expect(workflowRun.status).toBe('completed');
expect(queue).not.toHaveBeenCalled();
expect(list).toHaveBeenCalledTimes(interleaved ? 1 : 0);
const registrations = create.mock.calls.filter(
(call) => call[1].eventType === 'hook_created'
);
expect(registrations).toHaveLength(2);
for (const call of registrations) {
expect(call[2]?.eventCount).toBeUndefined();
expect(call[2]?.sinceCursor).toBeUndefined();
}
});
});

describe('workflowEntrypoint max-deliveries terminal write', () => {
afterEach(() => {
setWorld(undefined);
Expand Down
Loading
Loading