Skip to content

Local World starts an interrupted pending run without restoring its missing run_created event #4836

Closed

Description

@nsxdavid

Summary

The Local World bundled with eve@0.73.0 can advance a persisted pending run to running without restoring its missing run_created event. The surviving run metadata contains the original workflow input and identity, but the resulting journal starts with run_started.

Run creation writes the run file before publishing the creation event through a temporary file and hard link. An interruption between those writes leaves this partial state. The resilient run_started path creates both records when the run file is absent, but skips creation-event recovery when the run file already exists.

Environment

  • eve@0.73.0, bundled @workflow/world-local implementation
  • Linux x86_64, Node.js v26.8.2
  • Workflow spec version 8, local filesystem
  • Standalone current @workflow/world-local package has not been tested

Isolated reproduction

Save as repro.mjs beside an installation of eve@0.73.0. This uses a new temporary directory and synthetic run only. Moving the event into an ignored temporary filename models the published-run/unpublished-event interruption window; it does not simulate an actual process crash.

import assert from 'node:assert/strict';
import { createRequire } from 'node:module';
import { dirname, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { tmpdir } from 'node:os';
import { mkdtemp, readdir, rename, rm } from 'node:fs/promises';

const require = createRequire(import.meta.url);
const root = dirname(require.resolve('eve/package.json'));
const { createWorld } = await import(pathToFileURL(join(
  root, 'dist/src/compiled/@workflow/world-local/index.js'
)).href);
const dir = await mkdtemp(join(tmpdir(), 'workflow-run-gap-'));
let world = createWorld({ dataDir: dir, recoverActiveRuns: false });
try {
  const eventData = {
    deploymentId: 'dpl_local@1.0.0',
    workflowName: 'workflow//example//entry',
    input: new Uint8Array([1, 2, 3]),
    executionContext: {},
    attributes: {},
  };
  const { run } = await world.events.create(null, {
    eventType: 'run_created', specVersion: 8, eventData,
  });
  const runId = run.runId;
  await world.close();
  const events = join(dir, 'events');
  const filename = (await readdir(events)).find(f => f.endsWith('.json'));
  await rename(join(events, filename), join(events, filename + '.tmp.interrupted'));
  world = createWorld({ dataDir: dir, recoverActiveRuns: false });
  await world.events.create(runId, {
    eventType: 'run_started', specVersion: 8, eventData,
  });
  const journal = await world.events.list({
    runId, pagination: { limit: 100, sortOrder: 'asc' },
  });
  console.log(journal.data.map(event => event.eventType));
  console.log((await world.runs.get(runId)).status);
  assert.equal(journal.data[0]?.eventType, 'run_created');
} finally {
  await world.close();
  await rm(dir, { recursive: true, force: true });
}

Actual output:

[ 'run_started' ]
running
AssertionError: 'run_started' !== 'run_created'

Expected: recover the missing first creation event from the persisted original run metadata, then advance normally, or report an explicit unrecoverable partial-state error. Do not silently return a running run whose journal lacks the creation event.

A recovery needs to preserve original workflow/deployment/input/execution context/attributes/encryption metadata and creation timestamp rather than trust a potentially changed redelivery payload. Please cover concurrent starts, an original creation writer finishing concurrently, and normal creation without duplicate creation events.

Related but different: #3656 addresses atomic Postgres run creation; #3081 addresses orphaned Postgres step entities. This report concerns Local World's existing-run run_started recovery branch.

Activity

  1. added a commit that references this issue on Oct 20, 2026
    5170f8f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions