Skip to content

world-local write() does not retry a transient Windows EPERM from its existence probe #4203

Closed

Description

@DevvoLazza

The existence check in write() calls fs.access(filePath) directly. A transient Windows EPERM, EBUSY, or EACCES escapes before the atomic write begins, even though the same module's withWindowsRetry handles these errors for rename/unlink. Current upstream source still has the direct access call in packages/world-local/src/fs.ts (file SHA af18b50b6d0d0315bd976fc33dffc2b0f8fbb0d7).

Measured with Node 24.16.0 on Windows and @workflow/world-local 4.2.4. A single injected EPERM is sufficient; no concurrent writes are needed. The reproducer temporarily intercepts the public Node filesystem API within its own process. It does not edit node_modules or require an actual Defender lock.

Install the pinned package in a disposable project and save the following as repro.mjs:

import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { createLocalWorld } from "@workflow/world-local";

const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), "world-access-repro-"));
const world = createLocalWorld({ dataDir, recoverActiveRuns: false });
const access = fs.promises.access;
let probes = 0;
fs.promises.access = async (file, ...args) => {
  if (typeof file === "string" && path.dirname(file) === path.join(dataDir, "streams", "chunks")) {
    probes++;
    if (probes === 1) {
      throw Object.assign(new Error("transient Windows file lock"), {
        code: "EPERM", syscall: "access", path: file,
      });
    }
  }
  return access(file, ...args);
};
try {
  await world.writeToStream("example", "run_1", "hello");
  console.log({ probes, saved: true });
} catch (error) {
  console.log({ probes, saved: false, code: error.code, syscall: error.syscall });
} finally {
  fs.promises.access = access;
}

Run node repro.mjs. Observed: { probes: 1, saved: false, code: "EPERM", syscall: "access" }. Expected on Windows: retry the existence probe, let the subsequent ENOENT indicate a missing file, and successfully save one chunk. If the file exists after a retry, overwrite=false must still raise the normal conflict rather than overwriting it. Exhausted permission/lock failures must still reject.

Suggested upstream handling: apply withWindowsRetry to the existence probe while retaining the existing ENOENT/conflict semantics.

Consumer impact: libredb/libredb-studio#900. With STORAGE_PROVIDER=sqlite, WORKFLOW_TARGET_WORLD=local, a writable WORKFLOW_LOCAL_DATA_DIR, and enabled model configuration, our availability probe is green. Injecting one EPERM during ordinary sequential AgentRunService calls can reject start, narrative append, or EOF publication; an index-write fault leaves a terminal run missing from history (our service logs that failure). This proves reachability under fault injection; it does not measure naturally occurring lock frequency or claim a live-model reproduction.

Activity

  1. pranaygp commented on Sep 23, 2026

    @pranaygp
    Contributor

    (AI)

    Severity: S3. This only affects the local world on Windows, and only when an antivirus scanner, indexer or concurrent reader briefly locks the file. When that happens the write() fails even though its sibling rename/unlink/read calls already retry these errors. Depending on which write fails, the result is a failed stream chunk, a failed event or a missing index entry. Other platforms and production worlds are unaffected.

    Reproduction. I confirmed it with your script (thanks for the precise repro), running on Linux with process.platform stubbed to win32 before the import, against both 4.2.4 and current main. The result was { probes: 1, saved: false, code: 'EPERM', syscall: 'access' }. On main the call is world.streams.write(runId, name, chunk) and chunks live in per-stream subdirectories under streams/chunks, so the probe filter needs startsWith instead of dirname ===. With the fix it prints { probes: 2, saved: true }.

    Related. #4051 (merged) made the same change for readJSON. Windows support in general was tracked in #117. I didn't find any duplicates.

    Fix: #4331 wraps the existence probe in withWindowsRetry. ENOENT still leads to a write, an existing file still raises the overwrite: false conflict, and a lock that persists past the retries still rejects. There are unit tests for all three cases, and the unit tests pass on Ubuntu and Windows CI. The only failing checks are Vercel-deployment E2E lanes that hit a CI rate limit during deploy, which is unrelated to this change.

    To see if this fix solves your issue, could you try to install the pre-release in your package.json? Just replace the workflow dep like so:

    {
      "dependencies": {
        "workflow": "https://workflow-tarballs-git-triage-workflow4203-world-local-ac-667d94.labs.vercel.dev/workflow.tgz",
        "@workflow/world-local": "https://workflow-tarballs-git-triage-workflow4203-world-local-ac-667d94.labs.vercel.dev/workflow-world-local.tgz"
      }
    }
  2. added a commit that references this issue on Nov 13, 2026
    c11fc40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions