Skip to content

Non-canonical os.tmpdir() roots affect 243 sites across 10 packages, not just packages/runtime #1024

Closed

Description

@mrthankyou

Summary

#1017 reported this for packages/runtime/test, and #1019 fixes it there. The same defect is present across the rest of the repository: 243 sites in 67 files across 10 packages, plus the release-validation script and the CI scripts.

os.tmpdir() is /var/folders/... on macOS and /var is a symlink to /private/var, so any root built directly on it is non-canonical. Ultrafuzz deliberately rejects non-canonical roots, so the code fails against its own temporary directories. Linux is unaffected (/tmp is already canonical), which is why CI does not see it.

Confirmed impact outside packages/runtime

@ultrafuzz/artifacts on main, macOS: 324 tests, 312 pass, 12 fail, with the failures naming the cause directly:

Error [ArtifactPathError]: generated test file root must be canonical: /var/folders/...
Error: prompt schema destination crosses a symlink: /var
Error: ENOENT: no such file or directory, scandir '/var/folders/.../ultrafuzz-sch...'
Error: ENOENT: no such file or directory, chmod '/var/folders/.../ultrafuzz-seale...'

@ultrafuzz/config (76) and @ultrafuzz/prompts (103) pass, so this is not uniform -- it bites wherever a path guard or a canonicality assertion is reached.

Scope

files: 67   sites: 243

artifacts  cli  config  dashboard  evals  evmbench  modal
prompts    references   runtime     scripts/ci  scripts/validate-packed-install.mjs

Seven are production files rather than tests:

packages/evmbench/src/runner.ts
packages/modal/src/node-provider.ts
packages/modal/src/runner.ts
packages/references/src/index.ts
packages/runtime/src/smithers.ts                          (fixed in #1019)
packages/runtime/src/templates/smithers/agents/kimi.tsx
packages/runtime/src/workspace-handoff.ts                 (fixed in #1019)

The runtime requires canonical roots, so the production sites should create them rather than leave it to chance.

Also affects the release script

scripts/validate-packed-install.mjs builds its consumer and initialized project under os.tmpdir(), so pnpm validate:pack fails on macOS before it gets to validating anything:

Error: packed validate failed (... ultrafuzz validate --project /var/folders/.../initialized-project --json)
  "code": "SYMLINK_PATH",
  "message": "Topology path contains a symlink component: /var",

With a canonical TMPDIR it passes for all 12 packages, which isolates the cause.

Suggested fix

Canonicalize the tmpdir at the point of use -- fs.realpathSync(os.tmpdir()) -- everywhere a temporary root is created. #1019 introduces temporaryRoot() for packages/runtime/test, which also handles removal of sealed trees; the other packages mostly need only the canonical half, since they do not create sealed snapshots.

The production sites and the release script are the ones worth fixing regardless of platform, because they encode an assumption that the system temporary directory is canonical, and that is not a property any platform guarantees.

Activity

  1. mrthankyou commented on Aug 31, 2026

    @mrthankyou
    CollaboratorAuthor

    Correction to the scope in the description: my original grep required mkdtempSync, which missed sites using mkdtemp, mkdirSync, and other calls. The real figure is 375 sites in 92 files, not 243 in 67.

    Fixed in #1025, which measures 92 tests repaired across five packages with no new failures (modal +52, dashboard +18, artifacts +12, cli +5, references +5).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions