[aviggiano]

Fixes #379.

What broke

R54 (aave-v4-v0012-main-issue368fix-r54-invariant-only) died 97 seconds after start, before a single task node ran — all 23 left pending, project-discovery never past wait_reason: ready. The run-level event was workflow-submit-failed:

npm error code E404
npm error 404 Not Found - GET https://registry.npmjs.org/@ai-sdk/provider/-/provider-4.0.7.tgz - Not found

@ai-sdk/provider@4.0.7 was published at 23:48:58.306Z. The install started at 23:51:06 and 404ed. At 23:58 the same URL returned 200. The packument entry goes live before the tarball reaches every registry CDN edge; inside that window npm resolves the version and then hard-fails fetching it.

The generated workspace installs with --package-lock=false, so every unpinned transitive dependency is re-resolved per install — smithers-orchestrator@0.32.0 → ai@^7.0.10 → @ai-sdk/provider@^4 floats onto whatever shipped most recently. Pinning does not close this: the exposure is the entire transitive set, not one named package.

The change

withTransientNpmRegistryRetry wraps the runner install and retries at 1s/5s/15s/45s while the failure is one the registry will fix on its own — a 404 on a tarball URL, a network error, or a registry 5xx. Up to ~66s of waiting against a multi-hour campaign.

Deliberately conservative, so a real problem still fails fast:

• a 404 on a package or version name is permanent — not retried
• ERESOLVE and anything unrecognized — not retried
• an abort, or a kill from the caller's timeoutMs — not retried (the deadline that stopped it has already passed)

The last error propagates unwrapped, so callers still report the registry's own diagnostic.

packages/modal/src/node-worker.ts ran the same unpinned, unretried install per durable node — same defect, smaller blast radius (a node attempt and a sandbox spin-up). It shares the helper rather than duplicating the logic, which is why packages/runtime/src/index.ts gains one export line.

Tests

• startRun retries a workflow runner install the registry fails transiently — verified red on origin/main, and the red diagnostic is byte-for-byte R54's WORKFLOW_SUBMISSION_FAILED payload.
• startRun does not retry a workflow runner install the registry rejects permanently — asserts exactly one npm invocation, pinning the fail-fast boundary.
• isTransientNpmRegistryFailure separates a waitable registry gap from a real dependency error — unit coverage for each classification above.

pnpm --filter @ultrafuzz/runtime test → 442 tests, 427 pass, 0 fail, plus the 14 bun contract tests. pnpm --filter @ultrafuzz/modal test → 352 pass. prettier --check clean on all touched files. Both packages build.

Heads-up for in-flight branches

This touches packages/modal/src/node-worker.ts, packages/runtime/src/index.ts and packages/runtime/src/smithers.ts. Open PRs #366 (release/v0.1.0) and #165 (bench/deepseek-v4-flash-results) carry changes to those paths as accumulation branches and will want a rebase after this merges. It does not touch workflow.tsx, workflow-sync.ts, node-provider.ts or public-worker.ts.

🤖 Generated with Claude Code

Greptile Summary

The PR adds a shared retry helper for transient npm registry failures and applies it to runtime and Modal dependency installation.

• Retries tarball 404s, selected network failures, and registry 5xx responses with bounded backoff.
• Preserves immediate failure for aborts, timeout kills, dependency-resolution errors, and unrecognized failures.
• Adds runtime coverage for transient retries and permanent-error handling.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
packages/runtime/src/npm-install-retry.ts	Introduces bounded retry and conservative classification for transient npm registry failures.
packages/runtime/src/smithers.ts	Applies registry retries to the runtime’s Smithers dependency installation path.
packages/modal/src/node-worker.ts	Reuses the runtime retry helper for per-node Smithers dependency installation.
packages/runtime/test/runtime.test.ts	Covers successful transient retry, permanent failure without retry, and error classification boundaries.
packages/runtime/src/index.ts	Exposes the retry helper through the runtime package’s public entry point.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Install Smithers dependencies] --> B{Install succeeds?}
  B -->|Yes| C[Continue workflow submission]
  B -->|No| D{Transient registry failure?}
  D -->|No| E[Propagate original error]
  D -->|Yes| F{Retry available?}
  F -->|No| E
  F -->|Yes| G[Wait with backoff]
  G --> A

Loading

_{Reviews (2): Last reviewed commit: "Merge origin/main into fix/transient-npm..." | Re-trigger Greptile}

Context used:

• Knowledge Base — Runtime: Run Orchestration Lifecycle
• Knowledge Base — Modal remote benchmark execution

[aviggiano]

Independently confirmed this PR's root cause against the R54 volume (ultrafuzz-aave-v4-v0012-main-issue368fix-r54-invar-36ca421822b1), and the diagnosis in #379 holds exactly:

• state.json: run status: failed at 2026-08-07T23:52:39.065Z, 97s after created_at 23:51:02.506Z. Node counts: 9 succeeded (seeded/reference), 23 pending, 0 failed — no node ever ran, so there is nothing in any node's last_error. The only diagnostic is events.jsonl.
• @ai-sdk/provider@4.0.7 published 2026-08-07T23:48:58.306Z; install ran 23:51:06. Gap 128s. Re-checked the tarball just now: HTTP 200. Registry reports unpublished: false and dist-tags.latest = 4.0.7, confirming propagation lag rather than a yanked version.

Heads-up: this PR and #384 will conflict, though both currently report MERGEABLE/CLEAN against main.

Both branch from 0088f80 and both rewrite the same two npm install call sites. git merge-tree between the two heads conflicts in three files:

CONFLICT (content): packages/modal/src/node-worker.ts
CONFLICT (content): packages/runtime/src/smithers.ts
CONFLICT (content): packages/runtime/test/runtime.test.ts

Whichever merges first leaves the other conflicted. The changes are complementary, not competing — this one wraps the call, #384 replaces its argument array — so the second one through should combine rather than take a side:

await withTransientNpmRegistryRetry(() =>
  runChecked(
    "install-smithers",
    "npm",
    smithersDependencyInstallArgs({ prefix: path.join(projectRoot, ".smithers") }),
    projectRoot
  )
);

and in ensureSmithersDependencies, withTransientNpmRegistryRetry(() => execFileAsync("npm", smithersDependencyInstallArgs({ prefix: packageRoot, registry: "https://registry.npmjs.org" }), {...}), signal).

Worth keeping both: the cutoff in #384 shrinks the window, but only a retry survives a 5xx or a network drop on an already-resolved tarball.

Read-only diagnosis; I have not modified either branch.