Skip to content

Modal worker does not retry an exhausted generated preparation task #747

Closed as not planned

Description

@aviggiano

Summary

On release/v0.1.0, a Modal worker cannot recover a retained inner run when the selected cloud task's exact generated preparation task exhausted its retries.

The validated selected-task DTO explicitly binds all three task IDs:

id:            node:<attempt>
preparationId: prepare:<attempt>
verifierId:    verify:<attempt>

After normal resume re-emits the retained failure, selectedInnerTaskRetriesExhausted() accepts only a blocker whose node ID equals id. A blocker for the DTO-bound preparationId is treated as unrelated, so the worker rethrows the original resume failure and never invokes the already-bounded retry path.

This occurs before agent inference.

Minimal generic reproduction

Use only the existing temporary Git fixture:

  1. Select node:attempt-one with validated preparationId: prepare:attempt-one.
  2. Make normal inner resume fail.
  3. Make the diagnostic return one blocker: { kind: "retries-exhausted", nodeId: "prepare:attempt-one" }.
  4. Make retry-task succeed only when --node-id prepare:attempt-one is used.
  5. Assert the worker resumes by retrying that exact DTO-bound preparation task.

Run:

pnpm --filter @ultrafuzz/modal exec vitest run test/node-provider.test.ts \
  -t 'retries the exact generated preparation task for the selected inner task' \
  --reporter=verbose

Observed:

1 failed | 204 skipped
cloud worker phase resume-workflow failed with code 1

The command log contains up and why, but no retry-task.

Expected behavior

The recovery selector should return the exact exhausted blocker ID only when it equals one of the validated selected-task DTO's id, preparationId, or verifierId. The retry command should target that exact returned ID. Every unrelated blocker must remain terminal.

Duplicate search

No issue matched the preparation-task retry-selection gap. Issues #745 and #746 cover earlier dependency relocation and handoff-identity boundaries, not selection of a DTO-bound generated prerequisite after resume.

Activity

  1. aviggiano commented on Oct 2, 2026

    @aviggiano
    CollaboratorAuthor

    Closing as not planned. The exact-ID check this issue asks for (matching the exhausted blocker against the selected task's id, preparationId or verifierId) reached main in #744. PR #1197 (d2b5cce) then removed per-node cloud execution, deleting node-worker.ts and the cloud selected-task DTO. The Modal eval runner that remains resumes with ultrafuzz resume --retry-failed, which resets every failed or stalled task, including generated prepare: nodes (smithersFailedTasks in packages/runtime/src/smithers.ts), so the gap does not exist on the remaining paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions