Skip to content

[Bug]: SDK starts new durable operations after the invocation deadline #1102

Closed

Description

@ParidelPooya

Expected Behavior

Once an invocation's deadline has passed, the SDK starts no new durable operation for that invocation.

Actual Behavior

The SDK reads the deadline only to bound polling timers and plugin drain waits. It does not read it when a new operation starts. So when the deadline passes during a step, the step finishes, the SDK starts the next step, and the wrapper builds a SUCCEEDED result.

On standard Lambda this has no effect, because the environment freezes after the timeout. On Lambda Managed Instances (LMI), an invocation timeout does not stop the running code (LMI execution environment). Lambda then retries the invocation. So the old invocation and the retry run steps for the same execution at the same time, and each step's side effects can happen twice.

Issue #927 reproduced this. It made getRemainingTimeInMillis() return -1 and then released a running step.

Steps to Reproduce

See row 3 of the table in #927.

Design note required

This changes what the SDK reports when the deadline passes. So the contribution rules require a design note that a maintainer agrees to before code is written. The note should answer:

  1. Where the check lives. The proposal is one check, at the single point where an operation starts, not one check in every handler.
  2. What happens to the caller of a refused operation. The proposal is the never-settling promise that the termination path already uses.
  3. What status the invocation reports. The status must let Lambda retry without recording a business failure.
  4. How a missing or non-finite deadline reading behaves. The SDK invariants say it must mean "no deadline".

Out of scope: interrupting synchronous code, cancelling I/O that application code started, and branches left running after an early map, parallel or race completion. Those need separate design work.

SDK Version

main (9de0c7f2)

Node.js Version

Any

Is this a regression?

No

Additional Context

Split out of #1014.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpkg:sdkPackage: aws-durable-execution-sdk-js

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions