Skip to content

[Bug]: Fix AI operation execution boundaries #934

Open

Description

@yaythomas

Expected Behavior

The experimental AI operations should enforce their public contract at each execution boundary:

  • The extras package requires a core SDK version that supports custom step subtypes.
  • An AiTimeoutError retries in a later invocation, not during the ten-second reserve left for checkpointing.
  • An interrupted HTTP response is a retryable transport failure.
  • The request thread preserves the step's context variables.
  • An unavailable invocation deadline counts as no deadline.
  • llm().ask checkpoints only a response known to be complete.
  • A decider input is serialized once before the durable step starts.
  • The documentation states the minimum usable function timeout.

Actual Behavior

Python SDK #928: add experimental decider and LLM operations merged these operations in the AI operations merge commit 8fbb2459. The following cases remain:

  1. The extras package allows core SDK 2.x, but every AI operation constructs StepConfig(sub_type=...). Core 2.x has no sub_type parameter, so a call fails with TypeError before it creates an operation.
  2. default_ai_retry_strategy schedules the first timeout retry after two seconds. A sibling operation can keep the invocation alive, allowing retries to start during the ten-second reserve and fail without calling the model.
  3. The HTTP transport catches URLError and OSError, but not http.client.HTTPException. IncompleteRead from either response-body read therefore becomes a non-retryable StepError.
  4. The deadline path starts a bare daemon thread. Context variables set by the step, including an active OpenTelemetry context, are absent inside boto3, urllib, and custom transport calls.
  5. Deadline property access and invocation are not protected. A throwing getter fails the step, and non-finite numeric values reach the wait calculation.
  6. llm().ask accepts a response with no stopReason and checkpoints any text in it, although the response is not known to be complete.
  7. Each decider call serializes its input for preflight validation, then serializes it again inside the step. A supported container subclass with observable iteration runs twice and can produce a different request on the second serialization.
  8. The retry documentation promises a new invocation, but it does not state that a function timeout of ten seconds or less leaves no request budget.

Steps to Reproduce

These results reproduce on main at 8fbb2459 with Python 3.14:

  • Constructing StepConfig(sub_type="LlmAsk") with core SDK 2.0.0 raises TypeError: StepConfig.__init__() got an unexpected keyword argument 'sub_type' while the extras dependency accepts that version.
  • In a local-runner parallel, give the AI call about 10.05 seconds remaining and keep a sibling active for seven seconds. Attempts 2 and 3 restart in the first invocation, during the reserve, without another model call. Attempt 4 runs in the next invocation.
  • Make an HTTP response's read() raise http.client.IncompleteRead. The exception escapes _http_transport, and default_ai_retry_strategy returns no retry.
  • Set a ContextVar before call_before_deadline. It has the set value without a deadline and its default value inside the deadline thread.
  • Return Bedrock output text without stopReason. _converse returns the text instead of raising LlmStoppedError.
  • Give decider().check a list subclass whose __iter__ records calls. It is iterated twice.

All 145 current AI tests pass, so they do not cover these outcomes.

SDK Version

main at 8fbb2459; introduced by Python SDK #928: add experimental decider and LLM operations. No release tag contains this commit as of 2026-11-13.

Python Version

3.14. The defects are not specific to 3.14.

Is this a regression?

No. These behaviors were introduced with the experimental AI operations.

Last Working Version

N/A

Required Changes and Tests

  • Require aws-durable-execution-sdk-python>=3.0.0 directly. Do not add a core 2.x fallback.
  • Give AiTimeoutError a minimum retry delay greater than INVOCATION_RESERVE_SECONDS, and test it with a decreasing deadline and a running parallel sibling.
  • Convert http.client.HTTPException failures from both response-body reads into DeciderError; preserve an HTTP error status when headers were received. Add truncated 200 and truncated error-response tests.
  • Run the request thread under contextvars.copy_context().run, with a test that reads a context variable inside a transport.
  • Treat an exception, non-number, or non-finite deadline reading as no deadline, with a test for each branch.
  • Raise LlmStoppedError when stopReason is missing, with and without response content.
  • Capture the validated serialized decider input and pass that string into the step so serialization occurs once.
  • State that the function timeout must leave more than the ten-second reserve.
  • Add an end-to-end test under tests/e2e that exercises the public decider and LLM operations through replay with deterministic fake boundaries.

Files

  • packages/aws-durable-execution-sdk-python-extras/pyproject.toml
  • packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/_shared.py
  • packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/retry.py
  • packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/transports.py
  • packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/llm.py
  • packages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/decider.py
  • packages/aws-durable-execution-sdk-python-extras/docs/ai.md
  • AI unit and end-to-end tests

Out of Scope

  • Changes to the core SDK.
  • New AI operations, model hosts, retry options, or fallback paths for core SDK 2.x.
  • Changes to checkpoint or error wire formats.

Acceptance Criteria

  • The extras package requires core SDK 3.0.0 or later.
  • A timeout retry cannot start during the invocation reserve.
  • Interrupted HTTP bodies produce the documented retry decision.
  • Request work sees the step's context variables.
  • Unavailable or unusable deadline readings do not fail an otherwise valid call.
  • A Bedrock response without stopReason is never checkpointed as a successful answer.
  • Each decider input is serialized exactly once.
  • The minimum function timeout is documented.
  • Unit and end-to-end tests fail when any corrected branch is removed or inverted.
  • Tests, type checks, formatting, and lint pass.

Activity

  1. SilanHe commented on Nov 16, 2026

    @SilanHe
    Contributor

    Triaged: all 8 items are still present on main (8fbb245). Item 1 is high priority: the extras package allows core >=2.0.0 (pyproject.toml#L23-L32), so every AI call fails on core 2.x. Suggest a quick fix that raises the core floor. The other items are medium and can follow separately.

    AI-assisted comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions