Repository navigation
Open
Labels
Description
Expected Behavior
The experimental AI operations should enforce their public contract at each execution boundary:
- The extras package requires a core SDK version that supports custom step subtypes.
- An
AiTimeoutErrorretries in a later invocation, not during the ten-second reserve left for checkpointing. - An interrupted HTTP response is a retryable transport failure.
- The request thread preserves the step's context variables.
- An unavailable invocation deadline counts as no deadline.
llm().askcheckpoints only a response known to be complete.- A decider input is serialized once before the durable step starts.
- The documentation states the minimum usable function timeout.
Actual Behavior
Python SDK #928: add experimental decider and LLM operations merged these operations in the AI operations merge commit 8fbb2459. The following cases remain:
- The extras package allows core SDK 2.x, but every AI operation constructs
StepConfig(sub_type=...). Core 2.x has nosub_typeparameter, so a call fails withTypeErrorbefore it creates an operation. default_ai_retry_strategyschedules the first timeout retry after two seconds. A sibling operation can keep the invocation alive, allowing retries to start during the ten-second reserve and fail without calling the model.- The HTTP transport catches
URLErrorandOSError, but nothttp.client.HTTPException.IncompleteReadfrom either response-body read therefore becomes a non-retryableStepError. - The deadline path starts a bare daemon thread. Context variables set by the step, including an active OpenTelemetry context, are absent inside boto3, urllib, and custom transport calls.
- Deadline property access and invocation are not protected. A throwing getter fails the step, and non-finite numeric values reach the wait calculation.
llm().askaccepts a response with nostopReasonand checkpoints any text in it, although the response is not known to be complete.- Each decider call serializes its input for preflight validation, then serializes it again inside the step. A supported container subclass with observable iteration runs twice and can produce a different request on the second serialization.
- The retry documentation promises a new invocation, but it does not state that a function timeout of ten seconds or less leaves no request budget.
Steps to Reproduce
These results reproduce on main at 8fbb2459 with Python 3.14:
- Constructing
StepConfig(sub_type="LlmAsk")with core SDK 2.0.0 raisesTypeError: StepConfig.__init__() got an unexpected keyword argument 'sub_type'while the extras dependency accepts that version. - In a local-runner
parallel, give the AI call about 10.05 seconds remaining and keep a sibling active for seven seconds. Attempts 2 and 3 restart in the first invocation, during the reserve, without another model call. Attempt 4 runs in the next invocation. - Make an HTTP response's
read()raisehttp.client.IncompleteRead. The exception escapes_http_transport, anddefault_ai_retry_strategyreturns no retry. - Set a
ContextVarbeforecall_before_deadline. It has the set value without a deadline and its default value inside the deadline thread. - Return Bedrock output text without
stopReason._conversereturns the text instead of raisingLlmStoppedError. - Give
decider().checkalistsubclass whose__iter__records calls. It is iterated twice.
All 145 current AI tests pass, so they do not cover these outcomes.
SDK Version
main at 8fbb2459; introduced by Python SDK #928: add experimental decider and LLM operations. No release tag contains this commit as of 2026-11-13.
Python Version
3.14. The defects are not specific to 3.14.
Is this a regression?
No. These behaviors were introduced with the experimental AI operations.
Last Working Version
N/A
Required Changes and Tests
- Require
aws-durable-execution-sdk-python>=3.0.0directly. Do not add a core 2.x fallback. - Give
AiTimeoutErrora minimum retry delay greater thanINVOCATION_RESERVE_SECONDS, and test it with a decreasing deadline and a running parallel sibling. - Convert
http.client.HTTPExceptionfailures from both response-body reads intoDeciderError; preserve an HTTP error status when headers were received. Add truncated 200 and truncated error-response tests. - Run the request thread under
contextvars.copy_context().run, with a test that reads a context variable inside a transport. - Treat an exception, non-number, or non-finite deadline reading as no deadline, with a test for each branch.
- Raise
LlmStoppedErrorwhenstopReasonis missing, with and without response content. - Capture the validated serialized decider input and pass that string into the step so serialization occurs once.
- State that the function timeout must leave more than the ten-second reserve.
- Add an end-to-end test under
tests/e2ethat exercises the public decider and LLM operations through replay with deterministic fake boundaries.
Files
packages/aws-durable-execution-sdk-python-extras/pyproject.tomlpackages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/_shared.pypackages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/retry.pypackages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/transports.pypackages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/llm.pypackages/aws-durable-execution-sdk-python-extras/src/aws_durable_execution_sdk_python_extras/ai/decider.pypackages/aws-durable-execution-sdk-python-extras/docs/ai.md- AI unit and end-to-end tests
Out of Scope
- Changes to the core SDK.
- New AI operations, model hosts, retry options, or fallback paths for core SDK 2.x.
- Changes to checkpoint or error wire formats.
Acceptance Criteria
- The extras package requires core SDK 3.0.0 or later.
- A timeout retry cannot start during the invocation reserve.
- Interrupted HTTP bodies produce the documented retry decision.
- Request work sees the step's context variables.
- Unavailable or unusable deadline readings do not fail an otherwise valid call.
- A Bedrock response without
stopReasonis never checkpointed as a successful answer. - Each decider input is serialized exactly once.
- The minimum function timeout is documented.
- Unit and end-to-end tests fail when any corrected branch is removed or inverted.
- Tests, type checks, formatting, and lint pass.
Activity
SilanHe commentedon Nov 16, 2026
Triaged: all 8 items are still present on main (8fbb245). Item 1 is high priority: the extras package allows core
>=2.0.0(pyproject.toml#L23-L32), so every AI call fails on core 2.x. Suggest a quick fix that raises the core floor. The other items are medium and can follow separately.AI-assisted comment.
refactor(extras): drop ai input checks (#937)