Repository navigation
feat(extras): add experimental decider and llm operations - #928
Merged
Merged
Conversation
The JavaScript extras package has an /ai subpath with decider() and llm(). Python had no counterpart. This commit adds aws_durable_execution_sdk_python_extras.ai with the same operations, step names, subtypes and errors. decider() returns check, classify, label and score. Each one asks a Strands Decider one typed question in one durable step. A Strands Decider can run on a server, on SageMaker, on Lambda, or on AgentCore Runtime. Every host takes the same request body. So decider() takes either endpoint, for plain HTTP, or a transport: - sagemaker_transport(endpoint_name) calls InvokeEndpoint. - lambda_transport(function_name, qualifier=) calls Invoke, with the request as the event. - agentcore_transport(runtime_arn, qualifier=) calls InvokeAgentRuntime. boto3 is already a dependency of this package. So the transports need no new dependency. llm() returns ask, which calls Bedrock Converse in one durable step. Python-specific behavior: - Every error extends AiError, which extends DurableOperationError and is registered with the SDK. So the call raises the same class on the first run and on replay. DeciderError.status and LlmStoppedError.stop_reason are stored in the error data, so they survive replay. Any other step failure stays a StepError. - boto3 and urllib cannot cancel a call. So each request runs in a daemon thread, and the step stops waiting 10 seconds before the invocation ends, with AiTimeoutError. This is the pattern that the MicroVM operations use. - An invalid argument raises TypeError or ValueError before any step starts. The module and every export are marked experimental.
hln33
reviewed
Nov 12, 2026
Fixes for hln33's review. - check and label now reject a probability outside 0 to 1 with DeciderInvalidResponseError. A value within 0.01 of the range is clamped, as score does. - decider() raises TypeError when headers is set with transport. The headers were dropped without an error. - DeciderError keeps status, and LlmStoppedError keeps stop_reason, when data is also given. A dropped status read as unreachable and was retried. - New tests check that a replay returns the checkpointed answer without calling the model, for decider and llm.
ParidelPooya
force-pushed
the
feat/extras-ai
branch
from
November 12, 2026 21:39
55e19dd to
21bdb75
Compare
hln33
approved these changes
Nov 13, 2026
ParidelPooya
marked this pull request as ready for review
November 13, 2026 18:27
10 tasks
ParidelPooya
added a commit
to aws/aws-durable-execution-sdk-js
that referenced
this pull request
Nov 16, 2026
Brings the parked `/ai` work to main: #1160 (`decider()` and `llm()`) and #1213 (SageMaker and Lambda transports), merged with current main, plus fixes for the open review threads on #1213. ## What it adds `@aws/durable-execution-sdk-js-extras/ai`. Every export is marked `@experimental`. | Operation | Step name | Subtype | Returns | |---|---|---|---| | `decider().check` | `decider.check` | `DeciderCheck` | `boolean` | | `decider().classify` | `decider.classify` | `DeciderClassify` | one choice | | `decider().label` | `decider.label` | `DeciderLabel` | the choices that apply | | `decider().score` | `decider.score` | `DeciderScore` | 0 to 1 | | `llm().ask` | `llm.ask` | `LlmAsk` | text | | Decision model host | Use | Permission | |---|---|---| | Any server | `endpoint` | none | | SageMaker | `transport: sageMakerTransport({ endpointName })` | `sagemaker:InvokeEndpoint` | | Lambda | `transport: lambdaTransport({ functionName, qualifier })` | `lambda:InvokeFunction` | | AgentCore Runtime | `transport: agentCoreTransport({ runtimeArn })` | `bedrock-agentcore:InvokeAgentRuntime` | The four AWS SDK clients are optional peer dependencies, loaded on first use. Reference: `packages/aws-durable-execution-sdk-js-extras/docs/ai.md`. ## Fixes on top of #1213 1. **esbuild bundles.** esbuild leaves a missing package to run time only when `.catch()` sits directly on the literal `import()`. The `lazySdk` helper in #1213 moved it into the helper. So bundling `/ai` with esbuild failed unless all four clients were installed. Each `import()` now has its own `.catch(sdkLoadError(...))`. `bundle.test.ts` bundles a copy of `src/ai` where no client can be resolved. It fails with the #1213 code and passes now. 2. **One default-client cache.** `lazyClient()` replaces the three copies (hln33). 3. **Default client test.** Each transport creates one default client on first use and reuses it. 4. **README typo** (hln33), now in `docs/ai.md`. ## Changes from merging main - The ai tests now run on vitest. - The AI reference moved from the README to `docs/ai.md`, as main did for `docs/microvm.md`. The README links to it. - `package-dependencies.test.ts` lists the four ai clients. It checks that the package root and `/ai` load without them. ## Testing - Extras `npm test` (tsc and vitest): 514 passed, 1 skipped (the live test, which needs `AI_LIVE=1`). - `npm run build` and `npm run lint` pass. - Not tested against a real SageMaker endpoint or Lambda function. There is no example in the examples package, because an example needs a deployed decision model. Python counterpart: aws/aws-durable-execution-sdk-python#928. ## Design note **Problem.** A durable function often needs one typed decision (yes or no, one of N, a score) or one short text answer from a model. Calling a model inside `context.step` by hand repeats the same work in every function: prompt shape, reply parsing, retries, the invocation deadline, and checkpoint size. **Cause.** The core SDK gives steps, not model calls. No existing library gives model calls that are durable steps on Lambda. Typed-answer libraries are not durable. Durable platforms ship model layers only for their own runtimes. **Options considered.** | Option | Result | |---|---| | 1. Document a `context.step` + client pattern only | No new API. Every function repeats parsing, retries and the deadline. | | 2. One `ai()` namespace with all operations | One import. A decision model and a text model share one config, so a default model is ambiguous. | | 3. `decider()` and `llm()` factories under `/ai` (chosen) | Config is set once per factory. Two deciders can differ. Fakes are easy in tests. | | 4. One `decider()` config key per host (`agentCore`, `sageMaker`, ...) | Each new host changes `DeciderConfig`. | | 5. `endpoint` or one `transport` (chosen) | A new host, such as Bedrock later, is one new export. `decider()` does not change. | **Removed before release.** The `agentCore` config key (now `agentCoreTransport`), and `AgentCoreInvokeClient` (now `AwsSdkClient`). Neither was released. **Defaults.** `llm()` uses a pinned Bedrock model ID, moved forward in later releases. `decider()` has no default host, because no managed host exists yet. It uses `STRANDS_DECIDER_ENDPOINT` when no option is set. Step names and subtypes are fixed, because a change breaks executions in flight. **Experimental.** Every export is `@experimental`.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python counterpart of the JavaScript
/aisubpath (aws/aws-durable-execution-sdk-js#1160, with the transports from aws/aws-durable-execution-sdk-js#1213). Draft: the JavaScript work waits for Bedrock to support decider models, so this one waits too.What it adds
aws_durable_execution_sdk_python_extras.ai. The module and every export are marked experimental.decider().checkdecider.checkDeciderCheckbooldecider().classifydecider.classifyDeciderClassifydecider().labeldecider.labelDeciderLabeldecider().scoredecider.scoreDeciderScorellm().askllm.askLlmAskStep names and subtypes match the JavaScript SDK.
Where the decision model runs
Every Strands Decider host takes the same request. So
decider()takesendpointor one transport:endpoint=transport=sagemaker_transport(endpoint_name)InvokeEndpointsagemaker:InvokeEndpointtransport=lambda_transport(function_name, qualifier=)Invokelambda:InvokeFunctiontransport=agentcore_transport(runtime_arn)InvokeAgentRuntimebedrock-agentcore:InvokeAgentRuntimeboto3 is already a dependency of this package. So the transports need no new dependency.
Python-specific behavior
AiError, which extendsDurableOperationErrorand is registered with the SDK. So the call raises the same class on the first run and on replay.DeciderError.statusandLlmStoppedError.stop_reasonare stored in the error data, so they survive replay. Any other step failure stays aStepError.AiTimeoutError. The MicroVM operations use the same pattern.TypeErrororValueErrorbefore any step starts.Testing
pyteston the extras package: 563 passed (127 new). The new module has 97% line coverage.Stubber, the error mapping, replay of each error class, the deadline, and a local HTTP server forendpoint.mypyon the extras package andhatch fmt --linter --checkpass.hatch fmt --checkreports one file,tests/compute/microvm/aws_errors_test.py, which this PR does not change.Reference:
packages/aws-durable-execution-sdk-python-extras/docs/ai.md.