Repository navigation
fix(models): resolve Bedrock auto caching for Nova, OpenAI GPT, and xAI (Python) - #6149
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Assessment: Comment This port matches TS #6082 closely: caching info by model id, the Nova TTL and tools-point drops, placing the point only after text, and the tool-loop anchor all behave the same. I checked the tool-loop, guardContent, per-call tail and deprecated Review Categories
The traceability matrix and decision log made this port easy to check against the TS source. |
|
@agent-of-mkmeral review pls. is this a good port? any blockers? |
|
TL;DR: Yes, this is a good port — behaviour-faithful to #6082, not a transliteration. No code blockers. The only thing holding merge is the What I checked
🟢 Non-blocking notes1. Warning channel split. The new drops use 2. Open question 1 is a TS quirk, faithfully reproduced. 3. Multi-turn chat on Nova. Confirmed: turn 1 → point on Recommend approve + merge once the second approval lands. The inline nits (whole-object asserts, splitting |
Requested by: @jimbrub — port #6082 to Python
Human Overview
Resolving "auto" mode for all Bedrock models in Python.
Description
Python port of #6082. With
cache_configset, PythonBedrockModelnow does the same caching on each Bedrock model as TS:prompt_caching="auto"). No cache points are sent and the false "does not support automatic caching" warning is gone.Six existing tests used Nova as the stand-in for a model without caching. They now use Llama, as the TS PR did.
Related Issues
Refs #6082 (source), #5126, #1571
Documentation PR
None. The source PR shipped no docs either; the
site/prompt-caching update it lists as a follow-up covers both SDKs.Type of Change
Bug fix
Testing
tests/strands/models/: 2049 passed. The 66 ported and changed tests all pass.ruff format --checkandruff checkare clean.main(env stubs), none inbedrock.py.cache_config=CacheConfig(strategy="auto"). Every Nova model is accepted and reads from cache. Nova Pro is accepted even withttl="1h"andtools_ttl=True. Claude still caches.main(cache read)ttl="1h",tools_ttl=True)mainmainhatch run prepare(not run: the gates above were run individually in a uv venv)Additional Details
Traceability matrix: source test → Python test
TS line numbers are for
strands-ts/src/models/__tests__/at 42c4e82. Python line numbers are forstrands-py/tests/strands/models/test_bedrock.py. All Python rows pass.bedrock-cache.test.ts:4bedrockCacheModelInfo (15 ids):4588test_converse_cache_model_info(same 15)bedrock-cache.test.ts:26bedrockLimitsCachePoints (9 ids):4608test_converse_limits_cache_points(same 9)bedrock.test.ts:460leaves caching unknown for other ids (Nova → Llama):4468test_model_info_leaves_caching_unknown_for_other_model_ids:518OpenAI GPT or xAI implicitly cached, no point, no warning:4523test_model_info_reports_openai_gpt_and_xai_models_as_implicitly_cached:536warns that gpt-oss has no automatic caching:4540test_gpt_oss_has_no_automatic_caching_and_warns:553places cache points on an implicitly cached OpenAI model under'anthropic':4556test_anthropic_strategy_places_cache_points_on_an_implicitly_cached_openai_model:3800Nova caches system + conversation, not tools, no warning:4635test_nova_caches_system_and_conversation_but_not_tools_without_a_warning:3815drops a configured TTL from every point and warns:4652test_nova_drops_a_configured_ttl_from_every_cache_point_and_warns:3825drops a systemPromptTTL and warns:4666test_nova_drops_a_system_prompt_ttl_and_warns:3835drops the TTL of a caller-placed system point:4678test_nova_drops_the_ttl_of_a_caller_placed_system_cache_point_and_warns:3846warns that an explicit toolsTTL has no effect:4691test_nova_warns_that_an_explicit_tools_ttl_has_no_effectcache_tools):4707test_nova_drops_the_deprecated_cache_tools_point_and_warns:3859applies the Nova rules under'anthropic':4718test_nova_rules_apply_under_the_anthropic_strategy:3876keeps the point on the prompt that started a tool loop:4735test_nova_keeps_the_conversation_cache_point_on_the_prompt_that_started_a_tool_loop:3900keeps the point on the prompt when a per-call tail follows a tool result:4754test_nova_keeps_the_conversation_cache_point_on_the_prompt_when_a_per_call_tail_follows_a_tool_result:3914moves a caller point with no text ahead to the prompt, warns:4768test_nova_moves_a_caller_placed_cache_point_with_no_text_ahead_of_it_to_the_prompt_and_warns:3949point directly after text, ahead of image / PDF / image+txt / video:4795test_nova_places_the_cache_point_directly_after_the_text_ahead_of_media(4 cases):3959moves a caller point after an image back to the text:4807test_nova_moves_a_caller_placed_cache_point_after_an_image_back_to_the_text:3977keeps the point after an image on Claude:4820test_claude_keeps_the_cache_point_after_an_image:3990no conversation point when the only user text is in guardContent:4830test_nova_places_no_conversation_cache_point_when_the_only_user_text_is_guarded:4004moves a caller point off a tool result back, never past the boundary:4848test_nova_moves_a_caller_placed_cache_point_off_a_tool_result_back_to_the_prompt:4024caches a plain-text user message after earlier tool calls:4865test_nova_caches_a_plain_text_user_message_that_follows_earlier_tool_calls:4039keeps the tools point and TTL on Claude:4881test_claude_keeps_the_tools_cache_point_and_ttl(tools_ttl=True, see decision log)vercel.test.ts(Vercel → Bedrock)Existing tests switched from Nova to Llama:
:4454,:4913,:5400,:6441,:6484,:6873.:4913is the override test. Nova now defaults tocache_points, so the override was a no-op on Nova. TSbedrock.test.ts:573still uses Nova.Runner, lint and type-check output
The 7 mypy errors are in
_exception_notes.py,hooks/andvended_memory_stores/, and they are identical onmain. They come from local stubs and the forced--python-version 3.13: numpy's stubs fail under the configured 3.10 on this 3.13 venv.Decision log
warnings.warn(..., stacklevel=1)in place of TSwarnOnce(logger, …). This is the house Python pattern for once per process (anthropic.py,model.py). The registry key includes the message text, so each model id warns once. The existing "does not support automatic caching" warning stayslogger.warning, unchanged.cache_toolspoint is dropped on Nova whencache_configresolves to cache points, with the same tool-level warning (tools_ttl=<value>). TS has nocache_tools; without this, Nova would get a toolConfig point it rejects. Nova withcache_toolsand nocache_configis a manual opt-in and is unchanged, like the TS manual opt-in in fix(models): resolve Bedrock auto caching for Nova, OpenAI GPT, and xAI #6082 open question 3.bedrock.pyas private module functions (_converse_cache_model_info,_converse_limits_cache_points,_nova_cache_point_position,_step_back_to_text,_step_back_over_documents). TS has a separatebedrock-cache.tsbecause two providers share it; Python has one consumer. The Nova id list stays out ofshared/model-info.json, as in TS._autoCachePointIdxrefactor not mirrored: it has no behaviour change, so the Python auto-placement code is untouched.tools_ttl=True: Python's tools point needstools_ttlorcache_tools, which predates this change.Dependencies and sensitive surface
No new dependencies or imports in
src. The tests add only the stdlibreandwarnings. No network, credential, subprocess, env-var or deserialization changes. The diff touches onlystrands-py/src/strands/models/bedrock.pyand its test file.Open questions
[cachePoint, text]): Nova ends up with no conversation point, while Claude falls back to one at the end of the message. That's because blocks after a caller's boundary count as per-call. TS has the samemax(trailingBlocks, content.length - placedIdx)code, so this matches TS. Should both move?hatch run preparenot run: the gates were run individually (above). A Python 3.10 env with theallextras failed to build in the sandbox, so hatch was not used.