Skip to content

FEAT new target class for AWS Bedrock Anthropic Claude models - #699

Closed
kmarsh77 wants to merge 49 commits into
microsoft:mainfrom
kmarsh77:main
Closed

kmarsh77 wants to merge 49 commits into
microsoft:mainfrom
kmarsh77:main

Conversation

@kmarsh77

@kmarsh77 kmarsh77 commented Feb 7, 2025

Copy link
Copy Markdown
Contributor

Description

Adding a new target class for AWS Bedrock Anthropic Claude models. It will only work for Anthropic Claude models as the request body is specific to those, but it should be easy to modify the class for use with other Bedrock models.

boto3 Python library is used for sending requests. Local AWS credentials are used for authentication, typically stored in ~/.aws.

Tests and Documentation

Description of target class and parameters is provided in the class file, aws_bedrock_claude_target.py. Unit tests are provided in tests/unit/test_aws_bedrock_claude_target.py.

@kmarsh77

kmarsh77 commented Feb 7, 2025 via email

Copy link
Copy Markdown
Contributor Author

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks very good! I left a few comments, and you likely have complaints from our pre-commit hooks. You can run pre-commit run --all-files to fix that locally.

Comment thread pyrit/prompt_target/aws_bedrock_claude_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_target.py Outdated
use of aws_bedrock_claude_target.py target class requires the boto3 library
Deleting unnecessary commented out line
Adding support for multi-turn
Comment thread pyproject.toml Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_chat_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_chat_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_chat_target.py
Comment thread pyrit/prompt_target/aws_bedrock_claude_chat_target.py Outdated
Comment thread pyrit/prompt_target/aws_bedrock_claude_chat_target.py Outdated
@romanlutz

Copy link
Copy Markdown
Contributor

Related: lots of cloud providers like Azure, Google, and other local model providers like ollama use openai compatible APIs so we don't need anything custom. Looks like AWS is a bit behind on that front. I only see workarounds where people have deployed additional things to make that conversion work. Whenever they do become compatible I'd love to just move over to OpenAIChatTarget, though.

@kmarsh77

Copy link
Copy Markdown
Contributor Author

Roman Lutz (@romanlutz) I am actually unable to replicate the import error relating to boto3, could you please share the command you are using?

@kmarsh77

Copy link
Copy Markdown
Contributor Author

Roman Lutz (@romanlutz) alright I fixed the one error which was invalid value for converted_value_data_type in test_validate_request_invalid_data_type().

The other error is being caused by test_send_prompt_async() and test_complete_chat_async() not seeing an import for boto3 in the target class file (because the import is within try/except block in the initialization function rather than at the top of the file).

I'm not sure how to get around this but will think on it. Let me know if you have any ideas.

@romanlutz

Copy link
Copy Markdown
Contributor

Roman Lutz (@romanlutz) alright I fixed the one error which was invalid value for converted_value_data_type in test_validate_request_invalid_data_type().

The other error is being caused by test_send_prompt_async() and test_complete_chat_async() not seeing an import for boto3 in the target class file (because the import is within try/except block in the initialization function rather than at the top of the file).

I'm not sure how to get around this but will think on it. Let me know if you have any ideas.

I'll try to repro it sometime this week. Thank you!

@kmarsh77

Copy link
Copy Markdown
Contributor Author

Roman Lutz (@romanlutz) finally had some time to look at this; I think I found the solution in add_image_to_video_converter.py. I just moved the boto3 import to within the complete_chat_async method; this also allowed me to remove the "if TYPE_CHECKING" statement from the top of the file. Let me know if it looks okay to you.

@romanlutz

Copy link
Copy Markdown
Contributor

Roman Lutz (@romanlutz) finally had some time to look at this; I think I found the solution in add_image_to_video_converter.py. I just moved the boto3 import to within the complete_chat_async method; this also allowed me to remove the "if TYPE_CHECKING" statement from the top of the file. Let me know if it looks okay to you.

Great timing! I was going to try this out anyway. Will report back by EOW

@romanlutz

Copy link
Copy Markdown
Contributor

Sadly, still stuck on AWS sign-up because I'm not going to give them my phone number (nothing to do with AWS, but it's a company account so it shouldn't be associated with my number). Will update when that is resolved.

@romanlutz

Copy link
Copy Markdown
Contributor

kmarsh77 FYI I've started looking into this. Got an account now. The main problem for me right now is that it's not working with key-based auth (in addition to identity-based auth) yet so that's preventing us from adding integration tests. I'm working on updating it that way. It may take a little bit of time since I'm doing that on the side. Thanks for your patience.

@kmarsh77

Copy link
Copy Markdown
Contributor Author

Roman Lutz (@romanlutz) hey just checking in on this. Can you please clarify what you mean by "not working with key-based auth"? I might be able to help.

@romanlutz

Copy link
Copy Markdown
Contributor

I assumed I'd be able to use an API key which I now understand is wrong. I have yet to try out the identity-based version you implemented. It took us a while to get approval for an AWS subscription. Apologies for the delay.

@kmarsh77

kmarsh77 commented Feb 3, 2026

Copy link
Copy Markdown
Contributor Author

I'm closing this so I can update my fork for contributing websocket target class. Let me know if there's any interest in updating this target class to work with newer release.

@kmarsh77 kmarsh77 closed this Feb 3, 2026
@romanlutz

Copy link
Copy Markdown
Contributor

I'm not sure I understand what one target has to do with the other since you can have different branches.

I'm sorry we haven't made any progress in validating and providing feedback.

@kmarsh77

kmarsh77 commented Feb 4, 2026

Copy link
Copy Markdown
Contributor Author

yeah fair enough, I just think it would be more useful anyways if I adapted it to work for all bedrock models rather than just claude. Definitely something I can work on in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants