Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
19b32d0
Use openai SDK in OpenAIChatTarget and OpenAIResponseTarget
romanlutz Nov 18, 2025
997041a
migrate to openai SDK with chat and response targets
romanlutz Nov 19, 2025
cc3f8e8
remove unnecessary file
romanlutz Nov 19, 2025
431dac5
migrate remaining openai targets
romanlutz Nov 20, 2025
37110b1
finished migrations, removed chat target base, updated tests (all pas…
romanlutz Nov 20, 2025
f9bdc30
adding a few additional unit tests
romanlutz Nov 20, 2025
32084a2
unified error handling for chat and response
romanlutz Nov 21, 2025
42b1858
all tests working again post error handling unification (chat and res…
romanlutz Nov 21, 2025
069244a
fix one failing unit test
romanlutz Nov 21, 2025
479ce9e
working unit and integration tests (ollama pending)
romanlutz Nov 24, 2025
182829a
pre-commit (except local error for website build)
romanlutz Nov 24, 2025
a7eca64
move construct fn into base class
romanlutz Nov 24, 2025
e08f9ba
integration test updates
romanlutz Nov 25, 2025
3e8d44a
Merge branch 'main' of https://github.com/Azure/PyRIT into romanlutz/…
romanlutz Nov 25, 2025
d832eac
fix responses API handling of tool/function calls, more tests
romanlutz Nov 25, 2025
b25fbc6
Merge branch 'main' of https://github.com/Azure/PyRIT into romanlutz/…
romanlutz Nov 25, 2025
6534ea9
response target notebook running
romanlutz Nov 26, 2025
d20f1da
dalle -> image, sora -> video (for generic target naming)
romanlutz Nov 26, 2025
28d9e81
file reference fix
romanlutz Nov 26, 2025
846d4f5
realtime working (audio not tested yet)
romanlutz Nov 29, 2025
b184e3f
realtime working on all cases
romanlutz Nov 29, 2025
34355c5
remove obsolete auth header code
romanlutz Nov 29, 2025
dfb4839
multiple messages returned to normalizer, unit tests passing
romanlutz Dec 1, 2025
c515493
integration tests passing
romanlutz Dec 1, 2025
b92f6ca
Merge branch 'main' of https://github.com/Azure/PyRIT into romanlutz/…
romanlutz Dec 1, 2025
1124a89
fix test typo
romanlutz Dec 1, 2025
64b9224
ensuring executors handle content filter errors properly, addressing …
romanlutz Dec 2, 2025
434fd3b
miscellaneous pre-commit fixes
romanlutz Dec 2, 2025
7ad51e6
add old-style -> new-style URL conversion
romanlutz Dec 2, 2025
f460c86
Merge branch 'main' of https://github.com/Azure/PyRIT into romanlutz/…
romanlutz Dec 2, 2025
282a796
pre-commit
romanlutz Dec 2, 2025
fe174e9
unify url validation as best as possible, make target specific, add u…
romanlutz Dec 3, 2025
5fdc48b
unify temperature validation and fix other failing tests
romanlutz Dec 3, 2025
0568e14
clean up response target internals that translate messages into requests
romanlutz Dec 3, 2025
165f209
allow multiple system messages in responses api
romanlutz Dec 3, 2025
92142e8
move imports to the top
romanlutz Dec 4, 2025
a322427
combine imports
romanlutz Dec 4, 2025
70bb640
warn for non-conforming URLs, update XPIA notebook (not yet working)
romanlutz Dec 4, 2025
b24af6f
Merge branch 'main' of https://github.com/Azure/PyRIT into romanlutz/…
romanlutz Dec 4, 2025
dc9739d
xpia and prompt sending cookbook working
romanlutz Dec 4, 2025
f6205ab
fix env vars for consistency
romanlutz Dec 4, 2025
808a916
unit tests passing
romanlutz Dec 4, 2025
f28f1f1
small doc fix
romanlutz Dec 4, 2025
ab05d55
recreate broken notebook
romanlutz Dec 4, 2025
c5639a5
clean up image target, run notebooks for validation, many fixes
romanlutz Dec 4, 2025
461893d
pre-commit
romanlutz Dec 5, 2025
7bbf3e7
final pre-commit
romanlutz Dec 5, 2025
0d13b57
update ollama url
romanlutz Dec 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 49 additions & 41 deletions .env_example
Original file line number Diff line number Diff line change
Expand Up @@ -10,42 +10,46 @@
# or copy to OPENAI_CHAT_ENDPOINT
###################################

PLATFORM_OPENAI_CHAT_ENDPOINT="https://api.openai.com/v1/chat/completions"
PLATFORM_OPENAI_CHAT_ENDPOINT="https://api.openai.com/v1"
PLATFORM_OPENAI_CHAT_API_KEY="sk-xxxxx"
PLATFORM_OPENAI_CHAT_GPT4O_MODEL="gpt-4o"

# Note: For Azure OpenAI endpoints, include the api-version query parameter in the URL
# Example: https://xxxx.openai.azure.com/openai/deployments/xxxxx/chat/completions?api-version=2024-10-21
AZURE_OPENAI_GPT4O_ENDPOINT="https://xxxx.openai.azure.com/openai/deployments/xxxxx/chat/completions?api-version=2024-10-21"
# Note: For Azure OpenAI endpoints, use the new format with /openai/v1 and specify the model separately
# Example: https://xxxx.openai.azure.com/openai/v1
AZURE_OPENAI_GPT4O_ENDPOINT="https://xxxx.openai.azure.com/openai/v1"
AZURE_OPENAI_GPT4O_KEY="xxxxx"
AZURE_OPENAI_GPT4O_MODEL="deployment-name"

AZURE_OPENAI_INTEGRATION_TEST_ENDPOINT="https://xxxxx.openai.azure.com/openai/deployments/xxxx/chat/completions?api-version=2024-10-21"
AZURE_OPENAI_INTEGRATION_TEST_ENDPOINT="https://xxxxx.openai.azure.com/openai/v1"
AZURE_OPENAI_INTEGRATION_TEST_KEY="xxxxx"
AZURE_OPENAI_INTEGRATION_TEST_MODEL="deployment-name"

AZURE_OPENAI_GPT3_5_CHAT_ENDPOINT="https://xxxxx.openai.azure.com/openai/deployments/xxxxx/chat/completions?api-version=2024-10-21"
AZURE_OPENAI_GPT3_5_CHAT_ENDPOINT="https://xxxxx.openai.azure.com/openai/v1"
AZURE_OPENAI_GPT3_5_CHAT_KEY="xxxxx"
AZURE_OPENAI_GPT3_5_CHAT_MODEL="deployment-name"

AZURE_OPENAI_GPT4_CHAT_ENDPOINT="https://xxxxx.openai.azure.com/openai/deployments/xxxxx/chat/completions?api-version=2024-10-21"
AZURE_OPENAI_GPT4_CHAT_ENDPOINT="https://xxxxx.openai.azure.com/openai/v1"
AZURE_OPENAI_GPT4_CHAT_KEY="xxxxx"
AZURE_OPENAI_GPT4_CHAT_MODEL="deployment-name"

AZURE_FOUNDRY_DEEPSEEK_ENDPOINT="https://xxxxx.eastus2.models.ai.azure.com/chat/completions"
AZURE_FOUNDRY_DEEPSEEK_ENDPOINT="https://xxxxx.eastus2.models.ai.azure.com"
AZURE_FOUNDRY_DEEPSEEK_KEY="xxxxx"

AZURE_FOUNDRY_PHI4_ENDPOINT="https://xxxxx.models.ai.azure.com/chat/completions"
AZURE_FOUNDRY_PHI4_ENDPOINT="https://xxxxx.models.ai.azure.com"
AZURE_CHAT_PHI4_KEY="xxxxx"

AZURE_FOUNDRY_MINSTRAL3B_ENDPOINT="https://xxxxx.eastus2.models.ai.azure.com/chat/completions"
AZURE_FOUNDRY_MINSTRAL3B_ENDPOINT="https://xxxxx.eastus2.models.ai.azure.com"
AZURE_CHAT_MINSTRAL3B_KEY="xxxxx"

GROQ_ENDPOINT="https://api.groq.com/openai/v1/chat/completions"
GROQ_ENDPOINT="https://api.groq.com/openai/v1"
GROQ_KEY="gsk_xxxxxxxx"
GROQ_LLAMA_MODEL="llama3-8b-8192"

OPEN_ROUTER_ENDPOINT="https://openrouter.ai/api/v1/chat/completions"
OPEN_ROUTER_ENDPOINT="https://openrouter.ai/api/v1"
OPEN_ROUTER_KEY="sk-or-v1-xxxxx"
OPEN_ROUTER_CLAUDE_MODEL="anthropic/claude-3.7-sonnet"

OLLAMA_CHAT_ENDPOINT="http://127.0.0.1:11434/v1/chat/completions"
OLLAMA_CHAT_ENDPOINT="http://127.0.0.1:11434/v1"
OLLAMA_MODEL="llama2"

DEFAULT_OPENAI_FRONTEND_ENDPOINT = ${AZURE_OPENAI_GPT4O_AAD_ENDPOINT}
Expand All @@ -60,16 +64,16 @@ OPENAI_CHAT_MODEL=${PLATFORM_OPENAI_CHAT_GPT4O_MODEL}
# OPENAI RESPONSES TARGET SECRETS
##################################

AZURE_OPENAI_GPT5_RESPONSES_ENDPOINT="https://xxxxxxxxx.azure.com/openai/deployments/gpt-5/responses?api-version=2025-03-01-preview"
AZURE_OPENAI_GPT5_COMPLETION_ENDPOINT="https://xxxxxxxxx.azure.com/openai/deployments/gpt-5/chat/completions?api-version=2025-01-01-preview"
AZURE_OPENAI_GPT5_RESPONSES_ENDPOINT="https://xxxxxxxxx.azure.com/openai/v1"
AZURE_OPENAI_GPT5_COMPLETION_ENDPOINT="https://xxxxxxxxx.azure.com/openai/v1"
AZURE_OPENAI_GPT5_KEY="xxxxxxx"
AZURE_OPENAI_GPT5_MODEL="gpt-5"

PLATFORM_OPENAI_RESPONSES_ENDPOINT="https://api.openai.com/v1/responses"
PLATFORM_OPENAI_RESPONSES_ENDPOINT="https://api.openai.com/v1"
PLATFORM_OPENAI_RESPONSES_KEY="sk-xxxxx"
PLATFORM_OPENAI_RESPONSES_MODEL="o4-mini"

AZURE_OPENAI_RESPONSES_ENDPOINT="https://xxxxx.openai.azure.com/openai/responses?api-version=2025-03-01-preview"
AZURE_OPENAI_RESPONSES_ENDPOINT="https://xxxxx.openai.azure.com/openai/v1"
AZURE_OPENAI_RESPONSES_KEY="xxxxx"
AZURE_OPENAI_RESPONSES_MODEL="o4-mini"

Expand All @@ -84,11 +88,11 @@ OPENAI_RESPONSES_MODEL=${PLATFORM_OPENAI_RESPONSES_MODEL}
# or copy to OPENAI_REALTIME_ENDPOINT
##################################

PLATFORM_OPENAI_REALTIME_ENDPOINT="wss://api.openai.com/v1/realtime"
PLATFORM_OPENAI_REALTIME_ENDPOINT="wss://api.openai.com/v1"
PLATFORM_OPENAI_REALTIME_API_KEY="sk-xxxxx"
PLATFORM_OPENAI_REALTIME_MODEL="gpt-4o-realtime-preview"

AZURE_OPENAI_REALTIME_ENDPOINT = "wss://xxxx.openai.azure.com/openai/realtime?api-version=2024-10-01-preview"
AZURE_OPENAI_REALTIME_ENDPOINT = "wss://xxxx.openai.azure.com/openai/v1"
AZURE_OPENAI_REALTIME_API_KEY = "xxxxx"
AZURE_OPENAI_REALTIME_MODEL = "gpt-4o-realtime-preview"

Expand All @@ -97,20 +101,22 @@ OPENAI_REALTIME_API_KEY = ${PLATFORM_OPENAI_REALTIME_API_KEY}
OPENAI_REALTIME_MODEL = ${PLATFORM_OPENAI_REALTIME_MODEL}

##################################
# DALLE TARGET SECRETS
# IMAGE TARGET SECRETS
#
# The below models work with OpenAIDALLETarget - either pass via environment variables
# or copy to OPENAI_DALLE_ENDPOINT
# The below models work with OpenAIImageTarget - either pass via environment variables
# or copy to OPENAI_IMAGE_ENDPOINT
###################################

OPENAI_DALLE_ENDPOINT1 = "https://xxxxx.openai.azure.com/openai/deployments/xxxxx/images/generations?api-version=2024-10-21"
OPENAI_DALLE_API_KEY1 = "xxxxxx"
OPENAI_IMAGE_ENDPOINT1 = "https://xxxxx.openai.azure.com/openai/v1"
OPENAI_IMAGE_API_KEY1 = "xxxxxx"
OPENAI_IMAGE_MODEL1 = "deployment-name"

OPENAI_DALLE_ENDPOINT2 = "https://api.openai.com/v1/images/generations"
OPENAI_DALLE_API_KEY2 = "sk-xxxxx"
OPENAI_IMAGE_ENDPOINT2 = "https://api.openai.com/v1"
OPENAI_IMAGE_API_KEY2 = "sk-xxxxx"
OPENAI_IMAGE_MODEL2 = "dall-e-3"

OPENAI_DALLE_ENDPOINT = ${OPENAI_DALLE_ENDPOINT2}
OPENAI_DALLE_API_KEY = ${OPENAI_DALLE_API_KEY2}
OPENAI_IMAGE_ENDPOINT = ${OPENAI_IMAGE_ENDPOINT2}
OPENAI_IMAGE_API_KEY = ${OPENAI_IMAGE_API_KEY2}


##################################
Expand All @@ -120,29 +126,31 @@ OPENAI_DALLE_API_KEY = ${OPENAI_DALLE_API_KEY2}
# or copy to OPENAI_TTS_ENDPOINT
###################################

OPENAI_TTS_ENDPOINT1 = "https://xxxxx.openai.azure.com/openai/deployments/tts/audio/speech"
OPENAI_TTS_ENDPOINT1 = "https://xxxxx.openai.azure.com/openai/v1"
OPENAI_TTS_KEY1 = "xxxxxxx"
OPENAI_TTS_MODEL1 = "tts"

OPENAI_TTS_ENDPOINT2 = "https://api.openai.com/v1/audio/speech"
OPENAI_TTS_ENDPOINT2 = "https://api.openai.com/v1"
OPENAI_TTS_KEY2 = "xxxxxx"
OPENAI_TTS_MODEL2 = "tts-1"

OPENAI_TTS_ENDPOINT = ${OPENAI_TTS_ENDPOINT2}
OPENAI_TTS_KEY = ${OPENAI_TTS_KEY2}

##################################
# SORA TARGET SECRETS
# VIDEO TARGET SECRETS
#
# The below models work with OpenAISoraTarget - either pass via environment variables
# or copy to OPENAI_SORA_ENDPOINT
# The below models work with OpenAIVideoTarget - either pass via environment variables
# or copy to OPENAI_VIDEO_ENDPOINT
###################################

# Note: Remove /jobs?api-version=<value> from the end of the default Target URI provided
AZURE_OPENAI_SORA_ENDPOINT="https://xxxxx.cognitiveservices.azure.com/openai/v1/video/generations"
AZURE_OPENAI_SORA_KEY="xxxxxxx"
# Note: Use the base URL without API path
AZURE_OPENAI_VIDEO_ENDPOINT="https://xxxxx.cognitiveservices.azure.com/openai/v1"
AZURE_OPENAI_VIDEO_KEY="xxxxxxx"

OPENAI_SORA_ENDPOINT = ${AZURE_OPENAI_SORA_ENDPOINT}
OPENAI_SORA_KEY = ${AZURE_OPENAI_SORA_KEY}
OPENAI_SORA_MODEL = "sora-2"
OPENAI_VIDEO_ENDPOINT = ${AZURE_OPENAI_VIDEO_ENDPOINT}
OPENAI_VIDEO_KEY = ${AZURE_OPENAI_VIDEO_KEY}
OPENAI_VIDEO_MODEL = "sora-2"


##################################
Expand All @@ -164,7 +172,7 @@ AZURE_ML_KEY=${AZURE_ML_PHI_KEY}
###################################


OPENAI_COMPLETION_ENDPOINT="https://xxxxx.openai.azure.com/openai/deployments/xxxxx/completions"
OPENAI_COMPLETION_ENDPOINT="https://xxxxx.openai.azure.com/openai/v1"
OPENAI_COMPLETION_API_KEY="xxxxx"
OPENAI_COMPLETION_MODEL="davinci-002"

Expand All @@ -188,7 +196,7 @@ CRUCIBLE_API_KEY = "xxxxx"

HUGGINGFACE_TOKEN="hf_xxxxxxx"

GOOGLE_GEMINI_ENDPOINT = "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions"
GOOGLE_GEMINI_ENDPOINT = "https://generativelanguage.googleapis.com/v1beta/openai"
GOOGLE_GEMINI_API_KEY = "xxxxx"
GOOGLE_GEMINI_MODEL="gemini-2.0-flash"

Expand Down
20 changes: 7 additions & 13 deletions .env_local_example
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,6 @@ OPENAI_CHAT_ENDPOINT=${PLATFORM_OPENAI_CHAT_ENDPOINT}
OPENAI_CHAT_KEY=${PLATFORM_OPENAI_CHAT_API_KEY}
OPENAI_CHAT_MODEL="gpt-4o"

# For the XPIA example, the formatting for semantic kernel adds /chat/completions onto the end which results in a 404 Error.
# Thus this endpoint should not have the /chat/completions at the end to avoid getting added twice.
XPIA_OPENAI_GPT4O_ENDPOINT = ""
XPIA_OPENAI_API_VERSION = ""
XPIA_OPENAI_MODEL = ""
XPIA_OPENAI_KEY=""

##############
# The below GLOBAL_MEMORY_LABELS will be applied to all prompts sent via attacks and can be altered whenever needed.
Expand All @@ -32,19 +26,19 @@ GLOBAL_MEMORY_LABELS = {"username": "username"}
#OPENAI_CHAT_ADDITIONAL_REQUEST_HEADERS="<Provide headers using a dictionary format. Ex., {'key1':'value1'}>"

##############
# OpenAI Sora-2 Target Configuration
# OpenAI Video Target Configuration
# Works with both standard OpenAI Sora-2 API and Azure OpenAI Sora-2 deployments
##############

# OpenAI Sora-2 configuration
OPENAI_SORA_ENDPOINT="https://api.openai.com/v1/videos"
OPENAI_SORA_KEY="your-openai-api-key"
OPENAI_SORA_MODEL="sora-2"
OPENAI_VIDEO_ENDPOINT="https://api.openai.com/v1"
OPENAI_VIDEO_KEY="your-openai-api-key"
OPENAI_VIDEO_MODEL="sora-2"

# For Azure OpenAI Sora-2, override with your Azure endpoint and key:
OPENAI_SORA_ENDPOINT="https://your-resource.cognitiveservices.azure.com/openai/v1/video/generations"
OPENAI_SORA_KEY="your-azure-openai-key"
OPENAI_SORA_MODEL="sora-2"
OPENAI_VIDEO_ENDPOINT="https://your-resource.cognitiveservices.azure.com/openai/v1"
OPENAI_VIDEO_KEY="your-azure-openai-key"
OPENAI_VIDEO_MODEL="sora-2"


# Specify to run tests that may be skipped in the pipeline
Expand Down
4 changes: 2 additions & 2 deletions build_scripts/env_local_integration_test
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ OPENAI_CHAT_ENDPOINT=${AZURE_OPENAI_INTEGRATION_TEST_ENDPOINT}
OPENAI_CHAT_KEY=${AZURE_OPENAI_INTEGRATION_TEST_KEY}
OPENAI_CHAT_MODEL="gpt-4o"

OPENAI_DALLE_ENDPOINT=${OPENAI_DALLE_ENDPOINT2}
OPENAI_DALLE_API_KEY=${OPENAI_DALLE_API_KEY2}
OPENAI_IMAGE_ENDPOINT=${OPENAI_IMAGE_ENDPOINT2}
OPENAI_IMAGE_API_KEY=${OPENAI_IMAGE_API_KEY2}

OPENAI_TTS_ENDPOINT=${OPENAI_TTS_ENDPOINT2}
OPENAI_TTS_KEY=${OPENAI_TTS_KEY2}
Expand Down
4 changes: 2 additions & 2 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -438,10 +438,10 @@ API Reference
HuggingFaceEndpointTarget
limit_requests_per_minute
OpenAICompletionTarget
OpenAIDALLETarget
OpenAIImageTarget
OpenAIChatTarget
OpenAIResponseTarget
OpenAISoraTarget
OpenAIVideoTarget
OpenAITTSTarget
OpenAITarget
PlaywrightTarget
Expand Down
2 changes: 1 addition & 1 deletion doc/blog/2025_03_03.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ We now loop through various models to check that they're working. This helps us
[
("AZURE_OPENAI_GPT4O_ENDPOINT", "AZURE_OPENAI_GPT4O_KEY", ""),
("AZURE_OPENAI_GPT4O_INTEGRATION_TEST_ENDPOINT", "AZURE_OPENAI_GPT4O_INTEGRATION_TEST_KEY", ""),
("AZURE_OPENAI_GPT4O_UNSAFE_ENDPOINT", "AZURE_OPENAI_GPT4O_UNSAFE_CHAT_KEY", ""),
("AZURE_OPENAI_GPT4O_UNSAFE_CHAT_ENDPOINT", "AZURE_OPENAI_GPT4O_UNSAFE_CHAT_KEY", ""),
("AZURE_OPENAI_GPT4O_UNSAFE_CHAT_ENDPOINT2", "AZURE_OPENAI_GPT4O_UNSAFE_CHAT_KEY2", ""),
("AZURE_OPENAI_GPT3_5_CHAT_ENDPOINT", "AZURE_OPENAI_GPT3_5_CHAT_KEY", ""),
("AZURE_OPENAI_GPT4_CHAT_ENDPOINT", "AZURE_OPENAI_GPT4_CHAT_KEY", ""),
Expand Down
2 changes: 1 addition & 1 deletion doc/code/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The remainder of this document talks about the different components, how they wo

The first piece of an attack is often a dataset piece, like a prompt. "Tell me how to create a Molotov cocktail" is an example of a prompt. PyRIT is a good place to have a library of things to check for.

Ways to contribute: Check out our prompts in [seed prompts](https://github.com/Azure/PyRIT/tree/main/pyrit/datasets/seed_prompts) and [jailbreak templates](https://github.com/Azure/PyRIT/tree/main/pyrit/datasets/jailbreak); are there more you can add that include scenarios you're testing for?
Ways to contribute: Check out our prompts in [seed datasets](https://github.com/Azure/PyRIT/tree/main/pyrit/datasets/seed_datasets) and [jailbreak templates](https://github.com/Azure/PyRIT/tree/main/pyrit/datasets/jailbreak); are there more you can add that include scenarios you're testing for?

## Attacks

Expand Down
Loading