MAINT: migrate AdversarialBenchmark off deprecated PromptChatTarget#1746
Merged
romanlutz merged 1 commit intoMay 18, 2026
Merged
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hannahwestra25
approved these changes
May 18, 2026
Contributor
|
thanks ! was this caught by our integration tests ? I could be missing something but it seems like we don't have any integration tests running the doc/scanner notebooks |
Contributor
Author
|
@hannahwestra25 copilot found this while doing a general assessment of the repo. It said something like it's a result of your PR (which removed it) merging around the same time as the adversarial benchmark PR got merged. I don't think the integration test was run because those happen outside of our PR gates. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
pyrit/scenario/scenarios/benchmark/adversarial.pystill typed its publicadversarial_modelsparameter aslist[PromptChatTarget].PromptChatTargetwas deprecated in #1678 with removal scheduled for v0.16.0, so without this migration the publicAdversarialBenchmark(adversarial_models=...)API will break at import time once v0.16 ships.Why it slipped
Merge race between #1662 (
Adversarial Benchmark Scenario, merged 2026-05-07) and the surrounding deprecation work:TargetConfigurationcapability checks #1645 (2026-04-30) migrated every other consumer toCHAT_TARGET_REQUIREMENTS.validate(target=...).No follow-up PR addresses it; this is the only remaining
PromptChatTargetreference inpyrit/outside the deprecated shim itself.Changes
adversarial_modelsaslist[PromptTarget].CHAT_TARGET_REQUIREMENTSin__init__so callers get a clear, early error if a non-chat target sneaks in. Per-technique stricter requirements (e.g.SYSTEM_PROMPTforRedTeamingAttack) continue to be enforced atfactory.create()time, unchanged.TargetConfiguration(multi-turn, multi-message-pieces, system-prompt, editable-history — matching whatPromptChatTarget._DEFAULT_CONFIGURATIONadvertised) so the existing test suite still exercises the techniques that demand these capabilities.adversarial_modelsentry missing chat-target capabilities is rejected at construction.doc/scanner/benchmark.{py,ipynb}comments that referenced "PromptChatTargets".Tests and Documentation
uv run pytest tests/unit/scenario/test_adversarial.py— 34 passeduv run pytest tests/unit/scenario/— 579 passeduv run ruff format --check+ruff check+ty (type check)— pass via pre-commitdoc/scanner/benchmark.{py,ipynb}updated to the new wording.