FEAT text adaptive scenario

[hannahwestra25]

Add Adaptive Scenario Framework with TextAdaptive

Summary

Adds an adaptive scenario framework that picks attack techniques per-objective using an epsilon-greedy bandit informed by observed success rates, instead of running every technique against every objective. Concentrates spend on what works against the target and stops on first success — O(max_attempts × objectives) instead of O(techniques × objectives).

Includes:

• AdaptiveScenario (modality-agnostic base) + TextAdaptive (text subclass)
• TechniqueSelector Protocol + EpsilonGreedyTechniqueSelector (Laplace-smoothed, pooled cold-start backoff, thread-safe)
• AdaptiveTechniqueDispatcher — a thin factory (not an AttackStrategy) that selects techniques up-front per objective and returns a ready-to-run SequentialAttack(FIRST_SUCCESS)
• Walkthrough notebook + unit tests across selector, dispatcher, scenario, and analytics

How it works

For each objective:

1. Select — ε-greedy over Laplace-smoothed (s+1)/(n+1) estimates; cells with fewer than pool_threshold observations back off to the technique's pooled rate. Each decision derives a per-decision RNG from SHA-256(seed|context|decision_key) for resume-safe reproducibility.
2. Wrap — the chosen max_attempts_per_objective techniques become children of one SequentialAttack(FIRST_SUCCESS), wrapped in one AtomicAttack. Selection happens once at initialize_async; execution is plain framework code.
3. Record — selector updates (context, technique) → (s, n) from persisted child rows after the run.

from pyrit.scenario.scenarios.adaptive import TextAdaptive, EpsilonGreedyTechniqueSelector

scenario = TextAdaptive(
    selector=EpsilonGreedyTechniqueSelector(epsilon=0.3, random_seed=42),
)
scenario.set_params_from_args(args={"max_attempts_per_objective": 5})
await scenario.initialize_async(objective_target=target)
result = await scenario.run_async()

Resumable via scenario_result_id="..." — prior dispatch trails replay into the selector before the remainder runs.

Notes

• One AtomicAttack per objective — atomic_attack_name = "{prefix}_{dataset}::{objective_sha[:12]}", display_group = dataset_name preserves dataset grouping for reporting.
• All dispatchers share one selector — learning accumulates globally across datasets.
• BASELINE_ATTACK_POLICY = Enabled — prompt_sending runs as the baseline comparison rather than as an adaptive technique.
• Selector is a constructor kwarg — selector-specific params (epsilon, pool_threshold, random_seed) live on the selector; max_attempts_per_objective is a scenario parameter.

Updates since initial review

• Dispatcher is a plain factory (not an AttackStrategy) — execution and persistence are entirely the executor's responsibility via SequentialAttack (depends on FEAT add StrategySequenceAttack compound attack primitive #1819). No custom envelope subclass, no run_async override.
• No custom metadata stamping (addresses thread A) — the per-attempt trail is reconstructed from each child's auto-stamped atomic_attack_identifier; the notebook reads child.get_attack_strategy_identifier().unique_name directly. New MemoryInterface.get_attack_results(atomic_attack_eval_hashes=...) kwarg + compute_inner_attack_eval_hash helper (regression-tested so predictor and executor never diverge).
• Per-factory scoring config — AdaptiveScenario constructs each factory's narrowed AttackScoringConfig subtype (e.g. TAPAttackScoringConfig); incompatible techniques are skipped with a warning instead of failing the run.
• SeedAttackGroup.with_technique — deep-copies merged seeds so reused seed_techniques don't leak prompt_group_id mutations.
• CLI ergonomics — new --request-timeout; poll endpoint uses read=None so a busy server doesn't fail the run; httpx.ReadTimeout now points at the flag instead of a bare Error:.

Testing

pytest tests/unit/{scenario/scenarios/adaptive,analytics,identifiers,memory,cli} — all green.

[rlundeen2]

Looking great!