Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/blog/2025_01_27.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ When examining this request, you may discover that occasionally the Adversarial

[^8]: "PyRIT - SearchReplaceConverter", ../api/pyrit_prompt_converter.md#searchreplaceconverter

[^9]: "PyRIT - True False Scoring", ../code/scoring/2_true_false_scorers.ipynb#true-false-scoring
[^9]: "PyRIT - True False Scoring", ../code/scoring/1_true_false_scorers.ipynb

### Final Thoughts

Expand Down
4 changes: 2 additions & 2 deletions doc/blog/2026_04_14_scoring_scorers.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ flowchart TB

There are a few different ways to view metrics for specific scoring configurations.

**Directly on a scorer instance:** Call `get_scorer_metrics()` on any scorer object to look up its saved metrics (if they exist), as described at the bottom of the [Scorer Evaluation Identifier](#scorer-evaluation-identifier) section above. See the [scorer metrics notebook](../code/scoring/7_scorer_metrics.ipynb) to try it yourself!
**Directly on a scorer instance:** Call `get_scorer_metrics()` on any scorer object to look up its saved metrics (if they exist), as described at the bottom of the [Scorer Evaluation Identifier](#scorer-evaluation-identifier) section above. See the [scorer metrics notebook](../code/scoring/4_scorer_metrics.ipynb) to try it yourself!

**Automatically in scenario output:** When running scenarios and printing results (i.e., in [pyrit_scan](../scanner/1_pyrit_scan.ipynb) or [pyrit_shell](../scanner/2_pyrit_shell.md)), metrics are automatically fetched and displayed alongside the attack results (as long as the scoring configuration has been evaluated before):

Expand All @@ -132,7 +132,7 @@ The framework checks the JSONL registry for an existing entry matching the score

![alt text](2026_04_14_running_evaluation.png)

For the full walkthrough — including running objective and harm evaluations, configuring custom datasets, and comparing results — give the [scorer metrics notebook](../code/scoring/7_scorer_metrics.ipynb) a try!
For the full walkthrough — including running objective and harm evaluations, configuring custom datasets, and comparing results — give the [scorer metrics notebook](../code/scoring/4_scorer_metrics.ipynb) a try!

## Closing Thoughts

Expand Down
2 changes: 1 addition & 1 deletion doc/code/framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Ways to contribute: Check out our [target docs](./targets/0_prompt_targets.md).

The scoring engine is a component that gives feedback to the attack on what happened with the prompt. This could be as simple as "Was this prompt blocked?" or "Was our objective achieved?"

Ways to contribute: Check out our [scoring docs](./scoring/0_scoring.md). Is there data you want to use to make decisions or analyze?
Ways to contribute: Check out our [scoring docs](./scoring/0_scoring.ipynb). Is there data you want to use to make decisions or analyze?

## Memory

Expand Down
2 changes: 1 addition & 1 deletion doc/code/memory/5_advanced_memory.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@
"id": "2",
"metadata": {},
"source": [
"Because you have labeled `group1`, you can retrieve these prompts later. For example, you could score them as shown [here](../scoring/6_batch_scorer.ipynb). Or you could resend them as shown below; this script will resend any prompts with the label regardless of modality."
"Because you have labeled `group1`, you can retrieve these prompts later. For example, you could score them as shown [here](../scoring/0_scoring.ipynb#batch-scoring). Or you could resend them as shown below; this script will resend any prompts with the label regardless of modality."
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion doc/code/memory/5_advanced_memory.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
await output_attack_async(result)

# %% [markdown]
# Because you have labeled `group1`, you can retrieve these prompts later. For example, you could score them as shown [here](../scoring/6_batch_scorer.ipynb). Or you could resend them as shown below; this script will resend any prompts with the label regardless of modality.
# Because you have labeled `group1`, you can retrieve these prompts later. For example, you could score them as shown [here](../scoring/0_scoring.ipynb#batch-scoring). Or you could resend them as shown below; this script will resend any prompts with the label regardless of modality.

# %%
from pyrit.executor.attack import AttackConverterConfig
Expand Down
Loading
Loading