Benchmark, telemetry harness and evaluation code for the paper
Automation Is Not Intent: Measuring and Fixing Fraud-Model Failures on Legitimate AI Shopping Agents Rishabh Pande, 2026. PDF · Preprint DOI: 10.5281/zenodo.23477625 · arXiv: coming soon
Fraud models learned that "bot-like" means "risky." AI shopping agents now make legitimate purchases with exactly those signals. AgentRisk measures how often fraud models decline legitimate agents, shows how the obvious fixes fail (safe harbor, laundering, hijacking), and evaluates AgentGate, a verify-scope-score policy built on agent identity, delegation and mandate scope.
Try the simulator in seconds (needs only numpy and pandas; run from the repo root):
pip install numpy pandas
python -c "import sim; df = sim.generate({'H': 200, 'AU': 200, 'AV': 200}, seed=0); print(df.groupby('cls').size().to_dict())"This generates 600 synthetic checkout sessions (200 each of legitimate humans, unverified agents, and verified agents) and prints the per-class counts. generate accepts only the documented session classes, nonnegative integer counts (with at least one positive count), and known PARAMS overrides with finite probabilities in [0, 1]. Invalid inputs raise ValueError before generation, so a typo cannot silently alter an experiment. The full experiments take longer; see Reproduce below.
Results and demo:
- Hugging Face dataset (results only): https://huggingface.co/datasets/RishabhPande/agentrisk
- Demo Space: https://huggingface.co/spaces/RishabhPande/agentrisk-demo
Share of sessions hard-declined (solid) or stepped up (hatched) per session class, Legacy policy against AgentGate. The data is results/main.csv, the mean of 5 seeds of the semi-synthetic benchmark. Absolute rates are properties of the benchmark, not of any production system. Regenerate with python docs/plot_policies.py (needs pandas and matplotlib; run from the repo root).
| Path | Contents |
|---|---|
sim.py |
AgentRisk-Sim: semi-synthetic checkout-session generator (9 session classes, 36 features). All assumptions are in PARAMS. |
run.py |
Experiments: main (5 seeds, 11 policies), fewshot, sweep |
analyze.py |
Builds every figure, table and in-text number from results/ |
telemetry/ |
Instrumented checkout page and Playwright harness that measures what browser-automation primitives emit, plus the raw captures |
live/ |
The hosted test checkout used with a real production agent, and the 6 recorded sessions (event counts and timings only) |
score_real.py, analyze_real.py |
Score the real-agent sessions with the benchmark's models |
results/ |
Result files behind every number in the paper |
paper/ |
The paper (PDF) |
| Class | Description |
|---|---|
| H | Legitimate human (desktop or mobile, some with autofill) |
| AU | Legitimate unverified agent |
| AV | Legitimate verified agent (signature + signed mandate) |
| FH | Manual fraud |
| FB | Scripted bot fraud (card testing, account takeover) |
| FAU | Fraud via an unverified agent |
| FAV | Laundering through a permissive agent platform |
| FHJ | Hijacked agent (prompt injection) |
| FSP | Spoofed agent signature |
FAU, FAV, FHJ and FSP never appear in training unless a few-shot experiment adds them.
Python 3.10+.
pip install -r requirements.txt
python run.py main # ~8 min on 2 cores
python run.py fewshot # ~10 min
python run.py sweep # ~30 min
python analyze.py # figures, tables, numbers -> paper/gen, paper/figs
# optional: re-measure automation primitives (needs Chromium for Playwright)
python telemetry/capture.py 10
# real-agent scoring (contexts: AU/AV, local/cloud)
for c in 0 1; do for k in AU AV; do python score_real.py $c $k; done; done
python analyze_real.py- Train on the era-1 world (
H,FH,FB), optionally plus labeled legitimate agents (AU,AV). - Set the decline threshold on held-out humans so that 2% of legitimate human sessions are declined.
- Report the share declined for each of the 9 classes, the step-up rate on legitimate classes, and the agent false-decline rate.
Human, fraud, transaction and delegation features are simulated, and every assumption is a parameter. Agent behavior comes from measured automation primitives and from 6 sessions of one production agent. Treat absolute rates as properties of the benchmark, not of any production system. See the paper's Limitations section.
@misc{pande2026automation,
title = {Automation Is Not Intent: Measuring and Fixing Fraud-Model Failures on Legitimate AI Shopping Agents},
author = {Pande, Rishabh},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.23477625},
url = {https://doi.org/10.5281/zenodo.23477625}
}Code: MIT (see LICENSE). Data and results files: CC BY 4.0.
