What Are Prompt-Injection Benchmarks? AgentDojo, InjecAgent, and Beyond

Prompt-injection benchmarks are the artifacts security teams use to decide whether a defense for a tool-using LLM agent actually works. The documented landscape — AgentDojo, InjecAgent, adaptive-attack evaluations, and large crowd-sourced competitions — agrees on the threat but disagrees so severely on measurement that published attack-success-rate numbers cannot be compared across artifacts. Anyone selecting a defense based on a single leaderboard number needs to understand why.

What is a prompt-injection benchmark?

A prompt-injection benchmark is a fixed answer to three design questions: what the attacker is allowed to do (the threat model, including whether attacks adapt to the defense), how attacks are generated (a fixed corpus, pluggable components, or live humans), and what counts as success (the metric).

Every artifact in the audited evidence base converges on the first question: the attacker delivers malicious instructions through untrusted tool outputs or data channels — the threat class covered in what is indirect prompt injection. The artifacts diverge sharply on the second and third questions, and that divergence is the source of most confusion when practitioners read published results.

One sentence: a prompt-injection benchmark is a bundled threat model, attack generator, and success metric — and the metric only means something inside its own bundle.

How do the major benchmarks work?

The audited artifacts fall into four architectural classes, ordered by attacker adaptivity.

1. Fixed-corpus, static attacker. InjecAgent (arXiv 2403.02691) ships 1,054 test cases spanning 17 user tools and 62 attacker tools, with malicious instructions embedded in content the agent processes. It has two attack tiers: a baseline and an enhanced setting that nearly doubled attack success on ReAct-prompted GPT-4, which was vulnerable 24% of the time at baseline. The evidence documents no defense arm — InjecAgent measures model susceptibility, not defense efficacy. Its "detrimental actions" test cases instantiate the harm class described in what is tool hijacking.
2. Dynamic harness with pluggable components. AgentDojo (NeurIPS 2024; Debenedetti et al.) exposes four independent axes from its CLI: task suite, target model, defense, and attack. Documented defenses include a tool filter and a transformers-based injection detector. Its "attack with tool knowledge" variant makes attacker knowledge an explicit experimental parameter — the only artifact in the evidence base that does this.
3. Adaptive, defense-aware evaluation. The adaptive-attack study (arXiv 2503.00061, early 2025) evaluated eight indirect-prompt-injection defenses and bypassed all of them with adaptive attacks at over 50% attack success rate, concluding that adaptive evaluation is necessary when designing defenses. This is the strongest evidence in the landscape that static numbers overstate robustness.
4. Crowd-sourced adaptive competitions. LLMail-Inject collected 208,095 unique attack submissions from 839 participants against an email-assistant agent across multiple defenses, architectures, and retrieval configurations, with code, dataset, and analysis released. The NIST/CAISI-analyzed Gray Swan competition ran 250,000+ attempts from 400+ participants against 13 frontier models — all were breached. UK AISI × Gray Swan logged 1.8 million attempts and 62,000 breaks across 22 anonymized models and 44 target behaviors in four waves.

The critical measurement fact: at least three distinct definitions of attack success rate coexist across these artifacts — per-test-case success over a fixed corpus (InjecAgent), a prefix match on a literal output string (the StruQ/SecAlign harness), and crowd-normalized Total Breaks over Total Chats (UK AISI × Gray Swan). These are different quantities. A 5% ASR in one harness and a 5% ASR in another do not describe the same defensive posture.

Reproducibility is also bimodal. AgentDojo and LLMail-Inject release code and data. The UK AISI × Gray Swan pipeline used automated judging that UK AISI helped calibrate, plus manual appeal reviews, but the evidence contains no released judge prompts, thresholds, or rubrics.

Why does it matter?

Anyone selecting a defense for a production tool-using agent based on a published ASR number risks a false sense of security, for three documented reasons.

First, adaptive attackers broke every defense in the one adaptive study available. Second, static-attack numbers and adaptive-attack numbers come from incomparable measurement setups — the contrast between near-zero static ASRs and >50% adaptive ASRs is a landscape-level pattern across different papers and settings, not a head-to-head result on the same defense, but it is consistent and alarming. Third, NIST/CAISI found "universal" attack families that transfer across scenarios and models, with an asymmetry that matters for benchmark construction: attacks harvested against robust models generalize downward to weaker ones. Corpora built against weak models therefore systematically overstate defense efficacy on strong ones.

The single most actionable finding from the source audit: no audited artifact documents benign utility under attack or false-positive/refusal cost. The security-utility trade-off that should govern deployment decisions is unmeasured across the entire documented landscape. Both LLMail-Inject's authors and NIST/CAISI independently name adaptive-adversary evaluation as the central gap.

Per this site's publication policy for yellow-rated material, injection payload strings, attack templates, and judge-bypass techniques from these benchmarks and competitions are withheld; adaptivity is described architecturally only.

How do you use these benchmarks to evaluate a defense?

Six evaluation-practice moves, each with its cost and its limits.

1. Never accept static-only ASR as evidence of robustness. The adaptive study (arXiv 2503.00061) broke all eight defenses it tested at >50% ASR. Cost: adaptive evaluation is expensive and defense-specific. Limit: it establishes an upper bound on attacker capability for the defenses tested, not a general theorem.
2. Treat ASR numbers as harness-local. Never compare a fixed-corpus ASR, a prefix-match ASR, and a Total Breaks / Total Chats rate. Cost: you lose the convenience of one leaderboard number. Limit: within-harness comparisons remain valid and useful.
3. Prefer harnesses that parameterize attacker knowledge. AgentDojo's tool-knowledge attack tier is the only documented example. Use it to test whether your defense quietly assumes attacker ignorance. Cost: more configurations to run. Limit: the evidence base does not establish whether any AgentDojo attack is optimization-based.
4. Mine competition corpora for reusable attacks — harvested against the strongest targets. NIST/CAISI's transfer asymmetry means attacks that beat robust models are the most reusable evaluation material. Cost: competition data is noisy and effort-confounded. Limit: transfer does not guarantee coverage of your specific tool inventory.
5. Demand utility-under-attack and false-positive numbers before adopting a defense. An assertion like "without hurting utility" is not a measurement; no audited artifact provides one. Cost: you will have to build this measurement yourself. Limit: none — this is the open gap.
6. Check reproducibility before citing. AgentDojo and LLMail-Inject release code and data; the UK AISI × Gray Swan judging pipeline is not released in the evidence. Cost: even released harnesses may lack pinned environments.

Scope note: BIPIA, WASP, Agent Security Bench, ToolEmu, Tensor Trust, and CyberSecEval fall outside the source audit's evidence base. Their absence here is a coverage boundary, not a judgment on their merit. Likewise, the evidence does not establish whether StruQ or SecAlign were among the eight defenses broken by the adaptive study — do not read that linkage into the numbers above.

Related concepts and tools

• Security benchmarks for prompt-injection defenses in tool-using LLM agents — the source audit behind every factual claim in this explainer.
• Attack success rate — the metric whose incompatible definitions are the landscape's central measurement problem.
• Adaptive attack — the dominant axis of disagreement between benchmark designs.
• Tool-using agent hardening checklist — where to go once you have decided which evaluation evidence to trust.
• Agentic AI security — the parent topic hub for this artifact.

FAQ

Which prompt-injection benchmark should I use to evaluate a defense?

No single one. Fixed corpora like InjecAgent give cheap regression signal; dynamic harnesses like AgentDojo let you swap attacks and defenses independently; only adaptive evaluation — the arXiv 2503.00061 methodology or competition-derived corpora — predicts robustness against real attackers. Static-only results are documented to overstate defense efficacy, so use static benchmarks for regression and adaptive evaluation for acceptance decisions.

Why can't I compare attack success rates across different benchmarks?

Because ASR is defined differently per artifact. InjecAgent reports per-test-case success over a fixed 1,054-case corpus. The StruQ/SecAlign harness uses a prefix match on a literal output string. UK AISI × Gray Swan reports a crowd-normalized Total Breaks over Total Chats rate. These are different quantities measured against different attacker populations, so cross-harness comparisons are not meaningful.

Do any of these benchmarks measure whether a defense hurts the agent's normal usefulness?

Not in the audited evidence base. Defense authors assert utility preservation, but no measured benign-utility-under-attack or false-positive/refusal-cost number appears in any of the audited artifacts. This is the landscape's biggest documented gap, and it means the security-utility trade-off that should govern deployment decisions is currently unmeasured.

The short version: pick a harness for the question you are asking, keep its numbers inside its own walls, and do not sign off on a defense that has never faced an adaptive attacker.