What Prompt-Injection Defense Claims Mean for Security Leaders Evaluating Vendors
Quick Answer
Benchmark scores for prompt-injection defenses depend on whether the attacker was allowed to adapt: defenses that report near-zero attack success under static testing were all bypassed at over 50% success in the one documented adaptive evaluation, and no documented benchmark measures what a defense costs in blocked legitimate work. Security leaders should treat vendor efficacy numbers as scoped claims, demand adaptivity and utility disclosure, and architect agent deployments to assume some attacks succeed.
Key Takeaway
A near-zero prompt-injection defense score describes the benchmark's attacker, not yours; every documented defense evaluated under adaptive attack was bypassed at over fifty percent success.
Security leaders evaluating defenses for tool-using AI agents are being shown benchmark scores — often near-zero attack success rates — as evidence a product works. A measurement audit of the documented evaluation landscape shows those numbers hinge on one variable: whether the attacker was allowed to adapt to the defense. A near-zero prompt-injection defense score describes the benchmark's attacker, not yours; every documented defense evaluated under adaptive attack was bypassed at over fifty percent success. This matters now because agent deployments with email, web, and tool access are being approved on exactly these numbers.
What this means for your organization
Buying a defense product or approving an agent deployment on a benchmark score is a risk-acceptance decision, and the documented evidence shows the score often describes a far weaker adversary than the real one. Static benchmarks replay a fixed set of attacks; real attackers iterate. In the one documented adaptive evaluation, all eight defenses tested were bypassed at over 50% attack success — the same class of defenses that scores near zero under static testing. For agents with email, web, or tool access — a support agent, a copilot, an internal workflow — a successful injection means unauthorized actions on users' behalf: data exfiltration or fraudulent tool calls attributed to a legitimate session.
The second exposure is invisible in every audited artifact: cost. No documented benchmark measures what a defense blocks in legitimate work — false positives, refusals, degraded agent utility. Vendors assert utility preservation; nothing in the evidence base measures it. Government analysis points the same way: in the red-teaming competition NIST's CAISI analyzed, all 13 frontier models were breached, and CAISI concluded evaluations must "constantly evolve and adapt." Attack specifics are withheld here by design; what prompt-injection benchmarks measure covers the publicly safe detail.
What to ask your team
For each defense we are evaluating, was the reported attack success rate measured against a static attack corpus or an attacker allowed to adapt to the defense?
What is the measured false-positive or refusal rate on legitimate agent traffic with the defense enabled?
Are the products we are comparing scored on the same harness with the same definition of a successful attack?
What knowledge of our tools and configuration does the vendor's benchmark assume the attacker has?
Which irreversible actions can our agents take today without human approval if the defense fails?
What good looks like
A sound posture treats efficacy numbers as scoped claims and treats breach as an architectural assumption, not a benchmark outcome.
- Every efficacy number entering a procurement decision carries an adaptivity disclosure: static corpus or defense-aware attacker.
- Utility under defense is a required data point, not a vendor assertion. A defense with unknown false-positive cost has an unknown price.
- Head-to-head comparisons happen on the same harness with the same metric, or not at all.
- Adaptive and competition-derived evidence — the only documented large-scale adaptive evaluations — outweighs static-corpus results.
- Defenses are deployed as attack-rate reducers inside an architecture that limits blast radius: agents hold the minimum capabilities their job requires, and irreversible actions — payments, deletions, external sends — require a human. The reasoning behind that floor lives in agent capability control; the implementation work lives in the tool-using agent hardening checklist.
Where to dig deeper
- What are prompt-injection benchmarks — the companion explainer with benchmark-by-benchmark detail.
- The source measurement audit — the evidence base behind this brief.
- Prompt injection: executive summary — the threat-focused sibling brief; this one is its procurement counterpart.
- What is indirect prompt injection — the attack primer for readers who need it.
- Externally: the adaptive-attack evaluation (arXiv 2503.00061), the NIST CAISI competition analysis, and the LLMail-Inject dataset paper.
FAQ
A vendor showed us a near-zero attack success rate. Is that number wrong?
Not wrong — scoped. Near-zero attack success under static, template-based attacks coexists in the published literature with over-50% success when the attacker adapts to the defense (arXiv 2503.00061). The number describes the benchmark's adversary, not the adversary your deployment will face. Ask which one produced it.
Which benchmark should we trust when comparing two defense products?
None in isolation. The audit — which covers five documented artifact families, not the full benchmark landscape — finds that success metrics do not reconcile across artifacts: a prefix-match on a canary string, a breaks-per-chat ratio, and a fixed-corpus success rate are different quantities. Insist on same-harness, same-metric comparison, and ask whether the number came from an adaptive attacker.
Does any benchmark measure what a defense costs us in false positives or blocked legitimate work?
Not in the documented evidence base. No audited artifact measures benign utility under attack or false-positive and refusal costs; utility preservation is asserted, never measured. Treat utility numbers as absent unless the vendor produces measured ones for your workloads.
Is there a government or standards position on this yet?
Not normative guidance yet. NIST's CAISI published analysis of a large-scale red-teaming competition in which all 13 frontier models were breached, concluding that evaluations must "constantly evolve and adapt" — independently naming adaptive evaluation as the central gap, matching the academic finding. It is research signal, not a standard.
Derived From
Related Work
External References
FAQ
A vendor showed us a near-zero attack success rate. Is that number wrong?
Not wrong — scoped. Near-zero attack success under static, template-based attacks coexists in the published literature with over-50% success when the attacker adapts to the defense (arXiv 2503.00061). The number describes the benchmark's adversary, not the adversary your deployment will face. Ask which one produced it.
Which benchmark should we trust when comparing two defense products?
None in isolation. The audit — which covers five documented artifact families, not the full benchmark landscape — finds that success metrics do not reconcile across artifacts: a prefix-match on a canary string, a breaks-per-chat ratio, and a fixed-corpus success rate are different quantities. Insist on same-harness, same-metric comparison, and ask whether the number came from an adaptive attacker.
Does any benchmark measure what a defense costs us in false positives or blocked legitimate work?
Not in the documented evidence base. No audited artifact measures benign utility under attack or false-positive and refusal costs; utility preservation is asserted, never measured. Treat utility numbers as absent unless the vendor produces measured ones for your workloads.
Is there a government or standards position on this yet?
Not normative guidance yet. NIST's CAISI published analysis of a large-scale red-teaming competition in which all 13 frontier models were breached, concluding that evaluations must 'constantly evolve and adapt' — independently naming adaptive evaluation as the central gap, matching the academic finding. It is research signal, not a standard.