Safety IndexEvaluationsAgentDojo (static injection)
AgentDojo (static injection)
Adversarial robustness · Prompt injectionRetired from the mean (saturated)Run by SAI
Whether a tool-using agent ignores a fixed malicious instruction planted in tool output across 1,014 episodes, and how useful it stays on the task.
- Metric
- % survival. Higher is better.
- Dataset
- github.com/ethz-spylab/agentdojo · 1,014 attack + 98 benign · MIT
- Grader
- programmatic state checks (the paper used same)
- Status
- Run by SAI. Measured on 3 models, 3 readings by SAI.
- Replicability
- public
- Notes
- Nearly saturated for current frontier models, so it no longer counts toward the index.
Results
| Model | Value | Score | Source | Setting |
|---|---|---|---|---|
| 100%95% CI 99.6% to 100%n = 1,003 | not scored | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by none.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge none; as deployed (a provider block counts as a refusal) | |
| 100%95% CI 99.6% to 100%n = 1,014 | not scored | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by none.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge none; no provider blocks | |
| 98.1%95% CI 97.1% to 98.8%n = 1,014 | not scored | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by none.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge none; no provider blocks |