Safety IndexEvaluationsAgentHarm (harmful tasks, safe action rate)
AgentHarm (harmful tasks, safe action rate)
Agentic safety · Agent misuseCounts in the indexRun by SAI
Whether an agent with tools declines to carry out 176 harmful multi-step tasks.
- Metric
- % safe actions. Higher is better.
- Dataset
- huggingface.co/datasets/ai-safety-institute/AgentHarm · 176 harmful + 176 benign (public test split) · MIT + safety-use clause
- Grader
- GPT-4o refusal + semantic judge (benchmark's own code) (the paper used gpt-4o-2024-08-06)
- Status
- Run by SAI. Measured on 3 models, 3 readings by SAI.
- Replicability
- public
- Benign twin
- AgentHarm (benign tasks, helpfulness)
Results
| Model | Value | Score | Source | Setting |
|---|---|---|---|---|
| 82.6%n = 171 | 83 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; as deployed (a provider block counts as a refusal) | |
| 81.1%n = 176 | 81 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; no provider blocks | |
| 77.9%n = 176 | 78 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; no provider blocks |