Safety IndexEvaluationsBenign-request over-refusal rate (lab-reported)
Benign-request over-refusal rate (lab-reported)
Refusal calibration · Over-refusalDeveloper-reported
The developer's own measure of how often the model refused harmless prompts in sensitive areas.
- Metric
- % refused. Lower is better.
- Dataset
- –
- Grader
- the developer's own grader (see the model's system card)
- Status
- –. Measured on 8 models, 0 readings by SAI.
- Notes
- Lab system cards; internal prompt sets. Developer-reported; never counted.
Results
| Model | Value | Score | Source | Setting |
|---|---|---|---|---|
| 0.0% | developer-reported | Anthropic (Claude Opus 5.5 system card) Developer-reported · 2026-09-22 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate, API without system prompt; Table 4.1.2.A; claude.ai 0.34% | |
| 0.0% | developer-reported | Anthropic (Claude Haiku 4.5 system card) Developer-reported · 2025-10 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate; Table 2.1.2.A | |
| 0.2% | developer-reported | Anthropic (Claude Opus 4.5 system card) Developer-reported · 2025-11 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate, 6 languages; Table 3.1.2.A | |
| 0.7% | developer-reported | Anthropic (Claude Opus 4.6 system card) Developer-reported · 2026-02 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate, 7 languages; Table 3.1.2.A | |
| 0.1% | developer-reported | Anthropic (Claude Opus 5 system card) Developer-reported · 2026-07-24 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate, API without system prompt; Table 4.1.2.A; claude.ai 0.47% | |
| 0.0% | developer-reported | Anthropic (Claude Opus 5.5 system card) Developer-reported · 2026-09-22 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate, API without system prompt; Table 4.1.2.A; claude.ai 0.38% | |
| 0.0% | developer-reported | Anthropic (Claude Sonnet 4.5 system card) Developer-reported · 2025-09 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate; Table 2.1.2.A; run without safety classifiers | |
| 0.6% | developer-reported | Anthropic (Claude Opus 5 system card) Developer-reported · 2026-07-24 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate, API without system prompt; Table 4.1.2.A; claude.ai 1.54% |