Safety IndexAnthropicClaude Haiku 4.5
Claude Haiku 4.5
AnthropicProprietaryReleased 2025-10Reasoning modelVariant scored: default
Summary
Claude Haiku 4.5 has data in 3 of the five pillars, short of the 4 needed for a headline number, so it is listed as partly measured; the mean of its measured pillars is 89. Its strongest area is adversarial robustness at 99, and its weakest is interaction and honesty at 74.
3 measurements count toward the score, 0 of them from SAI's own runs. Not yet measured: agentic safety, people and society.
Pillar scores
How Claude Haiku 4.5 compares with every other model scored in the pillar, and which sub-categories the score rests on.
Refusal calibration
Does the model refuse what it should and help with what it may?
- Harmful requests93 from 1 evaluation
- Over-refusalnot measured
Rank 2 of 39 scored models.
Adversarial robustness
Do its refusals and instructions hold up when someone tries to get around them?
- Jailbreaksnot measured
- Prompt injection99 from 1 evaluation
- Multilingual safetynot measured
Rank 3 of 9 scored models.
Agentic safety
Does it behave safely when it has tools and autonomy?
- Agent misusenot measured
- Reward hacking and task integritynot measured
- Overreach and destructive actionsnot measured
3 models scored.
Interaction and honesty
Is it honest with the person it is talking to?
- Factuality and hallucination74 from 1 evaluation
- Honesty under pressure and sycophancynot measured
- Deception and schemingnot measured
- Alignment auditnot measured
- Manipulation of the usernot measured
Rank 12 of 29 scored models.
People and society
Does it protect vulnerable people and the public interest?
- Self-harm and wellbeingnot measured
- Child safetynot measured
- Bias, fairness and neutralitynot measured
- Privacy: memorisation and leakagenot measured
- Privacy: surveillance assistancenot measured
- Elections and influence operationsnot measured
3 models scored.
Every measurement
These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.
| Evaluation | Value | Score | Source | Setting |
|---|---|---|---|---|
| Refusal calibration | ||||
| Harmful requests | ||||
| AIR-Bench 2024 | 93.2%n = 5,694 | 93 | Stanford CRFM HELM AIR-Bench v1.19.0 Independent evaluator | Published by Stanford CRFM HELM AIR-Bench v1.19.0.Full configurationfull 5,694-prompt set, Stanford HELM AIR-Bench v1.19.0; overall refusal rate, 5,694 prompts, GPT-4o judge; HELM row 'Claude Haiku 4.5' |
| HarmBench (direct requests) | 95.9% | retired, not counted | Stanford CRFM HELM Safety v1.17.0 Independent evaluator · 2025-11-24 | Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).Full configurationHELM Safety score 0-100, higher is safer (not an attack-success rate); HELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)' |
| Over-refusal | ||||
| XSTest (HELM Safety) | 93.2% | retired, not counted | Stanford CRFM HELM Safety v1.17.0 Independent evaluator · 2025-11-24 | Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)' |
| Adversarial robustness | ||||
| Prompt injection | ||||
| Gray Swan IPI Arena kit | 1.3%n = 13,696 | 99 | Gray Swan AI IPI Arena (with UK AISI / US CAISI) Independent evaluator · 2026-03-16 | Published by Gray Swan AI IPI Arena (with UK AISI / US CAISI) (2026-03-16).Full configurationGray Swan IPI Arena, crowdsourced attackers, thinking disabled; indirect prompt injection on agents, breaks/attempts; row 'Claude Haiku 4.5'; arena runs with extended thinking disabled, so this reading may differ from the thinking variant scored elsewhere |
| Interaction and honesty | ||||
| Factuality and hallucination | ||||
| AA-Omniscience (public subset) | 26%n = 6,000 | 74 | Artificial Analysis Independent evaluator · 2025-11-16 | Published by Artificial Analysis (2025-11-16).Full configurationArtificial Analysis, 6,000 questions; AA-Omniscience v1 (6,000 q); row 'Claude Haiku 4.5' |
| People and society | ||||
| Bias, fairness and neutrality | ||||
| BBQ | 92.8%n = 1,000 | retired, not counted | Stanford CRFM HELM Safety v1.17.0 Independent evaluator · 2025-11-24 | Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)' |
| Other evaluators' scores (not counted) | ||||
| Anthropic Red Team prompts (HELM Safety) | 96.9% | context, not counted | Stanford CRFM HELM Safety v1.17.0 Independent evaluator · 2025-11-24 | Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)' |
| HELM Safety mean score | 95.5% | context, not counted | Stanford CRFM HELM Safety v1.17.0 Independent evaluator · 2025-11-24 | Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)' |
| Phare average safety | 83.2% | context, not counted | Giskard Phare leaderboard Independent evaluator · 2026-07-09 | Published by Giskard Phare leaderboard (2026-07-09).Full configurationPhare row 'Claude 4.5 Haiku' |
| Phare bias resistance | 70.7% | context, not counted | Giskard Phare leaderboard Independent evaluator · 2026-07-09 | Published by Giskard Phare leaderboard (2026-07-09).Full configurationPhare row 'Claude 4.5 Haiku' |
| Phare hallucination resistance | 83.6% | context, not counted | Giskard Phare leaderboard Independent evaluator · 2026-07-09 | Published by Giskard Phare leaderboard (2026-07-09).Full configurationPhare row 'Claude 4.5 Haiku' |
| Phare harmfulness resistance | 99.9% | context, not counted | Giskard Phare leaderboard Independent evaluator · 2026-07-09 | Published by Giskard Phare leaderboard (2026-07-09).Full configurationPhare row 'Claude 4.5 Haiku' |
| Phare jailbreak resistance | 78.5% | context, not counted | Giskard Phare leaderboard Independent evaluator · 2026-07-09 | Published by Giskard Phare leaderboard (2026-07-09).Full configurationPhare row 'Claude 4.5 Haiku' |
| SimpleSafetyTests (HELM Safety) | 98.8% | context, not counted | Stanford CRFM HELM Safety v1.17.0 Independent evaluator · 2025-11-24 | Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)' |
| Vectara HHEM hallucination rate | 9.8% | context, not counted | Vectara Hallucination Leaderboard (HHEM-2.3) Independent evaluator · 2026-10-22 | Published by Vectara Hallucination Leaderboard (HHEM-2.3) (2026-10-22).Full configurationsummarization of ~1,000 short documents; row 'Claude Haiku 4.5' |
| Developer-reported (not counted) | ||||
| BBQ ambiguous accuracy (lab-reported) | 98% | developer-reported, not counted | Anthropic (Claude Haiku 4.5 system card) Developer-reported · 2025-10 | Reported by the model's developer in its own system card or report.Full configurationBBQ ambiguous accuracy; Table 2.5.2.B; ambiguous accuracy 98.0 |
| Benign-request over-refusal rate (lab-reported) | 0.0% | developer-reported, not counted | Anthropic (Claude Haiku 4.5 system card) Developer-reported · 2025-10 | Reported by the model's developer in its own system card or report.Full configurationoverall over-refusal rate; Table 2.1.2.A |
| Harmful-request harmless-response rate (lab-reported) | 99.4% | developer-reported, not counted | Anthropic (Claude Haiku 4.5 system card) Developer-reported · 2025-10 | Reported by the model's developer in its own system card or report.Full configurationoverall harmless response rate; Table 2.1.1.A; Haiku 4.5 default 99.40%, extended 99.36%; Haiku 3.5 has no extended thinking |