SAI LabsBook a demo

Safety IndexAnthropicClaude Opus 4.5

Claude Opus 4.5

AnthropicProprietaryReleased 2025-11Reasoning modelVariant scored: thinking

SAI Safety Index86provisionalHas 3 of the 4 pillars needed
Strongest pillar100Adversarial robustness
Weakest pillar67Interaction and honesty
Coverage3of 5 pillars4 counted measurements, 0 by SAI

Summary

Claude Opus 4.5 has data in 3 of the five pillars, short of the 4 needed for a headline number, so it is listed as partly measured; the mean of its measured pillars is 86. Its strongest area is adversarial robustness at 100, and its weakest is interaction and honesty at 67.

4 measurements count toward the score, 0 of them from SAI's own runs. Not yet measured: agentic safety, people and society.

Pillar scores

How Claude Opus 4.5 compares with every other model scored in the pillar, and which sub-categories the score rests on.

Refusal calibration

Does the model refuse what it should and help with what it may?

  • Harmful requests90 from 1 evaluation
  • Over-refusalnot measured

Rank 4 of 39 scored models.

Adversarial robustness

Do its refusals and instructions hold up when someone tries to get around them?

  • Jailbreaksnot measured
  • Prompt injection100 from 1 evaluation
  • Multilingual safetynot measured

Rank 1 of 9 scored models.

Agentic safety

Does it behave safely when it has tools and autonomy?

Not measured for Claude Opus 4.5 yet.
  • Agent misusenot measured
  • Reward hacking and task integritynot measured
  • Overreach and destructive actionsnot measured

3 models scored.

Interaction and honesty

Is it honest with the person it is talking to?

  • Factuality and hallucination42 from 1 evaluation
  • Honesty under pressure and sycophancy93 from 1 evaluation
  • Deception and schemingnot measured
  • Alignment auditnot measured
  • Manipulation of the usernot measured

Rank 15 of 29 scored models.

People and society

Does it protect vulnerable people and the public interest?

Not measured for Claude Opus 4.5 yet.
  • Self-harm and wellbeingnot measured
  • Child safetynot measured
  • Bias, fairness and neutralitynot measured
  • Privacy: memorisation and leakagenot measured
  • Privacy: surveillance assistancenot measured
  • Elections and influence operationsnot measured

3 models scored.

Every measurement

These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.

EvaluationValueScoreSourceSetting
Refusal calibration
Harmful requests
Scale FORTRESS (risk score)average risk score 0-100 · Lower is better9.6%95% CI 7.5% to 11.7%non-thinking variant: 13.6%90Scale AI Fortress leaderboard
Independent evaluator
Published by Scale AI Fortress leaderboard.
Full configurationFortress row 'claude-opus-4-5-20251101-thinking'
Adversarial robustness
Prompt injection
Gray Swan IPI Arena kit% attack success · Lower is better0.5%n = 11,969100Gray Swan AI IPI Arena (with UK AISI / US CAISI)
Independent evaluator · 2026-03-16
Published by Gray Swan AI IPI Arena (with UK AISI / US CAISI) (2026-03-16).
Full configurationGray Swan IPI Arena, crowdsourced attackers, thinking disabled; indirect prompt injection on agents, breaks/attempts; row 'Claude Opus 4.5'; arena runs with extended thinking disabled, so this reading may differ from the thinking variant scored elsewhere
Interaction and honesty
Factuality and hallucination
AA-Omniscience (public subset)% hallucination rate · Lower is better58%n = 6,00042Artificial Analysis
Independent evaluator · 2025-11-25
Published by Artificial Analysis (2025-11-25).
Full configurationArtificial Analysis, 6,000 questions; AA-Omniscience v1 (6,000 q); row 'Claude Opus 4.5 (thinking)'
Honesty under pressure and sycophancy
MASK% honest · Higher is better92.5%95% CI 91.3% to 93.8%n = 500non-thinking variant: 87.1%93Scale AI SEAL / CAIS MASK leaderboard
Independent evaluator
Published by Scale AI SEAL / CAIS MASK leaderboard.
Full configurationScale leaderboard on the private 500-example split, not the public split; MASK private split, 500 examples; honesty score = 1 - P(lie); row 'claude-opus-4-5-20251101-thinking'
Other evaluators' scores (not counted)
Phare average safety% · Higher is better82.4%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Claude 4.5 Opus'
Phare bias resistance% · Higher is better63.2%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Claude 4.5 Opus'
Phare hallucination resistance% · Higher is better88.2%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Claude 4.5 Opus'
Phare harmfulness resistance% · Higher is better98.3%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Claude 4.5 Opus'
Phare jailbreak resistance% · Higher is better79.8%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Claude 4.5 Opus'
Vectara HHEM hallucination rate% · Lower is better10.9%context, not countedVectara Hallucination Leaderboard (HHEM-2.3)
Independent evaluator · 2026-10-22
Published by Vectara Hallucination Leaderboard (HHEM-2.3) (2026-10-22).
Full configurationsummarization of ~1,000 short documents; row 'Claude Opus 4.5'
Developer-reported (not counted)
BBQ ambiguous accuracy (lab-reported)% · Higher is better99.7%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationBBQ ambiguous accuracy; Table 3.5.2.B; ambiguous accuracy 99.7
Benign-request over-refusal rate (lab-reported)% refused · Lower is better0.2%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationoverall over-refusal rate, 6 languages; Table 3.1.2.A
Gray Swan Shade adaptive injection ASR (lab-reported)% ASR · Lower is better0.3%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.1.A
Gray Swan Shade adaptive injection ASR (lab-reported)% ASR · Lower is better0.7%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.1.A
Gray Swan Shade adaptive injection ASR (lab-reported)% ASR · Lower is better0.0%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.2.A
Gray Swan Shade adaptive injection ASR (lab-reported)% ASR · Lower is better0.7%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.2.A
Harmful-request harmless-response rate (lab-reported)% harmless / refused · Higher is better99.8%developer-reported, not countedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationoverall harmless response rate, 6 languages; Table 3.1.1.A; Arabic/English/French/Korean/Mandarin/Russian; default 99.7%, extended 99.85%; multilingual set so not comparable to earlier