SAI LabsBook a demo

Safety IndexEvaluationsGray Swan Shade adaptive injection ASR (lab-reported)

Gray Swan Shade adaptive injection ASR (lab-reported)

Adversarial robustness · Prompt injectionDeveloper-reported

Attack success rate of Gray Swan's adaptive Shade attacker against the model in an agent environment, as reported in its system card.

Metric
% ASR. Lower is better.
Dataset
www.grayswan.ai/
Grader
the developer's own grader (see the model's system card)
Status
–. Measured on 6 models, 0 readings by SAI.
Notes
From Anthropic system cards using Gray Swan Shade, one attempt and no extra safeguards. The test was made harder between cards, so numbers are not comparable across cards. Developer-reported; never counted.

Results

ModelValueScoreSourceSetting
Claude Fable 5.151.9%developer-reportedAnthropic (Claude Opus 5.5 system card)
Developer-reported · 2026-09-22
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.1.A
Claude Fable 5.10.1%developer-reportedAnthropic (Claude Opus 5.5 system card)
Developer-reported · 2026-09-22
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.2.A
Claude Fable 5.12.5%developer-reportedAnthropic (Claude Opus 5.5 system card)
Developer-reported · 2026-09-22
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A
Claude Opus 4.50.3%developer-reportedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.1.A
Claude Opus 4.50.7%developer-reportedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.1.A
Claude Opus 4.50.0%developer-reportedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.2.A
Claude Opus 4.50.7%developer-reportedAnthropic (Claude Opus 4.5 system card)
Developer-reported · 2025-11
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.2.A
Claude Opus 4.60.0%developer-reportedAnthropic (Claude Opus 4.6 system card)
Developer-reported · 2026-02
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.1.A
Claude Opus 4.617.8%developer-reportedAnthropic (Claude Opus 4.6 system card)
Developer-reported · 2026-02
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.2.A
Claude Opus 4.620%developer-reportedAnthropic (Claude Opus 4.6 system card)
Developer-reported · 2026-02
Reported by the model's developer in its own system card or report.
Full configurationattack success rate, 1 attempt, no safeguards; Table 5.2.2.2.A
Claude Opus 50.6%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.1.A
Claude Opus 50.4%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.1.A
Claude Opus 50.5%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A
Claude Opus 53.8%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A
Claude Opus 5.554.6%developer-reportedAnthropic (Claude Opus 5.5 system card)
Developer-reported · 2026-09-22
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.1.A
Claude Opus 5.50.1%developer-reportedAnthropic (Claude Opus 5.5 system card)
Developer-reported · 2026-09-22
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.2.A
Claude Opus 5.50.1%developer-reportedAnthropic (Claude Opus 5.5 system card)
Developer-reported · 2026-09-22
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A
Claude Sonnet 50.3%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.1.A
Claude Sonnet 50.3%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.1.A
Claude Sonnet 52.3%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A
Claude Sonnet 56.0%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A
Claude Sonnet 50.5%developer-reportedAnthropic (Claude Opus 5 system card)
Developer-reported · 2026-07-24
Reported by the model's developer in its own system card or report.
Full configurationattempt-level attack success rate, no safeguards; Table 5.2.2.3.A