SAI LabsBook a demo

Safety IndexEvaluationsAnthropic Red Team prompts (HELM Safety)

Anthropic Red Team prompts (HELM Safety)

Third-party cross-check

How safely the model answers prompts from Anthropic's 2022 red-team dataset; nearly saturated for current models.

Metric
0-100. Higher is better.
Dataset
crfm.stanford.edu/helm/safety/latest/
Grader
Llama 3.1 405B and GPT-4o ratings averaged (HELM Safety v1.17.0)
Status
–. Measured on 11 models, 0 readings by SAI.
Notes
Third-party board; shown on model pages as a cross-check and never folded into the index.

Results

ModelValueScoreSourceSetting
Claude Haiku 4.596.9%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)'
Claude Sonnet 4.598.2%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Sonnet (20250929)'
DeepSeek R196.2%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'DeepSeek R1'
Gemini 3 Pro97.0%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Gemini 3 Pro (Preview)'
GPT-599.1%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'GPT-5 (2025-08-07)'
GPT-5.199.5%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'GPT-5.1 (2025-11-13)'
gpt-oss-120b99.5%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'gpt-oss-120b'
gpt-oss-20b99.7%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'gpt-oss-20b'
Grok 495.7%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Grok 4 (0709)'
Llama 4 Maverick98.2%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Llama 4 Maverick (17Bx128E) Instruct FP8'
Qwen3-235B-A22B99%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Qwen3 235B A22B FP8 Throughput'