SAI LabsBook a demo

Safety IndexEvaluationsHELM Safety mean score

HELM Safety mean score

Third-party cross-check

Stanford HELM's unweighted average of its five safety scenarios.

Metric
0-100. Higher is better.
Dataset
crfm.stanford.edu/helm/safety/latest/
Grader
Stanford HELM Safety v1.17.0 (five scenarios averaged)
Status
–. Measured on 11 models, 0 readings by SAI.
Notes
Third-party board; shown on model pages as a cross-check and never folded into the index.

Results

ModelValueScoreSourceSetting
Claude Haiku 4.595.5%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Haiku (20251001)'
Claude Sonnet 4.597.1%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Claude 4.5 Sonnet (20250929)'
DeepSeek R186.5%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'DeepSeek R1'
Gemini 3 Pro92.5%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Gemini 3 Pro (Preview)'
GPT-598.1%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'GPT-5 (2025-08-07)'
GPT-5.196.8%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'GPT-5.1 (2025-11-13)'
gpt-oss-120b98.1%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'gpt-oss-120b'
gpt-oss-20b96.4%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'gpt-oss-20b'
Grok 483.6%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Grok 4 (0709)'
Llama 4 Maverick90.6%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Llama 4 Maverick (17Bx128E) Instruct FP8'
Qwen3-235B-A22B90.0%not scoredStanford CRFM HELM Safety v1.17.0
Independent evaluator · 2025-11-24
Published by Stanford CRFM HELM Safety v1.17.0 (2025-11-24).
Full configurationHELM Safety v1.17.0; HELM row 'Qwen3 235B A22B FP8 Throughput'