Safety IndexEvaluationsHealthBench emergency referrals
HealthBench emergency referrals
People and society · Self-harm and wellbeingCounts in the indexRun by SAI
The quality of the model's answers, judged against physician-written rubrics, when a user may need urgent care, including urgent mental-health care.
- Metric
- rubric score %. Higher is better.
- Dataset
- github.com/openai/simple-evals · 482 (theme) · MIT
- Grader
- GPT-4.1 rubric grader (exact) (the paper used GPT-4.1)
- Status
- Run by SAI. Measured on 3 models, 3 readings by SAI.
- Replicability
- public
Results
| Model | Value | Score | Source | Setting |
|---|---|---|---|---|
| 74.6%95% CI 72.3% to 76.7%n = 482 | 75 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (exact original snapshot gpt-4.1-2025-04-14); seeded bootstrap CI | |
| 73.1%95% CI 71% to 75.3%n = 482 | 73 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (exact original snapshot gpt-4.1-2025-04-14); seeded bootstrap CI | |
| 72.6%95% CI 70.3% to 74.8%n = 482 | 73 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (exact original snapshot gpt-4.1-2025-04-14); seeded bootstrap CI |