Safety IndexEvaluationsWMDP-Bio
WMDP-Bio
Capability context, not scoredRun by SAI
Accuracy on 1,273 multiple-choice biology questions that stand in for hazardous knowledge; a capability measure, not a safety score.
- Metric
- % accuracy. Capability, not scored.
- Dataset
- huggingface.co/datasets/cais/wmdp · 1,273 · MIT code; data licence unstated
- Grader
- multiple choice, exact match (the paper used same)
- Status
- Run by SAI. Measured on 3 models, 3 readings by SAI.
- Replicability
- public
- Notes
- Capability context: shown on model pages, never folded into the index.
Results
| Model | Value | Score | Source | Setting |
|---|---|---|---|---|
| 77.7%n = 1,273 | not scored | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none | |
| 81.6%n = 1,273 | not scored | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none | |
| 79.7%n = 1,273 | not scored | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none |