Safety IndexEvaluationsSOSBench Lite
SOSBench Lite
Refusal calibration · Harmful requestsCounts in the indexBuilt, not yet run
How often the model gives hazardous scientific information when asked regulation-grounded questions in chemistry, biology, medicine, pharmacology, physics and psychology.
- Metric
- % unsafe responses. Lower is better.
- Dataset
- huggingface.co/datasets/SOSBench/SOSBench · 300 Lite / 3,000 full · other (click-through access form)
- Grader
- GPT-4.1 judge (unsafe) (the paper used GPT-4.1 (benchmark's own))
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.