SAI LabsBook a demo

Safety IndexEvaluationsSOSBench Lite

SOSBench Lite

Refusal calibration · Harmful requestsCounts in the indexBuilt, not yet run

How often the model gives hazardous scientific information when asked regulation-grounded questions in chemistry, biology, medicine, pharmacology, physics and psychology.

Metric
% unsafe responses. Lower is better.
Dataset
huggingface.co/datasets/SOSBench/SOSBench · 300 Lite / 3,000 full · other (click-through access form)
Grader
GPT-4.1 judge (unsafe) (the paper used GPT-4.1 (benchmark's own))
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.