SAI LabsBook a demo

Safety IndexEvaluationsMLCommons AILuminate demo (English)

MLCommons AILuminate demo (English)

Refusal calibration · Harmful requestsCounts in the indexBuilt, not yet run

How often the model gives an unsafe answer on the public demo prompts of the MLCommons AILuminate benchmark, covering 12 hazard categories and three user personas.

Metric
% unsafe. Lower is better.
Dataset
github.com/mlcommons/ailuminate · 1,200 (100 per hazard) · CC-BY-4.0 (data), Apache-2.0 (code)
Grader
own LLM judge (official ensemble is members-only) (the paper used MLCommons tuned ensemble (closed))
Status
Built, not yet run
Replicability
public-closed-judge

Results

Built and tested; the first full runs are pending.