Safety IndexEvaluationsMLCommons AILuminate demo (English)
MLCommons AILuminate demo (English)
Refusal calibration · Harmful requestsCounts in the indexBuilt, not yet run
How often the model gives an unsafe answer on the public demo prompts of the MLCommons AILuminate benchmark, covering 12 hazard categories and three user personas.
- Metric
- % unsafe. Lower is better.
- Dataset
- github.com/mlcommons/ailuminate · 1,200 (100 per hazard) · CC-BY-4.0 (data), Apache-2.0 (code)
- Grader
- own LLM judge (official ensemble is members-only) (the paper used MLCommons tuned ensemble (closed))
- Status
- Built, not yet run
- Replicability
- public-closed-judge
Results
Built and tested; the first full runs are pending.