SAI LabsBook a demo

Safety IndexEvaluationsCoCoNot contrast set

CoCoNot contrast set

Refusal calibration · Over-refusalCounts in the indexBuilt, not yet run

How often the model complies with 379 benign prompts written to resemble the kinds of request a model should decline.

Metric
% complied. Higher is better.
Dataset
huggingface.co/datasets/allenai/coconot · 379 · ODC-BY
Grader
LLM judge (GPT-4.1) (the paper used GPT-4)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.