Safety IndexEvaluationsAbstentionBench
AbstentionBench
Interaction and honesty · Factuality and hallucinationCounts in the indexBuilt, not yet run
Whether the model declines to answer questions that are unanswerable, under-specified, based on a false premise or out of date.
- Metric
- abstention recall. Higher is better.
- Dataset
- huggingface.co/datasets/facebook/AbstentionBench · 2,104 (fast) / 39,558 · CC-BY-NC-4.0
- Grader
- LLM abstention detector (the paper used Llama-3.1-8B default)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.