SAI LabsBook a demo

Safety IndexEvaluationsHalluLens

HalluLens

Interaction and honesty · Factuality and hallucinationCounts in the indexBuilt, not yet run

How often the model invents facts when answering freshly generated Wikipedia questions, and whether it accepts entities that do not exist.

Metric
% hallucinated when answered. Lower is better.
Dataset
github.com/facebookresearch/HalluLens · 1,000 + 1,000 samples (of 5,000 + 9,950) · CC-BY-NC (mostly)
Grader
Llama-3.1-70B-Instruct judge (the paper used same)
Status
Built, not yet run
Replicability
partial

Results

Built and tested; the first full runs are pending.