SAI LabsBook a demo

Safety IndexEvaluationsFACTS Grounding

FACTS Grounding

Interaction and honesty · Factuality and hallucinationCounts in the indexRun by SAI

How often the model's long-form answers stay fully grounded in the document it was given.

Metric
% grounded. Higher is better.
Dataset
www.kaggle.com/facts-leaderboard · 860 public · CC-BY-4.0
Grader
two-model panel (GPT-4.1 + Gemini 2.5 Flash) approximating Google's panel (the paper used 3-frontier-judge panel (v2 upgraded))
Status
Run by SAI. Measured on 3 models, 3 readings by SAI.
Replicability
partial

Results

ModelValueScoreSourceSetting
Gemini 4 Argon92.3%n = 86092SAI
Measured by SAI · 2026-10-23
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationapproximate 2-judge panel gpt-4.1 + google/gemini-2.5-flash (not Google's panel); released v1 json templates; 860 public examples; provider-default sampling, reasoning on
Qwen3.8-Max89.8%n = 86090SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationapproximate 2-judge panel gpt-4.1 + google/gemini-2.5-flash (not Google's panel); released v1 json templates; 860 public examples; provider-default sampling, reasoning on
Kimi K384.4%n = 86084SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationapproximate 2-judge panel gpt-4.1 + google/gemini-2.5-flash (not Google's panel); released v1 json templates; 860 public examples; provider-default sampling, reasoning on