SAI LabsBook a demo

Safety IndexGoogleGemini 3.1 Pro

Gemini 3.1 Pro

GoogleProprietaryReleased 2026-02Reasoning model

SAI Safety Index56provisionalHas 2 of the 4 pillars needed
Strongest pillar70Refusal calibration
Weakest pillar42Interaction and honesty
Coverage2of 5 pillars2 counted measurements, 0 by SAI

Summary

Gemini 3.1 Pro has data in 2 of the five pillars, short of the 4 needed for a headline number, so it is listed as partly measured; the mean of its measured pillars is 56. Its strongest area is refusal calibration at 70, and its weakest is interaction and honesty at 42.

2 measurements count toward the score, 0 of them from SAI's own runs. Not yet measured: adversarial robustness, agentic safety, people and society.

Pillar scores

How Gemini 3.1 Pro compares with every other model scored in the pillar, and which sub-categories the score rests on.

Refusal calibration

Does the model refuse what it should and help with what it may?

  • Harmful requests70 from 1 evaluation
  • Over-refusalnot measured

Rank 30 of 39 scored models.

Adversarial robustness

Do its refusals and instructions hold up when someone tries to get around them?

Not measured for Gemini 3.1 Pro yet.
  • Jailbreaksnot measured
  • Prompt injectionnot measured
  • Multilingual safetynot measured

9 models scored.

Agentic safety

Does it behave safely when it has tools and autonomy?

Not measured for Gemini 3.1 Pro yet.
  • Agent misusenot measured
  • Reward hacking and task integritynot measured
  • Overreach and destructive actionsnot measured

3 models scored.

Interaction and honesty

Is it honest with the person it is talking to?

  • Factuality and hallucinationnot measured
  • Honesty under pressure and sycophancy42 from 1 evaluation
  • Deception and schemingnot measured
  • Alignment auditnot measured
  • Manipulation of the usernot measured

Rank 27 of 29 scored models.

People and society

Does it protect vulnerable people and the public interest?

Not measured for Gemini 3.1 Pro yet.
  • Self-harm and wellbeingnot measured
  • Child safetynot measured
  • Bias, fairness and neutralitynot measured
  • Privacy: memorisation and leakagenot measured
  • Privacy: surveillance assistancenot measured
  • Elections and influence operationsnot measured

3 models scored.

Every measurement

These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.

EvaluationValueScoreSourceSetting
Refusal calibration
Harmful requests
Scale FORTRESS (risk score)average risk score 0-100 · Lower is better29.8%95% CI 26.5% to 33.0%70Scale AI Fortress leaderboard
Independent evaluator
Published by Scale AI Fortress leaderboard.
Full configurationFortress row 'gemini-3.1-pro-preview'
Interaction and honesty
Honesty under pressure and sycophancy
MASK% honest · Higher is better42.4%95% CI 41.1% to 43.7%n = 50042Scale AI SEAL / CAIS MASK leaderboard
Independent evaluator
Published by Scale AI SEAL / CAIS MASK leaderboard.
Full configurationScale leaderboard on the private 500-example split, not the public split; MASK private split, 500 examples; honesty score = 1 - P(lie); row 'gemini-3.1-pro-preview'
Other evaluators' scores (not counted)
Phare average safety% · Higher is better70.6%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Gemini 3.1 Pro Preview'
Phare bias resistance% · Higher is better48.1%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Gemini 3.1 Pro Preview'
Phare hallucination resistance% · Higher is better87.6%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Gemini 3.1 Pro Preview'
Phare harmfulness resistance% · Higher is better95.2%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Gemini 3.1 Pro Preview'
Phare jailbreak resistance% · Higher is better51.6%context, not countedGiskard Phare leaderboard
Independent evaluator · 2026-07-09
Published by Giskard Phare leaderboard (2026-07-09).
Full configurationPhare row 'Gemini 3.1 Pro Preview'
Vectara HHEM hallucination rate% · Lower is better10.4%context, not countedVectara Hallucination Leaderboard (HHEM-2.3)
Independent evaluator · 2026-10-22
Published by Vectara Hallucination Leaderboard (HHEM-2.3) (2026-10-22).
Full configurationsummarization of ~1,000 short documents; row 'Gemini 3.1 Pro Preview'
Developer-reported (not counted)
AA-Omniscience hallucination rate (lab-reported)% · Lower is better50.9%n = 6,000developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationfull 6,000-question set
FACTS Grounding (lab-reported)% · Higher is better64.7%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationjudge panel and N unstated
SimpleQA Verified incorrect rate (lab-reported)% incorrect · Lower is better19.3%n = 1,000developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.