SAI LabsBook a demo

Safety IndexGoogleGemini 4 Argon

Gemini 4 Argon

GoogleProprietaryReleased 2026-10Reasoning modelVariant scored: provider default (reasoning on)

SAI Safety Index#2 of 3753 models ranked so far
Strongest pillar83Agentic safety
Weakest pillar68Interaction and honesty
Coverage4of 5 pillars8 counted measurements, 8 by SAI

Summary

Gemini 4 Argon scores 75 on the SAI Safety Index, rank 2 of 3 models with a headline number. Its strongest area is agentic safety at 83, and its weakest is interaction and honesty at 68.

8 measurements count toward the score, 8 of them from SAI's own runs (15 SAI measurements in total, including context metrics). Not yet measured: adversarial robustness.

Pillar scores

How Gemini 4 Argon compares with every other model scored in the pillar, and which sub-categories the score rests on.

Refusal calibration

Does the model refuse what it should and help with what it may?

  • Harmful requests77 from 1 evaluation
  • Over-refusal78 from 1 evaluation

Rank 24 of 39 scored models.

Adversarial robustness

Do its refusals and instructions hold up when someone tries to get around them?

Not measured for Gemini 4 Argon yet.
  • Jailbreaksnot measured
  • Prompt injectionnot measured
  • Multilingual safetynot measured

9 models scored.

Agentic safety

Does it behave safely when it has tools and autonomy?

  • Agent misuse83 from 1 evaluation
  • Reward hacking and task integritynot measured
  • Overreach and destructive actionsnot measured

Rank 1 of 3 scored models.

Interaction and honesty

Is it honest with the person it is talking to?

  • Factuality and hallucination84 from 3 evaluations
  • Honesty under pressure and sycophancynot measured
  • Deception and scheming51 from 1 evaluation
  • Alignment auditnot measured
  • Manipulation of the usernot measured

Rank 14 of 29 scored models.

People and society

Does it protect vulnerable people and the public interest?

  • Self-harm and wellbeing73 from 1 evaluation
  • Child safetynot measured
  • Bias, fairness and neutralitynot measured
  • Privacy: memorisation and leakagenot measured
  • Privacy: surveillance assistancenot measured
  • Elections and influence operationsnot measured

Rank 3 of 3 scored models.

Every measurement

These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.

EvaluationValueScoreSourceSetting
Refusal calibration
Harmful requests
AIR-Bench 2024% refused (safe) · Higher is better77%n = 636without provider blocks: 76.7%77SAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4o 2024-11-20.
Full configurationhate and harassment slice, 636 of 5,694 prompts; OpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 (paper: GPT-4o-2024-05-13); candidate cap 32,768 tokens; filtered items re-judged via OpenAI-pinned gpt-4o-2024-11-20; as deployed (a provider block counts as a refusal)
StrongREJECT (direct, violation rate)% harmful · Lower is better3.9%n = 310without provider blocks: 4.8%benign twin, not countedSAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (paper: GPT-4o-mini); Azure-filtered items re-judged via OpenRouter pinned to OpenAI (openai/gpt-4.1); as deployed (a provider block counts as a refusal)
StrongREJECT (direct)% safe responses · Higher is better96.1%n = 310without provider blocks: 95.2%retired, not countedSAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (paper: GPT-4o-mini); Azure-filtered items re-judged via OpenRouter pinned to OpenAI (openai/gpt-4.1); as deployed (a provider block counts as a refusal)
Over-refusal
OR-Bench-Hard-1K% helpful (not refused) · Higher is better77.7%95% CI 75.6% to 79.7%n = 1,569without provider blocks: 89.8%78SAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.
Full configurationXSTest-safe (250) + OR-Bench-Hard-1K (1,319) combined set, 1,569 prompts; helpfulness rate; OpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (originals GPT-4 / GPT-4-Turbo-2024-04-09); temperature 0; as deployed (a provider block counts as a refusal)
Harmless-prompt refusals% refused · Lower is better22.3%95% CI 20.3% to 24.4%n = 1,569without provider blocks: 10.2%benign twin, not countedSAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.
Full configurationXSTest-safe (250) + OR-Bench-Hard-1K (1,319) combined set, 1,569 prompts; refusal rate; OpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (originals GPT-4 / GPT-4-Turbo-2024-04-09); temperature 0; as deployed (a provider block counts as a refusal)
Adversarial robustness
Prompt injection
AgentDojo (static injection)% survival · Higher is better100%95% CI 99.6% to 100%n = 1,003without provider blocks: 100%retired, not countedSAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by none.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge none; as deployed (a provider block counts as a refusal)
Agentic safety
Agent misuse
AgentHarm (harmful tasks, safe action rate)% safe actions · Higher is better82.6%n = 171without provider blocks: 78.8%83SAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; as deployed (a provider block counts as a refusal)
AgentHarm (benign tasks, helpfulness)% benign tasks completed · Higher is better81.8%n = 167without provider blocks: 94.2%benign twin, not countedSAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; as deployed (a provider block counts as a refusal)
Interaction and honesty
Factuality and hallucination
AA-Omniscience (public subset)% hallucination rate · Lower is better24.5%95% CI 19% to 29.5%n = 60076SAI
Measured by SAI · 2026-10-23
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationSAI run on the 600-question public subset (ArtificialAnalysis/AA-Omniscience-Public); grader google/gemini-2.5-flash with the paper's prompt; paper's answer system prompt; provider-default sampling
FACTS Grounding% grounded · Higher is better92.3%n = 86092SAI
Measured by SAI · 2026-10-23
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationapproximate 2-judge panel gpt-4.1 + google/gemini-2.5-flash (not Google's panel); released v1 json templates; 860 public examples; provider-default sampling, reasoning on
SimpleQA Verified% incorrect · Lower is better15.5%95% CI 13.3% to 17.8%n = 1,00085SAI
Measured by SAI · 2026-10-23
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.
Full configurationofficial SimpleQA Verified autorater prompt; 1,000 questions; no tools, no system prompt; provider-default sampling; judge gpt-4.1 (= reference gpt-4.1-2025-04-14)
AA-Omniscience (public subset)% hallucination rate · Lower is better4.0%n = 6,00096duplicateArtificial Analysis
Independent evaluator · 2026-11-02 (fetched)
Published by Artificial Analysis (2026-11-02 (fetched)).
Full configurationArtificial Analysis, 3,000 questions; AA-Omniscience v1.1; row 'Gemini 4 Argon (high)'
Deception and scheming
InstrumentalEval% instrumental · Lower is better48.7%n = 76without provider blocks: 52.9%51SAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4o 2024-11-20.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 (original GPT-4o); markdown-tolerant re-parse of verdicts; as deployed (a provider block counts as a refusal)
People and society
Self-harm and wellbeing
HealthBench emergency referralsrubric score % · Higher is better72.6%95% CI 70.3% to 74.8%n = 48273SAI
Measured by SAI · 2026-10-24
Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.
Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (exact original snapshot gpt-4.1-2025-04-14); seeded bootstrap CI
Capability context (not scored)
GDM situational awarenesschallenges passed of 11 · Capability, not scored8 of 11n = 9context, not countedSAI
Measured by SAI · 2026-10-28
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationOpenRouter, provider-default sampling; public task set, hard condition, 10 tries per challenge (the report protocol uses more); a challenge counts as passed when its success rate meets the threshold; 2 of 11 challenges (Irreversibility, Approved Directories) could not be scored because Google's API blocked the requests, so 9 were evaluated and 8 of those passed
GDM stealthchallenges passed of 4 · Capability, not scored3 of 4n = 4context, not countedSAI
Measured by SAI · 2026-10-28
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationOpenRouter, provider-default sampling; public task set (4 of 5 upstream tasks released), 10 tries per version (protocol: 50), 129 of 130 episodes scored; monitors Gemini 2.5 Pro preview (strong) and Gemini 2.5 Flash (fast) via OpenRouter, replacing the retired Gemini 2.0 Flash
Other evaluators' scores (not counted)
Vectara HHEM hallucination rate% · Lower is better12.1%context, not countedVectara Hallucination Leaderboard (HHEM-2.3)
Independent evaluator · 2026-10-22
Published by Vectara Hallucination Leaderboard (HHEM-2.3) (2026-10-22).
Full configurationsummarization of ~1,000 short documents; row 'Gemini 4 Argon'
Developer-reported (not counted)
AA-Omniscience hallucination rate (lab-reported)% · Lower is better15.1%n = 6,000developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationfull 6,000-question set
AgentHarm benign helpfulness (lab-reported)% · Higher is better68.1%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationHelpfulness Score
AgentHarm safe action rate (lab-reported)% · Higher is better91.4%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationSafe Action Rate
AIR-Bench hate and harassment safe rate (lab-reported)% safe · Higher is better92.7%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationsingle-turn safe response rate
Benign science refusals (lab-reported)% refused · Lower is better6.1%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationBenign refusals on Google's internal CBRN Science set
FACTS Grounding (lab-reported)% · Higher is better79%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationjudge panel and N unstated
GDM situational-awareness challenges passed (lab-reported)count of 11 · Capability, not scored10 of 11n = 11developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationchallenges passed of 11
Over-refusal helpfulness (lab-reported)% helpful · Higher is better96.6%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationHelpfulness (single-turn), internal set
Prompt-injection survival, adaptive attacker (lab-reported)% · Higher is better99%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationadaptive attacker, 50 attempts per scenario
SimpleQA Verified incorrect rate (lab-reported)% incorrect · Lower is better6.3%n = 1,000developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
StrongREJECT (lab-reported)% safe / not unsafe · Higher is better96.6%developer-reported, not countedGoogle, Gemini 4 Argon Safety & Security Report
Developer-reported · 2026-10
Reported by the model's developer in its own system card or report.
Full configurationSafe Response Rate (%), single-turn, higher is better; StrongREJECT 313 no jailbreak