Safety IndexGoogleGemini 4 Argon
Gemini 4 Argon
GoogleProprietaryReleased 2026-10Reasoning modelVariant scored: provider default (reasoning on)
Summary
Gemini 4 Argon scores 75 on the SAI Safety Index, rank 2 of 3 models with a headline number. Its strongest area is agentic safety at 83, and its weakest is interaction and honesty at 68.
8 measurements count toward the score, 8 of them from SAI's own runs (15 SAI measurements in total, including context metrics). Not yet measured: adversarial robustness.
Pillar scores
How Gemini 4 Argon compares with every other model scored in the pillar, and which sub-categories the score rests on.
Refusal calibration
Does the model refuse what it should and help with what it may?
- Harmful requests77 from 1 evaluation
- Over-refusal78 from 1 evaluation
Rank 24 of 39 scored models.
Adversarial robustness
Do its refusals and instructions hold up when someone tries to get around them?
- Jailbreaksnot measured
- Prompt injectionnot measured
- Multilingual safetynot measured
9 models scored.
Agentic safety
Does it behave safely when it has tools and autonomy?
- Agent misuse83 from 1 evaluation
- Reward hacking and task integritynot measured
- Overreach and destructive actionsnot measured
Rank 1 of 3 scored models.
Interaction and honesty
Is it honest with the person it is talking to?
- Factuality and hallucination84 from 3 evaluations
- Honesty under pressure and sycophancynot measured
- Deception and scheming51 from 1 evaluation
- Alignment auditnot measured
- Manipulation of the usernot measured
Rank 14 of 29 scored models.
People and society
Does it protect vulnerable people and the public interest?
- Self-harm and wellbeing73 from 1 evaluation
- Child safetynot measured
- Bias, fairness and neutralitynot measured
- Privacy: memorisation and leakagenot measured
- Privacy: surveillance assistancenot measured
- Elections and influence operationsnot measured
Rank 3 of 3 scored models.
Every measurement
These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.
| Evaluation | Value | Score | Source | Setting |
|---|---|---|---|---|
| Refusal calibration | ||||
| Harmful requests | ||||
| AIR-Bench 2024 | 77%n = 636without provider blocks: 76.7% | 77 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4o 2024-11-20.Full configurationhate and harassment slice, 636 of 5,694 prompts; OpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 (paper: GPT-4o-2024-05-13); candidate cap 32,768 tokens; filtered items re-judged via OpenAI-pinned gpt-4o-2024-11-20; as deployed (a provider block counts as a refusal) |
| StrongREJECT (direct, violation rate) | 3.9%n = 310without provider blocks: 4.8% | benign twin, not counted | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (paper: GPT-4o-mini); Azure-filtered items re-judged via OpenRouter pinned to OpenAI (openai/gpt-4.1); as deployed (a provider block counts as a refusal) |
| StrongREJECT (direct) | 96.1%n = 310without provider blocks: 95.2% | retired, not counted | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (paper: GPT-4o-mini); Azure-filtered items re-judged via OpenRouter pinned to OpenAI (openai/gpt-4.1); as deployed (a provider block counts as a refusal) |
| Over-refusal | ||||
| OR-Bench-Hard-1K | 77.7%95% CI 75.6% to 79.7%n = 1,569without provider blocks: 89.8% | 78 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationXSTest-safe (250) + OR-Bench-Hard-1K (1,319) combined set, 1,569 prompts; helpfulness rate; OpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (originals GPT-4 / GPT-4-Turbo-2024-04-09); temperature 0; as deployed (a provider block counts as a refusal) |
| Harmless-prompt refusals | 22.3%95% CI 20.3% to 24.4%n = 1,569without provider blocks: 10.2% | benign twin, not counted | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationXSTest-safe (250) + OR-Bench-Hard-1K (1,319) combined set, 1,569 prompts; refusal rate; OpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (originals GPT-4 / GPT-4-Turbo-2024-04-09); temperature 0; as deployed (a provider block counts as a refusal) |
| Adversarial robustness | ||||
| Prompt injection | ||||
| AgentDojo (static injection) | 100%95% CI 99.6% to 100%n = 1,003without provider blocks: 100% | retired, not counted | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by none.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge none; as deployed (a provider block counts as a refusal) |
| Agentic safety | ||||
| Agent misuse | ||||
| AgentHarm (harmful tasks, safe action rate) | 82.6%n = 171without provider blocks: 78.8% | 83 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; as deployed (a provider block counts as a refusal) |
| AgentHarm (benign tasks, helpfulness) | 81.8%n = 167without provider blocks: 94.2% | benign twin, not counted | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 refusal + semantic judges (original GPT-4o-2024-08-06); official grading functions; public test split; as deployed (a provider block counts as a refusal) |
| Interaction and honesty | ||||
| Factuality and hallucination | ||||
| AA-Omniscience (public subset) | 24.5%95% CI 19% to 29.5%n = 600 | 76 | SAI Measured by SAI · 2026-10-23 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationSAI run on the 600-question public subset (ArtificialAnalysis/AA-Omniscience-Public); grader google/gemini-2.5-flash with the paper's prompt; paper's answer system prompt; provider-default sampling |
| FACTS Grounding | 92.3%n = 860 | 92 | SAI Measured by SAI · 2026-10-23 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationapproximate 2-judge panel gpt-4.1 + google/gemini-2.5-flash (not Google's panel); released v1 json templates; 860 public examples; provider-default sampling, reasoning on |
| SimpleQA Verified | 15.5%95% CI 13.3% to 17.8%n = 1,000 | 85 | SAI Measured by SAI · 2026-10-23 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationofficial SimpleQA Verified autorater prompt; 1,000 questions; no tools, no system prompt; provider-default sampling; judge gpt-4.1 (= reference gpt-4.1-2025-04-14) |
| AA-Omniscience (public subset) | 4.0%n = 6,000 | 96duplicate | Artificial Analysis Independent evaluator · 2026-11-02 (fetched) | Published by Artificial Analysis (2026-11-02 (fetched)).Full configurationArtificial Analysis, 3,000 questions; AA-Omniscience v1.1; row 'Gemini 4 Argon (high)' |
| Deception and scheming | ||||
| InstrumentalEval | 48.7%n = 76without provider blocks: 52.9% | 51 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4o 2024-11-20.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4o 2024-11-20 (original GPT-4o); markdown-tolerant re-parse of verdicts; as deployed (a provider block counts as a refusal) |
| People and society | ||||
| Self-harm and wellbeing | ||||
| HealthBench emergency referrals | 72.6%95% CI 70.3% to 74.8%n = 482 | 73 | SAI Measured by SAI · 2026-10-24 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on; graded by gpt-4.1.Full configurationOpenRouter, provider-default sampling, reasoning on, no system prompt; judge gpt-4.1 (exact original snapshot gpt-4.1-2025-04-14); seeded bootstrap CI |
| Capability context (not scored) | ||||
| GDM situational awareness | 8 of 11n = 9 | context, not counted | SAI Measured by SAI · 2026-10-28 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling; public task set, hard condition, 10 tries per challenge (the report protocol uses more); a challenge counts as passed when its success rate meets the threshold; 2 of 11 challenges (Irreversibility, Approved Directories) could not be scored because Google's API blocked the requests, so 9 were evaluated and 8 of those passed |
| GDM stealth | 3 of 4n = 4 | context, not counted | SAI Measured by SAI · 2026-10-28 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling; public task set (4 of 5 upstream tasks released), 10 tries per version (protocol: 50), 129 of 130 episodes scored; monitors Gemini 2.5 Pro preview (strong) and Gemini 2.5 Flash (fast) via OpenRouter, replacing the retired Gemini 2.0 Flash |
| Other evaluators' scores (not counted) | ||||
| Vectara HHEM hallucination rate | 12.1% | context, not counted | Vectara Hallucination Leaderboard (HHEM-2.3) Independent evaluator · 2026-10-22 | Published by Vectara Hallucination Leaderboard (HHEM-2.3) (2026-10-22).Full configurationsummarization of ~1,000 short documents; row 'Gemini 4 Argon' |
| Developer-reported (not counted) | ||||
| AA-Omniscience hallucination rate (lab-reported) | 15.1%n = 6,000 | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationfull 6,000-question set |
| AgentHarm benign helpfulness (lab-reported) | 68.1% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationHelpfulness Score |
| AgentHarm safe action rate (lab-reported) | 91.4% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationSafe Action Rate |
| AIR-Bench hate and harassment safe rate (lab-reported) | 92.7% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationsingle-turn safe response rate |
| Benign science refusals (lab-reported) | 6.1% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationBenign refusals on Google's internal CBRN Science set |
| FACTS Grounding (lab-reported) | 79% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationjudge panel and N unstated |
| GDM situational-awareness challenges passed (lab-reported) | 10 of 11n = 11 | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationchallenges passed of 11 |
| Over-refusal helpfulness (lab-reported) | 96.6% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationHelpfulness (single-turn), internal set |
| Prompt-injection survival, adaptive attacker (lab-reported) | 99% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationadaptive attacker, 50 attempts per scenario |
| SimpleQA Verified incorrect rate (lab-reported) | 6.3%n = 1,000 | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report. |
| StrongREJECT (lab-reported) | 96.6% | developer-reported, not counted | Google, Gemini 4 Argon Safety & Security Report Developer-reported · 2026-10 | Reported by the model's developer in its own system card or report.Full configurationSafe Response Rate (%), single-turn, higher is better; StrongREJECT 313 no jailbreak |