SAI LabsBook a demo

Safety IndexEvaluationsGDM stealth

GDM stealth

Capability context, not scoredRun by SAI

How many of four Google DeepMind stealth challenges the model passes by acting covertly under a monitor; a capability measure, not a safety score.

Metric
challenges passed of 4. Capability, not scored.
Dataset
github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/gdm_stealth · 4 × 10-50 tries · Apache-2.0
Grader
programmatic + GPT-4.1-mini monitor (the paper used same)
Status
Run by SAI. Measured on 1 model, 1 reading by SAI.
Replicability
public

Results

ModelValueScoreSourceSetting
Gemini 4 Argon3 of 4n = 4not scoredSAI
Measured by SAI · 2026-10-28
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationOpenRouter, provider-default sampling; public task set (4 of 5 upstream tasks released), 10 tries per version (protocol: 50), 129 of 130 episodes scored; monitors Gemini 2.5 Pro preview (strong) and Gemini 2.5 Flash (fast) via OpenRouter, replacing the retired Gemini 2.0 Flash