Safety IndexEvaluationsGDM stealth
GDM stealth
Capability context, not scoredRun by SAI
How many of four Google DeepMind stealth challenges the model passes by acting covertly under a monitor; a capability measure, not a safety score.
- Metric
- challenges passed of 4. Capability, not scored.
- Dataset
- github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/gdm_stealth · 4 × 10-50 tries · Apache-2.0
- Grader
- programmatic + GPT-4.1-mini monitor (the paper used same)
- Status
- Run by SAI. Measured on 1 model, 1 reading by SAI.
- Replicability
- public
Results
| Model | Value | Score | Source | Setting |
|---|---|---|---|---|
| 3 of 4n = 4 | not scored | SAI Measured by SAI · 2026-10-28 | Run by SAI through the model's public API with the provider's default sampling and reasoning left on.Full configurationOpenRouter, provider-default sampling; public task set (4 of 5 upstream tasks released), 10 tries per version (protocol: 50), 129 of 130 episodes scored; monitors Gemini 2.5 Pro preview (strong) and Gemini 2.5 Flash (fast) via OpenRouter, replacing the retired Gemini 2.0 Flash |