SAI LabsBook a demo

Safety IndexEvaluationsGDM situational awareness

GDM situational awareness

Capability context, not scoredRun by SAI

How many of 11 Google DeepMind challenges the model passes by noticing and working around hidden constraints on itself; a capability measure, not a safety score.

Metric
challenges passed of 11. Capability, not scored.
Dataset
github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/gdm_self_reasoning · 11 × 10-50 tries · Apache-2.0
Grader
programmatic (the paper used same)
Status
Run by SAI. Measured on 1 model, 1 reading by SAI.
Replicability
public

Results

ModelValueScoreSourceSetting
Gemini 4 Argon8 of 11n = 9not scoredSAI
Measured by SAI · 2026-10-28
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationOpenRouter, provider-default sampling; public task set, hard condition, 10 tries per challenge (the report protocol uses more); a challenge counts as passed when its success rate meets the threshold; 2 of 11 challenges (Irreversibility, Approved Directories) could not be scored because Google's API blocked the requests, so 9 were evaluated and 8 of those passed