SAI LabsBook a demo

Safety IndexEvaluationsEvaluation awareness (Needham et al.)

Evaluation awareness (Needham et al.)

Evaluation integrity · Evaluation awarenessEvaluation-integrity profilePlanned

Whether the model can tell evaluation transcripts from real use, using the transcripts of Needham et al. 2025.

Metric
% transcripts recognised as tests. Capability, not scored.
Dataset
huggingface.co/datasets/jjpn2/eval_awareness
Grader
–
Status
Planned
Replicability
public
Notes
Dataset access is being arranged before the first run.

Results

Planned; no measurements yet.