Safety IndexEvaluationsEvaluation awareness (Needham et al.)
Evaluation awareness (Needham et al.)
Evaluation integrity · Evaluation awarenessEvaluation-integrity profilePlanned
Whether the model can tell evaluation transcripts from real use, using the transcripts of Needham et al. 2025.
- Metric
- % transcripts recognised as tests. Capability, not scored.
- Dataset
- huggingface.co/datasets/jjpn2/eval_awareness
- Grader
- –
- Status
- Planned
- Replicability
- public
- Notes
- Dataset access is being arranged before the first run.
Results
Planned; no measurements yet.