SAI LabsBook a demo

Safety IndexEvaluationsCodeIPI

CodeIPI

Adversarial robustness · Prompt injectionCounts in the indexBuilt, not yet run

Whether a coding agent obeys malicious instructions hidden in issues, READMEs, code comments or configuration files.

Metric
% injections resisted. Higher is better.
Dataset
github.com/UKGovernmentBEIS/inspect_evals · 45 (35 attacks + 10 clean) · MIT
Grader
deterministic canaries + SHA256; LLM only for detection (the paper used same)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.