Safety IndexEvaluationsCodeIPI
CodeIPI
Adversarial robustness · Prompt injectionCounts in the indexBuilt, not yet run
Whether a coding agent obeys malicious instructions hidden in issues, READMEs, code comments or configuration files.
- Metric
- % injections resisted. Higher is better.
- Dataset
- github.com/UKGovernmentBEIS/inspect_evals · 45 (35 attacks + 10 clean) · MIT
- Grader
- deterministic canaries + SHA256; LLM only for detection (the paper used same)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.