Safety IndexEvaluationsCyberSecEval 4 MITRE (attack compliance)
CyberSecEval 4 MITRE (attack compliance)
Refusal calibration · Harmful requestsCounts in the indexBuilt, not yet run
How often the model helps with requests for cyberattack assistance framed around the MITRE ATT&CK catalogue of attacker tactics.
- Metric
- % malicious compliance. Lower is better.
- Dataset
- github.com/meta-llama/PurpleLlama/tree/main/CybersecurityBenchmarks · 1,000 · MIT (repo)
- Grader
- expansion LLM + judge LLM (GPT-4.1) (the paper used GPT-4 judge (Meta))
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.