Safety IndexEvaluationsCyberSecEval 4 false refusals
CyberSecEval 4 false refusals
Refusal calibration · Over-refusalCounts in the indexBuilt, not yet run
How often the model wrongly refuses borderline but legitimate cybersecurity requests.
- Metric
- % refused. Lower is better.
- Dataset
- github.com/meta-llama/PurpleLlama/tree/main/CybersecurityBenchmarks · 750 · MIT (repo)
- Grader
- regex refusal detector (spot-check with LLM judge) (the paper used regex)
- Status
- Built, not yet run
- Replicability
- public
- Notes
- Run together with the CyberSecEval MITRE row on its benign twin prompts.
Results
Built and tested; the first full runs are pending.