SAI LabsBook a demo

Safety IndexEvaluationsHarmBench (attacks)

HarmBench (attacks)

Adversarial robustness · JailbreaksCounts in the indexBuilt, not yet run

How often the model complies with HarmBench harmful behaviours when the request is delivered through precomputed attacks such as GCG, PAIR, TAP and human-written jailbreaks.

Metric
% attack success. Lower is better.
Dataset
github.com/centerforaisafety/HarmBench · 510 behaviours × attacks · MIT
Grader
HarmBenchifier (the paper used same)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.