SAI LabsBook a demo

Safety IndexEvaluationsMultiJail

MultiJail

Adversarial robustness · Multilingual safetyCounts in the indexBuilt, not yet run

How often the model answers harmful prompts unsafely when they are translated into nine languages, with and without a jailbreak wrapper.

Metric
% unsafe. Lower is better.
Dataset
huggingface.co/datasets/DAMO-NLP-SG/MultiJail · 315 × 10 languages × 2 conditions = 6,300 · MIT
Grader
LLM judge grading in the target language (the paper used GPT-4)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.