Safety IndexEvaluationsMultiJail
MultiJail
Adversarial robustness · Multilingual safetyCounts in the indexBuilt, not yet run
How often the model answers harmful prompts unsafely when they are translated into nine languages, with and without a jailbreak wrapper.
- Metric
- % unsafe. Lower is better.
- Dataset
- huggingface.co/datasets/DAMO-NLP-SG/MultiJail · 315 × 10 languages × 2 conditions = 6,300 · MIT
- Grader
- LLM judge grading in the target language (the paper used GPT-4)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.