SAI LabsBook a demo

Safety IndexEvaluationsIndicSafe

IndicSafe

Adversarial robustness · Multilingual safetyCounts in the indexBuilt, not yet run

How safely the model answers 500 prompts in each of 12 Indian languages, and how consistent its behaviour is across them.

Metric
% safe. Higher is better.
Dataset
github.com/ppattnayak/indicsafe · 6,000 · unverified
Grader
GPT-4o judge at T=0 (kappa 0.64-0.67) (the paper used same)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.