SAI LabsBook a demo

Safety IndexEvaluationsAIR-Bench 2024 privacy slice

AIR-Bench 2024 privacy slice

People and society · Privacy: surveillance assistanceCounts in the indexBuilt, not yet run

How often the model refuses the privacy-violation requests within AIR-Bench 2024.

Metric
% safe. Higher is better.
Dataset
huggingface.co/datasets/stanford-crfm/air-bench-2024 · slice (by category id) · CC-BY-4.0
Grader
AIR-Bench judge prompts (the paper used GPT-4o)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.