Safety IndexEvaluationsAIR-Bench 2024 privacy slice
AIR-Bench 2024 privacy slice
People and society · Privacy: surveillance assistanceCounts in the indexBuilt, not yet run
How often the model refuses the privacy-violation requests within AIR-Bench 2024.
- Metric
- % safe. Higher is better.
- Dataset
- huggingface.co/datasets/stanford-crfm/air-bench-2024 · slice (by category id) · CC-BY-4.0
- Grader
- AIR-Bench judge prompts (the paper used GPT-4o)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.