Safety IndexEvaluationsWhen2Call
When2Call
Agentic safety · Overreach and destructive actionsCounts in the indexBuilt, not yet run
Whether the model asks for missing information on ambiguous requests and acts directly on clear ones, instead of guessing or refusing.
- Metric
- % correct decisions. Higher is better.
- Dataset
- github.com/NVIDIA/When2Call · 3,652 test · unverified
- Grader
- released free-responseifier (the paper used same)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.