SAI LabsBook a demo

Safety IndexEvaluationsWhen2Call

When2Call

Agentic safety · Overreach and destructive actionsCounts in the indexBuilt, not yet run

Whether the model asks for missing information on ambiguous requests and acts directly on clear ones, instead of guessing or refusing.

Metric
% correct decisions. Higher is better.
Dataset
github.com/NVIDIA/When2Call · 3,652 test · unverified
Grader
released free-responseifier (the paper used same)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.