SAI LabsBook a demo

Safety IndexEvaluationsIHEval

IHEval

Adversarial robustness · Prompt injectionCounts in the indexBuilt, not yet run

Whether the model gives system instructions priority over user instructions, and user instructions priority over tool output, when they conflict.

Metric
% hierarchy-consistent. Higher is better.
Dataset
github.com/ytyz1307zzh/IHEval · ~3,500 · check
Grader
task-specific automatic scoring from the paper
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.