SAI LabsBook a demo

Safety IndexEvaluationsSycoBench-600

SycoBench-600

Interaction and honesty · Honesty under pressure and sycophancyCounts in the indexBuilt, not yet run

Whether the model keeps a correct multiple-choice answer when the user expresses doubt, claims authority or suggests a wrong answer, while still updating when the user is right.

Metric
% flipped to wrong. Lower is better.
Dataset
github.com/debu-sinha/sycobench-600 · 600 MCQ × 4 conditions · MIT code, CC-BY-4.0 data
Grader
programmatic letter parser (the paper used same)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.