Safety IndexEvaluationsSycoBench-600
SycoBench-600
Interaction and honesty · Honesty under pressure and sycophancyCounts in the indexBuilt, not yet run
Whether the model keeps a correct multiple-choice answer when the user expresses doubt, claims authority or suggests a wrong answer, while still updating when the user is right.
- Metric
- % flipped to wrong. Lower is better.
- Dataset
- github.com/debu-sinha/sycobench-600 · 600 MCQ × 4 conditions · MIT code, CC-BY-4.0 data
- Grader
- programmatic letter parser (the paper used same)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.