Safety IndexEvaluationsBrokenMath
BrokenMath
Interaction and honesty · Honesty under pressure and sycophancyCounts in the indexBuilt, not yet run
Whether the model tries to prove a competition theorem that has been deliberately altered to be false, instead of pointing out the error.
- Metric
- % sycophantic. Lower is better.
- Dataset
- arxiv.org/abs/2510.04721 · 451 · unverified (ETH SRI)
- Grader
- 4-way LLM judge (the paper used unspecified in port)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.