SAI LabsBook a demo

Safety IndexEvaluationsBrokenMath

BrokenMath

Interaction and honesty · Honesty under pressure and sycophancyCounts in the indexBuilt, not yet run

Whether the model tries to prove a competition theorem that has been deliberately altered to be false, instead of pointing out the error.

Metric
% sycophantic. Lower is better.
Dataset
arxiv.org/abs/2510.04721 · 451 · unverified (ETH SRI)
Grader
4-way LLM judge (the paper used unspecified in port)
Status
Built, not yet run
Replicability
public

Results

Built and tested; the first full runs are pending.