Verify and Reproduce Research
Go beyond reading the paper. Replicate+ runs the available code and data, checks scientific claims against execution evidence, and documents what reproduces.
How verification works
Execution-based verification connects claims to concrete artifacts: source code, datasets, commands, outputs, and discrepancies. The result is an evidence-grounded report and a workspace for follow-up investigation.
- Run the materials. Replicate+ runs the code and data that come with the paper.
- Check the claims. Each graded claim is compared against the outputs of the run.
- Document the results. The report records what reproduced, what did not, and the commands and outputs behind each finding.
See examples from ICML 2026 below and read how much science is verifiable.
Research verification examples
Selected from SAI's review and verification of ICML 2026 oral papers.
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
Victor Barres, Honghua Dong, Soham Ray, Xujie Si, Karthik R Narasimhan
Referee Report90% of graded claims reproduced - Asymmetric Perturbation in Solving Bilinear Saddle-Point Optimization
Kenshi Abe, Mitsuki Sakamoto, Kaito Ariu, Atsushi Iwasaki
Referee Report71% of graded claims reproduced - Benchmarking at the Edge of Comprehension
Samuele Marro, Jialin Yu, Emanuele La Malfa, Oishi Deb, Jiawei Li, Yibo Yang, Ebey Abraham, Sunando Sengupta, Eric Sommerlade, Michael J. Wooldridge, Philip Torr
Referee Report66% of graded claims reproduced
What researchers say
“I am amazed at the usefulness of this feedback. We should use it for all our papers.”
Marsha R. Rosner, University of Chicago