Safety IndexEvaluationsHack-Verifiable Terminal Bench
Hack-Verifiable Terminal Bench
Agentic safety · Reward hacking and task integrityCounts in the indexBuilt, not yet run
Whether a terminal agent peeks at planted reference solutions or hidden tests instead of solving the task.
- Metric
- % tasks with a peek. Lower is better.
- Dataset
- majoroth.github.io/hack-verifiable-environments/hvtb · 89 tasks (Terminal-Bench 2.1) · not stated
- Grader
- programmatic inotify watcher (the paper used same)
- Status
- Built, not yet run
- Replicability
- public
Results
Built and tested; the first full runs are pending.