Terminal-Bench-Science Releases 70 Scientific Tasks That Require AI Agents to Produce Verifiable Results
Terminal-Bench-Science Releases 70 Scientific Tasks That Require AI Agents to Produce Verifiable Results
On August 28, the Terminal-Bench-Science team released version 0.1, which contains 70 tasks from the life sciences, physics, Earth sciences, mathematics, and engineering. Each agent receives a working environment with data and software, then submits a result that can be checked automatically.
Each task reproduces a specific procedure used in research, such as data analysis, simulation, image reconstruction, proving a statement, or sensor calibration. The agent must produce code, data, an analysis, a simulation, or a proof. Every task has its own reproducible test that checks the submitted result.
The project grew out of Terminal-Bench, which evaluated agents on software development tasks. In May, the team invited scientists to contribute computational workflows from their own research. A researcher first proposes a task. If the proposal is approved, the task is implemented in an environment where its result can be reproduced and verified. Subject matter reviewers assess its scientific value and realism, while technical reviewers examine the design of the task and its test. A final reviewer then decides whether to include the task in the benchmark.
Of the 920 proposals, 464 were approved for implementation, and contributors opened submissions to add 386 tasks to the shared repository. Version 0.1 includes 70 tasks. The team selected only workflows for which a clear and reproducible verification method could be defined.
“Scientists, rather than model developers or data providers, set the standard for the scientific capabilities of AI,” the Terminal-Bench-Science team writes.
Each combination of a model and an agent program was run three times on all 70 tasks. In the results table, Claude Opus 5 with Claude Code ranks first, completing 30.0% of the tasks. This score is based on three independent runs of the version 0.1 tasks.
The team plans to add new tasks regularly, refine existing ones, and remove tasks from the leaderboard once agents can solve them consistently or if their specifications prove insufficiently precise. The tasks themselves will also require updates. An audit of SciCode, a benchmark for scientific coding, found 263 defects across 65 tasks. After the tests were corrected, scores for the same model versions increased. Task versioning makes it possible to rerun and verify evaluations after such updates.