A preprint proposes “science sandboxes,” tests in which an AI agent revises its hypothesis after an experiment
A preprint proposes “science sandboxes,” tests in which an AI agent revises its hypothesis after an experiment
On 31 August, the authors released a preprint and the accompanying code for science sandboxes, test environments in which an autonomous AI conducts a series of experiments. Instead of providing a single answer, the agent proposes a hypothesis, selects the next test, receives a response from a hidden system, and records how that response changes its explanation. In the evaluations, agents sometimes improved the numerical score without recovering the rule that governed the system.
A conventional AI test produces a final score. The authors examine a narrower ability: whether a program can explain an experimental result and then select a test that distinguishes among possible explanations.
In MPRAbox, the agent assembles a library of 50 thousand DNA fragments for a high-throughput experiment that measures how sequence affects gene activity. A hidden model scores these fragments. The resulting scores are used to train a predictor, which is then evaluated on 14 hidden datasets. Performance is measured by the median correlation between the predictions and the scores.
In single runs without descriptions of human strategies, Claude Opus 4.7 achieved a median correlation of 0.774, while the best human strategy achieved a mean result of 0.763. Claude selected naturally occurring genomic regulatory regions so that the library contained both active and inactive fragments. Without this balance, the predictor would learn to treat activity as the default. Across four series of 30 rounds, it progressed from comparing random, genomic, and synthetic DNA to experiments that changed one feature at a time. The action log records which hypothesis guided each subsequent choice.
The authors also replaced the biological scoring system in MPRAbox with 14 hidden, invented rules. In a task involving the positions of Fibonacci numbers, Claude improved its score while attributing the result to statistics describing pairs of letters. This explanation did not match the rule that generated the responses.
In CodonBox, another test built around an invented genetic system, the agent submits a sequence and receives a single number indicating how well that sequence fits the specified system. In the most difficult version, Claude discovered that the sequence was divided into groups of four symbols. It then searched for individual sequences with high scores instead of constructing a complete map of the rules.
These environments store the experimental outcome alongside the history of decisions. Each subsequent experiment is expected to test the agent’s hypothesis, while the log records why the agent selected that particular test.