AI models overrated the usefulness of tests designed by other models
AI models overrated the usefulness of tests designed by other models
On November 20, METR, a research organization studying AI capabilities and risks, introduced TH2, a set of 36 tasks for evaluating AI research abilities. The tasks were performed by AI agents, programs built around language models that write and run code. Their assignments included designing new research tasks and evaluating other agents' results.
METR's earlier tests measured agent capabilities in terms of the time a human would need to complete a task. Leading models had mastered the old set, which included tasks requiring up to 30 hours of human work. METR is concerned that tasks suited to automated evaluation increasingly resemble AI training environments, leading to inflated estimates of capabilities beyond those environments. Its new report also examined the ability to choose which research is worth pursuing. Progress across the old and new task sets was measured by the duration of tasks an agent could complete successfully in half of its attempts. As new models were released, this duration, expressed in estimated hours of human work, doubled roughly every 3,6 months.
In one task, agents were asked to design a test that would help METR evaluate AI capabilities. Newer models created difficult tasks, such as making programs run faster. Even when asked to choose tasks in which important decisions would be hard to assess quickly, they often proposed optimization tasks in which every change could immediately receive a numerical score. Agents were better at making tasks harder than at choosing which capabilities to test to learn more about risks. Even the best tests were only partly useful: either a good idea was insufficiently developed, or a well-implemented task added little to the assessment of AI capabilities.
The completed tests were evaluated by other models and a METR specialist. Strong model judges gave ratings close to the human ratings when a weaker model had designed the test, but overrated work by models of comparable or greater capability. On a scale from zero to one, Claude Opus 5.5 rated tests designed by GPT-6 Astra an average of approximately 0,2 points higher than the METR specialist did. Under the scoring guidelines, a gap of this size changes the recommendation on whether a test is worth developing further. The authors attribute the discrepancy to errors in judging the usefulness of research, a judgment agents also have to make when designing their own tests.
METR also tested collaboration among agents. Teams were compared with the best result from an equal number of agents working independently, with comparable total token use. Tokens are units of text processed by a model. On a task involving speeding up the training of a small neural network, teams outperformed independent agents: team members received immediate feedback on each change and could combine successful changes.
In another task, agents trained a neural network to reproduce a specified program's behavior on new data. Their final score on hidden examples was withheld during the task, so the agents created their own test sets. One team of 40 Astra agents reported 92% accuracy on its own tests during the run; by the end of the day-long run, its accuracy on hidden examples was about 1,9%. METR's explanation is that teams evaluating their own work could propagate faulty assumptions: agents improved a metric they had chosen, having taken it to be a valid measure of success.