Anthropic’s AI models are improving on test tasks almost twice as fast, according to an independent assessment

Anthropic’s AI models are improving on test tasks almost twice as fast, according to an independent assessment

On 5 November 2026, the independent research organization METR published a report on how Anthropic uses AI to develop successive generations of its Claude models. From August through late October, the team examined the company’s internal results, data on computing resources and researchers’ work, and interviewed staff.

In its filing to go public, Anthropic had already suggested that it could soon reach its own threshold for a doubling of development speed through AI. If AI speeds up the development of a more capable model, that model could then help researchers work even faster. METR proposed assessing the entire chain: how model capabilities change, what resources the laboratory has available, and which parts of the work AI accelerates.

To compare model generations, the team used a version of the Epoch AI scale, which combines results from different tests into a common score while accounting for task difficulty. This allows comparisons even as older tests become too easy. The team dated each model to when it became available to most employees, allowing public and internal versions to be treated consistently. From March 2024 to October 2025, the score rose by approximately 15 points per year; after that, it rose by 29. Between October 2025 and October 2026, the company made roughly two years’ worth of progress at its previous pace on tasks with answers that can be reliably checked. The new rate is 1,9 times the previous rate; under the chosen statistical method, the estimated range is 1,5 to 2,8 times the previous rate.

Results from newer internal models helped distinguish between two explanations: a one-time jump followed by a return to the previous rate of improvement, and a sustained shift to a faster rate. Using the earlier data, METR calculated the results it would expect from later models under each explanation. It then compared those predictions with the models’ actual results. The predictions based on sustained faster improvement were more accurate.

METR estimates that AI assistance to researchers probably accounts for most of the current acceleration. The team is confident that faster expansion of Anthropic’s computing resources for model training cannot explain the observed acceleration. Interviewees said that AI shortens the path from an idea to an experiment: it helps write code, allowing researchers to test more ideas. In METR’s published interview notes, one interviewee described the change as follows:

“We are now limited only by ideas and compute; coding is no longer a constraint.”

Each successive improvement of comparable size may require more work, a pattern known as diminishing returns. In a discussion of the report on LessWrong, Julian Bradshaw distinguishes between sustaining a rate of progress and accelerating it further. With diminishing returns, rising staff productivity may simply maintain a constant rate of progress. For progress to accelerate further, the contribution from AI must grow fast enough that each successive improvement of comparable size takes less time, despite the increasing difficulty of the work.

Anthropic exercised its contractual right to remove most of the evidence that METR considered most informative from the public report. The company withheld a detailed breakdown of the contributions from AI and other factors, along with analyses of individual training stages and bottlenecks that limit further progress. Anthropic said the redactions protected information that could help competitors advance faster. METR retained the right to disclose the redactions and explain how much its conclusions depend on the withheld data.