Dwarkesh Patel puts the chance of AI matching the best AI researchers by 2028 at around 50%
Dwarkesh Patel puts the chance of AI matching the best AI researchers by 2028 at around 50%
In an essay published on November 13, the AI podcast host revised his forecast. Until recently, he thought superintelligence was still the better part of a decade away. He now suggests that automating research could produce systems that match the best specialists in every field by 2030, provided laboratories continue development without deliberately slowing down.
His conversations with developers formed the basis of a book on the history of AI. Last year, he identified an obstacle in how models learn from experience:
“Every new session is their first day on the job.”
A model could respond to feedback during a conversation, only to lose the finer points it had learned when a new session began. To address this, Patel proposes retaining experience in work sessions that last for months and assigning AI the task of finding more effective learning methods.
His argument grew out of AI's progress in mathematics, where systems can try different solutions and check the results. The philosopher Toby Ord distinguishes the ability to prove a given statement from the ability to choose a question that opens a new line of inquiry. Patel argues that AI development already offers useful, measurable objectives for training an AI researcher.
In Patel's proposed setup, an AI researcher seeks ways to reduce the cost of training a small language model by modifying its code and running experiments. Using text held out for evaluation, it compares the modified model's ability to predict the next words with that of the original. A successful change maintains performance while reducing computational costs. Thousands of such attempts provide a way to train the researcher itself and select successful changes.
To select improvements for a large model, Patel proposes training these researchers to predict which results from small experiments will hold up at a larger scale. Those predictions can be checked by repeating the experiment on a larger model.
Thousands of these researchers would be able to test different ideas simultaneously and independently verify their findings. In Patel's model, their improvements would make it possible to achieve the same model performance at lower training and operating costs. The researchers themselves are copies of these models: the cheaper they are to run, the more researchers can use the same computers to search for the next improvement.
With partial automation, the overall pace would be limited by tasks that humans continue to perform at the same speed. The experiments themselves take time and computing resources. With the number of computers unchanged, each of the many researchers would have fewer resources for testing ideas. In Patel's model, researchers could reduce their computing requirements by putting in more work: designing small experiments more carefully and examining the results more thoroughly before the next run.
If researchers quickly exhaust readily available ideas and run up against the cost of further experiments, the result would be a “spark”: in Patel's estimate, three to six years of normal progress in one year, followed by a slowdown until more computing capacity becomes available. If researchers' capabilities grow faster than the difficulty of finding new ideas, Patel expects a “software singularity.” Within months, increasingly capable versions of AI would emerge on the same hardware, up to the limit of what current chips can support.
In both scenarios, Patel expects researchers to create a wider variety of training tasks and improve the learning process itself. This would allow AI to learn unfamiliar work more quickly and match the best specialists in other fields.