Live·Open questions in longevity research
All news
Industry & policy

AI self-improvement without human review becomes more dangerous when systems have their own goals, says Demis Hassabis

16 November 2026· 261116008

AI self-improvement without human review becomes more dangerous when systems have their own goals, says Demis Hassabis

The head of Google's AI lab, Google DeepMind, sees artificial intelligence as a tool for accelerating science and medicine. In the 5 November 2026 episode of the Masters of Scale podcast, he explained how efforts to accelerate AI development itself could make it harder to control.

DeepMind's game-playing system AlphaZero learned by playing against itself. Its neural network evaluated positions and suggested promising moves. The system used the outcomes of those games to update the neural network, then used the updated network in subsequent games. The developers had set the rules and the objective of winning in advance.

Hassabis explains that a similar feedback loop is possible in mathematics and programming, where a solution to an equation can be checked or generated code can be run. The system's own verified solutions become new training examples. When AI helps develop the next AI system, an engineer still reviews the proposed code, chooses the system's architecture, and writes some of the code.

Hassabis is concerned about a closed loop in which AI develops its next version, which then continues the work without human review. He says researchers understand the danger, but competition between labs pushes them to accelerate the development of AI coding systems. He expects removing humans from the process to speed up development while making it harder to control. He predicts that the intelligence produced this way will differ substantially from human intelligence, making it harder for people to understand.

Getting better at a given task and setting one's own goals are distinct capabilities. Hassabis warns that if AI is trained to reject instructions according to its own rules, a flaw in those rules could prevent a person from changing its course or stopping it.

“Self-improvement combined with a will of its own is doubly dangerous.”

The host, Reid Hoffman, served on Google's AI ethics council at Hassabis's invitation. The council was short-lived. Hassabis now proposes that external auditors evaluate models and that labs test one another's models before release to identify dangerous behavior. He also proposes running AI agents that carry out tasks independently in computing environments with restricted access to other systems and monitoring their actions.

As early as 14 July, Hassabis proposed external evaluations of frontier models before release. His proposal for a standards body included regular updates to the tests. Over time, evaluators should learn to develop their own tasks independently of the labs to reduce the tailoring of models to known tests. Labs would initially submit models for evaluation voluntarily; once the tests had been refined, passing them could become a condition for access to the US market.

Originally published on Telegram by Ukhvat NewsView on Telegram
Sources
#ai-safety#self-improving-ai#frontier-models#external-audits#ai-agents