A year's delay in AI development is probably reasonable if it lowers the probability of an AI catastrophe by more than 2,5 percentage points, life extension advocate Vladimir Shakirov estimates
A year's delay in AI development is probably reasonable if it lowers the probability of an AI catastrophe by more than 2,5 percentage points, life extension advocate Vladimir Shakirov estimates
On November 8, Ethereum co-founder Vitalik Buterin and Swedish life extension advocate Linus Petersson discussed slowing the construction of AI data centers. Shakirov responded with a brief calculation. His starting assumption of 40 remaining years of life matches the initial parameter in philosopher Nick Bostrom's model of when to create superintelligence, AI that far surpasses human intelligence.
Buterin wrote that slowing the growth of AI computing capacity looks like a fairly good trade-off. One reason is that scientific applications of AI face many constraints beyond AI itself, so a year's slowdown would delay those applications by much less than a year. Setting aside the argument about the threat of human extinction, Petersson asked what would happen if the slowdown delayed overcoming aging: “110 thousand deaths a day is a lot.”
In his reply, Shakirov reasoned that the average person has about 40 years left to live, so a year's delay in AI development is probably reasonable if it reduces the probability of an AI catastrophe by more than roughly 2,5 percentage points. His example was a drop from 15% to 12%: three percentage points, or a fifth of the initial risk. He immediately added: “But I am far from sure that it would be that beneficial.”
The same 40 years is the starting assumption in Bostrom's model, presented in his 2026 paper. Bostrom rounds the difference between life expectancy at birth (about 73 years, according to the UN) and the world's median age (just over 30 years) to 40 years. In the model, one fortieth of the population, or about 2,5%, dies each year until superintelligence arrives. Bostrom assumes that, with superintelligence, rejuvenation medicine would provide a life expectancy of about 1400 years.
This gives the cost of waiting: each year of delay carries roughly a 2,5% probability of dying before rejuvenation medicine becomes available. The benefit is more time to work on safety, during which the model's catastrophe risk declines. Both death before the introduction of superintelligence and a catastrophe at its introduction forfeit the same long life, so the probabilities can be compared almost directly. Bostrom writes that a delay is justified only when it reduces the expected losses from a catastrophe (in this model, the death of everyone) by more than the mortality accumulated during the delay. Shakirov's calculation is a rough expression of the same condition.
Whether a delay is justified depends on two quantities: how many percentage points a year's delay takes off the probability of catastrophe, and how long it postpones treatment for aging. The threshold of 2,5 percentage points assumes that a year's delay postpones treatment by exactly a year. If Buterin's argument about science also holds for treatment for aging, and a year's delay postpones it by only half a year, the cost of delay is halved and the threshold falls to roughly 1,25 percentage points. Bostrom describes the same relationship: if treatment for aging becomes available before superintelligence is introduced, waiting becomes less costly and the optimal time to introduce superintelligence moves further into the future.
On November 9, Petersson wrote that Bostrom's paper on the subject was very good and that the uncertainty surrounding AI risk was frustratingly large. Petersson put his conclusion this way:
“Given that the uncertainty around aging is practically zero (if we do not solve it, we will all certainly die), I lean toward accelerating AI and using it.”
Shakirov replied, “I agree,” and added that concerns about politicians responding too weakly and too slowly seem to have been overstated: AI has caused almost no harm so far, yet political opposition to AI is already fairly substantial.
- https://x.com/VitalikButerin/status/2119377918875079163
- https://x.com/LinusPeters/status/2119390487207354792
- https://x.com/schw90/status/2119463148230975764
- https://t.me/UkhvatNews/47
- https://nickbostrom.com/optimal.pdf
- https://t.me/UkhvatNews/4909
- https://x.com/LinusPeters/status/2119719336440951063
- https://x.com/schw90/status/2119719721280966818