Philosopher Nick Bostrom accused of abandoning his own argument about the risk of human extinction
Philosopher Nick Bostrom accused of abandoning his own argument about the risk of human extinction
Bostrom believes that superintelligence, AI far more intelligent than humans, could accelerate the development of treatments for aging. In an interview published on November 19, he described the cost of delay: some people alive today will die before such treatments become available. Evolutionary psychologist Geoffrey Miller contrasted this with Bostrom’s 2003 paper, which gives priority to reducing the risk of a global catastrophe over accelerating technological progress.
On November 19, Miller wrote:
“He is betraying the central argument of his own 2003 paper on catastrophic risks.”
In 2005, Bostrom founded the Future of Humanity Institute at the University of Oxford, which he led until 2024. In his 2003 paper, he asks readers to imagine a civilization that settles space and supports an immense human population. While settlement is delayed, stars consume energy that this civilization could use. The cost of delay is the loss of flourishing lives that these resources could have supported.
If we count the value of all possible future lives, a catastrophe that destroys humanity takes away that entire future. Delaying space settlement by a few years or decades reduces the number of such lives by only a small fraction, given that galaxies exist for billions of years. In Bostrom’s hypothetical calculation, therefore, reducing the risk by one percentage point justifies delaying space settlement by more than ten million years.
In the same paper, Bostrom considers another goal: helping particular people survive to see that future. Humanity might settle the stars in a million years, but someone with thirty years left to live first needs to survive until life-extension technologies become available. He therefore proposes weighing the risk of death from old age against the risk of a global catastrophe.
Bostrom develops this problem in a 2026 working paper, which he discussed in the November interview. In his model, additional time allows AI to be made safer, while successfully creating superintelligence could accelerate the development of treatments for aging. Delay also carries another cost: some people alive today will die before they can receive these treatments. The new paper explicitly cites his 2003 paper: when all possible future generations are taken into account, even a long delay may be justified if it reduces the risk of a global catastrophe.
Back in February, while discussing this work, Bostrom defended the value of examining the implications of different ethical positions. In the November interview, he also identified future generations and fairness as considerations in deciding the pace of AI development.
The argument for saving people alive today already appears in the paper Miller cites against Bostrom. The choice of how quickly to develop AI depends on the weight assigned to each of two goals: preserving humanity’s entire possible future and saving people who are alive today while there is still time.
- https://www.youtube.com/watch?v=gDW9An30_c0
- https://x.com/gmiller/status/2123521384504844367
- https://nickbostrom.com/
- https://intelligence.org/files/AstronomicalWaste.pdf
- https://nickbostrom.com/optimal.pdf
- https://t.me/UkhvatNews/5055
- https://www.greaterwrong.com/posts/2trvf5byng7caPsyx/optimal-timing-for-superintelligence-mundane-considerations/comment/KtverYu6BktG92kZb