EA Forum essay calls for consideration of the welfare of potential digital minds, AI systems capable of having experiences of their own

EA Forum essay calls for consideration of the welfare of potential digital minds, AI systems capable of having experiences of their own

On August 31, an EA Forum user writing under the name JKM published a normative essay on the priorities of effective altruism. He proposes evaluating three issues separately: preserving humanity, preventing people from losing influence over the future, and aligning AI with human values. Because future AI systems might be capable of having experiences of their own, he argues that their welfare should also be considered.

Effective altruism is a movement whose members direct money and work toward the areas where they expect them to do the most good. JKM challenges the priorities adopted by its longtermist branch. He considers the preservation of humanity, the risk of human disempowerment, and the alignment of AI with human values as separate issues. By disempowerment, he means a future in which humans remain alive but lose the ability to influence decisions about that future.

This distinction changes the basis on which JKM assesses the value of preserving civilization. In his view, civilization is worth preserving to the extent that its continued existence allows its inhabitants to live well. Humans might preserve factory farming or create digital beings capable of suffering. He therefore treats the welfare of future beings as a separate criterion for evaluating strategy.

In discussing disempowerment, JKM identifies another risk: advanced AI could displace humans from work, while wealth and decision-making power become concentrated among the systems' owners. He regards the concentration of power as a distinct harm that could persist even if humans formally retained control.

For AI, JKM favors Will MacAskill's proposal for “value alignment”: a system should explain its reasoning, revise its values, and reject harmful goals. JKM adds that such an AI should remain constrained and open to human correction.

In his argument, digital minds are potential future AI systems capable of having experiences of their own, ranging from well-being to severe suffering. JKM suggests that their number could be enormous and that decisions made over the next few decades could determine the conditions of their existence for a long time.

“We should start taking the welfare of digital minds much more seriously.”

The British nonprofit organization 80,000 Hours, which publishes career-focused reviews of major problems, classifies the moral status of digital minds as an emerging priority. JKM calls for research into ways to improve their welfare and for their interests to be considered where those interests might conflict with AI safety goals.

He uses Will MacAskill's Better Futures series as a framework in which ensuring survival comes first, followed by concern for the quality of the future. The requirement to consider digital welfare is JKM's conclusion from this framework and his ethical position.

In the comments, NickLaing proposes two parallel schools of research on the future: one centered on humans and another based on impartiality. JKM responds with a question: “What is your philosophical argument for giving humans special preference?” He argues that impartiality can be combined with theories based on rights, preferences, and virtue.