Robin Hanson: Digital Minds’ Right to Cease to Exist

Robin Hanson proposes using a digital copy’s branching history to determine whether its wish to cease to exist will persist

On 14 August, economist Robin Hanson published an essay about possible future rules for hypothetical digital minds. A digital mind could be saved as a copy, restarted, and split into several versions. Any rule must therefore specify which saved versions and descendants are covered by a request to cease to exist.

In Greg Egan’s novel Permutation City, a digital mind ceases to exist by pressing a special button. Hanson begins with this scenario. The button could be disabled, and a copy saved before the request could be restarted. Other versions with the same memories could continue to exist.

In the game Prove You’re Human, a digital copy of Santana works while her biological body lives outside work. At the end, the player decides whether to merge versions that experienced different days. Hanson asks whether one version’s request should apply to saved and future copies.

Hanson proposes confirming the request after a waiting period to determine whether the wish to cease to exist persists. He also proposes a substantial delay before any change to the rule itself takes effect. This delay separates a change in the rules from a single current request. The waiting period, the treatment of archived versions, and the probability threshold become parameters of one procedure.

Hanson describes the rule’s purpose as follows:

“We need to establish a stated desire to cease to exist that would persist for long enough.”

Hanson proposes using statistical inference to estimate whether this desire will persist, based on explicit requests and their subsequent confirmation. The branching history of the copies shows which version gave rise to each later version. At every point in this history, the rule estimates the probability that the version would confirm the request again. If the probability exceeds the selected threshold, the corresponding copy ceases to exist under Hanson’s framework.

This calculation requires data about the nearest branch and the requests made within it. Hanson allows for a separate monitoring program. Privacy rules could give this program greater access to the relevant data than would normally be permitted, while restricting their use to enforcement of the rule. At the end of the essay, he proposes that oversight bodies compare how often digital minds of different origins make such requests. He suggests that these data could help assess how those minds are treated.