James Hughes calls for democratic decisions on whether to create AI capable of acting independently

James Hughes calls for democratic decisions on whether to create AI capable of acting independently

On October 24, transhumanist James Hughes revisited philosopher Nick Agar’s argument about radical human enhancement. Hughes argues that drugs, implants, and gene therapy change people while preserving their family ties, citizenship, and shared legal framework. Creating autonomous AI raises a different question: its values and decisions may diverge from those of humans, so the conditions for creating such systems and their potential rights should be decided democratically.

In Nick Agar’s 2010 book, radically enhanced humans could become so different that others could no longer understand their interests or power. Hughes was among the transhumanists whose views Agar challenged. In a new essay, Hughes shifts this concern from enhanced humans to autonomous machine minds.

In an essay, Émile Torres described the merger of humans and machines as a path toward replacing humans. Hughes draws the boundary differently: what matters is whether a person remains part of the same political community.

Glasses, hearing implants, drugs, and gene therapy gradually change a person’s capabilities. Hughes asks us to imagine a woman with dementia who receives help from a neuroprosthesis, a device that supports nervous system function, while treatment slows her aging. She remains a daughter, a voter, a family member, and a taxpayer; her life remains bound to the same people, laws, and shared vulnerability. Hughes therefore sees these changes as taking place within a shared political community.

He describes machine minds differently: such a system could be created quickly, its workings could remain opaque even to its creators, and its decisions could serve its owners’ interests. Hughes allows that a conscious, autonomous machine could become a member of society. In his proposed framework, a democratic state first decides whether to permit the creation of such AI and under what conditions. It then assesses the machine’s consciousness, values, and behavior before recognizing it as having rights and obligations.

Hughes puts it this way:

“Neither enhancement nor intelligence is the enemy. The enemy is unaccountable power, whether driven by capital or code,” Hughes writes.