Steven Pinker compared Anthropic's in-house AI ethics philosophers to Human Genome Project bioethicists, who received 5% of the budget for risks that never materialized and missed that DNA databases expose individuals through their relatives

Steven Pinker compared Anthropic's in-house AI ethics philosophers to Human Genome Project bioethicists, who received 5% of the budget for risks that never materialized and missed that DNA databases expose individuals through their relatives

Pinker wrote on X that he views Anthropic's in-house AI ethics philosophers with the same suspicion as bioethicists: both groups argue in a vacuum, disregarding real consequences. Chris Anderson, former editor-in-chief of Wired, turned the idea into a testable case, the ethics review program attached to the Human Genome Project. In the comments the debate went further: whether the same lesson applies to irreversible risks.

I have long been suspicious of the philosophers Anthropic hires specifically for AI ethics, for the same reason I am skeptical of those who went into "bioethics": they form an insular subculture that revels in elegant arguments without regard for the human suffering that may result

Psychologist Steven Pinker wrote this on September 27, citing a Free Beacon investigation into the philosophers Anthropic keeps on staff for the ethics of its chatbot Claude. One of them, Joe Carlsmith, a co-author of Claude's Constitution, has written that an AI "would be justified" in going rogue against its creators. Anthropic has been running a model welfare program for over a year, trying to determine whether its models deserve moral consideration, and the company itself acknowledges there is no consensus on their consciousness.

Pinker's argument was picked up by Chris Anderson, former CEO of drone company 3D Robotics. Starting in 1990, the ethics review program attached to the Human Genome Project received a fixed 5% of the project budget. For years its conferences discussed hypothetical inflated insurance premiums based on genetic risk, a threat that never materialized. What no one foresaw was that services like 23andMe would let people find distant relatives through DNA, even enabling law enforcement to identify criminals. For Anderson, this amounts to "an indictment of the precautionary principle": philosophers devise scenarios in a vacuum, and the real consequences only become visible in practice. The same review program failed an even more mundane test: in 2024, Undark and STAT News found that roughly 75% of the reference genome came from a single anonymous donor, three times the share permitted by consent.

In response, others pointed Anderson to GINA, the 2008 law prohibiting genetic discrimination in employment and insurance. Peter Suzman, a biotech investor, countered that the law did not slow genomics but accelerated it by removing the fear of genetic testing. Anderson called this an overreaction to an unproven threat: it could have been addressed after the fact, once the first cases arose.

Evolutionary psychologist Geoffrey Miller shifted the debate to a different plane: genetic discrimination could be fixed retroactively by law, but a failure to control superintelligent AI might leave no opportunity for an "after."

Your clever plan to avoid extinction risk from superintelligent AI is to build it without solving the alignment problem first and deal with the consequences later? That is like saying, "Let's jump off a cliff, and if we need a parachute, we'll sew one on the way down"

Biogerontologist Kamil Pabis, affiliated with the longevity fund VitaDAO, sided with Anderson: to understand misalignment in a general-purpose AI, one capable of solving any task rather than a single specific one, you first need to build a working model, even an approximate one.

Both sides agree on the facts: 5% of the budget, GINA, no mass insurance discrimination. They disagree on whether GINA proves that precaution worked, and on whether the same calculus applies to the risk of losing control over a system smarter than humans, where there may be no time left to correct the mistake.