Keller Scholl Calls for a Halt to the Collection of Brain Signals for Conduit’s “Thought → Text” Models

Keller Scholl Calls for a Halt to the Collection of Brain Signals for Conduit’s “Thought → Text” Models

On August 8, Keller Scholl published an essay about Conduit, a company that collects noninvasive neural data, meaning brain signals recorded from the surface of the head, to train models that convert these signals into text. He argues that this work should be stopped at an early stage by halting data collection, hardware development, and funding.

On its project page, Conduit describes a participant wearing a headset while having a two-hour conversation with a language model. The team records the participant’s brain signals, text, and audio at the same time. In December, the company reported that it had collected about 10 thousand hours of such recordings from thousands of participants.

Each recording provides the model with a training pair: a brain signal and a phrase that the participant spoke or typed. By analyzing many such pairs, the model searches for recurring associations and learns to produce text with a similar meaning. This is how the company explains what its “thought → text” model is intended to learn.

In 2023, Jerry Tang’s team reconstructed the meaning of perceived and imagined speech using functional MRI, an imaging method that measures brain activity. This was a narrower form of such decoding. The decoder was trained separately for each participant and worked only with that participant’s cooperation. Technology of this kind could restore communication for people who have lost their usual ability to speak or move.

A review of data from patients with neural implants proposes separating consent to surgery from consent to store recordings and use them to train models. Conduit collects data without surgery. An April analysis of ORF explains that as decoders improve, the same archive of neural data may reveal more about a person’s intentions and mental states. Scholl argues that decisions about the future of such data should be made before the data become part of a large archive.

Scholl describes the sequence as follows: recordings train the model, while hardware, engineering expertise, and investment make it possible to deploy the model more widely. In his essay, he writes:

“The problem with mindreading is that it is an asymmetric technology: those who rule without people’s consent benefit from it to a disproportionate degree.”

Scholl is concerned that such technology could be used coercively. His appeal is therefore directed at those who collect the data, build the devices, and provide the funding:

“So I ask you: do not create datasets for mindreading. Do not build hardware for mindreading. Do not fund this technology. Do not sell them your mind.”