A neural network selects visual cortex stimulation based on the brain's predicted response

A neural network selects visual cortex stimulation based on the brain's predicted response

On August 7, the scientific journal Neuron published a study involving one blind participant with a 96-channel implant in the visual cortex. A neural network learned to predict how electrical pulses would change cortical activity and then select multielectrode stimulation to produce a specified activity pattern.

A cortical visual prosthesis delivers short electrical pulses directly to the part of the brain that processes visual information. The participant perceives phosphenes, which may appear as points, flashes, or arcs of light. To turn these sensations into an image, the device needs a way to evoke the required pattern of neural activity reliably.

In July, the NEvo system selected videos expected to activate a specified region of the visual cortex more strongly, but it estimated their effects using a computational model. In the new study, a neural network selects the electrical currents and then uses recorded cortical activity to determine immediately how closely the resulting response matches the target.

The same pulse produces slightly different responses on different days, and stimulation through multiple electrodes produces interacting effects. Study coauthor Michael Beyeler explains:

“The electrodes interact, neural responses change, and what we deliver to the brain may differ from what the person perceives.”

The team conducted 26 sessions over four months. The 96-channel implant stimulated selected electrodes and recorded the cortical response. The neural network received the combination of electrical currents and the mean baseline activity before the pulse, then predicted how the cortical response would change after stimulation.

The researchers then specified a target activity pattern and asked the system to select a corresponding set of electrode currents. One algorithm tested alternatives through sequential search. The other, an inverse neural network, generated a pattern in a single pass. In the implanted system, both methods reproduced the target cortical response more accurately than conventional calibration and required less total current. Sequential search found a more accurate set of currents in approximately 10–20 seconds, while the inverse neural network required approximately 50 microseconds.

The recorded cortical response also helped predict what the participant saw. Across 4 416 responses to the question of whether he had seen a phosphene, prediction accuracy increased from 74,2% when the model used stimulation parameters alone to 88,7% when it also received the recorded activity and the brain's baseline state. The recorded cortical response therefore gave the model more information about the visual percept than the specified currents alone.