NEvo synthesizes videos that a model predicts will provide the strongest stimuli for a selected area of the visual cortex

AI has been trained to design videos for testing hypotheses about visual cortex function

A team from EPFL and Johns Hopkins University has introduced NEvo, a system that creates two-second videos for a selected area of the visual cortex. It generates multiple candidates, and an fMRI model selects the video that it predicts will produce the strongest response in that area.

When neuroscientists study the visual cortex, they usually select the stimuli themselves, including faces, landscapes, moving dots, and gestures. A participant then lies in a scanner and watches these images or videos while the researcher compares the responses of different brain areas. This type of experiment tests a hypothesis that the scientist has already formulated.

In a preprint dated July 2, NEvo changes this sequence. The researcher selects a cortical area, and the system first creates still images before turning the best candidates into short videos. A separate model evaluates each candidate. From the video, it predicts a map of the fMRI signal, which reflects slow changes in blood flow associated with brain activity. Videos with the highest predicted responses remain in the search, while their descriptions are combined and modified in subsequent generations.

For the face recognition area, the system selected faces more often. For the area associated with processing places, it selected scenes and spaces. For motion-related areas, it selected rapidly changing images. The authors’ most interesting exploration followed a path from the early visual cortex to the superior temporal sulcus, which contributes to the perception of other people’s actions and social interactions. At the beginning of this path, NEvo found textures and movement. Farther along, it found bodies and joint actions. Near the end, it found faces and scenes of communication.

This is an inverse problem for a brain model. A conventional model asks, “What will happen in the cortex if this video is shown?” NEvo asks, “Which video should engage this area most strongly?” The answer is not a verbal guess, but a specific stimulus that can be used in the next human experiment.

This approach grew from two lines of research. Models such as TRIBE v2 can already predict how the brain will respond to a new film, sound, or text. Earlier studies also showed that static images and motion are distributed across visual pathways in a more complex way than a simple division in which one area processes images and another processes motion. NEvo adds a search process to prediction: the model constructs a video that can refine its own hypothesis.

For now, however, the video excites a computational model, not the measured brain of a new participant. The authors trained the evaluator on video datasets paired with corresponding fMRI data. Any errors or biases in that model may therefore affect the result. The decisive experiment will be to show the synthesized videos to people undergoing fMRI. If the predictions are confirmed, neuroscientists will have a faster way to select experimental stimuli instead of repeatedly testing ideas through trial and error.

This is an important working principle for future brain repair. Before intervening in a damaged network, researchers need to formulate testable hypotheses about its function. NEvo currently works with individual visual areas and two-second videos. Still, it shows how an additional step can be placed between brain data and an expensive experiment: the machine proposes the experiment, and the living brain determines whether the prediction is correct.