In a blinded trial of 511 antibodies, one AI-designed variant matched the laboratory benchmark, but most algorithms performed worse than random selection
In a blinded trial of 511 antibodies, one AI-designed variant matched the laboratory benchmark, but most algorithms performed worse than random selection
On August 19, Nature Biotechnology published a paper on AIntibody, a blinded trial in which 29 organizations submitted 511 antibody variants in advance for three tasks. Independent laboratories synthesized the variants and tested them under identical conditions.
An antibody is a protein that recognizes and binds to a specific target. A drug candidate must bind strongly, be produced in sufficient quantities, remain stable, and bind selectively to its intended target. The competition therefore measured both the rare best result and the proportion of variants suitable for further laboratory work.
The authors based the study on the logic of blinded CASP assessments: each team selects a candidate before the experiment reveals the answer. AIntibody followed the 2024 competition, and participants selected their antibody sequences in advance. All variants were first tested by surface plasmon resonance, which measures how an antibody binds to its target. The strongest variants were also tested using KinExA, a second method for assessing binding strength. Five tests excluded unstable and nonspecifically adhesive proteins.
In the task of improving an existing antibody, a variant from Aureka bound to the target as strongly as the best laboratory variant: 94.7 versus 113 pM by KinExA. Their 95% uncertainty intervals overlapped. This result came from one team in a task for which participants received data from the first stage of laboratory selection.
In the second task, the algorithms had to select the best variant from similar antibodies in three groups.
“For most participants, selecting antibodies with AI produced worse results than a strategy that chose the most frequent clone and supplemented it with randomly selected clones.”
With random selection, 39% of clones bound to the target more strongly than the most frequent variant in their group. Among AI submissions, this proportion was 9.8–13.8%.
The third task extended the search beyond the original library. Teams modified the regions of the antibody that contact the target. Here, 46.4% of the submitted variants either did not bind to the target or failed the panel of properties required for further development. The best individual result shows that an algorithm can identify a successful molecule. The proportion of suitable variants indicates how many candidates from the next laboratory batch are likely to reach the next stage.
The target was the receptor-binding domain (RBD) of SARS-CoV-2, the region of the coronavirus S protein that the virus uses to attach to a cell. Extensive sequence data were already available for this target, and participants in the second and third tasks also received binding strength measurements. With such a data-rich target, AIntibody measures the entire path from a computer-selected sequence to a protein that passes a shared laboratory screening process.