Flagship drug-discovery AI models Boltz-2 and Nesso-1 gain accuracy as similar proteins accumulate in public databases, while a far simpler algorithm beats both on rare proteins

Flagship drug-discovery AI models Boltz-2 and Nesso-1 gain accuracy as similar proteins accumulate in public databases, while a far simpler algorithm beats both on rare proteins

On September 13, researchers unaffiliated with any of the developer companies published the MIRAGE benchmark: on proteins that were nearly absent from public databases, both neural networks lose to a simpler and cheaper method.

When MIT and Recursion (Nasdaq: RXRX) released Boltz-2 in June 2025, the press release promised accuracy approaching that of expensive physics-based binding-energy calculations, while running up to a thousand times faster. Recursion's Chief Scientific Officer Najat Khan framed the stakes:

"Getting the right molecules early on is one of the most fundamental problems in drug discovery, and it determines whether a development program survives or fails"

A little over a year later, Recursion's research arm Valence Labs released Nesso-1, a lighter model that, in Khan's words, "matches or outperforms Boltz-2 on public benchmarks."

The MIRAGE preprint tested exactly that claim. The suspicion that such models memorize a protein's family rather than the physics of binding had been voiced before, but no one had yet evaluated already released neural networks across the full range of family representation. Trained on all available structures, the models could have learned to recognize a familiar binding pocket instead of general chemical rules. The authors compared both models against controls that were deliberately denied family data, testing them on proteins ranging from singletons to families with hundreds of relatives.

The result matched the suspicion: accuracy of Nesso-1 and Boltz-2 rose together with family representation, while the controls, which had no way to recognize the family, showed no such increase. The effect persisted even after removing cases where the model could have seen the exact same protein before, which means the issue is recognition of the entire family, a broader and subtler effect than simple repetition of a single entry.

On families the neural networks had rarely encountered, the ranking reverses: a simple random forest with no access to family data outperforms both Nesso-1 and Boltz-2.

To test the effect beyond historical datasets, the authors chose a real target from a poorly represented family: binding data for it were published in May 2026. Neither Boltz-2, nor Nesso-1, nor classical docking tools could reliably beat the most primitive baseline, prediction by the candidate's molecular weight. Boltz-2 scored below this baseline. Nesso-1 scored marginally above it, although the Nesso-1 report itself describes the difference as statistically unconvincing.

For Boltz-2, the authors ran an experiment that explains part of the failure: they supplied the model with its missing input, an evolutionary multiple sequence alignment (a comparison of the same protein across species that reveals which regions are conserved and which are variable). With this input, the success rate on unfamiliar families jumped from 2% to 46%, while performance on familiar families remained unchanged. The released version of the model simply does not receive this signal.

The MIRAGE authors themselves write:

"Selection by average accuracy rewards familiarity with protein families that have already been solved"

Before trusting a model's prediction for a new target, it is worth asking how well represented its family is in the data the model was trained on.