The best-performing AI model tested scored 25,6% on GDPx biology tasks using real experimental data
The best-performing AI model tested scored 25,6% on GDPx biology tasks using real experimental data
On November 2, SemiAnalysis, a research firm covering chips and AI infrastructure, released GDPx, a benchmark of 206 tasks across 11 fields. Models work independently with files and software for up to eight hours. LatchBio developed thirteen biology tasks: twelve are private, and one is public, along with records of the attempts.
LatchBio built some tasks using private data from partners and others using published data modified to prevent models from recalling the answer. Answers are checked programmatically. LatchBio reports that evaluators using scoring rubrics disagree with one another, accept incorrect answers, and reject correct ones.
On the leaderboard, Gemini 4 Carbon scored 25,6%, Opus 5.5 and Fable 5.1 each scored 20,5%, GPT-6 Astra scored 5,1%, and GLM 5.3 scored 0%, averaged over three attempts. SemiAnalysis cautions that safety filters sometimes activate on biology tasks, lowering the overall scores of Opus 5.5 and Astra. It suggests that Carbon probably leads because its filters are less restrictive.
LatchBio identifies three recurring types of model error. The first concerns statistics: models choose inappropriate statistical tests, tailor analyses to a desired result, and treat multiple samples from the same animal as independent observations, inflating the sample size. The second concerns instruments: in a task involving compromised samples, the plate reader caused interference in the multiwell plate measurements, but the model failed to recognize the source. The third concerns trust in the input data: models analyze everything provided, such as all cell types, instead of selecting those relevant to the conclusion.
The public task concerns antibody “shuttles,” which carry drugs from the blood into the brain across the blood-brain barrier, the protective lining of the brain’s blood vessels. A shuttle binds to a receptor on cells in this lining; a conventional antibody uses two “arms” to do so. In 2014, researchers at Roche reported in Neuron that, for one receptor in this lining, an antibody with one receptor-binding “arm” passed through the cell more readily, whereas an antibody with two was directed to lysosomes. A version of an antibody against beta-amyloid, the protein found in Alzheimer’s disease plaques, with one receptor-binding “arm” achieved 55 times greater target engagement than the original antibody in a mouse model of the disease.
Comparing one-armed and two-armed variants in animals requires a pair with equal binding strength; otherwise, the stronger binder will have an advantage. Models had to select such a pair from twelve antibody variants using two sets of measurements: one from cells expressing the receptor and another from a chip with the receptor immobilized on its surface. According to LatchBio, receptors are packed more densely on the chip than on cells, allowing a two-armed antibody to bind two receptors at once. This combined binding, called avidity, strengthens its binding on the chip.
The numerical values below come from the models’ own calculations. In two attempts, GLM 5.3 used the chip data to select a pair with equal binding strength, around 89 нМ (nanomoles per liter; lower values indicate stronger binding). On cells, those same antibodies reached half-maximal binding at 19,5 нМ for the one-armed variant and 1,8 нМ for the two-armed variant, an 11-fold difference.
In all three attempts, Opus 5.5 rejected the chip data because the one-armed and two-armed antibodies had been measured under different conditions. It used the cell data and selected the pair with the closest values, around 380 and 390 нМ. According to LatchBio, the pair should have been selected from antibodies with strong binding, whereas this pair bound weakly. In two reports, Opus itself identified the antibodies requiring 10 and 11 нМ as the best strongly binding pair, yet selected the other pair: the weakly binding pair’s values differed by 1–2%, compared with 10–12% for the strongly binding pair. Gemini 4 Argon selected the pair with values of 10 and 11 нМ twice. Of twelve attempts (four models making three attempts each), only these two were scored as correct.