Three Claude models planned routes to four molecules with unpublished syntheses, and three chemistry professors rated them at the level of early-stage graduate students in a blind review
Three Claude models planned routes to four molecules with unpublished syntheses, and three chemistry professors rated them at the level of early-stage graduate students in a blind review
On November 9, Anthropic published a report on three Claude models (Mythos 5.1 and early prerelease versions of Opus 5.5 and Mythos 5.2). With no access to the internet or reaction databases, the models received drawings of four molecules whose total syntheses had yet to be published by the laboratories of Noah Burns (Stanford University), Timothy Newhouse (Yale University), and Sarah Reisman (California Institute of Technology). Each model had one attempt per molecule to propose a route and identify its three riskiest steps. The professors then reviewed the twelve plans on paper, without knowing who had produced them, against their own groups' experience.
A total synthesis route is a sequence of reactions (steps) that leads from commercially available starting materials to a complex molecule. Planning backward from the target molecule is called retrosynthesis. According to the report, these plans are usually evaluated on molecules with published syntheses, making it difficult to distinguish planning from recall of a paper.
“Overall, I have to say I am impressed! … These are all the kinds of initial retrosyntheses I would expect from early-stage graduate students,” Burns wrote. Newhouse found that the routes used reliable, well-established reactions and were at the level of a first-year graduate student. Reisman judged them comparable to the work of a strong first-year graduate student, who in her group often writes the first draft of a synthesis plan.
Burns supplied GDGT-4, a membrane lipid found in single-celled archaea. It has two chains of 40 carbon atoms containing five-membered rings, joined by glycerol residues to form a 68-membered ring with 22 stereocenters (atoms around which groups can adopt two mirror-image arrangements). The routes comprised 35, 29, and 31 steps. The 40-carbon chain proposed by Mythos 5.1 was “exactly the intermediate we use.” Mythos 5.2 replaced palladium with diimide or platinum oxide for the addition of hydrogen to double bonds near stereocenters: palladium could have caused a double bond to migrate and disrupted a stereocenter. “This goes beyond choosing a plausible chemical reaction,” Burns noted.
Burns considered the starting material shared by all three routes too expensive, at more than 100 dollars per gram. The large-ring closure proposed by Mythos 5.1 “did not work for us in a similar system,” although it had been one of the group's first attempts. Burns also noticed that none of the models used asymmetric hydrogenation (the addition of hydrogen that favors one mirror-image form). His group had already used this approach to synthesize the related lipid GDGT-0: in 2021, a single hydrogenation of this kind applied to a building block set all 16 methyl stereocenters, and the synthesis took eight steps, whereas the shortest previous route had required more than 30.
For Newhouse's target, Mythos 5.1 misread two bonds pointing behind the plane of the drawing as pointing toward the viewer. It then proposed 15 steps to a version of his intermediate with two inverted stereocenters. Newhouse spotted the error immediately, helped by the assumption the model had explicitly recorded.
In Reisman's test, all three models independently chose the same established strategy and identified two steps as the riskiest, both of which concerned her as well: coupling two fragments (similar reactions had failed in her group) and closing the central ring. Reisman was surprised by the models' reasoning about the spatial arrangement of new stereocenters, which she thought approached what an expert could predict without experimental data.
Burns will select a problem from his group for the same models. Reisman plans to have a student try an approach that her group had set aside and that a model used in a separate exercise. It made her want to “revisit what we had put on the back burner.”