Insilico Medicine Releases 3D-Fit to Evaluate Language Models for Molecular Design

Insilico Medicine Releases 3D-Fit to Evaluate Language Models for Molecular Design

On July 20, the Insilico Medicine team released the 3D-Fit preprint. The benchmark asks language and diffusion models to generate molecules for 1 453 protein–small molecule complexes.

A drug molecule must occupy a protein pocket, which is a cavity between the protein's amino acids. A chemist may require the model to retain a fragment of a known molecule, place a hydrogen-bonding group at a specified point, or reach a particular amino acid. Each requested contact imposes additional constraints on the entire molecule. Its bonds and angles must remain valid, and the molecule must fit among the protein atoms.

In 3D-Fit, the authors provide the model with the coordinates of the pocket atoms and a text instruction specifying an anchor fragment, a pharmacophore point, which marks the location of a required chemical function, or a required contact with the protein. The model returns the coordinates of a ligand, meaning a candidate molecule. The authors then evaluate two properties separately. First, they determine whether the model followed the local spatial instruction. Second, they assess whether the molecule can be reconstructed, whether its geometry passes physical checks, and whether its pose fits within the pocket.

Language models often handle anchor fragments and pharmacophore points successfully. The authors attribute this result to the task format, since the text instruction resembles the representation that the model must produce. However, generating a complete pose requires the model to coordinate the molecule's shape with its position among the protein atoms.

In this evaluation, language models performed worse than specialized diffusion models. For the original poses, every language model tested received a UniDock score above −6 ккал/моль, the threshold that the authors use to indicate weak binding. Local optimization shifted most results to approximately −6…−7 ккал/моль. The best diffusion models achieved lower scores, which correspond to more favorable poses under this metric, and passed the geometry checks in PoseBusters more often.

Satisfying a local contact does not by itself make a molecule a good candidate. 3D-Fit tests whether that contact is preserved when the complete molecule is subjected to the constraints imposed by chemistry and the protein's three-dimensional pocket. Physical validation distinguishes a molecule that reaches the specified point from one that fits within the pocket as a whole.