Abhishek Mahajan proposes evaluating AI in biology by the new biological signals it helps obtain
Abhishek Mahajan proposes evaluating AI in biology by the new biological signals it helps obtain
On August 20, in an episode of the Core Memory podcast, Owl Posting biology blogger and OpenAI Foundation employee Abhishek Mahajan explained why robots and models accelerate drug discovery in different ways. The value of AI depends on whether the resulting data help researchers choose the next experiment.
Laboratory automation handles repetitive operations. A robot dispenses compounds into plates and increases the number of runs for an assay that researchers have already selected. A new assay or instrument changes the question that a laboratory can ask a biological system. According to Mahajan, such a tool is valuable when its data help researchers select an experiment that tests one explanation and rules out another.
“Good methods beat scaling bad methods every time.”
Mahajan cites next-generation sequencing, a method for reading large numbers of genetic sequences, as a historical example. In some viral libraries, it allowed researchers to collect data on billions of viral variants from a single sample. Laboratories could then obtain data more quickly, but they still had to determine which variants worked in a living system.
Mahajan uses the term virtual cells for programs that predict how a cell will respond to an intervention. These programs typically receive a cell type and a chemical or genetic intervention as inputs, then produce a profile of gene activity in individual cells. Data on proteins and signaling between cells require different measurements. Mahajan proposes evaluating AI used with such models and instruments by whether it helps measure the biological signal needed for the next decision.
Mahajan points to tiragolumab, an antibody that blocks the TIGIT protein on immune cells. In a large patient trial, adding it to atezolizumab did not extend survival in non-small cell lung cancer. In Mahajan’s view, this case illustrates the gap between data from cell and animal models and the human response to a drug.
Mahajan proposes testing AI ideas for assays and instruments retrospectively. Researchers could limit a model to knowledge available five or ten years ago and ask whether it would propose a new method or instrument that later changed laboratory practice. This test would assess whether the model directs researchers toward a new measurement and the experiment that the measurement makes possible.