Tahoe-100M atlas preprint: models barely improved predictions of cellular drug responses

Tahoe-100M atlas preprint: models barely improved predictions of cellular drug responses

A preprint published on September 1 examined Tahoe-100M, an atlas measuring how drugs alter gene activity in cells. The author tested whether a cancer cell line’s gene activity profile before treatment could predict its response to a drug. Six models barely outperformed a simple prediction based on the drug’s mean effect. The author attributed this result to the low reproducibility of differences between cell lines.

Each drug has a mean effect: it changes gene activity in a similar way across different cell lines. A simple prediction uses this mean effect. In a PerturbLDM evaluation using the same atlas, the test set withheld combinations of drug, dose, and cell line, although each of the three components had appeared separately in the training data. The new preprint withheld an entire cell line for testing and assessed whether its baseline gene activity profile could explain the difference between its response and the drug’s mean effect.

Each evaluation excluded one complete cell line from training. For that line, the model received only the gene activity profile of control cells treated with DMSO, the solvent used in the control condition, instead of a drug. The drug’s mean response was calculated from the remaining lines and then subtracted from the response of the test line. The resulting difference, called the contextual response, became the prediction target.

The benchmark included 47 cancer cell cultures and 377 drugs at a concentration of 5 мкМ. Across the full dataset, the best of the six variants improved on the simple prediction by an average of 0.004 R², where R² measured predictive accuracy specifically for the contextual response.

For 107 drugs, the author compared the contextual response between two experimental batches. The median similarity between replicates was 0.067, and only four drugs exceeded 0.2. This variation between replicates limits the achievable prediction accuracy for a new cell line.

Among the five drugs with reproducibility above 0.15, one model using the baseline gene activity profile achieved a mean R² of +0.058. In the author’s evaluation, three of these five results outperformed the prediction based on the drug’s mean effect.

This benchmark measures both model quality and the stability of the difference being modeled. Reporting reproducibility alongside R² helps distinguish a stable contextual signal from variation between replicate experiments.