InverseBench preprint: gene ranking scores differ for RNA and cell growth, 0,959 versus 0,445

InverseBench preprint: gene ranking scores differ for RNA and cell growth, 0,959 versus 0,445

On October 23, the authors of the InverseBench preprint proposed documenting which candidate data a model receives and how the desired outcome is measured. Using 7 354 genes in the K562 cell line, they compared an RNA score with independently measured cell growth suppression.

To change a cell's state, a researcher selects a gene to target. InverseBench distinguishes two settings for this choice. In the first, the ranking model has access to the measured RNA response to each candidate perturbation: which genes became more or less active after the perturbation. In the second, it evaluates a perturbation whose effects have yet to be measured using indirect features, such as changes in the target gene's own activity following other perturbations.

In the K562 comparison, the authors kept the same 7 354 genes, input RNA data, and training and test splits. They selected a model separately for each prediction target. One target was a composite RNA score derived from the same experiment as the input features. The other was the extent to which knocking out a gene slowed cell population growth, measured using data from large-scale CRISPR screens in the public DepMap database. RNA responses were measured using CRISPRi, which represses gene activity, whereas growth was measured after CRISPR-knockout, which disables the gene.

The authors measured agreement between gene rankings using Spearman correlation. The correlation was 0,959 for the RNA score and 0,445 for growth suppression. The first value describes recovery of the RNA score; the second describes the ranking of genes by an independently measured reduction in growth.

The authors also evaluated a shortlist of genes for a follow-up experiment. Among the top 5% of candidates, the mean functional percentile, which describes where the selected genes fall in the ranking by growth suppression, was nearly identical: 0,841 for models selected using the RNA score and 0,839 for models selected using the growth score. At thresholds of 1% and 10%, the relative order changed. Overall ranking accuracy and shortlist quality address different practical questions.

The authors propose reporting, alongside a model's score, which candidate responses the model can access, how the prediction target was obtained, and which outcomes were withheld for testing. This ties the score to a specific gene selection task and a specific measured outcome.