A new single-cell analysis method raises rare cell type detection from zero to 79–85%: standard analysis conflated scarcity with irrelevance
A new single-cell analysis method raises rare cell type detection from zero to 79–85%: standard analysis conflated scarcity with irrelevance
On September 24, Livnat Jerby's lab at Stanford published a single-cell data analysis method called DR-GEM in Nature Communications. Standard analysis minimizes the average error across all cells and therefore routinely loses rare cell types. Three existing algorithms designed to counter this bias failed to solve the problem.
When biologists study tissue at single-cell resolution, they obtain a table of tens of thousands of cells, each with the activity levels of tens of thousands of genes. To find structure in these data, standard analysis compresses each cell's profile into a compact set of numbers, groups similar cells into clusters, and labels each cluster with a cell type name.
Kristin Yeh and Livnat Jerby showed that this analysis, by default, conflates a cell's abundance with its importance. It represents frequent cells more accurately, while rare cells receive a coarse representation and dissolve into neighboring clusters. Across several datasets, rare cells were 2.85–9.18 times more likely to land in the worst 10% of representation quality.
Yeh and Jerby first tried imbalance-aware methods, which penalize the model for ignoring rare groups. Two of them did not help. The third was given something unavailable in a real task (exact cell type labels) and still failed to collect rare cells into a distinct cluster: the penalty reduced compression error but did not solve the recognition problem.
"Standard analysis can't find rare but important cells? Your clusters look like one round blob instead of distinct clusters?" This is how Yeh described the problem she set out to solve in her doctoral work.
DR-GEM takes a different approach: instead of penalizing missed rare cells, it changes the composition of the data the model trains on. The algorithm identifies the cells with the coarsest representation, retrains on them, then assembles thousands of random but composition-balanced subsamples and re-clusters each one. A cell's final label is determined by majority vote.
In images of 47 ovarian cancer tumors, mast cells (0.4% of all cells) were not grouped into a cluster by the standard method. DR-GEM detected them with 85% accuracy. On the full cell set from the same tumor (seven types, from cancer cells to immune cells), the standard method found only four; DR-GEM found all seven, including the mast cells. The same tumor was profiled with a different technology, MERFISH, without mast cells in the annotation, and the pattern repeated: four out of six for the standard method, all six for DR-GEM. In mouse brain, rare pericytes (0.87% of cells) went from zero to 79% accuracy. The method also works beyond cell types: in CRISPR experiments with the same kind of imbalance (from 2 to nearly 2,000 cells per perturbation against 75,000 controls), DR-GEM outperforms the standard method.
Jerby's broader research program, reprogramming immune cells against cancer, degeneration, and aging, earned her status as a Chan Zuckerberg Biohub investigator, a private scientific initiative of Mark Zuckerberg and Priscilla Chan. The compression-and-clustering pipeline that DR-GEM repairs is used in virtually every single-cell study, including aging and stem cell research, where the population of interest is almost always small: senescent cells, which make up less than 5% of tissue but whose inflammatory signal is disproportionately strong. Blindness to rare cells may have been quietly losing the very populations researchers were looking for.