GPT-5.6 helped disprove a 20-year-old hypothesis about testing thousands of scientific results at once
GPT-5.6 helped disprove a 20-year-old hypothesis about testing thousands of scientific results at once
Statistician Edgar Dobriban found a data model in which the Benjamini-Hochberg procedure promises slightly more reliability than it can provide. Researchers use this procedure when a single experiment tests thousands of genes or other features at once and they need to limit the proportion of false discoveries. Dobriban published the proof and the code for a numerical certificate. The model helped with the search, and the author verified the result.
On July 14, Edgar Dobriban published a proof concerning a standard safeguard against such coincidences. When a biologist compares thousands of genes across groups of cells, chance associations are inevitable. Researchers therefore usually apply the Benjamini-Hochberg procedure. It ranks the p-values, which indicate how consistent each observation is with chance, and selects a threshold intended to keep the expected proportion of false positives among the reported findings below a specified level.
The procedure has long had clear guarantees for independent results and for some forms of positive dependence. Real genomic data, however, are correlated. Nearby DNA variants are inherited together, genes operate in networks, and cells share common causes of change. It remained uncertain whether the standard procedure retained its guarantee for all correlated two-sided tests, in which an effect can occur in either direction.
Dobriban constructed a specific model with a shared latent factor, meaning one unobserved cause that shifts many results at the same time. This factor changes the distributions of both null results and true signals. Under these conditions, the procedure selects a threshold that produces slightly more false discoveries among the selected findings than its guarantee allows.
At a nominal level of 1%, the rigorous numerical certificate gives a value of at least 1.0416829%. The difference is small, but it disproves the universal hypothesis. For correlated two-sided data, the guarantee established for independent tests cannot be applied automatically.
On July 10, OpenAI published a candidate proof of a graph theory conjecture, which mathematicians still need to examine. In Dobriban's case, the verification chain already includes an open numerical certificate. According to the statistician, GPT-5.6 received the mathematical formulation and produced a counterexample, a proof strategy, and certificate code in about 90 minutes. Dobriban checked the argument, and the open program recomputes the lower bound using interval arithmetic, with rounding directed toward the conservative bound. The result can therefore be verified independently of any claims about the model's capabilities.
For the biology of aging, this work concerns part of the field's scientific infrastructure. Large-scale measurements of genes, proteins, and cells constantly require researchers to distinguish genuine signals from random noise. An AI system that finds a verifiable counterexample can help determine where a statistical method truly guarantees reliability and where the assumptions required for that guarantee no longer hold.
Source: Edgar Dobriban's manuscript · open code and certificate