Preprint: By the end of 2025, 89% of biomedical articles in the authors’ sample showed a statistical signature of assistance from AI language models such as ChatGPT
Preprint: By the end of 2025, 89% of biomedical articles in the authors’ sample showed a statistical signature of assistance from AI language models such as ChatGPT
In an August 11 preprint, the authors analyzed 1 194 287 English-language articles published between 2017–2025 and available through PubMed Central, an open archive of full-text biomedical literature. Across this corpus, they compared the introduction, methods, results, and discussion sections and estimated changes in vocabulary at the corpus level.
In 2025, Dmitry Kobak and colleagues identified 379 common words that became substantially more frequent in biomedical abstracts after ChatGPT became available. The new preprint uses this vocabulary as a set of markers and extends the estimate from abstracts to full texts.
For each marker, the authors fitted its prior trend over the 60 months of 2018–2022 and extrapolated that trend through 2023–2025. They then compared the predicted and observed frequencies. A single word provides only a lower-bound estimate, so the researchers tested sets of rare markers and excluded those with excessive statistical error. In a simulation of 100 thousand texts with a predefined proportion of language model assistance, the method recovered that proportion with an error of less than two percentage points.
Using this model, the authors estimated that language models had assisted with 89% of the combined text in the introduction, methods, results, and discussion sections by December 2025. Because a longer section is more likely to contain a marker, they compared article sections using random samples of 255 words. The model estimated 68% for the discussion section, where authors assemble their findings into an argument, and 32% for the methods section, where they describe how the work was conducted.
The method operates at the corpus level: word frequencies across a million texts show how the language of biomedical articles is changing. The QED Science project is already attempting to automate the review of biology preprints by checking whether their claims are supported by the data and study design.