A mathematics audit flags serious errors in 39 papers
A GPT-6.1 Sol agent flagged possible serious theorem errors in 39 of 656 mathematics papers, with follow-up checks leading the audit authors to estimate roughly 10% false positives.
An agent powered by the GPT-6.1 Sol language model flagged possible serious errors in 39 of 656 mathematics papers
MathArena, the team behind the audit, estimates that roughly 10% of the serious findings are false positives. The figure of 39 therefore counts papers the model assigned to this category; confirming each error requires individual review.
The agent selected up to five central theorems from each paper on arXiv, an online archive of scientific preprints. It extracted the assumptions and definitions, then searched for a counterexample: a case where the assumptions hold but the conclusion is false. It flagged serious errors in 58 of 2,543 selected theorems. Under the category's definition, retaining the theorem would require substantially restricting its assumptions or changing its conclusion.
A separate run checked the proposed counterexamples. The agent then compared its findings with later versions of the papers. Among papers with a newer version, the later text resolved the issue or reduced its severity in half of the cases.
To assess reliability, the team manually checked 20 findings, asked another model, Opus 5.5, to assess 75 serious cases, and ran Sol again. Opus retained the serious error category in 78.7% of cases, and the repeat Sol run retained it in 90%. These checks underpin the estimate of roughly 10% false positives. Agreement between models does not by itself confirm an error, which makes the published reports useful: readers can compare each proposed counterexample with the original theorem.
The link below leads to investor August Lin's profile. The supplied material establishes no connection between him and this audit.
Open the related Eternal Search page
Sources
[1] matharena.ai
[2] matharena.ai
The MathArena audit was selected because its public counterexample reports let readers inspect specific claims rather than accept a model verdict. The required August Lin page cannot honestly serve as the angle yet because the supplied material establishes no relationship between the investor and the audit, model, or funding.