A review of seven rare-disease diagnostic systems extracted a shared six-phase workflow: on 50 benchmark cases, it yielded 11 correct top-ranked diagnoses compared to five from the same GPT‑5 with a single prompt
A review of seven rare-disease diagnostic systems extracted a shared six-phase workflow: on 50 benchmark cases, it yielded 11 correct top-ranked diagnoses compared to five from the same GPT‑5 with a single prompt
On September 14, a review appeared on the preprint platform Research Square covering seven studies on AI systems that diagnose rare diseases by working through cases step by step, consulting medical references, and only then naming a diagnosis. Across 19 evaluations on 33 738 cases, the mean top-ranked diagnostic accuracy was 51.2%, with individual systems ranging from 22.0% to 77.5%.
Rare diseases are difficult to recognize: symptoms combine differently across patients, and clinicians struggle to match a free-text case description against the vast body of medical knowledge. Earlier tools typically required the patient's complaints to be pre-coded into a standard medical ontology, while a standalone language model without that preprocessing would suggest diagnoses from raw text and perform no further verification. The hypothesis the review tests is that accuracy improves when the model generates candidate diagnoses, refines them step by step through external medical sources, and documents the reasoning behind each choice, rather than producing an instant answer.
The authors screened 1 193 publications, read 111 full texts, and included seven studies that together produced 19 accuracy evaluations. Results across them ranged from 22.0% to 77.5% because of differences in architecture, underlying model, and case set: the aggregate figure of 51.2% describes a mixed corpus, not the accuracy of any single system. Six months earlier, a March review of agentic AI across all of medicine identified the same seven eligible studies spanning emergency care, oncology, radiology, and rehabilitation, but found only one full-scale trial involving actual patients. The present review narrowed those seven studies to a single task and added the first direct paired experiment on an identical model.
The paper calls this sequence of actions an agentic system: it parses the case description, extracts clinically significant features, generates and revises candidate diseases, consults medical reference databases, reorders the candidates, and records the rationale for each ranking. The authors found these steps in all seven systems they studied and consolidated them into a six-phase workflow.
They tested this workflow against a single-prompt call to the same GPT‑5 on 50 cases from RareBench, a benchmark dataset of rare-disease descriptions. Both versions received the same cases and ran with identical parameters: one produced a ranked list of diagnoses in a single model call, the other completed all six phases with reference lookups. The workflow placed the correct diagnosis first in 11 of 50 cases; the single call did so in five. An exact paired test yielded p = 0.0703 against the authors' preset threshold of 0.05.
In this work, the vague term "agentic" becomes an observable sequence of actions: how the system parses a case, where it looks for information, and by what path it arrives at its top candidate. Future comparisons will be able to test the contribution of each step individually, provided they keep explicit the order of operations, the access to sources, and the ranking rule.