Preprint: An AI agent analyzed medical records, and experts revised the outcome measure
Preprint: An AI agent analyzed medical records, and experts revised the outcome measure
On October 14, the authors of a preprint described how an AI agent helped conduct a study using a licensed electronic health record database, from patient selection to figures and a manuscript draft. The initial calculation treated a series of prescription records as treatment duration. Experts replaced that measure with the interval between the first and last records.
An electronic health record stores diagnoses, prescriptions, clinical notes, and laboratory results. A prescription record documents a treatment plan; it may be canceled, corrected, or duplicated. Working with these data therefore begins with establishing what event each record actually represents.
In the preprint, the authors described using the Muse Spark language model to work with a licensed health record database. The model wrote and executed code, selected a patient cohort for analysis, checked data quality, searched the literature, and prepared figures and a manuscript draft. The researchers defined the clinical question, reviewed the code and sources, and decided which changes to accept.
For the demonstration, the authors selected two anticancer antibody-drug conjugates. In these drugs, an antibody carries an active substance to cells with a specific target. Applied to a database of 4,95 million patients, the selection criteria identified 106 people with prescription records for both drugs: trastuzumab deruxtecan and sacituzumab govitecan. The first record was for trastuzumab deruxtecan in 73 patients and sacituzumab govitecan in 33.
The initial calculation filtered prescription sequences, added 21 days to the date of the last record, and labeled the result treatment duration. Expert review found that the filter depended on the dosing schedule of the particular drug and that the 21-day interval entered the calculation as an addition to the date of the last record. After the revision, the authors calculated the interval between the first and last prescription records for the second drug.
Among patients whose records listed trastuzumab deruxtecan first and sacituzumab govitecan second, the median interval was 63 days; for the reverse order, it was 49 days. These numbers describe the sequence of records in the chart, and the authors revised the calculation, figures, and text to reflect that meaning.
In OncologyBench, a test in which agents solve oncology tasks, two thirds of attempts with correctly performed individual analyses ended in an incorrect overall decision. In the preprint, experts checked the measure before the number became a study result by comparing the formula with what the health record actually documents.