medRxivJClub
Large language models detect methodological errors that human peer review misses in clinical trials
medRxiv · · Preprint · Open access
Brian, Pereira Hernandez + 3 more
Abstract ↗AI summary
The abstract is read at the publisher; the summary is JClub's.
Large language models detect methodological errors in randomized clinical trial reports with major-error detection scores up to 99%, far exceeding untrained human reviewers’ 27% in earlier tests.
- Why it matters: Peer review can miss methodological errors, leaving avoidable problems in published clinical trials undetected; this work benchmarks whether LLMs can catch the same issues.
- What they did: Fifteen LLMs (including four open-weight models) were tested on error-inserted randomized clinical trial reports, then repeated on three 2026 trials without inserted errors to limit memorization.
- The result: Major-error detection ranged from 53% to 96% initially and 51% to 99% in 2026 trials (open-weight: 66% to 87%); models also flagged 2% to 29% of error types in control reports.
The findingWhy it mattersWhat they didThe result
- Open access