bioRxivJClub
Auditing Biomedical Literature with Blinded AI Peer Review: Where Peer Review Works, Where It Fails, and How to Strengthen It
bioRxiv · · Preprint · Open access
Kosakovsky Pond, Pond + 4 more
Abstract ↗AI summary
The abstract is read at the publisher; the summary is JClub's.
BARS (Blinded AI Review System) produced 1,874 blind reviews and achieved 100% pass rates on verified sound controls while detecting 77%–91% of internally flawed papers.
- Why it matters: Biomedical publishing has expanded far faster than human peer review, and unblinded language models can’t audit literature objectively due to memorizing famous authors and journals.
- What they did: BARS strips titles, author identities, institutions, journal headers, and dates, then grades full texts on a 10-point rubric; it also audited 320 open-access articles (2020–2025) and 915 date-blinded papers (1991–2025).
- The result: Core rigor failures were common (54% scored <5.0/10; 41% excluding megajournals), scores declined from 7.6/10 to 5.8/10 over time, and BARS recalled 98% of unresolved human referee critiques.
The findingWhy it mattersWhat they didThe result
AI summary of the abstract · may contain errors · not medical advice · read the paper.
- Open access