Clinical Prediction Model Performance Metrics Cluster at Arbitrary Thresholds and are Incorrectly Reported in High-Ranking Journals: An Observational Meta-Research Study

A meta-research of 2,001 clinical prediction model articles found AUC, sensitivity, and specificity cluster near perfect values and at multiples of five and ten, suggesting likely manipulation.

• Why it matters: Arbitrary threshold “hacking” can inflate reported performance and distort the evidence clinicians rely on, making genuinely useful prediction models harder to identify.
• What they did: Full texts from a random sample of Q1 journal articles for 2025 were extracted using a validated large language model, yielding 15,656 reported AUC values and checks against confidence intervals and ROC curves.
• The result: About 1 in 40 AUC values were reported outside their confidence intervals, and about 1 in 8 ROC curves did not match the reported AUC, implying unreliable reporting beyond AUC alone.

AI summary of the abstract · may contain errors · not medical advice · read the paper.