Systematic Benchmarking and Optimization of Retrieval-Augmented Generation (RAG) for Clinical Phenotyping: A Type II Diabetes Case Study in an ICD-Screened Cohort

In an ICD-screened cohort for type 2 diabetes mellitus, GPT-4o RAG phenotyping across 24 prompt–configuration comparisons achieved mean sensitivity 0.9019 and F1-score 0.9204.

• Why it matters: EHR phenotyping often needs disease-specific upkeep, and RAG design choices are not well quantified for clinical phenotyping performance tradeoffs.
• What they did: The study processed 170,678 clinical notes from 275 patients, testing seven embedding models and three prompt strategies, then evaluating 350 embedding–chunk–retrieval-depth combinations to select eight configurations.
• The result: RAG outperformed ICD codes on PPV (0.9405 vs 0.7838), showed significantly higher sensitivity than PheNorm (Holm-adjusted p < 0.001), and paired bootstrap indicated higher NPV and F1-score without significant PPV change.

AI summary of the abstract · may contain errors · not medical advice · read the paper.