Evaluating uncertainty for detecting performance loss and guiding deferral after model transfer between hospital systems

After transferring neural-network mortality models from MIMIC-IV to eICU-CRD, AUROC dropped from 0.86–0.87 to 0.76–0.77, while predictive entropy and mutual information were lower.

• Why it matters: Performance loss after hospital transfer is often detected only after outcomes are observed, so clinicians need uncertainty signals that can flag which predictions to review.
• What they did: Three neural architectures were trained on MIMIC-IV for in-hospital mortality and tested on eICU-CRD, using predictive entropy, mutual information via Monte Carlo dropout and deep ensembles, and representation density.
• The result: None of the uncertainty measures reliably indicated reduced performance, but deferring the 20% highest-entropy cases lowered error rate from 17.5–20.1% to 6.3–8.0%, capturing 91–99% of false alarms while missing only 2–7% of deaths.

AI summary of the abstract · may contain errors · not medical advice · read the paper.