Development and cross-cohort evaluation of a multimodal deep-learning model for ER, PR and HER2 status classification in breast cancer

A multimodal deep-learning model for ER, PR, and HER2 classification achieved external AUCs of 0.952, 0.937, and 0.918 in CPTAC-BRCA, but showed systematic HER2 overprediction.

• Why it matters: ER, PR, and HER2 guide systemic therapy, yet multimodal models are seldom benchmarked against molecular baselines or tested for cross-cohort calibration and reliability.
• What they did: The model combined whole-slide images, transcriptomes, age, and stage, trained in TCGA-BRCA (1051 patients) and evaluated in CPTAC-BRCA (120 patients) with paired comparisons, transcript removal, alternative losses, and recalibration.
• The result: External macro-AUC was 0.936 versus 0.957 for logistic regression using ESR1, PGR, and ERBB2 (difference -0.021), and HER2 recalibrated ECE was 0.236 (ceiling 0.125) with calibration-in-the-large -1.657.

AI summary of the abstract · may contain errors · not medical advice · read the paper.