Similar discrimination, different behaviour under synthetic occlusion: confidence and attribution reproducibility of deep learning models for intraoral ultrasonography

EfficientNet-B0 shows markedly different behavior under synthetic occlusion despite similar baseline discrimination, with 49/50 boundary-sweep conditions dropping below 0.60 AUC.

• Why it matters: Similar ROC performance can mask instability when occlusion occurs, and it is unclear whether confidence and spatial attributions remain reliable for intraoral ultrasonography models.
• What they did: Five deep-learning architectures were tested on a subject-independent set (97 images, eight volunteers) with constant-intensity occlusion bands spanning 30–70% depth, across five training seeds. Attribution reproducibility was evaluated across seeds and class-activation methods, including attribution-guided versus random insertion.
• The result: EfficientNet-B0’s incorrect-prediction confidence under 50% occlusion rose to 0.989 and expected calibration error increased from 0.094 to 0.478, while occluded AUC varied by up to 0.161 between inference runs. Attribution patterns changed across seeds and methods, indicating limited reproducibility.

AI summary of the abstract · may contain errors · not medical advice · read the paper.