Estimating Uncertainty of a Deep Learning–Based Breast Cancer Risk Prediction Model Using Test-Time Augmentations

Introduction. Deep-learning breast cancer risk (BCR) models are nearing clinical use, yet reliable uncertainty estimates are required for patient-level decisions. To address this gap, the aleatoric uncertainty of state-of-the-art BCR model was assessed using test-time augmentations (TTA).

Methods. The MIRAI model was used to predict 5-year BCR on 1,269 mammography exams from the Slovenian screening program. Aleatoric uncertainty was probed with crops (1–10 px) and small rotations (−4° to +5°), yielding 100 augmented inputs per exam. Predictions across TTA were fitted to a beta distribution; the mean was taken as the final risk, and the standard deviation (SD) was used to estimate aleatoric uncertainty. Discrimination performance was assessed with 5-year AUC for baseline and TTA-averaged predictions.

Results. Baseline 5-year BCR discrimination AUC was 0.77 [0.75–0.80] and remained the same after applying TTA-averaged predictions. Aleatoric uncertainty across exams was 0.03 [0.01–0.09] (95% CI). Visual review of the most uncertain exams showed consistent patterns: dense parenchyma, presence of a cardiac device, lesions near the chest wall, suboptimal positioning, and artifacts.

Conclusions. Realistic TTA provided a practical, discrimination-preserving estimation for aleatoric uncertainty: averaging over small acquisition-like perturbations maintained discrimination while revealing case-level variability linked to imaging complexity and technical quality. Reporting TTA-based uncertainty alongside risk may support individualized screening.