Source-linked AI summary
Federated Learning for Breast Density Classification: A Real-World Implementation
Holger R. Roth, Ken Chang, Praveer Singh, Nir Neumark, Wenqi Li, Vikash Gupta, Sharut Gupta, Liangqiong Qu, Alvin Ihsani, Bernardo C. Bizzo, Yuhong Wen, Varun Buch, Meesam Shah, Felipe Kitamura, Matheus Mendonça, Vitor Lavor, Ahmed Harouni, Colin Compas, Jesse Tetreault, Prerna Dogra, Yan Cheng, Selnur Erdal, Richard White, Behrooz Hashemian, Thomas Schultz, Miao Zhang, Adam McCarthy, B. Min Yun, Elshaimaa Sharaf, Katharina V. Hoebel, Jay B. Patel, Bryan Chen, Sean Ko, Evan Leibovitz, Etta D. Pisano, Laura Coombs, Daguang Xu, Keith J. Dreyer, Ittai Dayan, Ram C. Naidu, Mona Flores, Daniel Rubin, Jayashree Kalpathy-Cramer
TL;DR
Robust medical imaging models need large, diverse datasets, while centralizing healthcare data creates privacy, logistical, ethical, cost, and institutional barriers. This paper evaluates federated learning for BI-RADS breast density classification across real-world clinical sites without sharing raw data. Federated models improved local performance and cross-site generalizability, although global performance remained below local-test performance and required local adaptation or selection.
Problem
Robust clinically useful deep learning requires large quantities of diverse medical data, but centralized healthcare data sharing faces privacy, ethical, logistical, storage, and institutional barriers.
Method
Seven clinical institutions collaboratively trained a BI-RADS breast density classification model through federated learning, sharing model parameters while retaining data locally.
Results
Federated learning improved local test performance by 6.3% on average and cross-site generalizability by 45.8% on average.
Takeaways & Limitations
Real-world federated learning can train mammography models without centralized data and achieve performance comparable to reported centralized-data models.
Takeaways & Limitations
The final global model was not near acceptable performance on local test datasets, so local model selection or fine-tuning remained necessary amid site heterogeneity.
Abstract
from arXiv · showhide
Building robust deep learning-based models requires large quantities of diverse training data. In this study, we investigate the use of federated learning (FL) to build medical imaging classification models in a real-world collaborative setting. Seven clinical institutions from across the world joined this FL effort to train a model for breast density classification based on Breast Imaging, Reporting & Data System (BI-RADS). We show that despite substantial differences among the datasets from all sites (mammography system, class distribution, and data set size) and without centralizing data, we can successfully train AI models in federation. The results show that models trained using FL perform 6.3% on average better than their counterparts trained on an institute's local data alone. Furthermore, we show a 45.8% relative improvement in the models' generalizability when evaluated on the other participating sites' testing data.
1 Introduction
Deep learning for clinically useful medical imaging requires substantial, diverse data, but healthcare data sharing is constrained by privacy, ethics, logistics, cost, and institutional reluctance. Federated learning offers a decentralized alternative, motivating this real-world study of BI-RADS breast density classification across collaborating institutions.
- Robust and clinically useful deep learning models require large quantities of data.
- Centralized healthcare databases create logistical, approval, privacy, ethical, storage, and data-sharing challenges.
- The study develops a breast density classification model with international hospitals and imaging centers in a purely data-decentralized setting.Unlike previous studies that simulated federation, this effort trained collaboratively without sharing data or using centralized training references.
- BI-RADS classifies mammograms into fatty, scattered, heterogeneously dense, and extremely dense categories for breast density assessment.Breast density relates to fibroglandular tissue and breast cancer risk, and dense tissue may mask tumors.
- Subjective BI-RADS assessment produces substantial variability between radiologists, while prior deep learning achieved a linear kappa of 0.67 against the original interpreting radiologist.
- Federated learning enables collaborative decentralized training by sharing model weights rather than raw patient data.
2 Method
The study implements client-server federated learning for BI-RADS breast-density classification across seven international clients without sharing raw data. The method accommodates heterogeneous mammography datasets while aggregating local model updates centrally.
- Study setup: Seven international clients participated in a real-world federated-learning implementation for breast-density classification.The study used a purely data-decentralized collaborative setting.
- Data: Each client contributed retrospectively selected mammography data with BI-RADS labels obtained from radiologists’ medical records.Clients used digital screening mammography, with one multi-institutional dataset split by acquisition system.
- Data: Client datasets differed in detector type, image resolution, mammography type, size, and BI-RADS class distribution.These differences produced non-independent, non-identically distributed data across sites.
- Federated learning: Federated learning trains the same architecture locally, sends model updates to a server, aggregates them, and redistributes the updated weights for subsequent rounds.Only model weight differences are aggregated; local data remain at each client.
- Model and training: The model used DenseNet-121 with a four-output fully connected layer, trained on normalized 224 × 224 mammography images using softmax cross-entropy.Training included all available breast views, patient-level data splits, Adam optimization, weighted aggregation, and augmentation.
- Evaluation: Performance was evaluated with Cohen’s linear weighted kappa against radiologists’ ground-truth assessments before and after federated learning.The metric was selected for comparability with prior breast-density-classification studies.
3 Results
Federated learning improved locally evaluated breast density classification and substantially increased generalizability across participating institutions. Local fine-tuning further improved performance on nearly every client.
- 6.3% relative improvement was observed on average when federated models were applied to clients’ own test data.The comparison used locally best models trained with local data alone versus after federated learning.
- 45.8% average relative improvement was observed in generalizability on other clients’ test data.Generalizability was measured using the off-diag. mean across participating clients.
- Local training and federated learning were compared using locally best models selected by validation performance on local data.Table 2 reports both local-training and post-federated-learning performance.
- The average diagonal kappa increased from 0.64 with local training to 0.68 after federated learning.The corresponding off-diagonal mean increased from 0.18 to 0.26.
- Additional local fine-tuning improved a model’s local test performance for all but one client.Figure 5 summarizes weighted linear kappa before federated learning, after federated learning, and after local fine-tuning.
4 Discussion & Conclusions
In a real-world, data-decentralized setting, federated learning produced more accurate local models and improved generalizability across institutions without directly sharing data. The study also identifies domain, data-size, class-imbalance, and privacy limitations that motivate further work.
- Federated learning improved local accuracy and generalizability to test data from other institutions without accumulating a centralized dataset.The authors attribute the improvement to the effectively larger training set available through federation without direct data sharing.
- Federated learning achieved an average local-model performance of 0.68, comparable to the reported centralized-dataset result of 0.67.The paper notes that direct comparison with centralized training was not possible in this real-world setting.
- Generalizability improved but remained below local-test performance, and the final global model was not acceptable on any local test dataset.The authors state that local fine-tuning or model selection using local validation data is needed in practice.
- The study deliberately omitted data harmonization, leaving intensity-distribution differences across mammography systems as an investigated domain-shift challenge.Future work could examine histogram equalization or domain-adaptation strategies within federated learning.
- The framework did not fully address data-size heterogeneity, class imbalance, or privacy-preservation techniques that could reduce model-inversion and data-leakage risks.The paper suggests client-specific training, alternative sampling or loss functions, and differential privacy as future directions.