Source-linked AI summary
Split Learning for collaborative deep learning in healthcare
Maarten G. Poirot, Praneeth Vepakomma, Ken Chang, Jayashree Kalpathy-Cramer, Rajiv Gupta, Ramesh Raskar
TL;DR
Healthcare deep learning faces limited medical datasets and barriers to sharing patient information across institutions. This paper applies split learning to two medical imaging tasks and compares it with centralized and non-collaborative configurations. Split learning performed better than non-collaborative settings across both tasks, supporting distributed collaboration in healthcare while motivating future comparisons and real-world deployments.
Problem
Small medical datasets and ethical, regulatory, organizational, storage, and bandwidth barriers limit conventional multi-center healthcare collaboration.
Method
The study applies split learning to retinal fundus and chest X-ray classification and compares it with centrally hosted and non-collaborative configurations across distributed participants.
Results
Split learning performed better than non-collaborative configurations on both datasets, with CheXpert performance significantly lower in non-collaborative settings especially beyond two clients.
Takeaways & Limitations
Distributed machine learning can provide benefits for medical collaboration across entities without relying on conventional centralized training.
Takeaways & Limitations
Future work will compare split learning with federated learning and LS-SGD in medical settings and investigate privacy enhancements and alternative configurations through controlled real-world deployments.
Abstract
from arXiv · showhide
Shortage of labeled data has been holding the surge of deep learning in healthcare back, as sample sizes are often small, patient information cannot be shared openly, and multi-center collaborative studies are a burden to set up. Distributed machine learning methods promise to mitigate these problems. We argue for a split learning based approach and apply this distributed learning method for the first time in the medical field to compare performance against (1) centrally hosted and (2) non collaborative configurations for a range of participants. Two medical deep learning tasks are used to compare split learning to conventional single and multi center approaches: a binary classification problem of a data set of 9000 fundus photos, and multi-label classification problem of a data set of 156,535 chest X-rays. The several distributed learning setups are compared for a range of 1-50 distributed participants. Performance of the split learning configuration remained constant for any number of clients compared to a single center study, showing a marked difference compared to the non collaborative configuration after 2 clients (p < 0.001) for both sets. Our results affirm the benefits of collaborative training of deep neural networks in health care. Our work proves the significant benefit of distributed learning in healthcare, and paves the way for future real-world implementations.
1 Introduction
Healthcare deep learning is constrained by small medical datasets and barriers to centralizing patient data. Distributed learning, including split learning, is presented as an alternative for collaborative training without conventional centralization.
- Small medical sample sizes limit the training data traditionally available for clinical models.
- Centralizing multi-center datasets can increase data amount and diversity but may conflict with ethical or regulatory restrictions on patient data leaving institutions.
- Institutional data value concerns, storage demands, and bandwidth requirements further impede traditional healthcare collaboration.
- Distributed learning methods can be compared by centralized performance, privacy, bandwidth usage, and computational-load distribution.
- This study applies split learning in medicine and compares it with centrally hosted and non-collaborative configurations across retinal fundus and chest X-ray datasets.
2 Related work
Split learning partitions a neural network into linked components with different access properties. The U-shaped configuration is selected for healthcare because it avoids raw-data and label sharing while distributing computation across clients and a central server.
- Earlier comparisons reported split learning outperforming federated learning and LS-SGD on convergence, client-side computation, and information leakage.
- Split learning divides a neural network into links forming a chain, with links hosted centrally or kept local to proprietary clients.
- U-shaped split learning: The U-shaped configuration uses local and central links and requires no raw-data or label sharing.
- U-shaped split learning: Sequential client training transfers local-link states without waiting for every client, avoiding synchronous-training logistics.
- U-shaped split learning: Placing most trainable layers in the central link reduces shared local-state bandwidth and client-side computational cost.
3 Methods
The study evaluates split learning on diabetic-retinopathy fundus images and CheXpert chest X-rays using collaborative, non-collaborative, and distributed-participant settings. Performance is assessed with task-specific validation metrics.
- Data collection: The diabetic-retinopathy dataset contains 9000 downsampled fundus images for binary normal-versus-abnormal classification.
- Data collection: CheXpert is posed as multi-label classification of 14 common chest-radiographic observations from chest radiographs labeled across patients.
- Data partitioning: Both datasets use random 75% training and 25% validation partitions, with training data split equally across clients when distributed.
- Model implementation: The experiments use ResNet-34 with binary cross-entropy for diabetic retinopathy and pretrained DenseNet121 with sigmoid binary cross-entropy for CheXpert.
- Training configurations: Collaborative clients train sequentially for one epoch, while non-collaborative training uses the same per-client sample size.
- Performance analysis: Diabetic-retinopathy performance uses averaged validation accuracy, whereas CheXpert performance uses mean AUROC across five competition tasks.
4 Results
Split learning outperformed non-collaborative configurations on both medical tasks, with the difference especially pronounced beyond two clients for CheXpert. The comparison used accuracy for diabetic retinopathy and AUROC for CheXpert.
- Split learning performed better than non-collaborative configurations on both CheXpert and diabetic-retinopathy datasets.
- For CheXpert, non-collaborative mean performance was significantly lower than collaborative performance, especially with more than two clients, at α = 0.005.
5 Discussion and future work
The study reports benefits of distributed split learning for medical collaboration and identifies privacy, efficiency, and real-world deployment as areas for future work.
- Distributed machine learning can enhance collaboration across healthcare entities.
- Split learning performed better than conventional single- and multi-institution setups in the reported medical applications.
- Future work includes improving weight-transfer efficiency, enhancing privacy, and evaluating alternative configurations in controlled real-world healthcare deployments.