Source-linked AI summary
Split learning for health: Distributed deep learning without sharing raw patient data
Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, Ramesh Raskar
TL;DR
Health entities need collaborative deep-learning methods that avoid sharing sensitive raw patient data and can accommodate heterogeneous modalities, labels, and tasks. The paper proposes practical SplitNN configurations for these settings and reports higher accuracy with lower client-side computational requirements than federated learning and large-batch synchronous SGD.
Problem
Health entities need to collaboratively train models across sensitive, heterogeneous patient data without sharing raw data or, in some settings, labels.
Method
The paper develops SplitNN configurations for vanilla collaboration, label-free learning, vertically partitioned modalities, multitask learning, and multihop collaboration.
Results
SplitNN achieves higher validation accuracies with drastically lower client-side computational requirements than federated learning and large-batch synchronous SGD.
Takeaways & Limitations
SplitNN provides versatile, resource-efficient distributed learning configurations for practical health collaborations without exchanging raw patient data.
Abstract
from arXiv · showhide
Can health entities collaboratively train deep learning models without sharing sensitive raw data? This paper proposes several configurations of a distributed deep learning method called SplitNN to facilitate such collaborations. SplitNN does not share raw data or model details with collaborating institutions. The proposed configurations of splitNN cater to practical settings of i) entities holding different modalities of patient data, ii) centralized and local health entities collaborating on multiple tasks and iii) learning without sharing labels. We compare performance and resource efficiency trade-offs of splitNN and other distributed deep learning methods like federated learning, large batch synchronous stochastic gradient descent and show highly encouraging results for splitNN.
1 Introduction
Health collaboration is constrained by trust, regulation, and limited patient consent, despite the need to train distributed models across sensitive and heterogeneous data. The paper motivates distributed learning that pools information without exchanging raw patient data.
- Trust deficits, data-sharing regulations, and limited patient consent impede collaboration among health entities.
- Health institutions may hold complementary EHR, imaging, pathology, or genetic data that could support collaborative model training.
- The motivating setting combines distributed learning over patient images with slow communication links while avoiding raw-data exchange.
- The paper builds on SplitNN to address practical health collaborations with lower computational-resource and communication-bandwidth demands than prior approaches.
2 SplitNN configurations for health
The paper proposes SplitNN configurations that keep raw patient data local while supporting vanilla collaboration, label-free learning, and vertically partitioned multimodal data.
- The section presents several SplitNN configurations for practical health settings.
- Vanilla configuration: SplitNN divides training at a cut layer: clients send intermediate outputs to a server, which returns cut-layer gradients without accessing raw data.
- Learning without label sharing: U-shaped SplitNN configurations avoid label sharing by returning server outputs to clients, which generate gradients from end layers.
- Learning without label sharing: The label-free setup is intended for cases where labels, such as patient disease status, are highly sensitive.
- Vertically partitioned data: Vertically partitioned SplitNN lets institutions holding different patient-data modalities train partial models whose cut-layer outputs are combined.
3 Results about resource efficiency
Across the reported comparisons, SplitNN achieves higher validation accuracies with substantially lower client-side computation than federated learning and large-batch synchronous SGD. Its communication advantage depends on the number of clients.
- SplitNN outperforms federated learning and large-batch synchronous SGD in validation accuracy while requiring drastically less client computation.The comparisons use CIFAR 10 with VGG for 100 clients and CIFAR 100 with ResNet-50 for 500 clients.
- SplitNN can exploit the distribution of network parameters found in most modern CNNs to improve client-side resource efficiency.
- Table 1 reports drastically lower per-client computation resources for SplitNN than large-batch SGD and federated learning on CIFAR 10 with VGG.Resources are measured in teraflops.
- For CIFAR 100 with ResNet, SplitNN requires lower per-client bandwidth than the alternatives with a large number of clients.For smaller client counts, federated learning requires lower bandwidth than SplitNN, while large-batch SGD remains heavy in both settings.
4 Conclusion and future work
The paper concludes that practical health collaborations require more than simple distributed-learning configurations and presents SplitNN as a versatile, resource-efficient alternative. It also identifies scalability, architectural flexibility, and compression-based extensions as future directions.
- Simple distributed-learning configurations do not cover the practical collaboration settings encountered across health entities.
- SplitNN is described as dramatically more resource efficient than federated learning and large-batch synchronous SGD, with plug-and-play configurations.
- SplitNN is scalable to large-scale settings and compatible with state-of-the-art deep-learning architectures.
- Combining SplitNN with neural-network compression methods is proposed to further improve distributed learning with edge devices.
5 Supplementary material:
The supplementary material extends SplitNN with configurations for chained processing, multi-task learning, and multi-hop collaboration while avoiding raw-data transfer between health entities.
- Extended vanilla split learning processes concatenated client outputs at another client before sending them to the server.
- Multi-task split learning concatenates outputs from modality-specific client networks and sends them to multiple servers for different supervised learning tasks.
- Multi-hop split learning chains clients sequentially, with each client transferring cut-layer outputs to the next until the final client sends activations to a server.
- Across these health configurations, SplitNN avoids transferring raw data between client and server entities during distributed model training and inference.