Source-linked AI summary
Decentralized Federated Learning through Proxy Model Sharing
Shivam Kalra, Junfeng Wen, Jesse C. Cresswell, Maksims Volkovs, Hamid R. Tizhoosh
TL;DR
Highly regulated institutions need collaborative learning without centralizing sensitive data, while conventional federated learning can constrain model autonomy and provide insufficient privacy guarantees. ProxyFL uses private and shared proxy models with decentralized, differentially private communication. Across image benchmarks and a medical histopathology task, it is reported as competitive or superior in accuracy, communication efficiency, and privacy preservation.
Problem
Regulatory and privacy constraints prevent many healthcare and finance institutions from pooling data, while centralized federated learning can conflict with institutional model autonomy and gradient updates may expose private examples.
Method
ProxyFL maintains private and proxy models, trains them jointly with differential privacy, and exchanges proxies over a decentralized communication graph.
Results
ProxyFL’s private models achieve the best overall performance on all evaluated image datasets, with statistically significant improvements over other methods (p-value < 1e-5).
Takeaways & Limitations
ProxyFL provides a decentralized and communication-efficient mechanism for multi-institutional training while preserving data and model privacy, including in a four-institution histopathology collaboration.
Takeaways & Limitations
The study assumes clients collaborate in good faith and focuses on privacy rather than security against malicious participants.
Abstract
from arXiv · showhide
Institutions in highly regulated domains such as finance and healthcare often have restrictive rules around data sharing. Federated learning is a distributed learning framework that enables multi-institutional collaborations on decentralized data with improved protection for each collaborator's data privacy. In this paper, we propose a communication-efficient scheme for decentralized federated learning called ProxyFL, or proxy-based federated learning. Each participant in ProxyFL maintains two models, a private model, and a publicly shared proxy model designed to protect the participant's privacy. Proxy models allow efficient information exchange among participants without the need of a centralized server. The proposed method eliminates a significant limitation of canonical federated learning by allowing model heterogeneity; each participant can have a private model with any architecture. Furthermore, our protocol for communication by proxy leads to stronger privacy guarantees using differential privacy analysis. Experiments on popular image datasets, and a cancer diagnostic problem using high-quality gigapixel histology whole slide images, show that ProxyFL can outperform existing alternatives with much less communication overhead and stronger privacy.
1 Introduction
Healthcare and finance need collaborative learning without pooling sensitive data, but centralized federated learning can undermine institutional model autonomy and lacks sufficient privacy guarantees. ProxyFL addresses these constraints through decentralized proxy exchange, differential privacy, and communication-efficient collaboration.
- Motivation: Healthcare and finance cannot freely pool data because regulations and privacy concerns restrict sharing.This limits research and model development despite the value of larger, more diverse datasets.
- Motivation: Histopathology collaboration is especially challenging because gigapixel images often occupy one or more gigabytes and vary across preparation sites.Such heterogeneity can cause models to focus on imaging artifacts and fail to generalize to new sources.
- Federated learning constraints: Centralized federated learning is unsuitable when hospitals require autonomy over models for regulatory compliance and specialty-specific tailoring.Decentralized federated learning is therefore preferred for this multi-institutional setting.
- Privacy: Raw-data locality does not guarantee security because clients send unaudited gradient updates that deep networks may use to memorize individual examples.Differential privacy instead provides quantitative guarantees by obscuring the effect of individual data points.
- Proposed approach: ProxyFL proposes decentralized collaboration adapted to heterogeneous data sources while preserving participant model autonomy, differential privacy, and communication efficiency.The stated contributions include privacy analysis and reducing the communication overhead required for collaboration.
2 Related Work
Related decentralized learning methods address data locality or coordination, but they retain important limitations for highly regulated, multi-institutional collaboration. These include reliance on central authorities, homogeneous architectures, or insufficient quantitative privacy guarantees.
- Decentralized federated learning: Decentralized federated learning is more suitable than centralized federated learning when regulatory constraints make a shared central model undesirable.Its main challenge is enabling peer-to-peer information passing among participants.
- Alternative methods: Cyclical Weight Transfer avoids data centralization, while split learning relies on a central party for inference and swarm learning elects a central authority each round.Swarm learning also requires homogeneous model architectures and inherits federated learning’s weaker performance under strict privacy measures.
3 Method - ProxyFL
ProxyFL is a decentralized federated-learning method in which clients retain private models while sharing differentially private proxy models through peer-to-peer communication. It supports heterogeneous private-model structures, privacy accounting, and communication-efficient proxy aggregation.
- ProxyFL design: ProxyFL lets clients preserve private-model autonomy while collaborating through shared proxy models in decentralized, multi-institutional settings.Private models may have different structures, provided their input/output specifications match.
- Joint training: Each client jointly trains its private and proxy models, using a DML-style objective so each model can learn from the other.The private objective combines cross-entropy with a KL-divergence term, while the proxy objective is similarly defined.
- Privacy protection: Proxy models are trained with DP-SGD: per-example gradients are clipped, aggregated over mini-batches, and perturbed with Gaussian noise before sharing.The clipping threshold C and noise level σ determine the per-step privacy cost.
- Privacy protection: Clients individually track (ϵ, δ) privacy for proxy training and may leave the protocol when their prespecified privacy budget is reached.Privacy composition is tracked using Rényi differential privacy and converted to (ϵ, δ)-DP.
- Communication efficiency: PushSum exchanges proxies over a sparse, potentially asymmetric and time-varying graph, reducing communication overhead without a central parameter server.De-biasing weights correct the limiting distribution so decentralized aggregation can mimic model averaging.
- Communication efficiency: Proxy training and communication alternate by round rather than running proxy averaging to convergence after every local update.This interleaves local optimization with decentralized proxy exchange.
4 Empirical Evaluation
ProxyFL is evaluated on benchmark image datasets and medical imaging tasks using decentralized proxy communication, differential privacy, and heterogeneous-client settings. Across these experiments, it combines strong performance with scalable communication and robustness to distribution shifts.
- Benchmark Image Classification: ProxyFL uses an exponential communication protocol in which each client sends and receives one proxy per round, supporting scalability as client numbers grow.The protocol communicates with peers at periodically increasing distances, while its per-round time complexity remains constant with respect to the number of clients.
- Benchmark Image Classification: ProxyFL-private achieves the best overall performance on MNIST, Fashion-MNIST, and CIFAR10, with statistically significant improvements over other methods (p-value < 1e-5).Joint training is described as an upper bound because it combines the private datasets.
- Benchmark Image Classification: Decentralized schemes are more robust to DP training: AvgPush outperforms FedAvg, while ProxyFL consistently outperforms FML.The comparison is reported across the benchmark datasets in the DP-training evaluation.
- Ablation: ProxyFL-private is the most robust to increasing non-IID skew, while its proxy model achieves performance similar to FML-private without DP guarantees.The ablation also evaluates heterogeneous private architectures across clients.
- Gastrointestinal Disease Detection: On Kvasir, decentralized AvgPush and ProxyFL learn substantially more than centralized FedAvg and FML-proxy, and ProxyFL-private consistently outperforms FML-private during training.The dataset contains 8,000 endoscopic images from eight gastrointestinal-disease classes distributed across eight clients.
- Histopathology Image Analysis: In histopathology classification, ProxyFL and FML achieve higher accuracy than other approaches, while ProxyFL continues improving marginally through the end of training.The task uses whole-slide-image data from four institutions and evaluates models on a merged multi-centric test set.
5 Conclusion and Future Work
ProxyFL addresses regulated-domain data constraints with decentralized, privacy-preserving federated learning and is evaluated on both benchmarks and histopathology data. The paper identifies public-dataset dependence, practical privacy validation, and malicious participants as future-work boundaries.
- Conclusion: ProxyFL targets decentralized multi-institutional learning without revealing participants’ private data.The scheme is presented as preserving data and model privacy while supporting communication-efficient distributed training.
- Conclusion: Experiments include a real-world collaboration among four medical institutions to identify healthy versus cancerous tissue from histopathology images.The histopathology evaluation records performance over four clients across fifteen independent runs.
- Conclusion: ProxyFL is reported as competitive with baselines in model accuracy, communication efficiency, and privacy preservation.The supplied conclusion states this comparison at the paper level without giving a specific metric value.
- Future Work: Some private-knowledge-transfer federated-learning methods require a public dataset, whereas ProxyFL’s work focuses on privacy without that stated requirement.The paper notes that public datasets are not readily available in highly regulated domains and leaves their use as future research.
- Future Work: The work focuses on privacy rather than security and assumes that all clients collaborate in good faith.Handling malicious participants is identified as an important direction for future research.
- Future Work: The paper argues that differential-privacy guarantees should be validated in practice against reconstruction and membership-inference attacks.It notes that theoretical guarantees alone do not establish practical resistance to these attacks.
Data Availability
The paper uses publicly available image datasets for its experiments, including standard classification benchmarks, gastrointestinal endoscopy images, and Camelyon-17 histopathology whole-slide images.
- Data Availability: All datasets used in the study are publicly available.The listed datasets include MNIST, Fashion-MNIST, CIFAR-10, Kvasir, and selected Camelyon-17 whole-slide images.
- Data Availability: The histopathology study uses some of the 1,399 whole-slide images from Camelyon-17.The passage identifies Camelyon-17 as the source for the histopathology study.
A.1 Benchmark Image Classification
The benchmark comparison uses specified MLP and CNN architectures from prior work, with explicit layer structures for reproducible direct comparison.
- A.1 Benchmark Image Classification: The experiments use model architectures from Shen et al. for direct comparison.The passage specifies the MLP and CNN architectures used in the benchmarks.
- A.1 Benchmark Image Classification: The MLP has two hidden layers with 200 units each and ReLU activations.This is the architecture definition given for the benchmark MLP.
- A.1 Benchmark Image Classification: CNN1 consists of convolution, ReLU, max-pooling, and fully connected layers with the specified channel and unit counts.The passage expands the meanings of convolutional, pooling, and fully connected layer notation.
B.1 Benchmark Image Classification
Additional benchmark analyses examine macro-accuracy, architecture homogeneity, batch size, and DML weights. The reported results support robust class performance, model-structure flexibility, and a privacy–training-time trade-off.
- Macro-accuracy: Macro-accuracies are very similar to the main-text accuracies, indicating that methods are not biased toward clients’ majority classes.Macro-accuracy averages per-class accuracies, making it informative for the imbalanced client settings used here.
- Architecture ablation: ProxyFL-private achieves the best performance regardless of the private model structure when all models use the same MLP architecture.This is shown as an additional MNIST ablation relative to the main-text setup.
- Batch size: Smaller batch sizes dramatically improve the privacy guarantee without significantly affecting ProxyFL-private’s final accuracy.The trade-off is longer training time, while the main experiments used B = 250 images per DP-SGD step.
- Batch size: Strong privacy guarantees can be obtained with small batch sizes at the cost of longer training time.The passage contrasts the privacy benefit of smaller batches with their computational consequence.
- DML weights: ProxyFL outperforms its centralized counterpart FML for all tested values of the DML weights α and β.The experiments fix α = β; increasing α narrows the performance gap between private and proxy models.
B.2 Histopathology Image Analysis
ProxyFL reduces communication time per client by avoiding the central-server bottleneck present in FedAvg and FML. Its decentralized protocol requires each client to send and receive only one model per round.
- Figure 13 reports mean and standard deviation across 4 clients for each of 15 independent runs.
- ProxyFL is more communication efficient than FedAvg and FML because it avoids a bottleneck at the central server.
- Centralized methods sequentially exchange models between the server and all clients, whereas decentralized methods use one model exchange per client each round.