Source-linked AI summary
NVIDIA FLARE: Federated Learning from Simulation to Real-World
Holger R. Roth, Yan Cheng, Yuhong Wen, Isaac Yang, Ziyue Xu, Yuan-Ting Hsieh, Kristopher Kersten, Ahmed Harouni, Can Zhao, Kevin Lu, Zhihong Zhang, Wenqi Li, Andriy Myronenko, Dong Yang, Sean Yang, Nicola Rieke, Abood Quraini, Chester Chen, Daguang Xu, Nic Ma, Prerna Dogra, Mona Flores, Andrew Feng
TL;DR
Federated learning needs practical ways to support collaboration across private, heterogeneous data and deployment settings. This paper presents NVFlare, an open-source, flexible SDK for simulation and real-world federated workflows, and reports that advanced algorithms can outperform FedAvg and FedProx under high data heterogeneity. The overview also identifies omitted NVFlare features as a scope limitation.
Problem
Federated workflows must accommodate distinct training schemes, model representations, and APIs across machine-learning libraries and collaborative settings.
Method
NVFlare uses a specification-based, componentized Python architecture that supports federated workflows, diverse training libraries, privacy mechanisms, and simulation-to-production deployment.
Results
Under α=0.1 data heterogeneity, FedOpt and SCAFFOLD show markedly better convergence rates and performance than FedAvg and FedProx with the same training steps.
Takeaways & Limitations
NVFlare supports customizable federated collaboration across research and real-world healthcare use cases, including COVID-19 analysis and medical-image applications.
Takeaways & Limitations
The paper does not cover all NVFlare features, including homomorphic encryption, TensorBoard streaming, provisioning dashboards, and MONAI integration.
Abstract
from arXiv · showhide
Federated learning (FL) enables building robust and generalizable AI models by leveraging diverse datasets from multiple collaborators without centralizing the data. We created NVIDIA FLARE as an open-source software development kit (SDK) to make it easier for data scientists to use FL in their research and real-world applications. The SDK includes solutions for state-of-the-art FL algorithms and federated machine learning approaches, which facilitate building workflows for distributed learning across enterprises and enable platform developers to create a secure, privacy-preserving offering for multiparty collaboration utilizing homomorphic encryption or differential privacy. The SDK is a lightweight, flexible, and scalable Python package. It allows researchers to apply their data science workflows in any training libraries (PyTorch, TensorFlow, XGBoost, or even NumPy) in real-world FL settings. This paper introduces the key design principles of NVFlare and illustrates some use cases (e.g., COVID analysis) with customizable FL workflows that implement different privacy-preserving algorithms. Code is available at https://github.com/NVIDIA/NVFlare.
1 Introduction
Federated learning supports robust, generalizable models by enabling collaboration without centralizing private data. NVFlare is introduced as an open-source SDK spanning research simulation and real-world applications.
- NVFlare helps data scientists develop robust and generalizable models by sharing model weights rather than private data.
- Healthcare is a prominent FL application because patient data requires protection.
- Existing FL efforts differ in whether they prioritize simulated research settings or production deployment.
- NVFlare aims to support both efficient, extensible simulation and production-oriented federated learning.
2 NVIDIA FLARE Overview
NVFlare is a flexible SDK that adapts diverse machine-learning workflows to federated settings. Its specification-based, componentized architecture supports customizable collaboration, privacy mechanisms, simulation, and deployment across training libraries.
- NVFlare enables researchers and data scientists to adapt machine-learning and deep-learning workflows to federated settings.
- The Python framework is lightweight, scalable, and agnostic to training libraries including PyTorch, TensorFlow, and NumPy.
- A controller coordinates client tasks and processes returned results, supporting workflows such as FedAvg and cyclic weight transfer.
- NVFlare includes simulation, dashboard, reference algorithms, differential privacy, homomorphic encryption, and pluggable integrations.
- Specification-based design exposes clear APIs and extensible components, including controllers, task executors, and filters.
3 System Concepts
NVFlare organizes federated computing around client-server components, task-based communication, and extensible data-processing mechanisms. It also provides configurable communication, simulation, privacy, security, scheduling, and availability features for real-world deployment.
- An NVFlare system connects FL servers, FL clients, and admin clients through a client-server architecture.
- Controllers assign client tasks, while workers execute them and return Shareable results containing standardized data objects.
- Communication drivers abstract transport protocols such as GRPC and TCP and support server-centric or peer-to-peer patterns.
- In the multi-cloud experiment, concurrent global-model downloads were slower because they increased server load, while AWS clients experienced additional cross-cloud delay.
- Filters transform shared data and provide privacy mechanisms including variable exclusion, percentile truncation, sparse-vector differential privacy, and homomorphic encryption.
- The simulator runs server and clients locally while preserving real-world job execution, allowing simulated components to be deployed in federated scenarios.
- NVFlare addresses security through authentication, confidentiality, authorization, privacy protection, auditing, and client policies, with mutual TLS provisioning for communicating parties.
4 Federated Data Science
NVFlare supports federated data science across deep-learning, traditional machine-learning, split-learning, and federated-statistics workflows. Its examples cover heterogeneous-data experiments, tree-based XGBoost, privacy-preserving coordination, communication patterns, and global statistical aggregation.
- Federated algorithms: Lower Dirichlet alpha values create greater client-data heterogeneity in CIFAR-10 experiments, affecting FedAvg performance.
- Federated algorithms: NVFlare supports multiple federated algorithms, including FedAvg, FedProx, FedOpt, SCAFFOLD, federated XGBoost, Ditto, FedSM, and Auto-FedRL.
- Federated algorithms: Under high heterogeneity with alpha=0.1, FedOpt and SCAFFOLD converge faster and perform better than FedAvg and FedProx.SCAFFOLD adds a client-model correction term, while FedOpt uses server-side SGD with momentum.
- Federated machine learning: Federated XGBoost shares locally added trees, bags received trees into a global forest, and continues boosting from the updated global model.
- Split learning: Split learning exchanges activations and gradients between clients holding images and labels, using private set intersection to align them.NVFlare’s PSI implementation can extend to multiple parties and other privacy-preserving database-alignment scenarios.
- Split learning: Direct peer-to-peer split-learning communication is more efficient than routing through the server, with only slight overhead versus standalone PyTorch.The comparison uses CIFAR-10 with 50,000 training samples, 15,625 communication rounds, and batch size 64.
- Federated statistics: NVFlare aggregates local feature statistics across clients and datasets to produce complete global statistics for numeric features.
5 Real-world Use Cases
NVFlare has supported real-world federated-learning studies in healthcare, including imaging and electronic-health-record applications. These collaborations evaluated federated learning for several clinically relevant prediction and segmentation tasks.
- Real-world collaborations used NVFlare for breast mammography classification, prostate segmentation, pancreas segmentation, and COVID-19 chest-X-ray and electronic-health-record analysis.
6 Summary & Conclusion
NVFlare is presented as an open-source SDK supporting federated learning from research through real-world deployment. Its flexibility spans training libraries and includes ongoing work toward large-model federated fine-tuning, while the overview does not cover every available feature.
- NVFlare is an open-source SDK designed to help data scientists use federated learning in research and real-world applications.
- Its design supports an easy transition from research workflows to real-world deployment, including handling unstable connections and client failures.
- NVFlare integrates with different deep learning frameworks and is intended to support federated fine-tuning of models with billions of parameters.
- The overview does not detail features including homomorphic encryption, TensorBoard streaming, a provisioning dashboard, and MONAI integration.
- NVFlare remains an open-source project that invites community contributions.