Source-linked AI summary
OpenFL: An open-source framework for Federated Learning
G Anthony Reina, Alexey Gruzdev, Patrick Foley, Olga Perepelkina, Mansi Sharma, Igor Davidyuk, Ilya Trushkin, Maksim Radionov, Aleksandr Mokrov, Dmitry Agapov, Jason Martin, Brandon Edwards, Micah J. Sheller, Sarthak Pati, Prakash Narayana Moorthy, Shih-han Wang, Prashant Shah, Spyridon Bakas
TL;DR
The paper addresses the difficulty of collaborative machine-learning training when data are too large or sensitive to centralize. It presents OpenFL, an open-source framework that keeps data at remote owners while coordinating federated training, and describes healthcare and competition deployments. OpenFL is intended to support production-oriented, secure, and reusable federated-learning collaborations.
Problem
Large and diverse annotated datasets can be too large or sensitive to transmit to centralized servers, limiting collaborative machine-learning training.
Method
OpenFL provides an open-source, framework-agnostic federated-learning platform that trains models at remote data owners while sharing model updates and metrics through an aggregator.
Results
The paper describes OpenFL’s use for consensus brain-tumor boundary models in international healthcare organizations and for the first computational competition on federated learning.
Takeaways & Limitations
OpenFL supports reusable federated-learning collaborations across healthcare and other industries while keeping training data at collaborator nodes.
Takeaways & Limitations
Federated learning introduces additional privacy, security, and confidentiality challenges, including protecting model intellectual property and preventing data extraction from model weights.
Abstract
from arXiv · showhide
Federated learning (FL) is a computational paradigm that enables organizations to collaborate on machine learning (ML) projects without sharing sensitive data, such as, patient records, financial data, or classified secrets. Open Federated Learning (OpenFL https://github.com/intel/openfl) is an open-source framework for training ML algorithms using the data-private collaborative learning paradigm of FL. OpenFL works with training pipelines built with both TensorFlow and PyTorch, and can be easily extended to other ML and deep learning frameworks. Here, we summarize the motivation and development characteristics of OpenFL, with the intention of facilitating its application to existing ML model training in a production environment. Finally, we describe the first use of the OpenFL framework to train consensus ML models in a consortium of international healthcare organizations, as well as how it facilitates the first computational competition on FL.
1 Intel Corporation, Santa Clara, CA 95052, USA
The paper identifies federated learning and OpenFL among related machine-learning and collaborative-computing concepts.
- OpenFL is associated with federated learning, machine learning, deep learning, distributed computing, collaborative learning, and secure computation.
1 Motivation
AI progress depends on data, but large or sensitive datasets are difficult to centralize. Federated learning addresses this by moving models to local data, and OpenFL applies the approach to healthcare collaboration and federated competition.
- Large or sensitive annotated datasets are difficult to transmit to centralized servers for machine-learning training.
- Federated learning lets organizations collaborate without sharing sensitive information by moving the model to the data.
- Federated-learning models can achieve similar accuracy to models trained with centralized learning.
- The paper presents OpenFL, explains how to convert existing machine-learning training into federated pipelines, and describes healthcare and competition applications.
2 The ‘Open Federated Learning’ framework
OpenFL is an open-source, framework-agnostic platform that trains models where data resides while coordinating collaborators through an aggregator, federation plan, and secure connections.
- Synopsis: OpenFL trains machine-learning models on remote data owners’ nodes while keeping training data at collaborator nodes and sharing model updates and metrics.
- Synopsis: OpenFL supports TensorFlow and PyTorch and can be extended to other machine-learning frameworks.
- Architecture: The federation uses a star topology in which collaborators train locally and an aggregator combines updates into a global model.
- Architecture: A shared YAML FL plan specifies federation settings, training parameters, and remote procedure calls.
- Architecture: The backend coordinates collaborator tasks and receives model and metric updates for aggregation.
- Security: Collaborators and aggregators communicate through mutually authenticated TLS connections, with production deployments requiring trusted PKI certificates.
Trusted Execution Environments
The paper describes hardware-backed execution security for federated learning and provides installation and deployment paths for OpenFL.
- Trusted execution environments aim to provide confidentiality, integrity, and remote attestation for federated-learning execution.
- Graphene with SGX runs OpenFL with Intel SGX without modifying the OpenFL code.
- OpenFL can be installed through Python packages, a built wheel, or Docker containers.
- The fx command-line interface is available after installation for developing federations.
2.5 Running a Federation
OpenFL supports federation development through both a Python API and the fx CLI. The CLI is presented as better suited to scaling federations in production, while the Python API is easier for newcomers.
- OpenFL provides two federation-development methods: the Python API and the fx CLI.
- The fx CLI is considered the better path for scaling federations within a production environment.
- The Python API is easier for data scientists working with OpenFL for the first time.
Python API
OpenFL’s Python API is introduced through a Jupyter Notebook workflow. The tutorial wraps and shards MNIST data, instantiates an FL model, and uses fx commands to initialize a workspace.
- The Python API tutorials run in a Jupyter Notebook server after OpenFL is installed in a Python virtual environment.
- The FederatedDataSet class wraps MNIST and shards it equally across collaborators in the tutorial example.In real-world deployments, each collaborator’s dataset already exists locally and would not require sharding.
- The FederatedModel class instantiates the federated-learning model, while fx commands initialize the workspace directory.
Experimental: Interactive Python API
The Interactive API is being developed to simplify multi-node FL experiments through a single Jupyter Notebook or Python-script entry point. Its components define federation settings, models, tasks, data loaders, and experiments that can launch a federation.
- The Interactive API lets users create and launch multi-node FL experiments from a single Jupyter Notebook or Python script.
- Federation setup uses Python objects to configure network parameters, model and optimizer, FL tasks, and data loaders.
- The Experiment API compiles the defined components, packages them into a distributable archive, and starts the aggregator.
Command Line Interface (fx CLI)
The fx CLI provides a stepwise workflow for creating, securing, distributing, and running an OpenFL federation. The process establishes aggregator and collaborator certificates before starting coordinated training.
- Workspace setup: The fx CLI workflow creates a federation workspace, installs requirements, and initializes the plan with the aggregator’s FQDN.
- Certificate setup: The aggregator acts as the federation’s certificate authority and generates signed certificates for itself and collaborators.The workflow creates a CA, generates certificate requests, signs them, and returns signed collaborator certificates.
- Workspace distribution: The aggregator exports the workspace as a zip archive, which collaborators import along with the required Python packages.
- Collaborator enrollment: Collaborators generate certificate requests, send request packages to the aggregator for signing, and import the signed certificates and chain.
- Federation execution: After the aggregator and collaborators start, the aggregator waits for all collaborators to connect before training begins.
- Federation execution: After the final training round, the aggregator stores the resulting model weights in the protobuf file specified by the YAML configuration.
3 Use Cases
The FeTS initiative applies federated learning to brain-tumor segmentation across international healthcare institutions without sharing patient data. Its challenge separates consensus-model training from evaluation on unseen, institutionally diverse datasets.
- 3.1 Federated Tumor Segmentation Initiative: FeTS partners with 56 clinical sites worldwide to improve brain-tumor sub-region segmentation without sharing patient data.The initiative combines pre-trained segmentation models and label-fusion approaches with federated learning for multi-institutional collaboration.
- 3.1 Federated Tumor Segmentation Initiative: International challenges often provide limited evidence about real-world clinical performance because their data come from controlled settings at few institutions.Collecting more institutional data centrally does not scale well because of privacy and ownership hurdles.
- 3.2 First Computational Competition on Federated Learning: The FeTS 2021 challenge is the first proposed for federated learning and addresses hurdles in creating and evaluating tumor-segmentation models.It uses clinically acquired, multi-institutional MRI scans from BraTS 2020 and remote independent institutions.
- 3.2 First Computational Competition on Federated Learning: Task 1 evaluates weight-aggregation methods for creating a consensus model, while Task 2 evaluates segmentation algorithms on unseen datasets from remote institutions.Task 1 can optionally account for network outages; Task 2 fixes the weight-aggregation method during testing.
4 Discussion
Federated learning is presented as a way to access larger and more diverse datasets while preserving data privacy. Its advantage over centralized learning comes from broader data access rather than an inherent algorithmic benefit.
- 4 Discussion: Federated learning enables AI training on large, diverse datasets when privacy requirements prevent direct data sharing.The approach is motivated by the difficulty of transmitting large or sensitive datasets to a centralized training server.
- 4 Discussion: Greater data access may improve accuracy and reduce bias, but federated learning offers no inherent guarantee that additional data are better.The discussion attributes FL's advantage over centralized learning to access to more data, not to an inherent algorithmic benefit.
- 4 Discussion: Federated learning can achieve similar accuracy to centralized learning while supporting privacy-preserving access to larger and more diverse datasets.The cited discussion also contrasts FL with similar collaborative learning techniques.
5 Conclusion and future work
OpenFL is a production-ready, open-source framework for training models on remote data-owner nodes across machine-learning libraries. The authors envision federations becoming permanent networks for continuously managing AI systems across industries.
- 5 Conclusion and future work: OpenFL lets developers port TensorFlow, PyTorch, or other machine-learning models into distributed training pipelines on remote data-owner nodes.The framework emphasizes security, open-source development, and interoperability with other federated-learning software efforts.
- 5 Conclusion and future work: The authors envision permanent federations that train, deploy, monitor, and update AI algorithms across healthcare, financial, industrial, and retail industries.This ambition extends federations beyond ad hoc collaborations for individual research efforts.