PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning for Medical Image Analysis: A Survey

Hao Guan, Pew-Thian Yap, Andrea Bozoki, Mingxia Liu

arXiv:2306.05980v4cs.CVeess.IV

TL;DR

Medical image analysis suffers from small sample sizes, while privacy restrictions limit pooling data across sites. This paper surveys federated learning methods and organizes them by system components, resources, experiments, challenges, and opportunities. Its experimental study empirically compares representative federated-learning methods for medical image analysis.

  • Problem

    Medical image analysis faces small sample sizes, while privacy restrictions make cross-site medical-image sharing and pooling difficult.

  • Method

    The paper surveys federated learning for medical image analysis, organizing methods into client-end, server-end, and client-server communication categories.

  • Results

    The paper empirically compares representative federated-learning methods for medical image analysis.

  • Takeaways & Limitations

    The survey summarizes current federated-learning methods, benchmark datasets, software platforms, challenges, and potential research opportunities for medical image analysis.

  • Takeaways & Limitations

    Federated-learning systems can require substantial computational resources, and large-scale federations spanning diverse sites worldwide remain scarce.

Abstract

from arXiv · show

Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. In this survey, we first introduce the background knowledge of federated learning for dealing with privacy protection and collaborative learning issues in medical imaging. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field.

1. Introduction

Medical image analysis is constrained by small sample sizes and privacy rules that make pooling data across sites difficult. This survey reviews federated learning methods, resources, experiments, and future directions through a system-oriented organization.

  • Medical image analysis faces a fundamental small-sample-size problem that hinders robust and reliable model building.
  • Pooling images across multiple sites could create larger training datasets, but privacy policies make direct cross-site sharing typically infeasible.
  • Federated learning collaboratively trains a global model through local training and model aggregation without exchanging data among sites.
  • The survey covers papers published from 1 January 2017 to 31 October 2023, including recent advances such as transformer-based methods.
  • It reviews federated-learning software platforms and benchmark medical-imaging datasets and empirically evaluates typical federated-learning methods.
  • The survey discusses newly arisen problems such as generalizability to unseen clients and federated learning for medical video analysis, alongside future directions.
  • Methods are organized into client-end, server-end, and client-server communication groups, with common questions used to explain their motivations.

2. Background

Medical image analysis is constrained by privacy rules and limited, site-specific datasets, motivating federated learning as a distributed alternative to centralizing raw images. FL trains a shared model through client-server updates while keeping data local.

  • Motivation: Privacy regulations constrain collecting, sharing, and processing medical images across sites.The passages cite laws including the CCPA and GDPR and note potential penalties for violations.
  • Medical imaging challenges: Single-site datasets often have few images because imaging and labeling are costly, limiting statistical significance and model performance.Site-specific sampling can also introduce demographic and class imbalance bias.
  • Motivation: FL aims to combine diverse site data in a distributed way while preserving privacy and improving the model’s generalizability beyond individual local models.The final global model is broadcast to clients for deployment, although hospital integration and user adoption remain practical considerations.
  • Problem formulation: Federated learning collaboratively trains a global model from N clients without exchanging their raw datasets.The server aggregates client information and broadcasts the learned model for deployment.
  • Typical process: A typical FL workflow selects clients, initializes a global model, performs local training, uploads model updates, aggregates them, and repeats broadcasting and iteration.Only model updates—not medical data itself—are shared during the upload step.

3.1. Methods Overview: A System Perspective

The survey organizes federated learning methods for medical image analysis from a system perspective. Existing approaches are grouped by client-end methods, server-end methods, and communication methods, with each group organized around its target research problems.

  • System perspective: Federated learning provides a privacy-preserving distributed-learning framework into which common medical imaging models can be integrated.The survey gives U-Net-based segmentation as an example of a model trained federatively across clients.
  • System perspective: The survey categorizes medical-image FL approaches into client-end methods, server-end methods, and communication methods.Methods within each group are clustered according to the specific research problems they address.

3.2. Client-End Learning

Client-end methods address heterogeneous medical sites, limited data and labels, and unequal client resources. The reviewed strategies personalize models, adapt or harmonize domains, augment scarce data, and accommodate client differences.

  • Domain Shift Among Clients: Client domain shift can cause difficult global-model convergence, performance degradation, and biased performance across sites.The shift may arise from scanning settings, subject populations, and differing intensity distributions.
  • Domain Shift Among Clients: Domain-specific learning locally fine-tunes shared models or combines shared and client-specific components to capture both invariant and site-specific features.Examples include shared encoders with client-specific decoders and shared CNN weights with locally adapted GNNs.
  • Domain Shift Among Clients: Domain adaptation and image harmonization reduce cross-client distribution differences by aligning features or translating images toward common styles.Reviewed approaches use discriminators, batch normalization, reference-site alignment, VAEs, cycleGANs, and frequency-domain normalization.
  • Limited Data and Labels: Limited images and labels can cause poorly trained client models and biased federation-wide results, making small-sample mitigation an important FL topic.The survey discusses contrastive learning and synthetic graph or image data as strategies for reducing the effects of scarce labeled data.
  • Limited Data and Labels: Unequal data volumes and computational resources, including GPU availability, create client disparities that can slow overall federation convergence.Clients may therefore perform different numbers of local updates between synchronization steps.

3.3. Sever-End Learning

Server-end methods modify aggregation and global optimization to address uneven client contributions, domain shift, label imbalance, and corrupted clients. Their goals include more consistent performance, convergence, and robustness across the federation.

  • Weight Aggregation: Server-side aggregation methods adapt client weighting or parameter selection to combine decentralized updates more effectively.Examples include low-frequency Fourier aggregation and loss-based weighting that assigns smaller weights to relatively poor-performing clients.
  • Client Shift: Server optimization methods address client shift by promoting convergence or uniform performance across heterogeneous medical centers.One fairness-motivated objective gives greater loss contribution to clients where the global model performs poorly.
  • Client Shift: FedSLD mitigates label-distribution differences by using federation-wide category counts to construct weighted cross-entropy losses during local training.The batch loss weights are computed from local-batch distributions relative to label distributions across the federation.
  • Client Corruption: Corrupted clients can damage the global model, motivating server-side outlier detection and suppression for noisy-label or malicious-client settings.Copula-based anomaly weighting gives high-outlier clients tiny aggregation weights, and experiments with noisy labels report effectiveness.

3.4. Client-Server Communication

Client-server communication in federated learning must address privacy leakage while also improving training efficiency. Reviewed approaches include partial weight sharing, differential privacy, attack analysis, and dynamic client selection.

  • Privacy and Data Leakage: Privacy protection is a central concern because leaked gradients can reconstruct pixel-level medical images.The survey motivates proactive defenses against data leakage during server-client interactions.
  • Privacy and Data Leakage: Partial weight sharing protects privacy by aggregating only selected model components while keeping later layers private.Clients share the feature-learning part for server aggregation and retain the last several layers locally.
  • Privacy and Data Leakage: Differential privacy adds Gaussian noise to client gradients to reduce the certainty of inferring an individual’s presence in training data.The approach is intended to protect privacy from the server and other clients.
  • Privacy and Data Leakage: Gradient attacks and inversion methods demonstrate that federated learning communications can expose training images or batch-normalization statistics.These studies reconstruct or synthesize images by matching gradients from real and synthesized data.
  • Communication Efficiency: Communication-efficiency methods target faster convergence and more effective training through client selection and aggregation policies.A dynamic fusion approach selects clients by local-model performance and aggregates according to participating clients’ training time, excluding late uploads.

4. Software Platforms and Tools

The survey reviews open-source platforms that provide APIs and frameworks for building federated learning systems in medical image analysis. These tools support privacy-preserving, decentralized, and multi-center experimentation across common machine-learning frameworks.

  • Platform Overview: Federated learning platforms provide APIs intended to improve the efficiency and robustness of building large FL systems.The reviewed tools target medical image analysis and related real-world applications.
  • PySyft: PySyft is an open-source library for secure and private machine learning that wraps popular deep-learning frameworks.It is implemented in Python and runs on Linux, MacOS, and Windows.
  • PySyft: PySyft-based federated learning achieved performance comparable to centralized-data training in an empirical medical-dataset study.The cited study evaluates models trained with federated learning against models trained on centralized data.
  • OpenFL: OpenFL is an open-source federated-learning framework initially developed for medical imaging through Intel and UPenn collaboration.It supports PyTorch and TensorFlow and underlies the Federated Tumor Segmentation platform.
  • PriMIA and Fed-BioMed: PriMIA and Fed-BioMed extend the platform ecosystem to privacy-preserving medical imaging and real-world multi-center applications.PriMIA supports decentralized deep learning and diverse imaging formats, while Fed-BioMed supports PyTorch, Scikit-Learn, and NumPy; cited studies report pediatric chest-X-ray and structural-brain-imaging applications.

5. Medical Image Datasets for Federated Learning

Medical-image federated-learning experiments commonly use multi-center databases or partition large datasets into client subsets. The survey catalogs benchmarks spanning brain, chest, cardiac, and skin-imaging applications, with substantial variation in modalities, institutions, and scanners.

  • Dataset Construction: Federated-learning experiments use either databases collected across medical sites or partitions of large datasets treated as client datasets.Multi-center databases are described as ideal for FL simulation, while partitioning provides another common experimental setup.
  • Brain Imaging: ADNI provides structural and functional brain MRI plus PET from more than 1,900 subjects and 59 centers for Alzheimer’s disease research.It is described as the largest and most influential Alzheimer’s disease benchmark.
  • Brain Imaging: UK Biobank and IXI provide multi-center brain and cardiac imaging resources with differing scales and acquisition settings.UK Biobank includes half a million participants and four imaging centers, while IXI contains around 600 MR images from three London hospitals.
  • Chest Imaging: Chest-imaging benchmarks include CheXpert, ChestX-ray14, COVID-19 CXR, and COVIDx, covering large radiograph collections and disease-specific labels.Their reported sizes range from 13,975 COVIDx images to 224,316 CheXpert radiographs, with labels for normal, pneumonia, COVID-19, and other thoracic conditions.
  • Cardiac Imaging: Cardiac benchmarks include ACDC and M&Ms, with M&Ms spanning 375 participants, six hospitals, and four scanner vendors.These datasets support cardiac MRI assessment and segmentation research.

5.5. Others

The survey’s remaining benchmarks cover retinal, prostate, pathology, reconstruction, and lightweight multi-modality studies. The listed research uses diverse datasets, tasks, modalities, and neural architectures.

  • Retinal Imaging: Retinal benchmarks support diabetic-retinopathy classification and fundus segmentation using labeled color retinal images.The Kaggle Retina dataset contains 17,563 image pairs with severity labels from 0 to 4.
  • Prostate Imaging: PROMISE12 provides 100 prostate MRIs acquired by different scanners from four independent medical centers.The dataset is intended to evaluate prostate MRI segmentation methods.
  • Pathology: TCGA supplies whole-slide images for histology and microscopy research involving normal controls and cancers.The cited studies use TCGA for cancer classification and image harmonization.
  • Image Reconstruction: fastMRI supports machine-learning-based medical image reconstruction with more than 1,500 knee MRIs and 10,000 clinical knee MRIs.The data include 1.5T and 3T acquisitions, and related studies also use BraTS and IXI.
  • Lightweight Benchmarks: MedMNIST is a lightweight classification dataset with 28 × 28 images across 10 pre-processed subsets and multiple modalities.Its modalities include MR, CT, X-ray, Ultrasound, and OCT, making it suitable for rapid prototyping.
  • Research Coverage: The surveyed applications span classification, segmentation, diagnosis, synthesis, harmonization, reconstruction, and age prediction across CNN, U-Net, GAN, transformer, GNN, and MLP models.Examples include HAM10000, ISIC, ABIDE, RSNA, TCGA, fastMRI, and several prostate and cardiac datasets.

6. Experiment

The experiment compares conventional and federated learning settings for Alzheimer’s disease versus normal-control classification using ADNI MRI data. Mix performs best, cross-site transfer performs worst, and federated methods achieve satisfactory performance without cross-site data sharing.

  • Experimental Setup: The study evaluates three conventional and three federated learning methods for AD-versus-normal-control classification on ADNI data.The data comprise ADNI-1 and ADNI-2 T1-weighted MRIs, represented by mean gray-matter volumes from 90 AAL-atlas regions.
  • Experimental Setup: The evaluation uses cross, single, mix, FedAvg, FedSGD, and FedProx settings with client-specific training and testing procedures.Cross transfers a model between client datasets, single trains separately within each client, mix pools training data, and federated methods aggregate weights or gradients.
  • Result and Analysis: Mix achieves the best performance because it combines all clients’ training data and exposes the model to the largest amount of data information.This setting requires sharing data across sites.
  • Result and Analysis: Cross achieves the worst performance, which the authors attribute to domain shift caused by different scanning parameters between ADNI-1 and ADNI-2.Directly transferring a model between the two datasets may therefore produce poor classification results.
  • Result and Analysis: Federated learning methods achieve satisfactory performance without cross-site data sharing by leveraging more data information than cross and single baselines.Among federated methods, model-weight aggregation methods include FedAvg and FedProx, while FedSGD aggregates client gradients.

7. Discussion

The discussion identifies data heterogeneity, privacy and security, deployment constraints, and limited real-world scale as major challenges for medical-imaging federated learning. It also outlines research opportunities in generalization, multimodal and video analysis, interpretability, decentralization, and sustainable deployment.

  • Data Heterogeneity: Data heterogeneity from scanners, protocols, and patient populations can cause domain shift, degraded performance, and biased results across federated clients.Alleviating this influence is identified as a major research problem.
  • Privacy and Security: Federated learning does not eliminate privacy and security risks, including privacy leakage and poisoning attacks at both server and client ends.Protecting individual privacy and data security remains challenging because many entities communicate and interact in an FL system.
  • Technological and Long-Term Limitations: Practical FL deployment faces substantial computational, communication, network-allocation, synchronization, scalability, sustainability, regulatory, and connectivity challenges.Long-term operation must also accommodate clients joining or leaving the federation.
  • Future Research Directions: Future work includes federated domain adaptation and personalized FL for client shift, multimodal learning, and generalization to unseen sites or classes.Classic domain adaptation may require access to source and target domains, which can conflict with FL privacy constraints.
  • Future Research Directions: Additional opportunities cover imperfect labels, stronger defenses, data auditing and unlearning, decentralized or blockchain-based FL, medical video analysis, large-scale benchmarks, and interpretability.The survey notes that existing work is often algorithmic or simulated, with few large-scale federations spanning multiple cities or countries.

8. Conclusion

This survey reviews recent federated learning advances for medical image analysis through a system-level categorization, benchmark datasets, and empirical comparison. It also discusses challenges and future research opportunities to clarify the field’s development and inspire further work.

  • The survey categorizes existing federated learning methods into client-end, server-end, and client-server communication methods.
  • A novel question-answer paradigm explains the motivation and mechanism of different federated learning methods in medical image analysis.
  • The paper introduces benchmark medical imaging datasets used for federated learning research.
  • An experiment empirically compares representative federated learning methods on the ADNI benchmark imaging database.
  • The survey discusses current challenges, research opportunities, and future directions for federated learning in medical image analysis.
  • The authors aim to provide researchers with a clear picture of federated learning’s development and inspire further research.
Loading 2306.05980v4…