Source-linked AI summary
Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning
Pengfei Guo, Puyang Wang, Jinyuan Zhou, Shanshan Jiang, Vishal M. Patel
TL;DR
Deep learning-based MR reconstruction needs large, diverse datasets that are difficult to collect and share, while federated learning can preserve privacy during collaboration. This paper adds cross-site modeling to align latent feature distributions across institutions and reports improved generalization and reconstruction quality across four diverse datasets.
Problem
Large, diverse datasets for deep learning-based MR reconstruction are costly and difficult to share because of patient-privacy constraints, while federated models can still suffer from institutional domain shifts.
Method
The paper proposes a federated MR reconstruction framework with cross-site modeling that aligns latent-space distributions between source and target domains without directly sharing data.
Results
Experiments on four datasets with diverse characteristics demonstrate better generalization, improved reconstruction quality, and benefits from multi-institutional collaboration under federated learning.
Takeaways & Limitations
The framework supports privacy-preserving use of multi-institutional MR data for improved image reconstruction.
Abstract
from arXiv · showhide
Fast and accurate reconstruction of magnetic resonance (MR) images from under-sampled data is important in many clinical applications. In recent years, deep learning-based methods have been shown to produce superior performance on MR image reconstruction. However, these methods require large amounts of data which is difficult to collect and share due to the high cost of acquisition and medical data privacy regulations. In order to overcome this challenge, we propose a federated learning (FL) based solution in which we take advantage of the MR data available at different institutions while preserving patients' privacy. However, the generalizability of models trained with the FL setting can still be suboptimal due to domain shift, which results from the data collected at multiple institutions with different sensors, disease types, and acquisition protocols, etc. With the motivation of circumventing this challenge, we propose a cross-site modeling for MR image reconstruction in which the learned intermediate latent features among different source sites are aligned with the distribution of the latent features at the target site. Extensive experiments are conducted to provide various insights about FL for MR image reconstruction. Experimental results demonstrate that the proposed framework is a promising direction to utilize multi-institutional data without compromising patients' privacy for achieving improved MR image reconstruction. Our code will be available at https://github.com/guopengf/FLMRCM.
1. Introduction
Deep learning improves MR reconstruction but requires difficult-to-share data, motivating privacy-preserving federated collaboration. Because institutional domain shifts can limit generalization, the paper proposes cross-site modeling to align latent feature distributions and improve reconstruction.
- MRI reconstruction from under-sampled data is clinically important, while deep learning methods require large, diverse datasets that are costly to collect and difficult to share because of privacy constraints.The paper motivates collaborative training as an alternative to building a large centralized dataset.
- Federated learning enables institutions to collaboratively train a global model while retaining local patient data and communicating model parameters or gradients.The server periodically aggregates local updates and shares the resulting model with participating institutions.
- Domain shifts from different sensors, disease types, and acquisition protocols can produce dataset-specific latent-feature clusters and suboptimal cross-site generalization.The paper illustrates these differences using t-SNE visualizations of features from four datasets.
- The paper conducts extensive experiments and reports that cross-site modeling enables improved reconstructions while preserving privacy in multi-institutional collaboration.The conclusion states that experiments use four datasets with diverse characteristics.
- FL-MR with Cross-site Modeling aligns latent-space distributions between source and target domains to address domain shift in federated MR reconstruction.The method is intended to leverage data from multiple institutions without directly sharing the underlying images.
2. Related Work
Prior work frames MR reconstruction as an ill-posed inverse problem addressed with image- or k-space-domain deep learning, while federated learning enables collaborative model training without sharing local data. Existing federated approaches include parameter averaging and extensions targeting learning bias and domain shift.
- MR reconstruction from under-sampled k-space data is an ill-posed inverse problem addressed by sparsity-based compressed sensing and deep learning mappings in image or k-space domains.The proposed method follows the image-domain mapping approach.
- Federated learning lets multiple institutions collaboratively learn a shared model without sharing local training data through repeated local updates, server aggregation, and broadcasts.The process can continue through multiple exchanges until a global model is learned.
- FedAvg learns a global model by averaging local model parameters, while FedProx and Agnostic Federated Learning address learning bias for local entities.These methods are presented as extensions or common frameworks within federated learning.
- Prior medical-imaging federated studies include segmentation, adversarial alignment for domain shift, and privacy-preserving functional MRI classification.These examples establish related applications beyond the paper’s MR reconstruction setting.
3. Methodology
The methodology learns MR reconstruction from under-sampled to fully sampled images through federated local training and global parameter aggregation. FL-MRCM additionally aligns source and target latent feature distributions adversarially to address cross-site domain shift while preserving data locality.
- MR reconstruction: The reconstruction model learns a mapping from an under-sampled image x to a fully sampled image y, with Fourier operators and noise defining the acquisition relationship.The acceleration factor controls the ratio of fully sampled to accelerated k-space data, and the goal is to estimate y from x.
- Federated MRI reconstruction: Each institution trains a local model on paired under-sampled and fully sampled images from its private dataset.The local reconstruction loss is minimized iteratively during local epochs.
- Cross-site modeling: Because institution-specific sensors, disease types, and acquisition protocols bias local models, cross-site modeling aligns source and target latent feature distributions.The approach uses encoder outputs and an adversarial domain identifier for each source-target pair.
- Federated MRI reconstruction: FedAvg aggregates updated local model parameters at a central server and broadcasts the resulting global model without accessing institutional private data.After repeated communication rounds, the global model incorporates information from multiple domains.
- Cross-site modeling: The adversarial domain identifier learns to distinguish source from target latent features, while the encoders are trained to confuse it.The identifier receives latent features rather than direct patient data, supporting data-sharing constraints.
- Cross-site modeling: FL-MRCM combines reconstruction and adversarial losses, with λadv controlling the adversarial contribution during source-site training.The source-site training procedure is described in Algorithm 2; experiments use a U-Net-style encoder-decoder and specified optimization settings.
4. Experiments and Results
The experiments evaluate FL-MRCM across two scenarios, testing generalization to unseen institutions and the benefit of privacy-preserving multi-institution collaboration. Results show improved reconstruction quality, with cross-site modeling reducing errors under domain shift.
- Experimental scenarios: Experiments compare training and evaluation strategies across Scenario 1, which tests generalization to an unobserved dataset, and Scenario 2, which evaluates multi-institution collaboration.Scenario 1 uses one dataset as the target while training on other sites; Scenario 2 compares single-site, mixed-data, and federated strategies.
- Scenario 1: Generalizability: FL-MR outperforms other privacy-preserving alternatives in Scenario 1, while FL-MRCM further improves reconstruction quality across four datasets.The comparison uses SSIM and PSNR, with qualitative results reported for T1- and T2-weighted images.
- Scenario 1: Generalizability: FL-MRCM produces reconstructions with visual similarity to reference images across four datasets with diverse characteristics.Figure 5 reports qualitative comparisons and absolute image differences for T1- and T2-weighted images.
- Scenario 2: FL-based collaboration: FL-MRCM reaches the mixed-data upper bound in SSIM and PSNR and outperforms single-site and other methods in Scenario 2.The mixed-data model uses all available data directly, whereas FL-MRCM preserves the federated setting.
- Scenario 2: FL-based collaboration: On HPKS T2-weighted images, FL-MRCM improves SSIM from 0.9001 to 0.9232 and PSNR from 31.29 to 32.44.The method also suppresses errors around skull and lesion regions, consistent with the quantitative results.
- Ablation study: Cross-site modeling achieves smaller errors in ablations comparing models with and without the proposed modeling component.The ablation study evaluates T2-weighted images, with the target site providing test data.
5. Conclusion
The paper presents a privacy-preserving federated framework for multi-institutional MR reconstruction and adds cross-site modeling to address domain shift. Experiments across four diverse datasets show better generalization and benefits from multi-institutional collaboration.
- Conclusion: The proposed framework leverages multi-institutional data for MR image reconstruction without directly sharing patient data.Cross-site modeling aligns latent-space distributions between source and target domains within the federated framework.
- Conclusion: Experiments on four datasets with diverse characteristics demonstrate better generalization for the proposed method.The conclusion also reports benefits from multi-institutional collaboration under federated learning.