PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning of Generative Image Priors for MRI Reconstruction

Gokberk Elmas, Salman UH Dar, Yilmaz Korkmaz, Emir Ceyani, Burak Susam, Muzaffer Özbey, Salman Avestimehr, Tolga Çukur

arXiv:2202.04175v2eess.IVcs.CVcs.LG

TL;DR

Multi-site MRI reconstruction must address privacy concerns and heterogeneity in images and imaging operators, while conditional models generalize poorly to operator differences. FedGIMP federates a generative image prior and adapts it with each subject’s imaging operator. Across imaging scenarios, it outperforms federated conditional methods, with reported gains under mismatched operators.

  • Problem

    Federated training avoids sharing local data, but heterogeneous image distributions and imaging operators can impair model performance, and conditional models generalize poorly to imaging heterogeneity.

    Source: §I · p. 1, A. Federated Learning of Conditional… · p. 3

  • Method

    FedGIMP learns a shared unconditional generative MRI prior with site-specific latent variables, then combines and adapts it with each test subject’s imaging operator.

    Source: §I · pp. 1–2, Contributions · p. 2

  • Results

    Across imaging scenarios, FedGIMP outperformed federated conditional methods; under mismatched R, it improved PSNR by 4.75 dB and SSIM by 4.00% over the second-best FL method.

    Source: Domain shifts in the imaging operator · p. 7, §VI · p. 8

  • Takeaways & Limitations

    The reported multi-site results show improved generalization against domain shifts and make FedGIMP a promising candidate for multi-site accelerated MRI collaborations.

    Source: §VII · p. 9

Abstract

from arXiv · show

Multi-institutional efforts can facilitate training of deep MRI reconstruction models, albeit privacy risks arise during cross-site sharing of imaging data. Federated learning (FL) has recently been introduced to address privacy concerns by enabling distributed training without transfer of imaging data. Existing FL methods for MRI reconstruction employ conditional models to map from undersampled to fully-sampled acquisitions via explicit knowledge of the imaging operator. Since conditional models generalize poorly across different acceleration rates or sampling densities, imaging operators must be fixed between training and testing, and they are typically matched across sites. To improve generalization and flexibility in multi-institutional collaborations, here we introduce a novel method for MRI reconstruction based on Federated learning of Generative IMage Priors (FedGIMP). FedGIMP leverages a two-stage approach: cross-site learning of a generative MRI prior, and subject-specific injection of the imaging operator. The global MRI prior is learned via an unconditional adversarial model that synthesizes high-quality MR images based on latent variables. Specificity in the prior is preserved via a mapper subnetwork that produces site-specific latents. During inference, the prior is combined with subject-specific imaging operators to enable reconstruction, and further adapted to individual test samples by minimizing data-consistency loss. Comprehensive experiments on multi-institutional datasets clearly demonstrate enhanced generalization performance of FedGIMP against site-specific and federated methods based on conditional models, as well as traditional reconstruction methods.

I. INTRODUCTION

Deep MRI reconstruction relies on learned priors and large paired datasets, while privacy concerns and domain shifts complicate multi-site training. FedGIMP separates federated prior learning from subject-specific operator injection and adapts the prior at inference.

  • Privacy concerns and the cost of compiling diverse datasets can limit centralized training of deep MRI reconstruction models.Deep models learn data-driven priors, and successful training procedures use large input-output collections.Source: §I · p. 1
  • Conditional reconstruction methods can generalize poorly across imaging operators, requiring matched acceleration rates and sampling densities across training, testing, and sites.These requirements can constrain multi-institutional flexibility and necessitate retraining after substantial operator changes.Source: §I · p. 1
  • FedGIMP learns a global MRI prior across sites, then injects the subject-specific imaging operator in a two-stage reconstruction approach.An unconditional adversarial model synthesizes images from low-dimensional latents produced by a mapper.Source: §I · p. 1, Figure 1 · p. 2
  • At inference, the global prior is combined with subject-specific operators and adapted by minimizing data-consistency loss on acquired k-space samples.Source: §I · p. 2
  • The mapper produces site-specific latents while the generator is shared across sites, preserving site-specific and site-general image representations.Source: Contributions · p. 2, Figure 1 · p. 2

II. RELATED WORK

Prior federated MRI reconstruction work addressed site variability but remained dependent on conditional models and explicit imaging operators. FedGIMP instead federates a generative prior decoupled from the operator, including for multi-coil reconstruction.

  • Related FL approaches for other imaging tasks include data harmonization, episodic learning, adversarial alignment, and split networks, but MRI reconstruction must form images from raw data.Harmonization can discard patient-level information of diagnostic value.Source: §II · p. 2
  • Earlier FL MRI reconstruction methods used adversarial alignment or global encoders with local decoders, but both relied on conditional models informed by the imaging operator.Such methods prescribe matched acceleration rates and sampling densities across training and testing, often across sites.Source: §II · pp. 2–3
  • Conditional reconstruction models are vulnerable to imaging-operator shifts, potentially limiting performance and requiring retraining when operators change substantially.Source: §II · p. 2
  • Prior centralized work separated image priors from imaging operators, but this separation had not been studied in federated learning.FedGIMP is presented as the first FL method to decouple the MRI prior from the operator and the first FL method for multi-coil MRI data.Source: §II · p. 3

A. Federated Learning of Conditional MRI Models

Federated conditional MRI models train a shared reconstruction model through communication between sites and a server. Their dependence on imaging operators limits generalization across heterogeneous imaging settings, so operators are typically matched across training, testing, and sites.

  • Each site initializes its local model from the server’s global model and updates it using a local reconstruction loss.Local data include undersampled acquisitions, fully sampled reference images, and the site's imaging operator and adjoint.Source: A. Federated Learning of Conditional… · p. 3
  • At each communication round’s end, updated local models are aggregated by federated averaging, with site weights typically proportional to each site’s share of training samples.Source: A. Federated Learning of Conditional… · p. 3
  • The conditional model receives a zero-filled Fourier reconstruction of the undersampled acquisition, while the imaging operator incorporates coil sensitivities and partial Fourier transformation.Source: A. Federated Learning of Conditional… · p. 3
  • Conditional models generalize poorly to imaging heterogeneity, and imaging operators are typically matched across training-test sets and sites.Source: A. Federated Learning of Conditional… · p. 3
  • For reconstruction, FedGIMP combines its trained generator with the test subject’s imaging operator and adapts the generator, mapper, and noise to minimize data-consistency loss on acquired k-space samples.The loss compares acquired samples with synthesized samples after coil projection and undersampling.Source: Figure 2 · p. 3

B. Federated Learning of Generative MRI Priors

FedGIMP learns a global generative MRI prior across sites, then combines it with each subject’s imaging operator and adapts it to the reconstruction task. Its unconditional generator uses site-specific latent variables to preserve prior specificity.

B. Federated Learning of Generative MRI Priors

  • FedGIMP separates cross-site prior learning from subject-specific operator injection, adapting the prior to each test reconstruction through data consistency.The training losses do not use imaging operators, so the trained generator synthesizes MR images but cannot directly map undersampled data to an image.Source: B. Federated Learning of Generative… · p. 3, §B.3 · pp. 4–5

B.1 Unconditional Adversarial Model:

  • The mapper transforms random variables augmented with a site-index encoding into latent variables, which the synthesizer converts into realistic MR images.The synthesizer progressively upsamples feature maps and applies convolution, noise injection, and adaptive instance normalization; the discriminator distinguishes actual from synthetic images.Source: §B.1 · pp. 3–4

B.2 Training of the Global MRI Prior:

  • Sites update local generators and discriminators using adversarial losses, then send generator parameters for server aggregation into the global prior.Local discriminators are not exchanged, limiting communication load and augmenting data privacy.Source: §B.2 · p. 4

B.3 Inference at a Test Site:

  • At inference, the learned prior is combined with the test subject’s imaging operator and adapted by minimizing data-consistency loss on acquired k-space samples.The per-cross-section optimization adjusts synthesizer and mapper parameters and noise; a gradient penalty is included to prevent noise amplification.Source: §B.3 · p. 5

A. Architectural Details

  • The generator uses an eight-layer mapper and synthesizer, with a discriminator that also has eight layers; images are padded for training when needed and centrally cropped at inference.For complex MR images, generator output and discriminator input use separate real and imaginary channels.Source: A. Architectural Details · p. 5

B. Competing Methods

The comparison includes traditional, centralized non-federated, and federated conditional reconstruction methods alongside FedGIMP. Training settings were selected for near-optimal performance across tasks, and strict data consistency was enforced before reporting.

B. Competing Methods

  • The comparison spans LORAKS, centralized conditional GANcond and unconditional GIMP, and federated conditional methods FL-MRCM, FedGAN, LG-Fed, and FedMRI.GANcond and GIMP serve as privacy-violating benchmarks for conditional and unconditional reconstruction, respectively.Source: B. Competing Methods · pp. 5–6
  • Hyperparameters were selected using validation performance, with one setting per method chosen for near-optimal performance across tasks.Federated models trained for 100 communication rounds with one local epoch per round; non-federated models trained for 100 epochs.Source: B. Competing Methods · p. 5
  • FedGIMP inference used Adam optimization for 1200 iterations, while all reconstructions underwent strict data consistency before reporting.LORAKS used different matrix ranks for single-coil and multi-coil data.Source: B. Competing Methods · p. 5
  • Federated conditional baselines vary their site-sharing designs: FL-MRCM aligns latent representations, FedGAN shares an encoder and decoder, LG-Fed uses site-specific encoders, and FedMRI uses site-specific decoders.FedMRI also includes a contrastive loss.Source: B. Competing Methods · pp. 5–6

C.1 Datasets:

Experiments use in-house and public multi-institutional MRI datasets, with single-coil and multi-coil reconstruction settings. Reconstruction quality is assessed using PSNR and SSIM on test subjects.

C.1 Datasets:

  • Experiments used an in-house Bilkent dataset and IXI, fastMRI, and BRATS, retrospectively undersampled with variable- or uniform-density patterns at 3x or 6x acceleration.There was no subject overlap among training, validation, and test sets.Source: §C.1 · p. 6

C.2 Single-Coil Reconstruction:

  • Single-coil experiments used 40, 10, and 5 subjects per dataset for training, validation, and testing, with 2520 cross-sections used for training.The acquisitions covered multiple contrasts, including T1-, T2-, and PD-weighted images in IXI and BRATS and T1c-, T2-, and FLAIR-weighted images in fastMRI.Source: §C.2 · p. 6

C.3 Multi-Coil Reconstruction:

  • Multi-coil experiments used fastMRI brain, fastMRI knee, and in-house brain data, with 2592 training cross-sections and coil compression to five virtual coils.Conditional models mapped undersampled multi-coil complex zero-filled reconstructions to references; GIMP and FedGIMP synthesized complex coil-combined images, with the operator injected only at inference.Source: §C.3 · p. 6

D. Quantitative Assessments

  • Reconstruction quality was measured using PSNR and SSIM against Fourier reconstructions of fully sampled acquisitions, with average metrics across test subjects and Wilcoxon sign-rank tests.Figure 3 shows representative 3x uniform-density reconstructions for IXI, fastMRI, and BRATS alongside reference images and seven methods.Source: D. Quantitative Assessments · pp. 6–7, Figure 3 · p. 7

A. Single-Coil Reconstruction

FedGIMP was evaluated against intrinsic image-distribution shifts and mismatches in acceleration rate, sampling density, and site-specific imaging operators. It outperformed competing methods in these settings, with reported gains over the second-best federated method.

A. Single-Coil Reconstruction

  • Across sites and acceleration rates, FedGIMP improved average PSNR by 3.66 dB and SSIM by 3.00% over the second-best FL method.It outperformed competing methods except on BRATS at R=6x, where the privacy-violating GIMP benchmark performed similarly.Source: Domain shifts in the MR image… · p. 7
  • 4.75 dB PSNR and 4.00% SSIM under mismatched acceleration rates, and 4.50 dB PSNR and 4.76% SSIM under mismatched sampling density, over the second-best FL method.FedGIMP was the top performer among the methods, except that GIMP performed similarly.Source: Domain shifts in the imaging operator · p. 7
  • Compared with the second-best FL method, FedGIMP improves PSNR by 1.31 dB and SSIM by 2.70% under matched operators, and by 5.08 dB and 2.98% under mismatched operators.Source: Domain shifts in the imaging operator · p. 7

B. Multi-Coil Reconstruction

In multi-coil reconstruction, FedGIMP outperformed competing methods under matched and mismatched imaging operators, with GIMP performing similarly. Ablations also showed that its trained prior, inference adaptation, and site-specific mapper each matter to performance.

B. Multi-Coil Reconstruction

  • Under mismatched imaging operators with different acceleration rates across training and test sets, FedGIMP improved PSNR by 6.93 dB and SSIM by 4.45% over the second-best FL method, except GIMP performed similarly.Representative R=6x brain and knee reconstructions are shown; conditional models were trained at R=3x.Source: B. Multi-Coil Reconstruction · p. 8, Figure 4 · p. 8
  • 6.26 dB PSNR and 4.21% SSIM improvement over the second-best FL method in multi-coil tests with matched imaging operators.The experiments included brain and knee datasets; GIMP performed similarly to FedGIMP.Source: B. Multi-Coil Reconstruction · pp. 7–8, Table IV · p. 8

C. Ablation Study

  • FedGIMP consistently outperformed variants using a randomly initialized generator, a static mapper, or a site-general mapper across sites.The ablations indicate contributions from the high-quality image prior, subject-specific latent-distribution adaptation, and site-specific latent distributions.Source: C. Ablation Study · p. 8

VI. DISCUSSION

FedGIMP is presented as more robust to imaging-operator heterogeneity, with a simpler training procedure, but iterative inference adds computational burden and privacy resilience warrants further study.

VI. DISCUSSION; VII. CONCLUSION

  • Across varying acceleration rates, sampling densities, and train-test operators, FedGIMP outperformed federated conditional models; the conclusion also reports benefits over traditional methods.Source: §VI · p. 8, §VII · p. 9

VI. DISCUSSION

  • One MRI prior can reconstruct with different imaging operators, whereas conditional models are retrained for each acceleration-rate and sampling-density configuration.FedGIMP also avoids the additional cross-site alignment or weighted-contrastive losses described for conditional methods.Source: §VI · p. 8
  • FedGIMP uses iterative inference, unlike conditional methods’ single forward pass; transferring optimized weights across neighboring cross-sections can accelerate the procedure.Source: §VI · p. 9
  • Privacy risks remain: inference attacks can target training-data leakage, and the authors call for further study of FedGIMP’s privacy-preserving abilities.They note differential-privacy procedures as a possible way to improve resilience against inference attacks.Source: §VI · p. 9

VI. DISCUSSION; VII. CONCLUSION

  • The federated prior may extend to MRI super-resolution or synthesis, and to CT, PET, or ultrasound if the imaging operator is changed.For inference adaptation in other tasks, the corresponding physical signal model would replace the MRI imaging operator.Source: §VI · p. 9, §VII · p. 9

Related papers

Loading 2202.04175v2…