PaperlayerView exact revision on arXiv

Source-linked AI summary

An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstruction

Nicola Pezzotti, Sahar Yousefi, Mohamed S. Elmahdy, Jeroen van Gemert, Christophe Schülke, Mariya Doneva, Tim Nielsen, Sergey Kastryulin, Boudewijn P. F. Lelieveldt, Matthias J. P. van Osch, Elwin de Weerdt, Marius Staring

arXiv:2004.07339v2eess.IVcs.CV

TL;DR

MRI acquisition is relatively slow, motivating methods to accelerate scans while reconstructing images from undersampled k-space. Adaptive-CS-Net unrolls an iterative reconstruction scheme with learned multiscale sparsification, neighboring-slice inputs, and MRI-domain priors. Adaptive-CS-Net was leading in the two clinically relevant multi-coil tracks and ranked 3rd on a simulated single-coil track in the 2019 fastMRI challenge.

  • Problem

    MRI acquisition is relatively slow, motivating methods to accelerate scans while reconstructing images from undersampled k-space.

    Source: §I · p. 1

  • Method

    Adaptive-CS-Net unrolls an iterative reconstruction scheme with learned multiscale sparsification, neighboring-slice inputs, and MRI-domain priors.

    Source: C. Adaptive-CS-Network · p. 4, §VI · p. 9

  • Results

    Adaptive-CS-Net was leading in the two clinically relevant multi-coil tracks and ranked 3rd on a simulated single-coil track in the 2019 fastMRI challenge.

    Source: §VII · p. 11

  • Takeaways & Limitations

    The authors conclude that the multiscale structure and many reconstruction blocks were the main drivers of performance.

    Source: §VI · p. 11

Abstract

from arXiv · show

Adaptive intelligence aims at empowering machine learning techniques with the additional use of domain knowledge. In this work, we present the application of adaptive intelligence to accelerate MR acquisition. Starting from undersampled k-space data, an iterative learning-based reconstruction scheme inspired by compressed sensing theory is used to reconstruct the images. We adopt deep neural networks to refine and correct prior reconstruction assumptions given the training data. The network was trained and tested on a knee MRI dataset from the 2019 fastMRI challenge organized by Facebook AI Research and NYU Langone Health. All submissions to the challenge were initially ranked based on similarity with a known groundtruth, after which the top 4 submissions were evaluated radiologically. Our method was evaluated by the fastMRI organizers on an independent challenge dataset. It ranked #1, shared #1, and #3 on respectively the 8x accelerated multi-coil, the 4x multi-coil, and the 4x single-coil track. This demonstrates the superior performance and wide applicability of the method.

I. INTRODUCTION

MRI scan times can be long, and further acceleration remains needed; compressed sensing and parallel imaging accelerate acquisition, but reconstruction from undersampled data remains central. Adaptive-CS-Network builds on ISTA with learned operations and MRI-specific priors, and performed strongly across fastMRI challenge tracks.

I. INTRODUCTION

  • MRI acquisition can take 15 minutes for knee imaging or an hour or more for cardiac imaging, motivating faster acquisition and reconstruction.Long scans challenge patients, can produce motion artifacts, and limit temporal resolution for dynamic imaging.Source: §I · p. 1
  • Compressed sensing reconstructs missing k-space data iteratively by exploiting image sparsity, while parallel imaging uses multiple receive coils as additional signal encoding.Both approaches accelerate acquisition through undersampling.Source: §I · p. 1
  • Prior deep-learning reconstruction spans data-driven networks and hybrid methods that integrate learning into existing iterative algorithms.Examples include learned filters in variational networks and learnable operators in Deep ADMM-Net.Source: §I · p. 2
  • Adaptive-CS-Network extends ISTA with learnable multiscale sparsification and MRI-specific priors, including data consistency, phase behavior, and background location.The design also uses neighboring slices through a 2.5D approach and a hierarchical training strategy.Source: §I · pp. 2–3
  • The method won one fastMRI challenge track and co-won a second, while domain-specific information was identified as crucial for high-quality MR reconstruction.The proposed network was evaluated across the challenge tracks.Source: §I · p. 3

II. FASTMRI CHALLENGE

The fastMRI challenge, organized by Facebook AI Research and NYU Langone Health, compares reconstruction across three acceleration tracks.

II. FASTMRI CHALLENGE

  • The challenge has 4x single-coil, 4x multi-coil, and 8x multi-coil tracks, with 17 teams in the single-coil track and eight in the multi-coil track.Its aim is to advance AI-based MR reconstruction and accelerate acquisition to reduce examination time.Source: §II · p. 3

A. Dataset

The fastMRI dataset comprises knee MRI volumes across training, validation, test, and challenge splits, with the challenge set reserved for independent organizer evaluation. The study describes the sampling setup and image-reconstruction metrics used for evaluation.

A. Dataset

  • The training set contains 973 volumes and the validation set 199; the test and challenge sets contain 118 and 104 multi-coil volumes, respectively.Single-coil test and challenge sets have 10 and 12 fewer volumes, respectively; organizers evaluated the final model on the independent challenge set.Source: A. Dataset · p. 3
  • The challenge uses randomly selected Cartesian k-space lines, with a fully sampled central region of 8% at 4x acceleration and 4% at 8x.The central region supports computation of coil sensitivity maps.Source: A. Dataset · p. 3

1) Normalized Mean Square Error (NMSE):

2) Peak Signal-to-Noise Ratio (PSNR):

3) Structural Similarity Index Metric (SSIM):

The paper evaluates image similarity and radiological quality, then introduces an ISTA-based reconstruction approach. ISTA iteratively refines a starting estimate, but nonlinear transforms, parameter tuning, and convergence can constrain its use.

3) Structural Similarity Index Metric (SSIM):

  • SSIM assesses image similarity through luminance, contrast, and structure, and the study computes it on magnitude images as grayscale scans.The distortion weights α, β, and γ are each set to 1.Source: 3) Structural Similarity Index Metric… · p. 3

C. Radiological Evaluation On The Challenge Dataset

  • The four highest-SSIM submissions were rated by seven radiologists on artifacts, sharpness, perceived contrast-to-noise ratio, and diagnostic confidence.Ratings used a 1-to-5 scale, with 1 best and 5 worst, and were averaged into a final ranking.Source: C. Radiological Evaluation On The… · p. 3

III. METHODS

  • The paper presents ISTA, then ISTA-Net, before introducing Adaptive-CS-Network as an ISTA-Net extension with domain-knowledge inductive biases.ISTA-Net is described as a deep-learning variant of ISTA.Source: §III · p. 3

A. ISTA Background

  • ISTA iteratively solves the reconstruction optimization from an initial estimate, often obtained by inverse Fourier reconstruction of zero-filled undersampled k-space.Its update corrects measured-data error by a step ρ, followed by a proximal-mapping step with regularization weight λ.Source: A. ISTA Background · pp. 3–4
  • For nonlinear sparsity transforms, the proximal step is not straightforward; ISTA also requires tuning λ and ρ and may converge slowly.The passage notes that many iterations may be needed to reach the optimum of the stated objective.Source: A. ISTA Background · p. 4

B. ISTA-Net

ISTA-Net replaces the handcrafted transform in an ISTA-inspired reconstruction update with learned operations. It unrolls the update into trainable blocks and combines reconstruction and transform-consistency losses.

B. ISTA-Net

  • ISTA-Net replaces the handcrafted transform Ψ with a learned operator S built from convolutions and a ReLU, and defines a similarly structured inverse Ŝ.The update rule substitutes S for Ψ and uses Ŝ, with Ŝ ◦ S = I.Source: B. ISTA-Net · p. 4
  • The model unrolls a fixed number of ISTA iterations into reconstruction blocks, sharing learnable parameters across blocks and training them end to end.This fixed-depth network can be trained by backpropagation.Source: B. ISTA-Net · p. 4
  • Its training loss combines ground-truth reconstruction error with a discrepancy term encouraging Ŝ ◦ S = I; σ controls the discrepancy weight.The paper also notes an ISTA-Net+ extension that adopts residual computations.Source: B. ISTA-Net · p. 4

C. Adaptive-CS-Network

Adaptive-CS-Network extends ISTA-Net with multiscale residual learning, neighboring-slice inputs, and soft MRI priors. These additions are introduced as three innovations ordered by their importance to final network performance.

C. Adaptive-CS-Network

  • The three innovations are multiscale and multi-slice computations, followed by soft MRI priors, ordered by importance to final network performance.The proposed model builds on Zhang and Ghanem’s ISTA-Net solution.Source: C. Adaptive-CS-Network · p. 4
  • A multiscale transform and its inverse provide a compressed-sensing-inspired inductive bias, while the network learns to sparsify and transform only the residual.Scale-specific feature channels are thresholded before the inverse transform maps them back to the image domain.Source: C. Adaptive-CS-Network · p. 4
  • Neighboring slices are concatenated as channels in a 2.5D approach, using inter-slice information at scales where it is beneficial while reducing memory use versus 3D convolutions.The saved GPU memory can be reinvested in more unrolled iterations; Fig. 1 depicts stacks of three consecutive knee MR images as network input and output.Source: C. Adaptive-CS-Network · pp. 4–5, Figure 1 · p. 5
  • Soft MRI priors derived from the current estimate and measured k-space are supplied as extra input channels to capture image properties that a limited receptive field may not learn easily.The network can incorporate these priors when they benefit optimization of the loss function.Source: C. Adaptive-CS-Network · p. 5

D. Final Design

The final design combines block-specific transforms with data-consistency, phase, and background priors. These inputs guide reconstruction while allowing the network to learn scale-specific processing.

D. Final Design

  • Each reconstruction block learns its own transforms, allowing blocks to focus on different image properties and increasing network capacity.The block update applies a soft-thresholded transform and its inverse using the estimate and the supplied priors.Source: D. Final Design · p. 5
  • The soft data-consistency prior accompanies the current estimate, allowing the network to assess acquired-data reliability and potentially compensate for coil-combination errors.The final design supplies the current estimate rather than the data-consistent image as the transform input.Source: D. Final Design · p. 5
  • The phase prior encodes the slowly varying phase of spin-echo acquisitions, whose removal is expected to yield a real-valued reconstructed image.Its low-pass reference comes from the fully sampled center of k-space and can be derived once beforehand.Source: D. Final Design · p. 5
  • The background prior penalizes estimated signal where the low-pass image indicates background; that reference is artifact-free but low-resolution because it uses fully measured central k-space.The spin-echo and background priors are computed only for the central slice to save GPU memory.Source: D. Final Design · pp. 5–6

E. Network Training and Implementation Details

The block-count experiments found improved performance with more blocks, and the final model used the largest count that fit GPU memory. Challenge training used acceleration- and population-specific fine-tuning.

E. Network Training and Implementation Details

  • Challenge training ran for 25 epochs with randomly selected 2x–10x accelerations, then fine-tuned eight models across coil type, acceleration, protocol, and scanner field strength.Each final model had 33M trainable parameters; inference took approximately 327 ms for single-coil and 518 ms for multi-coil data on an NVIDIA V100 GPU. Quantitative comparisons were limited to validation because test ground truth was unavailable.Source: E. Network Training and Implementation… · p. 6

IV. EXPERIMENTS AND RESULTS: MODEL OPTIMIZATION

  • Optimization experiments compared block count, loss function, neighboring slices, optimizer, and soft priors using a smaller 10-block model and validation measures.The study used repeated-measures one-way ANOVA on SSIM with significance level p = 0.05.Source: §IV · p. 6

A. Number of blocks

  • Increasing the number of blocks improved performance, so the final network used 25 blocks, the maximum that fit GPU memory; optimization studies used 10.The block-count experiments used 4x and 8x single-coil validation data; Fig. 2 presents the effect and indicates variance bars and significance stars.Source: Figure 2 · p. 6, A. Number of blocks · p. 6

B. Loss functions

The section compares reconstruction losses and selects a weighted combination of MSSIM and ℓ1 for final training.

B. Loss functions

  • The study compared MSE, perceptual loss, ℓ1, Huber, and MSSIM using a 10-block single-slice network; perceptual loss used pretrained VGG-16 features.The perceptual loss used VGG-16 layers relu1_2, relu2_2, and relu3_3.Source: B. Loss functions · p. 6
  • MSSIM extends SSIM across multiple image resolutions, using weights adopted from prior work.The multiscale formulation combines luminance, contrast, and structure factors.Source: B. Loss functions · pp. 6–7
  • MSSIM and ℓ1 were combined for their distinct structural, contrast, and luminance sensitivities; discrepancy loss was omitted over tuning difficulty and GPU-memory costs.The combination used α = 0.84.Source: B. Loss functions · p. 7
  • The weighted MSSIM–ℓ1 loss yielded the best results and was selected for final training, with improvements consistent across scans despite small SSIM gains.MSSIM alone was used as the loss function in subsequent optimization experiments.Source: B. Loss functions · p. 7

C. Multi-slice Network

The optimization studies selected a 2.5D scheme with center-slice-only loss and the RAdam optimizer. Adaptive-CS-Net significantly outperformed ISTA-Net+, though the first qualitative example favored different methods by anatomy and SSIM.

C. Multi-slice Network

  • The 2.5D scheme consistently improved over center-slice 2D, and defining loss only on the center slice was selected for the final model.The tested alternatives included neighboring slices as inputs and varied whether neighboring slices also contributed to the loss.Source: C. Multi-slice Network · p. 7

D. Optimizer

  • RAdam consistently outperformed the other tested optimizers and was selected for the final network.The comparison included RMSprop, RAdam, LookAhead, and Ranger.Source: D. Optimizer · p. 7

E. Adaptive-CS-Net vs ISTA-Net+

  • Adaptive-CS-Net significantly outperformed ISTA-Net+ in the single-coil 4× comparison, including against a parameter-matched ISTA-Net+ variant.The comparison used a 2D network with 10 blocks and 16 feature maps per layer.Source: E. Adaptive-CS-Net vs ISTA-Net+ · p. 7
  • In the first qualitative example, Adaptive-CS-Net better preserved anatomy, whereas ISTA-Net+ had higher SSIM, making radiological assessment complementary.The figure compares the methods on the single-coil 4× dataset.Source: E. Adaptive-CS-Net vs ISTA-Net+ · p. 7, Figure 4 · p. 8

F. Soft Priors

The section reports limited gains from adding soft priors, then describes the submitted Adaptive-CS-Net and discusses its design drivers and evaluation scope.

F. Soft Priors

  • Adding all priors improved SSIM but worsened NMSE and PSNR, with only minimal overall improvement.The authors retained all priors because challenge-phase selection was based on SSIM.Source: F. Soft Priors · p. 8

V. ADAPTIVE-CS-NET: SUBMITTED MODEL

  • The final model used 25 reconstruction blocks, with mixed block designs to fit GPU memory, and contained 33M parameters.Its chosen configuration used a linear combination of MSSIM and ℓ1 loss, two neighboring slices with loss on the central slice, and the RAdam optimizer.Source: §V · p. 9

VI. DISCUSSION

  • The multi-scale architecture and many reconstruction blocks were the main performance drivers, while training remained memory- and time-bound.Final training took approximately 7 days on two V100 GPUs.Source: §VI · p. 9
  • The proposed 2.5D scheme outperformed 2D and all-slice-loss 2.5D networks; weighting neighboring-slice losses reduced effective capacity per slice and degraded performance.The authors expect neighboring-slice information to matter more for MRI acquisitions with less asymmetric voxel sizes.Source: §VI · pp. 10–11
  • Soft priors added only limited performance; high-SNR cases showed large SSIM variance, and radiological scores measured visual quality rather than diagnostic interchangeability.The authors identify better quality metrics and diagnosis-based network design as areas for further research.Source: §VI · pp. 10–11

V. ADAPTIVE-CS-NET: SUBMITTED MODEL

  • Adaptive-CS-Net outperformed U-net, cascade net, KIKI-net, and PD-net on single-coil, and i-RIM on multi-coil but not single-coil.The radiological scores were 2.285 for multi-coil 4x, 1.286 for multi-coil 8x, and 2.714 for single-coil 4x; closer to 1 is better.Source: §V · p. 9, §VI · p. 11

VII. CONCLUSION

Adaptive-CS-Net was developed in the context of the 2019 fastMRI challenge and led in its two clinically relevant multi-coil tracks, while ranking third on the simulated single-coil track.

VII. CONCLUSION

  • Adaptive-CS-Net led in the two clinically relevant multi-coil tracks and ranked third on the simulated single-coil track.The paper describes the multi-coil tracks as clinically relevant and the single-coil track as simulated.Source: §VII · p. 11
  • The proposed adaptive intelligence algorithm is called Adaptive-CS-Net.Source: §VII · p. 11
  • Adaptive-CS-Net was developed in the context of the 2019 fastMRI challenge.Source: §VII · p. 11
Loading 2004.07339v2…