PaperlayerView exact revision on arXiv

Source-linked AI summary

3D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Network

Hongming Shan, Yi Zhang, Qingsong Yang, Uwe Kruger, Mannudeep K. Kalra, Ling Sun, Wenxiang Cong, Ge Wang

arXiv:1802.05656v2cs.CV

TL;DR

Low-dose CT reduces radiation exposure but can increase noise and artifacts that compromise image quality and diagnosis. This paper introduces 2D and 3D CPCE denoisers within a GAN framework, extending a trained 2D model to 3D and fine-tuning with adjacent-slice information. The 3D transfer-learning model converges faster and denoises favorably while suppressing noise without compromising image texture.

  • Problem

    Reducing CT radiation dose increases noise and artifacts, motivating denoising methods that preserve diagnostic image quality.

  • Method

    The paper combines 2D and 3D CPCE denoisers with a GAN framework, initializing the 3D model from a trained 2D network and fine-tuning it with adjacent-slice spatial information.

  • Results

    10 epochs versus 40 epochs achieved the same perceptual-loss denoising performance, saving 75% computational time for the 9-slice transfer-learning case.

  • Takeaways & Limitations

    The CPCE 2D and 3D networks suppress image noise without compromising image texture, while 3D transfer learning provides improved and noticeable denoising performance.

Abstract

from arXiv · show

Low-dose computed tomography (CT) has attracted a major attention in the medical imaging field, since CT-associated x-ray radiation carries health risks for patients. The reduction of CT radiation dose, however, compromises the signal-to-noise ratio, and may compromise the image quality and the diagnostic performance. Recently, deep-learning-based algorithms have achieved promising results in low-dose CT denoising, especially convolutional neural network (CNN) and generative adversarial network (GAN). This article introduces a Contracting Path-based Convolutional Encoder-decoder (CPCE) network in 2D and 3D configurations within the GAN framework for low-dose CT denoising. A novel feature of our approach is that an initial 3D CPCE denoising model can be directly obtained by extending a trained 2D CNN and then fine-tuned to incorporate 3D spatial information from adjacent slices. Based on the transfer learning from 2D to 3D, the 3D network converges faster and achieves a better denoising performance than that trained from scratch. By comparing the CPCE with recently published methods based on the simulated Mayo dataset and the real MGH dataset, we demonstrate that the 3D CPCE denoising model has a better performance, suppressing image noise and preserving subtle structures.

I. INTRODUCTION

Low-dose CT reduces radiation exposure but increases noise and artifacts that can impair diagnosis. The paper introduces 2D and 3D CPCE denoising networks, extending a trained 2D model to 3D through transfer learning.

  • Reducing CT radiation dose increases image noise and artifacts, potentially adversely affecting diagnosis.
  • CNN denoising methods for LDCT use varied architectures and objectives, including 2D and 3D CNNs, MSE, adversarial, and perceptual losses.
  • The paper introduces a 2D Conveying Path-based Convolutional Encoder-decoder network for LDCT denoising.
  • A 3D CPCE is obtained by directly extending the 2D model to use 3D spatial information and enhance denoising performance.
  • Transfer learning initializes the 3D CPCE from a trained 2D model, starting at the 2D model’s denoising performance and improving relative to an established benchmark.

A. Noise reduction for LDCT

LDCT noise reduction methods operate in the sinogram, reconstruction, or image post-processing domains. Deep-learning approaches differ in architecture and objective, while the proposed CPCE uses conveying paths to preserve high-resolution features.

  • LDCT noise reduction methods include sinogram filtration, iterative reconstruction, and image post-processing.
  • Sinogram filtration operates before reconstruction, whereas iterative reconstruction combines sinogram statistics with image-domain prior information.
  • Traditional post-processing methods can produce uneven improvements, over-smoothing, and loss of subtle structural details.
  • Deep-learning denoising performance depends on network architecture and objective function, which determine model complexity and learning behavior.
  • The CPCE uses conveying paths to reuse early feature maps, preserving high-resolution features while 1 × 1 convolutions reduce concatenated feature maps from 64 to 32.
  • The paper combines adversarial and perceptual losses in a CPCE formulation and extends it to the 3D context.

B. Wasserstein GAN framework

The Wasserstein GAN framework models LDCT denoising with generator–discriminator training and addresses shortcomings of the original GAN. The paper iteratively optimizes the generator and discriminator while combining Wasserstein distance with gradient penalty and perceptual loss.

  • In LDCT denoising, the GAN generator takes LDCT images as input, while the discriminator distinguishes generated samples from real data.
  • The original GAN can suffer from low-quality images, convergence problems, and mode collapse.
  • WGAN uses Wasserstein distance and constrains the discriminator to 1-Lipschitz functions, with gradient penalty replacing weight clipping.
  • The framework iteratively optimizes the generator and discriminator, using interpolated images and a parameter λ to balance Wasserstein distance against gradient penalty.
  • The proposed method applies this GAN framework to a CPCE network initialized for 3D training from a trained 2D denoising model.

A. Proposed denoising model

The CPCE denoising model extends a convolutional encoder-decoder from 2D to a hybrid 2D/3D configuration, using adjacent CT slices to incorporate spatial information while preserving structural details.

  • Model formulation: LDCT denoising estimates the corresponding NDCT image by learning an approximate inverse of the noise-corrupting process.The model maps a low-dose image through G to approximate the normal-dose image.
  • CPCE-2D architecture: The CPCE architecture uses convolutional and deconvolutional layers with a conveying path that reuses early feature maps in later layers.The network has four convolutional and four deconvolutional layers, with 3 × 3 filters and a one-filter final layer.
  • 3D spatial information: Adjacent CT slices provide correlated 3D spatial information that can improve denoising and help recover structural details.The approach reflects how radiologists inspect neighboring slices in a CT volume.
  • 3D spatial information: The 3D extension augments one slice with adjacent slices and replaces 2D 3 × 3 filters with 3D 3 × 3 × 3 filters.The example uses three input LDCT slices, with the resulting feature-map depth reduced to 1.
  • 3D spatial information: For larger slice stacks, the conveying path copies middle-depth features so tensors with different depth dimensions can be combined while preserving structural details.The hybrid model is called CPCE-3D, distinguishing it from the 2D CPCE model.

C. Transfer learning strategy

The transfer-learning strategy initializes a 3D CPCE model by extending a trained 2D filter with zero padding, then fine-tunes it to use 3D spatial context.

  • Motivation: Training CPCE-3D from scratch is computationally expensive, motivating initialization from the easier-to-train CPCE-2D model.The strategy uses an available trained 2D denoising model instead of random initialization.
  • Initialization: A trained 2D convolutional filter is extended to a 3D filter by placing it at the middle depth and padding the other depth slices with zeros.The 3D filter has shape cin × cout × 3 × 3 × 3, while the trained 2D filter has shape cin × cout × 3 × 3.
  • Initialization: The zero-padded 3D filter is initially identical in operation to the trained 2D filter, so the initialized 3D model starts with the same denoising performance.This preserves the learned 2D behavior before fine-tuning.
  • Fine-tuning: Fine-tuning uses 3D spatial context and is described as enabling accelerated training and an improved chance of better denoising than random initialization.The paper characterizes the hybrid architecture as a trade-off between 2D and pure 3D networks.
  • Comparison: Unlike the pure 3D network in, CPCE-3D is hybrid, produces a 2D output, uses fewer parameters, and retains 3D spatial information.The comparison is presented as an architectural distinction and trade-off.

D. Objective function

The denoising network is optimized in a WGAN framework using adversarial and perceptual losses, combining distributional realism with high-level feature similarity to NDCT images.

  • Adversarial objective: The proposed denoising network serves as the GAN generator and is optimized within a WGAN framework.The discriminator is a convolutional network followed by fully connected layers.
  • Adversarial objective: Adversarial loss encourages generated denoised images to be indistinguishable from NDCT images.The loss is the generator objective associated with the adversarial component.
  • Perceptual objective: Perceptual loss measures the distance between denoised and NDCT images in a high-level feature space rather than pixel space.The feature representation uses a pre-trained VGG-19 network.
  • Combined objective: The final generator objective combines adversarial loss and perceptual loss as L = La + λpLp.The parameter λp controls the contribution of the perceptual term, with λp = 0 excluding it and λp = ∞ leaving only perceptual optimization in the reported validation analysis.

IV. EXPERIMENTS

The experiments evaluate CPCE denoising on simulated Mayo data, using separate patients for training and validation and adjacent slices for 3D models.

  • IV. EXPERIMENTS: The study evaluates CPCE denoising on simulated Mayo low-dose CT data and distinguishes CPCE-3D models by their number of input slices.CPCE-3D(i) denotes a model using i input slices, while transfer-learning models carry a superscript +.
  • IV. EXPERIMENTS: The dataset contains normal-dose abdominal CT images from 10 anonymous patients and corresponding simulated quarter-dose CT images.The slice thickness is 1.0 mm, and the reconstruction interval is 0.8 mm.
  • IV. EXPERIMENTS: Training uses 128K randomly selected 64 × 64 patches from five patients, while validation uses 64K patches from the remaining five.For CPCE-3D, adjacent low-dose patches are retained during training and testing.

1) Experimental setting:

The study selects the perceptual-loss weight, compares transfer learning with scratch training, and assesses convergence and clinical visualization on the Mayo dataset.

  • Experimental setting:: λp = 0.1 achieved the lowest validation MSE and was used in subsequent experiments.The perceptual loss had a more dominant influence on denoising performance than the adversarial loss.
  • Experimental setting:: Transfer learning initialized CPCE-3D from a trained CPCE-2D model, producing lower perceptual loss, MSE, and Wasserstein distance than scratch training.Fine-tuning then incorporated 3D spatial information from adjacent slices.
  • Experimental setting:: The CPCE-3D model required 17 minutes and 32 minutes per half-epoch iteration for three and nine input slices, respectively, compared with 14 minutes for CPCE-2D.The time saved by transfer learning depends on the number of adjacent slices.
  • Experimental setting:: With nine input slices, transfer learning reached the scratch model’s perceptual-loss performance in approximately 10 epochs instead of 40.This represented 75% computational-time savings excluding 2D training, or about 65% including it.

4) Denoising performance:

On Mayo images, CPCE-3D combines adjacent-slice information with the CPCE architecture to suppress noise while preserving texture and subtle anatomical structures.

  • 4) Denoising performance:: CPCE-3D best preserved liver blood vessels while suppressing noise and maintaining subtle structures in the clinical visual comparisons.The low-attenuation lesions were visible with all networks, but vessels were clearest with CPCE-3D against the NDCT benchmark.
  • 4) Denoising performance:: The evaluation uses PSNR, SSIM, perceptual loss, and texture matching loss on full-size testing slices.Texture matching loss measures texture similarity in a high-level feature space.
  • 4) Denoising performance:: RED-CNN achieved the highest PSNR, but its MSE-based optimization produced over-smoothed images and loss of texture information.High PSNR did not guarantee the best perceptual or texture similarity with NDCT images.
  • 4) Denoising performance:: CPCE-2D outperformed WGAN-VGG across PSNR, SSIM, perceptual loss, and texture matching loss on the Mayo dataset.These metrics evaluate complementary aspects of denoising quality.
  • 4) Denoising performance:: Adding adjacent-slice information improved CPCE-3D denoising relative to its 2D counterpart, while transfer learning consistently improved CPCE-3D over scratch training.Increasing the number of input slices further improved CPCE-3D performance.

B. Low-dose dataset with real noise

The method is also evaluated on real noisy MGH cadaver scans, where large slice thickness limits the observed 2D-to-3D improvement in pixel-space performance.

  • B. Low-dose dataset with real noise: The real MGH dataset contains 40 cadaver scans acquired at four dose levels: 10NI, 20NI, 30NI, and 40NI.Noise Index is referenced to the standard deviation of CT numbers within a water-phantom region of interest.
  • B. Low-dose dataset with real noise: The experiment uses eight resolution-matched cadavers, with four for training and four for validation.All selected scans have 2.5 mm slice thickness and reconstruction interval.
  • B. Low-dose dataset with real noise: Because the slice thickness was large, the MGH experiment used CPCE-3D models with three and five input slices.The initial 3D model was transferred from the CPCE-2D model at epoch 20.
  • B. Low-dose dataset with real noise: The real-data results aligned with the simulated Mayo findings, although the validation curves were less smooth because low-dose and normal-dose images were not perfectly registered.The large slice thickness also made 2D-to-3D pixel-space improvement less significant than on Mayo data.

C. Blind reader study for Mayo and MGH datasets

A blind reader study used randomly selected test-patient slices to compare denoising methods across Mayo and MGH datasets. Radiologists evaluated noise removal, artifact reduction, structural preservation, and overall quality, with objective functions showing different strengths.

  • Ten randomly selected image-slice groups per dataset were independently rated by two blinded radiologists using LDCT and NDCT references.Each group contained LDCT, NDCT, and denoised LDCT images.
  • MSE-based methods achieved the best noise removal, whereas WGAN methods with perceptual loss scored better for artifact reduction, structural preservation, and overall quality.
  • CPCE-2D received a slightly better score than WGAN-VGG in the reader-study comparison.

V. DISCUSSION AND CONCLUSION

The discussion presents CPCE as a conveying-path encoder-decoder combined with Wasserstein GAN and perceptual loss, and extends trained 2D models to 3D. The reported comparisons favor CPCE for denoising while preserving texture, although broader clinical validation remains future work.

  • Discussion and conclusion: CPCE-2D reuses early-layer feature maps through three conveying paths and combines this architecture with Wasserstein GAN training and perceptual loss.
  • Discussion and conclusion: The 3D CPCE network is initialized from a trained 2D model, yielding more stable, efficient training and better denoising than direct 3D training from scratch.
  • Discussion and conclusion: CPCE-2D and CPCE-3D performed favorably against RED-CNN and WGAN-VGG for suppressing image noise without compromising image texture.
  • Discussion and conclusion: Future work includes generalizing the transfer-learning scheme and conducting more experiments toward clinical applications.
  • Discussion and conclusion: The study was the authors' first attempt to transfer a trained 2D CNN to a 3D counterpart for low-dose CT denoising.
Loading 1802.05656v2…