PaperlayerView exact revision on arXiv

Source-linked AI summary

DU-GAN: Generative Adversarial Networks with Dual-Domain U-Net Based Discriminators for Low-Dose CT Denoising

Zhizhong Huang, Junping Zhang, Yi Zhang, Hongming Shan

arXiv:2108.10772v2eess.IVcs.CVphysics.med-ph

TL;DR

LDCT dose reduction introduces noise and artifacts that can compromise diagnosis, while conventional denoising discriminators may represent either global structure or local details. DU-GAN uses U-Net-based discriminators in image and gradient domains, with CutMix-based confidence maps, and reports effectiveness on simulated and real-world datasets. Clinical reader validation and the method’s added training cost remain limitations.

  • Problem

    LDCT dose reduction introduces noise and artifacts that compromise image quality and diagnosis, while conventional discriminators often focus on either global structure or local details.

  • Method

    DU-GAN uses U-Net-based discriminators in image and gradient domains to provide global and per-pixel feedback, enhance edges, reduce streak artifacts, and generate uncertainty maps through CutMix.

  • Results

    Experiments on two simulated datasets and one real-world dataset demonstrated DU-GAN’s effectiveness through qualitative and quantitative comparisons.

  • Takeaways & Limitations

    DU-GAN achieves better denoising performance than other GAN-based models and has potential for clinical use with uncertainty visualization.

  • Takeaways & Limitations

    Clinical reader studies are needed to validate diagnostic utility, and DU-GAN introduces slightly more computational cost during training.

Abstract

from arXiv · show

LDCT has drawn major attention in the medical imaging field due to the potential health risks of CT-associated X-ray radiation to patients. Reducing the radiation dose, however, decreases the quality of the reconstructed images, which consequently compromises the diagnostic performance. Various deep learning techniques have been introduced to improve the image quality of LDCT images through denoising. GANs-based denoising methods usually leverage an additional classification network, i.e. discriminator, to learn the most discriminate difference between the denoised and normal-dose images and, hence, regularize the denoising model accordingly; it often focuses either on the global structure or local details. To better regularize the LDCT denoising model, this paper proposes a novel method, termed DU-GAN, which leverages U-Net based discriminators in the GANs framework to learn both global and local difference between the denoised and normal-dose images in both image and gradient domains. The merit of such a U-Net based discriminator is that it can not only provide the per-pixel feedback to the denoising network through the outputs of the U-Net but also focus on the global structure in a semantic level through the middle layer of the U-Net. In addition to the adversarial training in the image domain, we also apply another U-Net based discriminator in the image gradient domain to alleviate the artifacts caused by photon starvation and enhance the edge of the denoised CT images. Furthermore, the CutMix technique enables the per-pixel outputs of the U-Net based discriminator to provide radiologists with a confidence map to visualize the uncertainty of the denoised results, facilitating the LDCT-based screening and diagnosis. Extensive experiments on the simulated and real-world datasets demonstrate superior performance over recently published methods both qualitatively and quantitatively.

I. INTRODUCTION

LDCT dose reduction introduces noise and artifacts that can compromise diagnosis, while existing denoising objectives and discriminators have limitations. DU-GAN addresses these issues with dual-domain U-Net discriminators and CutMix-based uncertainty visualization.

  • Reducing CT radiation dose inevitably introduces noise and artifacts that can compromise diagnosis and LDCT-based lung nodule classification.
  • MSE can achieve high PSNR but often produces over-smoothed images that correlate poorly with human-perceived image quality.
  • Adversarial loss dynamically measures similarity to normal-dose images, but classification discriminators typically emphasize either global structure or local regions.
  • DU-GAN uses U-Net discriminators to learn global and local differences between denoised and normal-dose images in both image and gradient domains.The encoder focuses on global structure, while the decoder produces per-pixel confidence feedback; the gradient branch targets photon-starvation artifacts and edges.
  • CutMix regularizes the U-Net discriminator and enables confidence maps that visualize uncertainty in denoised results for radiologists.
  • Experiments on simulated and real-world datasets report effectiveness through qualitative and quantitative comparisons.

B. Generative Adversarial Networks (GANs)

In LDCT denoising, GANs use a generator and discriminator to produce realistic images, but conventional discriminators may not retain both global and local representations. DU-GAN uses U-Net-based discriminators in image and gradient domains to address this limitation.

  • GANs alternate a generator that produces samples with a discriminator that distinguishes generated samples from real data.
  • For LDCT denoising, the generator produces photo-realistic results while the discriminator distinguishes denoised images from normal-dose CT images.
  • DU-GAN performs adversarial training in both image and gradient domains to reduce noise, enhance edges, and alleviate streak artifacts.
  • The framework combines a denoising generator with two U-Net-based discriminators operating on image and gradient representations.
  • The denoising model maps an LDCT image I_LD to its normal-dose CT counterpart I_ND by removing noise.
  • Unlike traditional scalar-output discriminators, U-Net-based discriminators capture global and local information simultaneously.

B. Dual-Domain U-Net Based Discriminator

DU-GAN replaces the conventional classification discriminator with a U-Net based discriminator that captures global and local differences between denoised and normal-dose images. Gradient-domain analysis motivates extending this design to dual-domain discrimination.

  • Conventional GAN denoising discriminators can forget previous samples as the synthetic-image distribution shifts during training.
  • DU-GAN introduces U-Net based discriminators in both image and gradient domains to characterize global and local differences.
  • Horizontal and vertical gradient representations make streak artifacts readily visible, motivating gradient-domain discrimination.
  • The image-domain U-Net encoder captures global structural context, while its decoder and skip connections enhance local-detail discrimination.
  • Unlike traditional discriminators that output only a global real/fake classification, the U-Net discriminator provides per-pixel and global structural feedback to the generator.

+ EILD

The image-domain adversarial competition alone may not preserve pathological edges or adequately suppress photon-starvation streaks. DU-GAN therefore adds a gradient-domain adversarial branch based on Sobel-estimated gradients.

  • Image-domain competition encourages photo-realistic denoising but is insufficient for preserving pathological edges and alleviating photon-starvation streak artifacts.
  • The gradient branch uses horizontal and vertical gradient magnitudes, where CT streaks and edges are highlighted.
  • Two independent GAN competitions operate in image and gradient domains; the image branch targets photo-realism, while the gradient branch encourages edges and reduces streak artifacts.

C. CutMix Regularization

CutMix regularization trains the image-domain U-Net discriminator to retain local discrimination while predicting global mixing structure. Its outputs can support confidence maps for denoised-result uncertainty.

  • CutMix regularization addresses declining local-discrimination capability and encourages the discriminator to learn intrinsic real–fake differences.
  • The binary mask M controls spatial mixing of NDCT and denoised images through element-wise multiplication.
  • The mixed sample is treated as globally fake by Dimg enc, while Dimg dec identifies pixel-level differences in the mixed regions.
  • CutMix outputs combine real and generated patches according to real/fake classification scores, demonstrating per-pixel discrimination despite patch mixing.
  • The trained discriminator can provide radiologists with a confidence map showing uncertainty in denoised results.
  • Dimg enc predicts the proportion of real patches, or mixed ratio, while focusing on global structures.

D. Network Architecture

The proposed framework combines a GAN generator with U-Net based discriminators, using global and local discrimination plus an additional gradient branch to guide LDCT denoising.

  • The framework contains a denoising generator and U-Net based discriminators operating in image and gradient domains.

1) RED-CNN based generator:

DU-GAN uses RED-CNN as its generator while focusing the framework's contribution on dual-domain U-Net-based adversarial discriminators.

  • 1) RED-CNN based generator:: RED-CNN serves as DU-GAN's generator because the framework focuses on adversarial loss from dual-domain U-Net-based discriminators.The framework is optimized in a GAN manner rather than with vanilla RED-CNN's MSE-only objective.
  • 1) RED-CNN based generator:: The image- and gradient-domain discriminators independently use U-Net architectures with encoder and decoder branches.The encoder produces a global confidence score, while the decoder processes upsampled features with skip connections.
  • 1) RED-CNN based generator:: The paper does not introduce a new network architecture for the generator or discriminator.Its key architectural contribution is applying U-Net discriminators in dual domains to capture local details and global structures.
  • 1) RED-CNN based generator:: The proposed U-Net discriminator captures both local details and global structures for LDCT denoising.This design distinguishes DU-GAN from approaches using conventional classification discriminators.

2) Pixel-wise loss:

DU-GAN trains with pixel- and gradient-level reconstruction losses alongside adversarial objectives to preserve image content and edge information.

  • 2) Pixel-wise loss:: The pixel-wise objective compares denoised LDCT images with NDCT images using both pixel and gradient losses.The gradient term is intended to preserve edge information at the pixel level.
  • 2) Pixel-wise loss:: MSE is used for pixel loss, while absolute mean error is used for gradient loss because image gradients are sparser than pixels.The choice avoids the computational cost of feature-level losses based on pretrained models.
  • 2) Pixel-wise loss:: The generator's final objective combines adversarial, image-domain, and gradient-domain losses.The loss weights λadv, λimg, and λgrd control the contributions of these three terms.
  • 2) Pixel-wise loss:: The discriminators are optimized with a mixed loss, using the same formulation independently for image and gradient domains.The gradient-domain discriminator additionally applies a Sobel operator to compute image gradients.

F. Complexity of DU-GAN

DU-GAN adds gradient-domain discrimination and U-Net-based discriminators, increasing training cost while retaining the inference efficiency of traditional methods.

  • F. Complexity of DU-GAN: DU-GAN introduces one extra hyperparameter to control the gradient branch relative to MSE- and conventional GAN-based methods.The additional branch complements the original pixel branch by encouraging clearer boundaries.
  • F. Complexity of DU-GAN: Its main computational costs come from the U-Net-based discriminator and gradient branch during training.These costs are described as affordable in relation to the reported denoising quality and performance.
  • F. Complexity of DU-GAN: Inference efficiency remains the same as traditional methods because the added computational costs occur only during training.The paper separates training-time discriminator and gradient-branch costs from deployment-time inference.
  • F. Complexity of DU-GAN: Experiments cover qualitative and quantitative evaluations, uncertainty visualization, and ablation studies.The study uses simulated Mayo-25% and Mayo-10% datasets plus the real-world Piglet-5% dataset.
  • F. Complexity of DU-GAN: The Mayo experiments use separate training and testing patients, with 20 patients per split for each dataset.The datasets contain 300K training patches and 64K testing patches selected from each set.
  • F. Complexity of DU-GAN: Models are trained on 64 × 64 patches for up to 100K iterations using Adam with a fixed learning rate of 10^-4.Training uses one NVIDIA V100 GPU, with λadv = 0.1, λimg = 1, and λgrd = 20.

C. Qualitative Evaluations

Qualitative evaluations show that DU-GAN better preserves diagnostically relevant structures and mitigates streak artifacts than the compared baselines across simulated and real-world CT data.

  • C. Qualitative Evaluations: The qualitative evaluation compares LDCT, NDCT, five baseline methods, and DU-GAN using full images and enlarged ROIs.Fig. 5 covers Mayo-10%, Mayo-25%, and Piglet-5% datasets, with arrows marking lesions and white-edge artifacts.
  • C. Qualitative Evaluations: MSE-trained RED-CNN and Q-AE produce over-smoothed, blurred images that lose structural details.Their results are contrasted with NDCT and DU-GAN, where low-attenuation lesions and bones are clearer.
  • C. Qualitative Evaluations: WGAN-VGG and CPCE-2D improve visual fidelity but retain minor streak artifacts and lose some local details.The paper attributes their limitation to global feedback from traditional classification discriminators and perceptual-loss processing.
  • C. Qualitative Evaluations: DU-GAN consistently preserves small structures and their boundaries with clear visual fidelity.The paper connects this result to its dual-domain U-Net-based discriminators.
  • C. Qualitative Evaluations: In neck CT with strong photon-starvation streaks, DU-GAN avoids the strongly blurry and ghosted results seen in baseline methods.Its gradient branch detects streak artifacts, while dual-domain adversarial training alleviates their impact.
  • C. Qualitative Evaluations: DU-GAN preserves more structural details important for diagnosis, while compromising PSNR and RMSE relative to some alternatives.The paper notes that these pixel-wise metrics correlate poorly with human perception of image quality.

E. Uncertainty Visualization

The U-Net discriminator produces both global scores and per-pixel confidence maps, enabling uncertainty visualization of denoised CT images. Experiments show that it detects blurriness and streak artifacts while distinguishing the proposed results from competing methods.

  • Uncertainty visualization: The U-Net discriminator learns local differences through its decoder and global structures through its encoder, enabling confidence-map visualization.CutMix regularization supports per-pixel uncertainty estimates for radiologists.
  • Uncertainty visualization: The discriminator accurately distinguishes LDCT from NDCT images using both global scores and per-pixel confidence.Its decoder confidence maps identify differences in denoised image quality.
  • Uncertainty visualization: RED-CNN and Q-AE receive low global discriminator scores because their over-smoothed images contain detectable blurriness.The discriminator’s global encoder score reflects structural differences between real and denoised images.
  • Uncertainty visualization: CPCE-2D and WGAN-VGG retain streak artifacts despite preserving more local detail, whereas DU-GAN produces the most photo-realistic results and highest global score.Unlike traditional classification discriminators, DU-GAN also provides per-pixel feedback to the generator.
  • Uncertainty visualization: DU-GAN produces smoother per-pixel confidence, indicating that its discriminator cannot distinguish real and generated samples at the pixel level.This confidence behavior is reported as evidence of improved local similarity.
  • Components analysis: Adding the image-domain U-Net discriminator increases SSIM, CutMix further emphasizes local details, and gradient-domain training improves PSNR, RMSE, and SSIM.The gradient branch is specifically associated with removing streak artifacts and sharpening denoised edges.

2) Architectures of discriminator:

The discriminator architecture combines global and local analysis through U-Net structure, while dual-domain training and CutMix support denoising quality and uncertainty visualization. The method generalized across simulated and real-world datasets, but clinical validation and lower-dose testing remain open.

  • Architectures of discriminator: U-Net discriminators characterize global and local differences, combining image-level structural analysis with per-pixel feedback.This architecture is the central discriminator design used by DU-GAN.
  • Patch size: Smaller training patches achieved better performance than larger patches among the evaluated 64×64, 128×128, 256×256, and 512×512 sizes.The study fine-tuned the generator from models trained on progressively larger patch sizes.
  • Generalization: DU-GAN was evaluated on two simulated low-dose CT datasets with different doses and one real-world dataset using the same architecture and hyperparameters.The authors report this as evidence of generalization across datasets.
  • Limitations: The study lacks a human reader validation, and the clinical utility of confidence maps remains to be examined through tasks such as liver lesion diagnosis.DU-GAN also incurs additional training cost from its U-Net discriminator and dual-domain strategy, while inference efficiency remains unchanged.
  • Discussion and conclusion: Qualitative and quantitative comparisons found better denoising performance than other GAN-based models, with potential for clinical use through uncertainty visualization.The paper frames this conclusion as potential rather than established clinical benefit.
Loading 2108.10772v2…