PaperlayerView exact revision on arXiv

Source-linked AI summary

CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)

Chenyu You, Guang Li, Yi Zhang, Xiaoliu Zhang, Hongming Shan, Shenghong Ju, Zhen Zhao, Zhuiyang Zhang, Wenxiang Cong, Michael W. Vannier, Punam K. Saha, Ge Wang

arXiv:1808.04256v3eess.IVcs.CVcs.LGstat.ML

TL;DR

CT super-resolution must recover fine anatomical detail from noisy LR images while addressing limited paired data and the practical costs of complex networks. GAN-CIRCLE uses semi-supervised cycle-consistent Wasserstein adversarial learning with structural constraints and lightweight CNN-based generators. Across three large-scale CT datasets, the networks achieve promising restoration results, including anatomical preservation, noise suppression, and reported diagnostic-quality improvements.

  • Problem

    Recovering HR CT images from noisy LR inputs is difficult because paired LR-HR data are limited and complex networks impose computational and memory overheads.

  • Method

    GAN-CIRCLE uses semi-supervised cycle-consistent Wasserstein adversarial learning with joint structural constraints, residual CNN feature extraction, and lightweight generators.

  • Results

    Evaluations on three large-scale CT datasets report promising results for restoring noisy LR images, preserving anatomical information, suppressing noise, and improving diagnostic quality.

  • Takeaways & Limitations

    The proposed networks provide a computationally efficient approach for CT super-resolution in supervised and semi-supervised settings, including settings with limited paired data.

Abstract

from arXiv · show

Computed tomography (CT) is widely used in screening, diagnosis, and image-guided therapy for both clinical and research purposes. Since CT involves ionizing radiation, an overarching thrust of related technical research is development of novel methods enabling ultrahigh quality imaging with fine structural details while reducing the X-ray radiation. In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this deep imaging process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the CT imaging performance, which limit its real-world applications by imposing considerable computational and memory overheads, we apply a parallel $1\times1$ CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. Quantitative and qualitative evaluations demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

I. INTRODUCTION

CT super-resolution is a challenging, ill-posed computational problem because low-dose LR images contain complex noise and artifacts, while hardware-based resolution improvements increase cost and radiation. GAN-CIRCLE addresses these limitations with cycle-consistent adversarial learning, structural constraints, hierarchical residual CNN features, and lightweight network design.

  • Motivation: Computational CT super-resolution seeks HR images from LR images while avoiding the cost, radiation, and speed compromises of hardware-oriented improvements.The reconstruction task is described as a seriously ill-posed inverse problem.
  • Challenges: Existing supervised methods require paired LR-HR CT images, which are scarce because obtaining them may require additional patient radiation exposure.The paper therefore identifies semi-supervised learning as essential for clinical practice.
  • Challenges: GAN-based SR can learn nonlinear interdomain mappings but may be under-constrained, collapse outputs, introduce unsupported features, or lack interpretable training progress.Cycle consistency is presented as a response to these mapping and collapse risks.
  • Challenges: Deep networks can exploit hierarchical features, but increasingly large models impose computational and memory overheads that hinder real-world CT applications.The paper also notes that local CT features occur at different scales and that L2 optimization may produce over-smoothed outputs.
  • Proposed approach: GAN-CIRCLE combines cycle consistency, Wasserstein distance, residual CNNs, hierarchical feature learning, skip connections, L1 refinement, and joint sparsifying-transform regularization.The design targets structural preservation, stabilized GAN training, reduced overhead, and deblurring.

II. METHODS

The paper first reviews medical-imaging super-resolution problems, then introduces its adversarial network framework and SR architecture, and finally describes optimization.

  • Problem review: The methods section begins by reviewing super-resolution problems in the medical imaging field.
  • Framework and architecture: It then introduces the proposed adversarial network framework and SR imaging network architecture.
  • Optimization: The section concludes by describing the optimization process.

A. Problem Statement

The paper formulates noisy LRCT-to-HRCT recovery as a nonlinear, potentially non-linear inverse problem and proposes a bidirectional GAN-CIRCLE framework with multiple structural constraints.

  • Problem motivation: Noisy LRCT images contain complex spatial variations and reconstruction-induced noise and artifacts that complicate HRCT recovery.
  • Cycle-consistent model: The proposed model jointly trains mappings G: X → Y and F: Y → X with adversarial discriminators to synthesize HRCT and reconstruct LRCT images.
  • Loss design: GAN-CIRCLE combines adversarial, cycle-consistency, identity, and joint sparsifying transform losses to regularize bidirectional training.
  • Loss design: Wasserstein adversarial losses match generated images to empirical source and target distributions while penalizing gradient-norm deviations.
  • Loss design: Cycle consistency promotes F(G(x)) ≈ x and G(F(y)) ≈ y, imposing shared-latent-space constraints that preserve source content.
  • Loss design: The joint sparsifying transform loss combines image sparsification with anatomical preservation, using τ = 0.5 in this paper.

C. Network Architecture

The architecture uses lightweight generative and discriminative networks for CT super-resolution, combining dense feature extraction, skip connections, nonlinear 1 × 1 projections, and residual reconstruction.

  • Generative networks: The lightweight generator architecture contains feature-extraction and reconstruction streams optimized for real-application SR CT imaging.
  • Feature extraction: Twelve nonlinear SR feature blocks use 3 × 3 convolutions, Leaky ReLU, and dropout to extract features while mitigating dead units and overfitting.
  • Feature extraction: A skip connection concatenates hidden-layer outputs before reconstruction, helping prevent training saturation and overfitting while retaining diverse HRCT details.
  • Reconstruction: Parallelized Network-in-Network CNNs use shallow MLPs and 1 × 1 convolutions to project densely connected features nonlinearly.
  • Reconstruction: The final convolution fuses feature maps into a residual image containing mostly high-frequency details; supervised outputs add it to bicubic interpolation.
  • Discriminative networks: The discriminator has four convolutional stages followed by fully connected layers with 1024 units and one output, without sigmoid cross-entropy.

III. EXPERIMENTS AND RESULTS

The experiments compare the proposed algorithms with state-of-the-art SR methods quantitatively and qualitatively, including robustness and diagnostic quality assessments.

  • The study evaluates proposed algorithms against state-of-the-art SR methods using quantitative and qualitative comparisons.
  • The experiments additionally assess robustness in real SR scenarios and perform detailed diagnostic quality assessment.

A. Training Datasets

The study uses two CT datasets with distinct characteristics and constructs noisy, downsampled LR images from high-quality HRCT images for training.

  • Dataset overview: The experiments use two high-quality CT image datasets with very different characteristics to evaluate GAN-CIRCLE fidelity and robustness.
  • Tibia dataset: The Tibia dataset contains 25 fresh-frozen cadaveric ankle specimens from 17 body donors scanned with a Siemens microCAT II.
  • Abdominal dataset: The Abdominal dataset contains 5,936 full-dose CT images from 10 patients in the NIH-AAPM-Mayo Clinic Low Dose CT Grand Challenge.
  • Preprocessing: Preprocessing scales CT images to [0,1], treats originals as HRCT ground truth, and creates LR images by adding noise and reducing spatial resolution by a factor of 2.
  • Preprocessing: Overlapping patches increase training samples while preserving local anatomy, using 64×64 HRCT patches and corresponding 32×32 LRCT patches.

B. Implementation Details

The implementation initializes convolutional weights using a variance based on filter size and filter count, then applies specified regularization and activation settings.

  • Convolutional weights use standard deviation 0.118 for 3×3 filters with 16 filters, with all biases initialized to zero.The standard deviation is computed as std = 2/m, where m = f_s^2 × n_f.
  • Training sets λ1, λ2, and λ3 to 1, 0.5, and 0.001, respectively.
  • Dropout with p = 0.8 is applied to each convolutional layer.
  • Convolutional and transposed convolutional layers use Leaky ReLU activations with slope α = 0.1.

C. Performance Comparison

The study compares GAN-CIRCLE and its variants with interpolation, dictionary-based, PSNR-oriented, and GAN-based methods across benchmark CT datasets using visual and quantitative evaluations.

  • The comparison includes nearest-neighbor, bilinear, bicubic, Lanczos, A+, FSRCNN, ESPCN, LapSRN, and SRGAN methods.The methods are grouped into interpolation-based, dictionary-based, PSNR-oriented, and GAN-based classes.
  • Visual comparisons include abdominal and real-dataset cases, with restored anatomical features or bony structures highlighted in red and yellow boxes.
  • BM3D-denoised images are also super-resolved with interpolation methods and A+ to assess denoising-plus-interpolation baselines.
  • The proposed method is evaluated through G-Forward, G-Adversarial, and full GAN-CIRCLE variants.
  • Supervised, semi-supervised, and unsupervised settings use paired and unpaired data splits with 100%, 50%, and 0% paired samples.
  • Performance is measured with PSNR, SSIM, and IFC on two benchmark datasets.

D. Experimental Results with the Tibia Dataset

On the tibia dataset, GAN-CIRCLE and related deep-learning methods reduce interpolation artifacts, while GAN-CIRCLE recovers fine details and anatomical information with competitive quantitative performance.

  • BM3D removes noise effectively but over-smooths noisy LR images, whereas interpolation methods produce partial-aliasing artifacts.
  • DL-based methods suppress interpolation artifacts, and GAN-CIRCLE recovers finer subtle details and more anatomical information.
  • GAN-CIRCLEs predicts sharper boundaries and richer textures than GAN-CIRCLE, while GAN-CIRCLEu learns additional anatomical information from unpaired samples.
  • G-Forward achieves the highest PSNR and SSIM scores among the evaluated methods.
  • GAN-CIRCLE achieves second-best SSIM and IFC values and performs similarly to fully supervised methods on the tibia dataset.

E. Experimental Results on the Abdominal Dataset

On the abdominal dataset, GAN-CIRCLE preserves anatomical information and high-frequency details while producing relatively lower noise than other GAN-based methods, although GAN-based outputs can introduce artifacts.

  • Table III reports subjective diagnostic-quality scores as mean±STDS, with red and blue indicating the best and second-best performances.
  • GAN-CIRCLE preserves anatomical information and more clearly visualizes the portal vein in abdominal CT images.
  • PSNR-oriented methods significantly suppress noise and artifacts but can receive lower human-observer image-quality judgments because noise sensitivity depends on local image properties.
  • GAN-based models improve structural identification but can introduce strong noise, including tiny artifacts in GAN-CIRCLEu results.
  • GAN-CIRCLE and GAN-CIRCLEs retain high-frequency details and reconstruct more realistic images with relatively lower noise than G-Adv and SRGAN.

F. Super-resolving Real-world Images

Real-world SRCT experiments target unmatched or limited paired data while adapting across noise distributions. The proposed methods recover anatomical structure and texture effectively, with radiologist assessments supporting their diagnostic quality.

  • F. Super-resolving Real-world Images: Semi-supervised SRCT addresses clinical settings where matched LRCT-HRCT pairs are scarce or mismatched.The authors demonstrate real applications using a small amount of mismatched paired data and flexibility across noise distributions.
  • F. Super-resolving Real-world Images: The mouse dataset used LRCT and HRCT scans acquired with two protocols on the same scanner.The HRCT images measured 1450 × 1450 with 600 slices at 48 µm isotropic voxel size, while LRCT images measured 725×725 with 300 slices.
  • F. Super-resolving Real-world Images: GAN-CIRCLEs and GAN-CIRCLEu achieved the highest IFC among the evaluated SR methods.GAN-CIRCLEs also obtained the second-best SSIM, while PSNR-oriented methods produced higher PSNR and SSIM overall.
  • F. Super-resolving Real-world Images: GAN-based methods produced sharper anatomical content and richer texture, whereas PSNR-oriented methods better suppressed noise.Visual comparisons examined lung regions and mouse bone structures, where the proposed methods preserved edges and structural information.
  • G. Diagnostic Quality Assessment: Three board-certified radiologists independently graded sharpness, noise, contrast resolution, diagnostic acceptability, and overall quality on a 1–5 scale.They assessed 10 randomized, deidentified image sets from the Tibia and Abdominal datasets while blinded to the post-processing algorithms.

IV. DISCUSSIONS

The discussion positions GAN-CIRCLE as an efficient CT-SR framework that uses unpaired data and joint constraints to preserve anatomical information. The authors also identify longer training, imperfect subtle-structure recovery, and incomplete consistency with HR images as limitations.

  • IV. DISCUSSIONS: SR imaging could support visualization of bony details, lung structures, implanted stents, and radiomics analysis.The paper frames these applications as potential practical benefits of higher-resolution CT.
  • IV. DISCUSSIONS: High-resolution micro-CT is well suited to bone imaging because bone microstructure strongly relates to bone strength and fracture risk.The discussion connects improved spatial resolution with evaluation of skeletal micro-architecture.
  • IV. DISCUSSIONS: Abdominal and pelvic CT detects diseases in internal organs, while radiation exposure motivates manipulation of scanning parameters and newer technologies.The discussion mentions the small bowel, colon, kidneys, and other internal organs as diagnostic targets.
  • IV. DISCUSSIONS: GAN-CIRCLE combines cycle-consistency, identity and supervision constraints, Wasserstein distance, and efficient CNN design to improve structure and stabilize training.Skip connections and network-in-network techniques reduce computational complexity while supporting hierarchical feature learning.
  • IV. DISCUSSIONS: The model requires substantially longer training than standard GAN-based methods, which generally require 1–2 days.The authors propose more efficient architectures to reduce computational cost and model complexity.
  • IV. DISCUSSIONS: Subtle structures may not always be faithfully recovered, and restored images cannot be uniformly consistent with the original HR images.The authors suggest adding sinogram consistency, low-dimensional manifold constraints, or adaptive task-specific losses.
  • V. CONCLUSIONS: The conclusion reports promising preservation of anatomical information and noise suppression in supervised and semi-supervised CT-SR settings.Expert radiologist evaluations are described as confirming superior diagnostic quality alongside quantitative image-quality results.
Loading 1808.04256v3…