PaperlayerView exact revision on arXiv

Source-linked AI summary

CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)

Chenyu You, Guang Li, Yi Zhang, Xiaoliu Zhang, Hongming Shan, Shenghong Ju, Zhen Zhao, Zhuiyang Zhang, Wenxiang Cong, Michael W. Vannier, Punam K. Saha, Ge Wang

arXiv:1808.04256v3eess.IVcs.CVcs.LGstat.ML

TL;DR

CT super-resolution must recover fine anatomical detail from noisy LR images while addressing limited paired data and the practical costs of complex networks. GAN-CIRCLE uses semi-supervised cycle-consistent Wasserstein adversarial learning with structural constraints and lightweight CNN-based generators. Across three large-scale CT datasets, the networks achieve promising restoration results, including anatomical preservation, noise suppression, and reported diagnostic-quality improvements.

Abstract

from arXiv · show

Computed tomography (CT) is widely used in screening, diagnosis, and image-guided therapy for both clinical and research purposes. Since CT involves ionizing radiation, an overarching thrust of related technical research is development of novel methods enabling ultrahigh quality imaging with fine structural details while reducing the X-ray radiation. In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this deep imaging process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the CT imaging performance, which limit its real-world applications by imposing considerable computational and memory overheads, we apply a parallel $1\times1$ CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. Quantitative and qualitative evaluations demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

I. INTRODUCTION

CT super-resolution is a challenging, ill-posed computational problem because low-dose LR images contain complex noise and artifacts, while hardware-based resolution improvements increase cost and radiation. GAN-CIRCLE addresses these limitations with cycle-consistent adversarial learning, structural constraints, hierarchical residual CNN features, and lightweight network design.

Motivation

  • Computational CT super-resolution seeks HR images from LR images while avoiding the cost, radiation, and speed compromises of hardware-oriented improvements.The reconstruction task is described as a seriously ill-posed inverse problem.Source: §I · p. 1

Challenges

  • Existing supervised methods require paired LR-HR CT images, which are scarce because obtaining them may require additional patient radiation exposure.The paper therefore identifies semi-supervised learning as essential for clinical practice.Source: §I · p. 3
  • GAN-based SR can learn nonlinear interdomain mappings but may be under-constrained, collapse outputs, introduce unsupported features, or lack interpretable training progress.Cycle consistency is presented as a response to these mapping and collapse risks.Source: §I · p. 3
  • Deep networks can exploit hierarchical features, but increasingly large models impose computational and memory overheads that hinder real-world CT applications.The paper also notes that local CT features occur at different scales and that L2 optimization may produce over-smoothed outputs.Source: §I · p. 4

Proposed approach

  • GAN-CIRCLE combines cycle consistency, Wasserstein distance, residual CNNs, hierarchical feature learning, skip connections, L1 refinement, and joint sparsifying-transform regularization.The design targets structural preservation, stabilized GAN training, reduced overhead, and deblurring.Source: §I · p. 4

II. METHODS

The paper first reviews medical-imaging super-resolution problems, then introduces its adversarial network framework and SR architecture, and finally describes optimization.

Problem review

  • The methods section begins by reviewing super-resolution problems in the medical imaging field.Source: §II · p. 4

Framework and architecture

  • It then introduces the proposed adversarial network framework and SR imaging network architecture.Source: §II · p. 4

Optimization

  • The section concludes by describing the optimization process.Source: §II · p. 4

A. Problem Statement

The paper formulates noisy LRCT-to-HRCT recovery as a nonlinear, potentially non-linear inverse problem and proposes a bidirectional GAN-CIRCLE framework with multiple structural constraints.

Problem motivation

  • Noisy LRCT images contain complex spatial variations and reconstruction-induced noise and artifacts that complicate HRCT recovery.Source: A. Problem Statement · p. 4

Cycle-consistent model

Loss design

C. Network Architecture

The architecture uses lightweight generative and discriminative networks for CT super-resolution, combining dense feature extraction, skip connections, nonlinear 1 × 1 projections, and residual reconstruction.

Generative networks

  • The lightweight generator architecture contains feature-extraction and reconstruction streams optimized for real-application SR CT imaging.Source: 1) Generative Networks · p. 5

Feature extraction

  • Twelve nonlinear SR feature blocks use 3 × 3 convolutions, Leaky ReLU, and dropout to extract features while mitigating dead units and overfitting.Source: 1) Generative Networks · p. 5
  • A skip connection concatenates hidden-layer outputs before reconstruction, helping prevent training saturation and overfitting while retaining diverse HRCT details.Source: 1) Generative Networks · p. 6

Reconstruction

  • Parallelized Network-in-Network CNNs use shallow MLPs and 1 × 1 convolutions to project densely connected features nonlinearly.Source: 1) Generative Networks · pp. 6–7
  • The final convolution fuses feature maps into a residual image containing mostly high-frequency details; supervised outputs add it to bicubic interpolation.Source: 1) Generative Networks · p. 7

Discriminative networks

  • The discriminator has four convolutional stages followed by fully connected layers with 1024 units and one output, without sigmoid cross-entropy.Source: 2) Discriminative Networks · p. 8

III. EXPERIMENTS AND RESULTS

The experiments compare the proposed algorithms with state-of-the-art SR methods quantitatively and qualitatively, including robustness and diagnostic quality assessments.

  • The study evaluates proposed algorithms against state-of-the-art SR methods using quantitative and qualitative comparisons.Source: §III · p. 8
  • The experiments additionally assess robustness in real SR scenarios and perform detailed diagnostic quality assessment.Source: §III · p. 8

A. Training Datasets

The study uses two CT datasets with distinct characteristics and constructs noisy, downsampled LR images from high-quality HRCT images for training.

Dataset overview

  • The experiments use two high-quality CT image datasets with very different characteristics to evaluate GAN-CIRCLE fidelity and robustness.Source: A. Training Datasets · p. 8

Tibia dataset

  • The Tibia dataset contains 25 fresh-frozen cadaveric ankle specimens from 17 body donors scanned with a Siemens microCAT II.Source: 1) Tibia dataset · p. 8

Abdominal dataset

  • The Abdominal dataset contains 5,936 full-dose CT images from 10 patients in the NIH-AAPM-Mayo Clinic Low Dose CT Grand Challenge.Source: 2) Abdominal dataset · p. 8

Preprocessing

  • Preprocessing scales CT images to [0,1], treats originals as HRCT ground truth, and creates LR images by adding noise and reducing spatial resolution by a factor of 2.Source: 2) Abdominal dataset · p. 8
  • Overlapping patches increase training samples while preserving local anatomy, using 64×64 HRCT patches and corresponding 32×32 LRCT patches.Source: 2) Abdominal dataset · p. 8

B. Implementation Details

The implementation initializes convolutional weights using a variance based on filter size and filter count, then applies specified regularization and activation settings.

C. Performance Comparison

The study compares GAN-CIRCLE and its variants with interpolation, dictionary-based, PSNR-oriented, and GAN-based methods across benchmark CT datasets using visual and quantitative evaluations.

  • The comparison includes nearest-neighbor, bilinear, bicubic, Lanczos, A+, FSRCNN, ESPCN, LapSRN, and SRGAN methods.The methods are grouped into interpolation-based, dictionary-based, PSNR-oriented, and GAN-based classes.Source: C. Performance Comparison · p. 8
  • Visual comparisons include abdominal and real-dataset cases, with restored anatomical features or bony structures highlighted in red and yellow boxes.Source: Figure 6 · p. 9, Figure 7 · p. 10
  • BM3D-denoised images are also super-resolved with interpolation methods and A+ to assess denoising-plus-interpolation baselines.Source: C. Performance Comparison · p. 9
  • The proposed method is evaluated through G-Forward, G-Adversarial, and full GAN-CIRCLE variants.Source: C. Performance Comparison · p. 9
  • Supervised, semi-supervised, and unsupervised settings use paired and unpaired data splits with 100%, 50%, and 0% paired samples.Source: C. Performance Comparison · p. 9
  • Performance is measured with PSNR, SSIM, and IFC on two benchmark datasets.Source: C. Performance Comparison · pp. 9–10

D. Experimental Results with the Tibia Dataset

On the tibia dataset, GAN-CIRCLE and related deep-learning methods reduce interpolation artifacts, while GAN-CIRCLE recovers fine details and anatomical information with competitive quantitative performance.

E. Experimental Results on the Abdominal Dataset

On the abdominal dataset, GAN-CIRCLE preserves anatomical information and high-frequency details while producing relatively lower noise than other GAN-based methods, although GAN-based outputs can introduce artifacts.

  • Table III reports subjective diagnostic-quality scores as mean±STDS, with red and blue indicating the best and second-best performances.Source: Table III · p. 12
  • GAN-CIRCLE preserves anatomical information and more clearly visualizes the portal vein in abdominal CT images.Source: E. Experimental Results on the… · p. 12
  • PSNR-oriented methods significantly suppress noise and artifacts but can receive lower human-observer image-quality judgments because noise sensitivity depends on local image properties.Source: E. Experimental Results on the… · p. 12
  • GAN-based models improve structural identification but can introduce strong noise, including tiny artifacts in GAN-CIRCLEu results.Source: E. Experimental Results on the… · p. 12
  • GAN-CIRCLE and GAN-CIRCLEs retain high-frequency details and reconstruct more realistic images with relatively lower noise than G-Adv and SRGAN.Source: E. Experimental Results on the… · p. 12

F. Super-resolving Real-world Images

Real-world SRCT experiments target unmatched or limited paired data while adapting across noise distributions. The proposed methods recover anatomical structure and texture effectively, with radiologist assessments supporting their diagnostic quality.

F. Super-resolving Real-world Images

  • Semi-supervised SRCT addresses clinical settings where matched LRCT-HRCT pairs are scarce or mismatched.The authors demonstrate real applications using a small amount of mismatched paired data and flexibility across noise distributions.Source: F. Super-resolving Real-world Images · p. 12
  • The mouse dataset used LRCT and HRCT scans acquired with two protocols on the same scanner.The HRCT images measured 1450 × 1450 with 600 slices at 48 µm isotropic voxel size, while LRCT images measured 725×725 with 300 slices.Source: 1) Practical SRCT Implementation Details · p. 12
  • GAN-CIRCLEs and GAN-CIRCLEu achieved the highest IFC among the evaluated SR methods.GAN-CIRCLEs also obtained the second-best SSIM, while PSNR-oriented methods produced higher PSNR and SSIM overall.Source: 2) Comparison With the… · p. 12
  • GAN-based methods produced sharper anatomical content and richer texture, whereas PSNR-oriented methods better suppressed noise.Visual comparisons examined lung regions and mouse bone structures, where the proposed methods preserved edges and structural information.Source: 2) Comparison With the… · pp. 12–13

G. Diagnostic Quality Assessment

  • Three board-certified radiologists independently graded sharpness, noise, contrast resolution, diagnostic acceptability, and overall quality on a 1–5 scale.They assessed 10 randomized, deidentified image sets from the Tibia and Abdominal datasets while blinded to the post-processing algorithms.Source: G. Diagnostic Quality Assessment · p. 13

IV. DISCUSSIONS

The discussion positions GAN-CIRCLE as an efficient CT-SR framework that uses unpaired data and joint constraints to preserve anatomical information. The authors also identify longer training, imperfect subtle-structure recovery, and incomplete consistency with HR images as limitations.

IV. DISCUSSIONS

  • SR imaging could support visualization of bony details, lung structures, implanted stents, and radiomics analysis.The paper frames these applications as potential practical benefits of higher-resolution CT.Source: §IV · p. 13
  • High-resolution micro-CT is well suited to bone imaging because bone microstructure strongly relates to bone strength and fracture risk.The discussion connects improved spatial resolution with evaluation of skeletal micro-architecture.Source: §IV · p. 13
  • Abdominal and pelvic CT detects diseases in internal organs, while radiation exposure motivates manipulation of scanning parameters and newer technologies.The discussion mentions the small bowel, colon, kidneys, and other internal organs as diagnostic targets.Source: §IV · p. 13
  • GAN-CIRCLE combines cycle-consistency, identity and supervision constraints, Wasserstein distance, and efficient CNN design to improve structure and stabilize training.Skip connections and network-in-network techniques reduce computational complexity while supporting hierarchical feature learning.Source: §IV · p. 13
  • The model requires substantially longer training than standard GAN-based methods, which generally require 1–2 days.The authors propose more efficient architectures to reduce computational cost and model complexity.Source: §IV · p. 13
  • Subtle structures may not always be faithfully recovered, and restored images cannot be uniformly consistent with the original HR images.The authors suggest adding sinogram consistency, low-dimensional manifold constraints, or adaptive task-specific losses.Source: §IV · pp. 13–14

V. CONCLUSIONS

  • The conclusion reports promising preservation of anatomical information and noise suppression in supervised and semi-supervised CT-SR settings.Expert radiologist evaluations are described as confirming superior diagnostic quality alongside quantitative image-quality results.Source: §V · p. 14

Related papers

Loading 1808.04256v3…