PaperlayerView exact revision on arXiv

Source-linked AI summary

Framing U-Net via Deep Convolutional Framelets: Application to Sparse-view CT

Yoseob Han, Jong Chul Ye

arXiv:1708.08333v3cs.CVcs.LGstat.ML

TL;DR

Sparse-view CT lowers radiation dose but causes severe FBP streaking artifacts, while the theoretical basis of successful U-Net reconstruction remains insufficiently understood. The paper applies deep convolutional framelet theory to identify U-Net’s frame-condition limitation and proposes dual frame and tight frame variants. Experiments with real patient data show better reconstruction performance for the new architectures, with tight frame U-Net providing strong high-frequency recovery and visual quality.

  • Problem

    Sparse-view CT reconstruction suffers severe FBP streaking artifacts, and the theoretical justification for U-Net’s strong performance remains lacking.

  • Method

    The paper analyzes U-Net using deep convolutional framelets and proposes dual frame and tight frame U-Nets that satisfy the frame condition.

  • Results

    Experiments show that the proposed architectures provide better reconstruction performance, including improved high-frequency recovery, streak suppression, detailed-structure preservation, and smaller NMSE values than TV in reported experiments.

  • Takeaways & Limitations

    Frame-condition-aware multi-resolution architectures offer a supported approach for improving sparse-view CT reconstruction beyond standard U-Net.

  • Takeaways & Limitations

    The study’s sparse-view CT evidence includes experiments with real patient data and specifically evaluates globally distributed streaking artifacts requiring large receptive fields.

Abstract

from arXiv · show

X-ray computed tomography (CT) using sparse projection views is a recent approach to reduce the radiation dose. However, due to the insufficient projection views, an analytic reconstruction approach using the filtered back projection (FBP) produces severe streaking artifacts. Recently, deep learning approaches using large receptive field neural networks such as U-Net have demonstrated impressive performance for sparse- view CT reconstruction. However, theoretical justification is still lacking. Inspired by the recent theory of deep convolutional framelets, the main goal of this paper is, therefore, to reveal the limitation of U-Net and propose new multi-resolution deep learning schemes. In particular, we show that the alternative U- Net variants such as dual frame and the tight frame U-Nets satisfy the so-called frame condition which make them better for effective recovery of high frequency edges in sparse view- CT. Using extensive experiments with real patient data set, we demonstrate that the new network architectures provide better reconstruction performance.

I. INTRODUCTION

Sparse-view CT reduces radiation dose by using fewer projection views, but insufficient views cause severe FBP streaking artifacts. The paper uses deep convolutional framelets to diagnose limitations of U-Net and propose frame-condition-satisfying variants.

  • Motivation: Sparse-view CT reduces radiation dose by reducing the number of projection views.The approach is relevant to applications including spectral CT, dynamic beam blockers, C-arm CT, and dental CT.
  • Problem: Insufficient projection views produce severe streaking artifacts in FBP reconstruction.Compressed sensing alternatives are computationally expensive because iterative updates repeatedly apply projection and back-projection.
  • Related Work: Deep learning has achieved strong performance across imaging tasks, including CT denoising and related low-dose CT applications.Prior work reported that directional-wavelet CNNs efficiently remove low-dose CT noise.
  • Theory: Deep convolutional framelets provide a mathematical framework for understanding deep learning approaches to inverse problems.The framework extends convolution framelets, which represent signals using fixed non-local bases convolved with data-driven local bases.
  • Limitation: Existing U-Net does not satisfy the frame condition and overemphasizes low-frequency components, producing blurry artifacts in sparse-view CT.The paper therefore investigates architectures designed to satisfy the frame condition.
  • Contribution: Dual frame and tight frame U-Nets are proposed as architectures satisfying the frame condition for improved high-frequency recovery.The dual frame variant adds a low-resolution bypass connection, while the tight frame variant uses an additional high-frequency path.

B. Frame

The frame section defines frame bounds and frame operators, then connects frame-based recovery and noise amplification to convolutional and Hankel-matrix representations. These concepts motivate learning-based signal representations for inverse problems.

  • Frame Definition: A family of functions is a frame when it satisfies lower and upper bounded representation inequalities.The positive constants α and β are the frame bounds; equal bounds define a tight frame.
  • Frame Operator: The frame operator is constructed from the frame elements and determines the associated frame representation.The supplied passage introduces the operator through the functions φk and its matrix form.
  • Frame Bounds: Frame bounds can be characterized using the minimum and maximum singular values of the frame operator.These singular values determine the lower and upper conditioning of the representation.
  • Signal Recovery: When the lower frame bound is non-zero, the original signal can be recovered from frame coefficients using a dual frame satisfying the frame condition.The recovered signal is reconstructed from coefficients c = Φ⊤f through the dual frame.
  • Noise Robustness: Noise in frame coefficients propagates through the dual frame, with amplification governed by the frame conditioning.A tight frame has minimum noise amplification, with β/α = 1.
  • Hankel Matrix: Hankel matrices support the theory because they often have low-rank structure related to Fourier-domain sparsity.CNN convolutions can be represented with Hankel-matrix operations, including single- and multi-channel cases.

D. Deep Convolutional Framelets: A Review

Deep convolutional framelets model CNN representations through learned local convolutional bases and user-defined non-local pooling bases. The framework links residual learning and low-rank Hankel structure while requiring a frame condition for recovery and controllable shrinkage.

  • Residual learning: After training, inference applies the learned network to produce a denoised solution associated with a rank-r Hankel-structured matrix.The framework connects convolution operations with Hankel matrices and uses low-rank structure as the representation constraint.
  • Residual learning: Residual learning trains filters to approximately annihilate the artifact-free signal while retaining the streaking artifact signal.For sparse-view CT, the input is modeled as artifact-contaminated and the network learns a representation associated with the residual.
  • Multi-layer extension: The multi-layer extension recursively narrows the search space of convolutional frames across layers.Layer-specific filter lengths and channel counts define the recursive encoder-decoder construction.
  • Framelet framework: Deep convolutional framelets represent signals using fixed non-local bases convolved with data-driven local bases.The non-local bases correspond to generalized pooling and unpooling, while local bases act as learnable convolutional filters.
  • Framelet framework: The frame condition on the non-local bases is the prerequisite for recovery conditions and controllable shrinkage behavior.This condition guides the construction of the U-Net variants studied later.

A. U-Net for Sparse-View CT and Its Limitations

U-Net’s pooling and unpooling provide a large receptive field suited to globally distributed sparse-view CT streaks, but its extended pooling operator fails the frame condition. This failure overemphasizes low-frequency components and can produce blurring artifacts.

  • A. U-Net for Sparse-View CT and Its Limitations: 48 projection views produce streaking artifacts across the image area, motivating convolutional filters with image-wide receptive fields.The artifact distribution is illustrated through reconstruction and artifact-only images.
  • A. U-Net for Sparse-View CT and Its Limitations: Pooling and unpooling enlarge the effective receptive field relative to a single-resolution CNN with the same convolutional filter size.The resulting multi-resolution architecture is suited to globally distributed sparse-view CT streaks.
  • A. U-Net for Sparse-View CT and Its Limitations: The standard U-Net filters the input with local convolutions, pools it to a half-size approximation, and uses bypass connections to compensate for lost high-frequency detail.The bypass and low-pass subband form the convolutional framelet coefficients.
  • A. U-Net for Sparse-View CT and Its Limitations: The extended pooling operator in standard U-Net does not satisfy the frame condition, resulting in artifacts and overemphasis of low-frequency components.The paper associates this behavior with blurring artifacts in sparse-view CT reconstructions.
  • A. U-Net for Sparse-View CT and Its Limitations: The dual frame U-Net adds a low-resolution residual bypass and thereby satisfies the frame condition, but its condition number introduces noise amplification.The reported condition number of I + ΦΦ⊤ = I + PR(Φ) is 2.
  • A. U-Net for Sparse-View CT and Its Limitations: The tight frame U-Net adds high-frequency subband paths and bypasses each subband to individual concatenation layers.The architecture uses an orthogonal wavelet frame, with Haar wavelets providing a tight frame.

B. Dual Frame U-Net

The dual frame U-Net adds a low-resolution bypass connection so the residual signal can be upsampled, satisfying the frame condition. This modification has a noise amplification factor of 2 and corresponds to the dual-frame reconstruction structure.

  • A dual frame provides a simple fix for the frame-condition limitation of the standard U-Net.
  • The dual frame reconstructs framelet coefficients using the dual frame associated with Φext.
  • The dual frame U-Net adds a bypass connection that upsamples the low-resolution residual signal through unpooling.This design is illustrated as an additional low-resolution bypass connection.
  • The added connection makes the network satisfy the frame condition while introducing a noise amplification factor of 2.The factor follows from the condition number of I + ΦΦ⊤ = I + PR(Φ).
  • After concatenation, multi-channel convolution applies the inverse Hankel operation to processed framelet coefficients multiplied by the local basis.

C. Tight Frame U-Net

The tight frame U-Net uses a tight filter-bank frame, with Haar wavelets providing low- and high-pass subbands. Unlike the standard U-Net, it adds a high-pass branch while bypassing each subband to concatenation layers.

  • The tight frame U-Net replaces the non-local basis with a tight filter bank whose subband operators form the frame representation.The filter bank is assumed to be tight, and the resulting extended basis is also tight.
  • Haar wavelets decompose the signal into low-pass and high-pass subbands, with the low-pass branch equivalent to average pooling.
  • The Haar wavelet frame is tight, and its two-dimensional implementation is used in the tight frame U-Net.
  • Unlike the standard U-Net, the tight frame U-Net includes an additional high-pass branch.
  • Each subband signal is bypassed to individual concatenation layers, where subsequent convolution can process the concatenated representation.
  • The architectures are evaluated for sparse-view CT because globally distributed streaking artifacts require multi-scale deep networks.

A. Data Set

The study uses synthetic sparse-view CT data derived from ten AAPM patient datasets, with separate training, validation, and test patients. Networks are trained on image patches and evaluated using PSNR and SSIM.

  • Ten patient datasets from the AAPM Low Dose CT Grand Challenge were used to construct the training data.Synthetic projection data were generated by re-projecting reconstructed images with the MATLAB Radon operator.
  • Artifact-free reference images used all 720 projection views, while sparse-view inputs used 60, 90, 120, 180, 240, and 360 views.
  • Eight patients were used for training, one for validation, and one for testing, yielding 3720 training, 254 validation, and 486 test slices.All slices were 512 × 512 images, and training data were augmented by horizontal and vertical flipping.
  • Reconstruction quality is assessed using PSNR and SSIM alongside the stated image-reconstruction setup.The passages define both metrics for comparing reconstructed and ground-truth images.
  • The networks use convolution, batch normalization, ReLU, and contracting-path concatenation, with 3 × 3 intermediate kernels and a 1 × 1 final kernel.
  • Training uses SGD for 150 epochs, with λ = 10^-4, learning rates from 10^-3 to 10^-5, and 256 × 256 image patches.Inference uses 512 × 512 inputs because the convolution filters are spatially invariant.

V. EXPERIMENTAL RESULTS

Experiments show that tight frame U-Net generally improves sparse-view CT reconstruction over standard and single-scale networks, while also outperforming TV reconstruction with substantially lower computational time.

  • Tight frame U-Net produced the best PSNR among the evaluated U-Net variants, while dual frame U-Net was best at the x2 downsampling factor.Within the body ROI, tight frame U-Net ranked first and dual frame U-Net ranked second.
  • Tight frame U-Net improved visual quality and achieved lower NMSE than standard U-Net, whose reconstructions often showed blurred edges.Dual frame and tight frame U-Nets enhanced high-frequency image characteristics in enlarged and difference views.
  • Tight frame U-Net consistently outperformed TV across view down-sampling factors in average PSNR and SSIM.Figures 6 and 7 also reported reduced streaking, preserved details, and much smaller NMSE values for the proposed method.
  • The proposed method required 250 ms per slice with GPU and 5 seconds per slice with CPU, compared with 20–50 seconds for TV on CPU.The reported timing corresponds to a 4–10 times speed advantage over TV.
  • Single-scale vs. Multi-scale residual learning: Tight frame U-Net consistently exceeded the single-scale network in PSNR and SSIM for every view down-sampling factor.The paper attributes this difference to the single-scale network’s smaller receptive field, which makes global streaking-artifact correction difficult.

B. Diversity of training set

Training the tight frame U-Net with FBP data from 60, 120, and 240 projection views provided the best reconstruction across a wide range of view downsampling factors. Experiments also assessed lesion detection and implementation trade-offs.

  • Diversity of training set: Combined training on FBP data from 60, 120, and 240 projection views offered the best reconstruction across wide ranges of view downsampling.Training on only 60 or 240 views was competitive at the corresponding view count, but combined training was selected for all downsampling factors.
  • Lesion detection: The AAPM Challenge dataset contains full- and quarter-dose CT images, but its original task targeted tube-current-modulated low-dose noise rather than sparse-view CT.The paper therefore generated projection data from quarter-dose images to test streaking-artifact removal.
  • Lesion detection: The proposed tight frame U-Net clearly detected lesion regions from 180-view quarter-dose projection data, unlike non-local means and AAPM-Net.The method also removed low-dose CT noise and produced clear images.
  • D. Max Pooling: Max-pooling variants require tracking all b_i,i+1 values at each pooling step, which increases memory requirements.The analysis therefore focuses mainly on average pooling and its corresponding high-pass branch.
  • VII. CONCLUSION: Extensive experiments showed that the dual frame and tight frame U-Net variants performed better than conventional U-Net for sparse-view CT reconstruction.The conclusion attributes the tight frame design’s robustness and directional processing to additional paths for subband signals.
Loading 1708.08333v3…