PaperlayerView exact revision on arXiv

Source-linked AI summary

DR2-Net: Deep Residual Reconstruction Network for Image Compressive Sensing

Hantao Yao, Feng Dai, Dongming Zhang, Yike Ma, Shiliang Zhang, Yongdong Zhang, Qi Tian

arXiv:1702.05743v4cs.CV

TL;DR

Traditional iterative compressive-sensing reconstruction is computationally intensive. DR2-Net combines linear mapping with residual learning to reconstruct images quickly and improve quality, outperforming existing iterative and deep-learning methods across the reported measurement rates.

  • Problem

    Traditional image compressive-sensing methods rely on computationally intensive iterative optimization, creating a bottleneck for application.

  • Method

    DR2-Net uses a linear mapping network to produce a preliminary reconstruction, followed by residual learning blocks that infer and add corrective residuals.

  • Results

    At measurement rate 0.25, DR2-Net exceeds NLP-CS and ReconNet by 0.87 dB and 3.14 dB in PSNR, respectively.

  • Takeaways & Limitations

    DR2-Net outperforms traditional iterative and deep-learning-based methods in reconstruction speed and quality across the reported measurement rates.

Abstract

from arXiv · show

Most traditional algorithms for compressive sensing image reconstruction suffer from the intensive computation. Recently, deep learning-based reconstruction algorithms have been reported, which dramatically reduce the time complexity than iterative reconstruction algorithms. In this paper, we propose a novel \textbf{D}eep \textbf{R}esidual \textbf{R}econstruction Network (DR$^{2}$-Net) to reconstruct the image from its Compressively Sensed (CS) measurement. The DR$^{2}$-Net is proposed based on two observations: 1) linear mapping could reconstruct a high-quality preliminary image, and 2) residual learning could further improve the reconstruction quality. Accordingly, DR$^{2}$-Net consists of two components, \emph{i.e.,} linear mapping network and residual network, respectively. Specifically, the fully-connected layer in neural network implements the linear mapping network. We then expand the linear mapping network to DR$^{2}$-Net by adding several residual learning blocks to enhance the preliminary image. Extensive experiments demonstrate that the DR$^{2}$-Net outperforms traditional iterative methods and recent deep learning-based methods by large margins at measurement rates 0.01, 0.04, 0.1, and 0.25, respectively. The code of DR$^{2}$-Net has been released on: https://github.com/coldrainyht/caffe\_dr2

I. INTRODUCTION

Compressive sensing reduces measurement requirements but traditional image reconstruction is computationally intensive. DR2-Net combines linear mapping with residual learning to reconstruct CS images quickly while improving quality, and experiments report gains over existing methods.

  • Compressive sensing acquires images from far fewer measurements than Nyquist theory requires, supporting applications such as cameras and medical scanners.
  • Iterative optimization in traditional compressive-sensing reconstruction creates an intensive-computation bottleneck for image applications.
  • DR2-Net takes CS measurements as input, first producing a preliminary reconstruction with linear mapping, then refining it through residual estimation.
  • Extensive experiments report that DR2-Net outperforms both iterative and deep learning-based methods.
  • The linear mapping stage provides a reasonably good reconstruction with faster speed and lower computational cost, suiting resource- and bandwidth-limited scenarios.
  • Residual learning further improves reconstruction quality and can be independently fused with existing deep learning-based methods.

II. RELATED WORK

Related work spans iterative compressive-sensing reconstruction, deep learning-based recovery, and residual networks. DR2-Net differs from ReconNet by explicitly using linear mapping for preliminary recovery followed by residual refinement.

  • Traditional compressive-sensing recovery uses sparsity priors and iterative optimization to reconstruct signals from underdetermined measurements.
  • Deep learning methods such as ReconNet and SDA provide non-iterative reconstruction and are reported to be about 3 orders of magnitude faster than traditional algorithms.
  • Residual networks use identity shortcut connections to reduce signal attenuation across stacked nonlinear transformations.
  • DR2-Net differs from ReconNet in motivation and structure by treating linear mapping as suitable for CS and training residual refinement after preliminary reconstruction.

III. DEEP RESIDUAL RECONSTRUCTION NETWORK

DR2-Net maps CS measurements of image patches to preliminary reconstructions and then estimates residuals to produce refined outputs. Its linear mapping network is implemented with a fully connected layer trained from paired measurements and source images.

  • DR2-Net maps CS measurements of 33 × 33 image patches to reconstructed 33 × 33 images using linear mapping and residual subnetworks.
  • A. Linear Mapping: The linear mapping subnetwork generates a preliminary image, while the residual subnetwork estimates the difference between that image and the ground truth.
  • A. Linear Mapping: The mapping matrix is estimated from paired training measurements and source signals by minimizing reconstruction error.
  • A. Linear Mapping: A fully connected layer simulates the linear mapping function and learns the mapping matrix through its training loss.
  • A. Linear Mapping: The linear mapping network contains one fully connected layer with 1,089 neurons and produces a preliminary reconstruction for a given CS measurement.

B. Residual Learning

DR2-Net first obtains an approximate reconstruction through linear mapping, then estimates and fuses a residual to narrow the gap to the source image.

  • Linear mapping produces an approximate source-image reconstruction because obtaining an accurate direct solution is difficult.
  • Residual learning estimates the difference between the ground-truth image and the preliminary reconstruction.
  • The estimated residual is fused with the linear-mapping output to produce the final reconstructed image.
  • The residual network takes the preliminary image as input and generates an estimated residual using parameters Wr.
  • DR2-Net is trained with SGD to optimize the parameters Wf and Wr.

C. DR2-Net Architecture

DR2-Net reconstructs each image block by mapping CS measurements to a preliminary image, then refining it with a residual network composed of stacked residual learning blocks.

  • The linear mapping network converts CS measurements into a 33 × 33 preliminary image through one fully connected layer with 1089 neurons.
  • The residual network receives the preliminary image and outputs a 33 × 33 image through four residual learning blocks.
  • Each residual block contains three convolutional layers with 11 × 11, 1 × 1, and 7 × 7 kernels.

D. Reconstruction Procedure

The reconstruction procedure applies DR2-Net independently to non-overlapping 33 × 33 patches, reassembles the reconstructed patches, and uses BM3D to remove block-processing artifacts.

  • The image is divided into non-overlapping 33 × 33 patches, whose CS measurements are reconstructed into image patches by DR2-Net.
  • The reconstructed patches are composed into an intermediate reconstructed image before post-processing.
  • BM3D is finally applied to remove artifacts caused by block-wise processing.

IV. DR2-NET TRAINING

DR2-Net training uses multiscale image patches and CS measurements, with separate optimization stages for the linear mapping and residual components.

  • 86,656 training patches are sampled from 273 images resized to scales 0.75, 1, and 1.5.
  • Each training pair consists of a random-Gaussian CS measurement of a luminance patch and that patch as the ground-truth label.
  • The first training step optimizes Ff with a learning rate of 0.001 for up to 1,000,000 iterations using a step strategy.
  • The second step trains DR2-Net end-to-end for up to 100,000 iterations at a fixed learning rate of 0.00001 while updating Wf and Wr.
  • The experiments use Caffe, Gaussian weight initialization, and batch size 128, with ImageNet Val used only for Sect V-G.

B. Comparison with Existing Methods

DR2-Net achieves the highest mean PSNR across four measurement rates and compares favorably with both iterative and deep learning-based methods. Its linear mapping stage also provides a fast, reasonably good preliminary reconstruction.

  • At measurement rate 0.25, DR2-Net exceeds NLR-CS by 0.87 dB and ReconNet by 3.14 dB in PSNR.
  • The fc1089 linear mapping network outperforms ReconNet in PSNR and testing loss at measurement rates 0.25, 0.10, and 0.04.At measurement rate 0.01, fc1089 achieves 17.26 dB versus ReconNet's 17.27 dB, while its testing loss is 5.42 versus 5.307.
  • fc1089 reconstructs a single image ten times faster than ReconNet while producing a reasonably good preliminary reconstruction.

D. Evaluation on Residual Learning

Residual learning improves the preliminary reconstruction, and the training analysis indicates that the residual component is effectively optimized. Four residual blocks are selected because additional blocks yield diminishing gains.

  • More than 95% of residual-learning activation values lie between -0.05 and 0.05.The small residual values follow the preliminary reconstruction produced by the linear mapping network.
  • DR2-Net loss decreases during training while the linear mapping network loss remains stable, indicating effective optimization of residual learning.DR2-Net loss combines the linear mapping and residual-learning losses.
  • Adding one residual block improves fc1089 PSNR by 0.09 dB, 0.54 dB, 0.92 dB, and 1.6 dB at measurement rates 0.01, 0.04, 0.10, and 0.25.
  • Four residual blocks achieve the highest performance at measurement rates 0.04, 0.10, and 0.25, while further additions provide relatively smaller improvements.At measurement rate 0.25, fc1089-Res4 improves over fc1089-Res1 by 0.25 dB, compared with 1.6 dB for fc1089-Res1 over fc1089.
  • The selected DR2-Net architecture is fc1089-Res4, combining the linear mapping network with four residual blocks.

E. Time Complexity

DR2-Net is evaluated for reconstruction speed and robustness to measurement noise. Its linear mapping is fastest, while residual-enhanced variants improve performance but require more computation; under intense noise, both compared networks degrade.

  • Comparison with deep learning methods: Deep learning-based methods are compared because their time complexity is substantially lower than traditional iterative reconstruction algorithms.The section compares DR2-Net with other deep learning-based methods rather than repeating the established speed advantage over iterative approaches.
  • DR2-Net structures: 3–4ms is sufficient for fc1089 to reconstruct a 256 × 256 image, making it the fastest of the five tested DR2-Net structures.fc1089 contains only one fully connected layer; deeper networks take longer.
  • DR2-Net structures: 30 times slower than fc1089, the deepest fc1089-Res4 adds one fully connected layer and 12 convolutional layers.Runtime increases linearly with network depth in the tested structures.
  • Comparison with deep learning methods: fc1089-Res1 is faster and achieves higher performance than ReconNet, whereas the three deeper DR2-Net variants are slower than ReconNet.fc1089 also spends less time than SDA and ReconNet while achieving comparable performance.
  • Robustness to noise: DR2-Net outperforms ReconNet at σ = 0.01, 0.05, and 0.1 across four measurement rates, but both perform poorly at σ = 0.25 and 0.5.The evaluation adds five Gaussian-noise levels to CS measurements without applying BM3D denoising.

G. Scalability on Large-scale Dataset

DR2-Net scales to ImageNet Val evaluation and outperforms ReconNet across four measurement rates. The experiments also show that reconstruction quality depends on image texture complexity, with smoother images generally reconstructed more successfully.

  • Scalability on Large-scale Dataset: ImageNet Val contains 50,000 images from 1,000 classes and is used to test DR2-Net scalability.Both DR2-Net and ReconNet are trained on the standard 91 images used in the referenced setup.
  • Scalability on Large-scale Dataset: DR2-Net achieves better performance than ReconNet at all four measurement rates on ImageNet Val.The advantage is especially large at higher measurement rates.
  • Reconstruction quality and texture: At MR 0.10, DR2-Net constantly outperforms ReconNet, while images with richer textures generally have lower reconstruction quality.The reported observation links reconstruction quality patterns to texture complexity for both methods.
  • Method context: DR2-Net combines a linear mapping network with residual learning blocks that enhance the preliminary reconstruction from CS measurements.The fully connected layer produces the preliminary image, and residual blocks infer its correction relative to the ground truth.
Loading 1702.05743v4…