PaperlayerView exact revision on arXiv

Source-linked AI summary

Convolutional Neural Networks for Non-iterative Reconstruction of Compressively Sensed Images

Suhas Lohit, Kuldeep Kulkarni, Ronan Kerviche, Pavan Turaga, Amit Ashok

arXiv:1708.04669v2cs.CV

TL;DR

Iterative compressive-sensing reconstruction can be computationally expensive and ineffective at low measurement rates. ReconNet learns a non-iterative mapping from block-wise measurements to image blocks; the paper reports faster reconstruction and improved quality in several tested settings.

  • Problem

    Traditional compressive-sensing recovery is computationally expensive and ineffective at low measurement rates, limiting its use where reconstruction must be fast.

    Source: §1 · p. 1

  • Method

    ReconNet is a data-driven network that learns an end-to-end mapping from block-wise compressive measurements to reconstructed image blocks.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    ReconNet is 3 orders of magnitude faster than conventional iterative approaches and produces superior-quality reconstructions across a range of measurement rates, including noisy and real sensor measurements.

    Source: Contributions · p. 2, §10 · p. 13

  • Takeaways & Limitations

    Reconstructions at measurement rates around 0.01 retain semantic content that the authors use for object tracking.

    Source: Contributions · p. 2

Abstract

from arXiv · show

Traditional algorithms for compressive sensing recovery are computationally expensive and are ineffective at low measurement rates. In this work, we propose a data driven non-iterative algorithm to overcome the shortcomings of earlier iterative algorithms. Our solution, ReconNet, is a deep neural network, whose parameters are learned end-to-end to map block-wise compressive measurements of the scene to the desired image blocks. Reconstruction of an image becomes a simple forward pass through the network and can be done in real-time. We show empirically that our algorithm yields reconstructions with higher PSNRs compared to iterative algorithms at low measurement rates and in presence of measurement noise. We also propose a variant of ReconNet which uses adversarial loss in order to further improve reconstruction quality. We discuss how adding a fully connected layer to the existing ReconNet architecture allows for jointly learning the measurement matrix and the reconstruction algorithm in a single network. Experiments on real data obtained from a block compressive imager show that our networks are robust to unseen sensor noise. Finally, through an experiment in object tracking, we show that even at very low measurement rates, reconstructions using our algorithm possess rich semantic content that can be used for high level inference.

1 INTRODUCTION

Compressive sensing can reduce sensing and bandwidth demands, but iterative reconstruction can be too slow and ineffective at low measurement rates. ReconNet uses an end-to-end learned network for block-wise reconstruction, with extensions for adversarial loss, joint measurement learning, and other settings.

1 INTRODUCTION

  • Compressive sensing integrates sampling and compression in hardware, reducing the number of measurements needed to reconstruct a signal and addressing sensor-cost, bandwidth, or power constraints.The introduction points to short- and medium-wave infrared imaging as settings where sensor costs can dominate system costs.Source: §1 · p. 1
  • Iterative reconstruction can take up to 5 minutes for a 256×256 image and performs poorly below a 0.1 measurement rate, motivating a real-time alternative.The introduction also notes that users may need to choose the sparsity level and sparsifying basis.Source: §1 · p. 1

Contributions

  • ReconNet learns an end-to-end mapping from compressive measurements of an image block to its reconstructed block.The paper describes adversarial-loss training as producing sharper reconstructions and higher PSNRs.Source: §1 · p. 2, Contributions · p. 2
  • ReconNet is non-iterative and 3 orders of magnitude faster than conventional iterative approaches, with reported superior PSNR at measurement rates of 0.1 and below and in measurement noise.The paper also reports high-quality reconstruction from real compressive measurements and semantic content sufficient for object tracking at measurement rates around 0.01.Source: Contributions · p. 2
  • The authors extend ReconNet to jointly learn the measurement matrix and reconstruction algorithm, and use circulant layers to reduce network complexity.They report that a 95% reduction in first-layer parameters keeps the PSNR drop within 2 dB across a range of measurement rates.Source: Contributions · p. 2
  • ReconNet is also evaluated on real sensor measurements and in object tracking, with the paper reporting robustness to arbitrary sensor noise and useful semantic content at very low measurement rates.The real-data network is trained on synthetic data; the tracking result is reported at a measurement rate of about 0.01.Source: Contributions · p. 2

2 BACKGROUND AND RELATED WORK

The section surveys compressive sensing, iterative recovery, CNNs for image prediction, GANs, and data-driven CS reconstruction. It frames the paper's approach against the limits of sparse recovery and the growing use of learned mappings for inverse problems.

2.1 Compressive Sensing

  • Compressive sensing acquires a small set of random projections and reconstructs the signal by solving an inverse problem; natural images are compressible in transform domains such as wavelets.The background states that recovery can be possible when the measurement count scales as O(s log(n/s)) under stated sparsity and measurement-matrix conditions.Source: §2.1 · p. 2

Iterative algorithms for reconstruction

  • Early CS methods used l1 minimization with transform-domain sparsity, but image compressibility rather than exact sparsity motivated more complex model-based and approximate-message-passing methods.Model-based approaches either project estimates onto structured-sparsity solution sets or use non-local regularization; approximate message passing refines estimates with a denoiser.Source: Iterative algorithms for reconstruction · p. 2

2.2 CNNs for per-pixel prediction tasks

  • CNNs learn nonlinear image features and have been applied to per-pixel prediction and inverse problems, including super-resolution; the paper treats CS reconstruction as a related inverse problem.The paper notes that CS measurements lack the spatial structure of images, motivating an architecture to map measurements to images.Source: §2.2 · p. 3

2.2.1 Generative Adversarial Networks

  • For inverse problems, GAN-based losses have been used to produce sharper results than Euclidean loss, which the section attributes to Euclidean loss's averaging effect.A GAN trains a generator and discriminator in alternating fashion, with the discriminator distinguishing real images from generated ones.Source: §2.2.1 · p. 3

2.3 Purely data-driven approaches to CS image and video reconstruction using deep learning

  • Prior learned CS-reconstruction work includes non-iterative autoencoder and convolutional methods, whole-image networks, residual connections, learned measurements, and CS video recovery.The cited discussion notes that independent block reconstruction does not use dependencies between different blocks; later work addresses this with whole-image measurements.Source: §2.3 · p. 3

3 RECONNET

ReconNet reconstructs an image by mapping compressive measurements of individual blocks to image patches, then denoising the assembled image. The section describes its CNN architecture, training data, and Euclidean and adversarial loss variants.

3 RECONNET

  • Each image is divided into blocks, reconstructed separately from their CS measurements, reassembled, and passed through a denoiser to reduce blocky artifacts.The figure specifies 33 × 33 blocks and BM3D as the post-processing denoiser.Source: §3 · pp. 3–4, Figure 1 · p. 4

3.1 ReconNet Unit Architecture

  • A fully connected layer maps the measurement vector to a 33 × 33 feature map, followed by ReconNet units of three convolutional layers with ReLU and zero-padding.Two units give six convolutional layers; the final layer outputs the reconstructed block.Source: §3.1 · p. 4

3.2 Training Data

  • Training uses 21,760 luminance patches of size 33 × 33 from 91 natural images, with simulated Gaussian CS measurements at rates 0.25, 0.10, 0.04, and 0.01.RGB reconstruction replicates the network across individual channels.Source: §3.2 · pp. 4–5

3.3.1 Euclidean Loss

  • The Euclidean-loss variant minimizes average squared reconstruction error between each network output and its ground-truth patch using mini-batch gradient descent with backpropagation.The paper selects between random and deterministic fully connected-layer initialization using validation loss.Source: §3.3.1 · p. 5

3.3.2 Euclidean + Adversarial Loss

  • The adversarial variant trains ReconNet as generator G alongside a discriminator that classifies real ground-truth blocks and fake reconstructed blocks.The generator combines Euclidean and adversarial losses; the authors use one ReconNet unit because reconstruction quality does not improve with more units and training becomes harder with the discriminator.Source: §3.3.2 · p. 5

4 SYNTHETIC EXPERIMENTS

In simulated CS experiments, ReconNet provides stronger reconstruction quality than iterative methods at low measurement rates, handles noisy measurements, and reconstructs images much faster. The results also show how adversarial loss affects visual quality and PSNR.

4.1 Reconstruction of simulated CS data

  • The simulated experiments compare two loss variants on 11 grayscale test images at measurement rates 0.25, 0.1, 0.04, and 0.01, with and without measurement noise.The iterative baselines use default parameters; BM3D is used to reduce block artifacts in reconstructions.Source: §4.1 · p. 6, §4 · p. 6
  • At measurement rates 0.25 and 0.10, adversarial loss raises PSNR by about 1 dB; at 0.04 and 0.01, PSNR decreases, while reconstructions remain sharper and preserve more detail.The comparison is between ReconNet (Euc + Adv) and ReconNet (Euc).Source: Reconstruction from noiseless CS… · p. 6
  • With Gaussian measurement noise at σ = 10, 20, and 30, ReconNet beats the other algorithms at high noise levels without training separate networks for each noise level.The same networks used in the noiseless experiments are tested at measurement rates 0.25, 0.10, and 0.04.Source: Performance in the presence of noise · p. 6
  • At measurement rates 0.1, 0.04, and 0.01, both ReconNet variants outperform TVAL3, NLR-CS, and D-AMP; mean PSNR drops 8.37 dB from MR 0.25 to 0.01, versus TVAL3’s 16.53 dB drop.At MR 0.1, TVAL3 has comparable PSNR but introduces undesirable artifacts.Source: Reconstruction from noiseless CS… · p. 6, Table 1 · p. 8

4.2 Gains in Time Complexity

  • For a 256×256 image, ReconNet takes about 0.02 seconds and is three orders of magnitude faster than traditional reconstruction algorithms.The reported GPU implementation is 145–390 times faster than TVAL3, 1,400–2,700 times faster than D-AMP, and 14,782–15,660 times faster than NLR-CS.Source: §4.2 · p. 7, Table 2 · p. 8

5 EFFICIENT TRAINING STRATEGY FOR NEW MEASUREMENT MATRIX

For a new measurement matrix, the authors adapt a pretrained network by training only its first fully connected layer while keeping the convolutional layers fixed. This reduces training overhead while maintaining comparable PSNR; fine-tuning the convolutional layers can improve reconstruction quality at additional training cost.

5 EFFICIENT TRAINING STRATEGY FOR NEW MEASUREMENT MATRIX

  • For a new measurement matrix, training only the first fully connected layer for 1,000 iterations—about 2 seconds on a Titan X GPU—keeps the pretrained convolutional layers fixed.The base network is pretrained for the same or a slightly higher measurement rate.Source: §5 · p. 9
  • Training only the fully connected layer has minimal computational overhead while maintaining PSNR comparable to the reported baseline results.The table summarizes mean PSNR without BM3D, training time, and the base network’s measurement rate.Source: Table 3 · p. 9, §5 · p. 9
  • Fine-tuning convolutional layers C1–C6 along with the fully connected layer can improve reconstruction quality, at the cost of more training time.Source: §5 · p. 9

6 LEARNING THE MEASUREMENT MATRIX

ReconNet can jointly learn a measurement matrix and reconstruction network by adding a linear fully connected encoder. At low measurement rates, learned measurement matrices improve PSNR over random Gaussian matrices, with further sharpening from adversarial loss.

6 LEARNING THE MEASUREMENT MATRIX

  • An added fully connected layer maps a 33 × 33 image patch to m measurements, and its learned weights form the measurement matrix while subsequent layers reconstruct the image block.Training pairs use the same image patch as input and desired output.Source: §6 · p. 9
  • The added encoder is constrained to a single linear layer because the single-pixel camera captures linear projections of the scene.Source: §6 · p. 9
  • At MR 0.10, 0.04, and 0.01, learning Φ improves PSNR over random Gaussian Φ by 2.83, 3.15, and 2.17 dB without adversarial loss, and by 3.25, 3.33, and 2.4 dB with it.The visual comparisons at MR 0.1 and 0.04 show more detail with learned Φ, with adversarial loss further sharpening reconstructions.Source: §6 · p. 9, Figure 6 · p. 9

7 RECONSTRUCTION OF REAL DATA FROM COMPRESSIVE IMAGER

ReconNet was tested on real block-compressive-imager measurements, using a scalable DMD-based optical setup. All four tested variants preserved more detail than iterative methods; learning the measurement matrix improved results, while adversarial loss produced sharper reconstructions.

7.1 Scalable Optical Compressive Imager Testbed

  • The testbed uses two imaging arms and a DMD to modulate light, with detector outputs combined into one measurement per block.DMD patterns are generated from rows of the projection matrix and scanned temporally, in parallel across blocks.Source: §7.1 · p. 10, Figure 7 · p. 10
  • Calibration dynamically identifies each block’s sensor mapping and measures blockwise bias and scaling non-uniformities.Known DMD patterns are displayed under uniform white illumination to locate sensor pixels associated with each block.Source: §7.1 · p. 10

7.2 Reconstruction experiments

  • At measurement rates 0.1 and 0.04, the experiment tests 8-bit Gaussian and learned matrices with Euclidean or Euclidean-plus-adversarial training.Networks trained on simulated measurements are applied directly to real measurements without further training.Source: §7.2 · p. 10
  • All four ReconNet variants preserve more detail than iterative methods on real measurements, with less quality degradation as measurement rate falls.Learning the measurement matrix significantly improves results, and adding adversarial loss produces sharper reconstructions.Source: §7.2 · pp. 10–11

8 REDUCING MEMORY FOOTPRINT WITH CIRCULANT LAYERS

Circulant layers replace ReconNet’s large first fully connected layer with a more compact parameterization. Multiple circulant layers can limit quality loss, though higher measurement rates may require them to avoid under-fitting.

8 REDUCING MEMORY FOOTPRINT WITH CIRCULANT LAYERS

  • For MR = 0.10 and 33 × 33 blocks, the first fully connected layer has 118701 weights, versus 22720 parameters across the remaining layers.A single circulant layer reduces first-layer parameters from MN to N; at MR = 0.10, that is a 99.1% reduction.Source: §8 · p. 11
  • Circulant layers constrain the weight matrix to be circulant, an alternative to the unconstrained first fully connected layer.The cited prior work reports reducing fully connected-layer complexity from O(d^2) to O(d) without performance loss.Source: §8 · p. 11
  • A circulant layer computes its output through circular convolution, equivalently using Fourier transforms and element-wise multiplication.The input is zero-padded to the output dimension before applying the circulant matrix.Source: §8 · p. 11
  • At higher measurement rates, one circulant layer can under-fit, so multiple layers are used and their feature maps are combined before the following convolution.The following convolutional filters expand to size 11 × 11 × γ for γ circulant layers.Source: §8 · p. 11
  • At most measurement rates, circulant layers reduce first-layer parameters by at least 95% while keeping test-set PSNR loss within 2 dB.The evaluation increases the number of circulant layers until the parameter-reduction target is met.Source: §8 · p. 11

9 REAL-TIME HIGH LEVEL VISION USING COMPRESSIVE IMAGERS

The paper evaluates ReconNet reconstructions as input to kernelized-correlation-filter tracking on 15 videos. Tracking remains effective at low measurement rates, although the effect of adversarial loss depends on measurement rate and matrix choice.

9 REAL-TIME HIGH LEVEL VISION USING COMPRESSIVE IMAGERS

  • The proof-of-concept simulates framewise block measurements on 15 tracking videos at MR = 0.01 and 0.10, then tracks as frames are reconstructed.The tracking algorithm uses its default values and is applied without a denoiser.Source: §9 · pp. 11–12
  • At a 20-pixel location-error threshold, ReconNet+KCF achieves 68.14%–77.46% average precision at MR = 0.01 and 79.49%–84.89% at MR = 0.10, versus 84.9% on original videos.The average precision curve summarizes tracking over 15 videos.Source: §9 · p. 12
  • Learning the measurement matrix boosts tracking performance by about 8 percentage points at MR = 0.01 and 5 points at MR = 0.10.This comparison is across variants of ReconNet+KCF without the denoiser.Source: §9 · p. 12
  • The adversarial-loss effect varies by setting: it lowers tracking performance by about 3 points in three settings but raises it by about 3 points at MR = 0.01 with a learned matrix.ReconNet+KCF runs at around 10 FPS for 480 × 720 frames and up to 56 FPS for 240 × 320 frames.Source: §9 · p. 12
  • Even at MR = 0.01, the reconstruction retains semantic content required for object tracking.The figure plots average precision against location-error threshold for ReconNet+KCF and original videos.Source: §9 · p. 11, Figure 9 · p. 13

10 CONCLUSION

ReconNet combines non-iterative CNN reconstruction with improvements from learned measurement matrices and adversarial training. The conclusion reports real-time tracking from compressive measurements even at MR = 0.01.

10 CONCLUSION

  • ReconNet is a CNN-based, non-iterative CS reconstruction algorithm that is three orders of magnitude faster than traditional iterative algorithms.The authors describe this speed as making reconstruction essentially real-time.Source: §10 · p. 13
  • Jointly learning the measurement matrix and using adversarial loss each improve reconstruction quality across a range of measurement rates.Source: §10 · p. 13
  • ReconNet+KCF enables efficient real-time tracking from CS measurements at a very low measurement rate of 0.01.The conclusion suggests image recognition could use a similar framework.Source: §10 · p. 13

APPENDIX A ADDITIONAL RESULTS

Appendix A adds PSNR results for the seven remaining test images and reconstruction examples across additional images and ReconNet variants.

APPENDIX A ADDITIONAL RESULTS

  • Table 6 reports PSNR values at various measurement rates for the seven test images not covered in Table 1.Table 1 covers four of the 11 test images.Source: Appendix A · p. 13
  • Figure 6 shows reconstructions from four algorithm variants for two test images.Source: Appendix A · p. 13
  • Figure 10 shows reconstructions for eight additional images.Source: Appendix A · p. 13
Loading 1708.04669v2…