Source-linked AI summary
Learning to Invert: Signal Recovery via Deep Convolutional Networks
Ali Mousavi, Richard G. Baraniuk
TL;DR
Compressive sensing must recover signals from limited measurements despite imperfect fixed-basis sparsity and slow conventional recovery. DeepInverse uses a trained deep convolutional network to learn both signal structure and an approximate inverse map. Experiments report recovery quality comparable to state-of-the-art methods with much faster runtime, while requiring intensive one-time offline training.
Problem
Compressive sensing faces real-world signals that are not exactly sparse in a fixed basis and recovery algorithms that converge too slowly for some real-time settings.
Method
DeepInverse trains a deep convolutional network on representative measurement-signal pairs to learn a signal representation and an inverse map approximating greedy or convex recovery.
Results
DeepInverse closely approximates state-of-the-art compressive sensing recovery solutions while running hundreds of times faster.
Takeaways & Limitations
The framework is attractive for sparse recovery applications requiring low-latency reconstruction and sensing matrices applied to entire signals.
Takeaways & Limitations
DeepInverse's recovery quality depends on undersampling ratio and model capacity: D-AMP can outperform the three-layer network at higher ratios, and larger networks remain future work.
Abstract
from arXiv · showhide
The promise of compressive sensing (CS) has been offset by two significant challenges. First, real-world data is not exactly sparse in a fixed basis. Second, current high-performance recovery algorithms are slow to converge, which limits CS to either non-real-time applications or scenarios where massive back-end computing is available. In this paper, we attack both of these challenges head-on by developing a new signal recovery framework we call {\em DeepInverse} that learns the inverse transformation from measurement vectors to signals using a {\em deep convolutional network}. When trained on a set of representative images, the network learns both a representation for the signals (addressing challenge one) and an inverse map approximating a greedy or convex recovery algorithm (addressing challenge two). Our experiments indicate that the DeepInverse network closely approximates the solution produced by state-of-the-art CS recovery algorithms yet is hundreds of times faster in run time. The tradeoff for the ultrafast run time is a computationally intensive, off-line training procedure typical to deep networks. However, the training needs to be completed only once, which makes the approach attractive for a host of sparse recovery problems.
1. INTRODUCTION
Compressive sensing recovers signals from undersampled measurements by exploiting structure, but fixed-basis sparsity is imperfect for real-world data and existing high-performance recovery algorithms are slow. DeepInverse addresses both challenges by learning a signal representation and an approximate recovery map with a deep convolutional network.
- Problem: Recovering x from y = Φx is generally ill-posed because undersampling requires signal structure that reduces dimensionality without losing information.In compressive sensing, this structure is represented through sparsity in a basis, with only K ≪ N coefficients nonzero.
- Challenges: Real-world data is not exactly sparse in a fixed basis, limiting the effectiveness of the standard compressive sensing formulation.Learned data-dependent dictionaries have been explored, but their redundancy degrades recovery performance.
- Challenges: High-performance recovery algorithms are slow to converge, restricting compressive sensing to non-real-time settings or cases with massive back-end computing.Existing recovery uses sparsity-regularized convex optimization or greedy algorithms.
- DeepInverse: DeepInverse learns the inverse transformation from measurement vectors to signals using a deep convolutional network trained on representative images.The network learns both a signal representation and an inverse map approximating greedy or convex recovery algorithms.
- Outcome: Hundreds of times faster in run time, DeepInverse closely approximates state-of-the-art compressive sensing recovery solutions.Its tradeoff is computationally intensive offline training completed only once.
2. PRIOR WORK
Prior deep-learning approaches to structured signal recovery used fully connected layers or blocky measurement matrices. DeepInverse differs by supporting arbitrary measurement matrices rather than only blockwise sensing.
- Earlier approaches: The first deep-learning approach used a stacked denoising autoencoder as an unsupervised feature learner for structured signal recovery.Its fully connected layers increase network size and training complexity as signal size grows, and can lead to overfitting.
- Earlier approaches: The stacked-autoencoder approach reconstructed signals by sensing and recovering smaller non-overlapping or overlapping blocks.This addresses dimensionality but makes the measurement matrix blocky, which is unrealistic in many applications.
- Subsequent approach: A subsequent method combined a fully connected layer with convolutional neural networks but also used a blocky measurement matrix.
- DeepInverse: DeepInverse works with arbitrary measurement matrices, unlike both prior approaches.
3. DEEP CONVOLUTIONAL NETWORKS PRIMER
Deep convolutional networks combine convolution, ReLU nonlinearities, and pooling, while sparse connectivity and shared weights make them suited to sparse recovery.
- Core layers: Convolutional layers apply learnable filters across the visual field to form feature maps.
- Core layers: ReLU layers introduce nonlinearity, and pooling layers downsample representations while providing translation invariance.
- Training: Backpropagation trains the network and fine-tunes the convolutional filters.
- Why DCNs fit recovery: Sparse neuron connectivity and shared weights across receptive fields increase learning speed relative to fully connected networks.
4. CONVOLUTIONAL NETWORKS FOR SIGNAL RECOVERY
DeepInverse maps undersampled measurements to signal estimates with a modified deep convolutional network trained on representative signal–measurement pairs. Its architecture uses a fixed adjoint-based dimensionality lift, avoids pooling, and learns the inverse mapping through supervised loss minimization.
- Framework: DeepInverse learns a nonlinear inverse map from measurement vectors y to signals x using a deep convolutional network trained on representative images.The network learns both a signal representation and an inverse map approximating greedy or convex recovery algorithms.
- Architecture: The network accepts measurements in R^M and produces signal estimates in R^N, typically with M < N.A fully connected linear layer performs the dimensionality increase from measurement space to signal space.
- Architecture: The initial fully connected layer implements the adjoint measurement operator Φ⊺ rather than learning its weights.This fixed layer boosts the input dimensionality from R^M to R^N.
- Architecture: DeepInverse removes max-pooling operations to preserve processing dimensionality in R^N and uses convolutional layers with ReLU nonlinearities.The experiments use one fixed fully connected layer and three convolutional layers.
- Convolutional processing: The signal proxy ˜x = Φ⊺y is processed as an n1 × n2 input, with n1 × n2 = N, through convolutional feature maps.The first-layer feature-map output applies convolution and bias, followed by ReLU and border-ignoring subsampling to restore the original signal size.
- Training objective: The model defines ˆx = M(y, Ω) and learns parameters Ω by minimizing mean squared error over the training data using backpropagation.The training pairs contain measurements and their corresponding original signals.
5. EXPERIMENTAL RESULTS
DeepInverse is evaluated against state-of-the-art compressive-sensing recovery algorithms across recovery quality, runtime, noise robustness, and training behavior. Results show comparable reconstruction quality with much lower runtime, while performance varies by undersampling ratio and the network’s capacity.
- Experimental setup: DeepInverse is compared with D-AMP, TV minimization, and P-AMP for signals recovered without block subdivision.The experiments use different undersampling ratios and exclude block-based methods designed specifically for subdivided recovery.
- Recovery quality: At low undersampling ratios such as 0.01, DeepInverse has better recovery performance, whereas D-AMP outperforms it as the ratio increases.Across undersampling ratios, no single method is uniformly best in reconstruction quality.
- Recovery quality: DeepInverse offers recovery probability and PSNR performance comparable to state-of-the-art CS recovery algorithms.Figure 4 also indicates that DeepInverse outperforms D-AMP for some test images.
- Runtime: DeepInverse has a runtime that is a tiny fraction of current algorithms, making it especially suitable for low-latency recovery.The runtime comparison is reported in Table 1 across sampling rates and algorithms.
- Noise robustness: For undersampling ratio 0.1 with 20 dB input noise, DeepInverse is more robust to noise than D-AMP.Table 3 reports average PSNR under added test-image noise and notes noise-folding after reconstruction.
- Training and limitations: After several training iterations at M/N = 0.1, DeepInverse starts to outperform TV minimization and P-AMP, while greater network depth is expected to improve recovery performance.The authors identify larger-capacity DeepInverse networks as future work and note that D-AMP performs better than a 3-layer DeepInverse in general.
6. CONCLUSIONS
The paper develops DeepInverse for sensing and recovering signals, learning structured representations from training data while approximating recovery algorithms efficiently.
- DeepInverse is a framework for sensing and recovering signals.
- The framework learns a structured representation from training data.
- DeepInverse approximates signal recovery at a small fraction of state-of-the-art algorithms' cost.