Source-linked AI summary
Deep Networks for Compressed Image Sensing
Wuzhen Shi, Feng Jiang, Shengping Zhang, Debin Zhao
TL;DR
Compressed sensing still faces difficult sampling-design and nonlinear-reconstruction problems. The paper addresses both with a deep network whose stages imitate traditional compressed sensing reconstruction, and reports significant performance improvements over current methods while running in real time.
Problem
Compressed sensing needs effective sampling-operator design and fast nonlinear reconstruction for recovering signals from limited measurements.
Method
The paper builds a three-stage deep network for compressed sampling, initial reconstruction, and deep reconstruction, with learned sampling and architecture informed by traditional block compressed sensing.
Results
CSNet achieves significant performance improvements over several current state-of-the-art methods and runs in real time.
Takeaways & Limitations
Learning the sampling operator within an end-to-end reconstruction network provides a practical compressed sensing approach with strong quality and speed.
Abstract
from arXiv · showhide
The compressed sensing (CS) theory has been successfully applied to image compression in the past few years as most image signals are sparse in a certain domain. Several CS reconstruction models have been recently proposed and obtained superior performance. However, there still exist two important challenges within the CS theory. The first one is how to design a sampling mechanism to achieve an optimal sampling efficiency, and the second one is how to perform the reconstruction to get the highest quality to achieve an optimal signal recovery. In this paper, we try to deal with these two problems with a deep network. First of all, we train a sampling matrix via the network training instead of using a traditional manually designed one, which is much appropriate for our deep network based reconstruct process. Then, we propose a deep network to recover the image, which imitates traditional compressed sensing reconstruction processes. Experimental results demonstrate that our deep networks based CS reconstruction method offers a very significant quality improvement compared against state of the art ones.
1. INTRODUCTION
The paper addresses compressed sensing’s two central challenges—learning an effective sampling operator and developing fast nonlinear reconstruction—through a deep network modeled on traditional reconstruction stages. Its learned sampling and end-to-end reconstruction achieve strong quality and speed relative to existing methods.
- 1. INTRODUCTION: CSNet is reported to achieve state-of-the-art reconstruction quality with high, competitive speed compared with existing compressed sensing methods.The comparison is based on Set5 results at a 0.1 sampling ratio, as summarized in Table 1.
- 1. INTRODUCTION: CSNet targets both sampling-operator design and fast nonlinear reconstruction, the two principal challenges identified for compressed sensing.The paper frames these challenges within an ill-posed recovery problem when measurements are fewer than signal dimensions.
- 1. INTRODUCTION: The network mirrors block compressed sensing smooth projected Landweber through compressed sampling, initial reconstruction, and nonlinear deep reconstruction sub-networks.This relationship provides the design basis for the proposed architecture.
- 1. INTRODUCTION: A convolutional sampling layer learns the sampling operator automatically, avoiding complicated artificial matrix designs.Prior approaches use random, definite, or signal-specific matrices, each with stated trade-offs in computation, implementation, or reconstruction quality.
- 1. INTRODUCTION: The convolutional network learns an end-to-end mapping from compressed measurements to target images while aiming for good reconstruction quality and fast speed.The proposed network includes compressed sampling, initial reconstruction, and deep reconstruction stages.
2. RELATED WORK
Projection-based compressed sensing methods reconstruct signals through successive projection and thresholding, while BCS combines block sampling with MMSE initialization and nonlinear reconstruction.
- Projection-based methods obtain reconstruction outputs by successively projecting and thresholding.
- The proposed CSNet is compared with traditional BCS through corresponding sampling, initial reconstruction, and nonlinear reconstruction stages.
- BCS divides images into B × B blocks, samples each block with a measurement matrix, and uses MMSE estimation for the initial reconstruction.
- Several improved BCS-SPL methods, including MC-BCS-SPL, MS-BCS-SPL, and MH-BCS-SPL, have been proposed.
3. PROPOSED DEEP NETWORK FOR COMPRESSED SENSING RECONSTRUCTION
CSNet learns compressed sampling and reconstruction jointly through sub-networks corresponding to traditional BCS stages. Its learned sampling, initial reconstruction, and deep nonlinear reconstruction components form an end-to-end trainable network.
- Compressed Sampling Sub-network: The sampling sub-network uses convolution filters as learned measurement matrices for non-overlapping B × B block sampling.
- Initial Reconstruction Sub-network: The initial reconstruction sub-network learns convolution filters instead of computing the MMSE linear estimator, then reshapes and concatenates blocks into an image.
- Deep Reconstruction Sub-network: The deep reconstruction sub-network applies a multilayer convolutional network to perform nonlinear reconstruction from the initial output.
- CSNet combines compressed sampling, initial reconstruction, and deep reconstruction sub-networks into an end-to-end network.
- Training: The sampling and reconstruction sub-networks are trained jointly with mean square error as the optimization objective.
4. EXPERIMENTAL RESULTS
CSNet is evaluated on standard datasets against five representative recovery methods using multiple sampling ratios, with quantitative and qualitative comparisons. It achieves the best reconstruction quality and fastest runtime in the reported experiments.
- 4.1. Datasets: The experiments use BSDS500 for training and validation, with Set5 and Set14 serving as benchmark test datasets.Training uses 200 BSDS500 training images, 200 test images, and 100 validation images; Set5 and Set14 are used for benchmarking.
- 4.3. Comparisons with State-of-the-Art Methods: Visual comparisons include Set5 image baby at sampling ratio 0.1 and image butterfly at sampling ratio 0.3.These examples provide qualitative CS recovery comparisons under two sampling conditions.
- 4.3. Comparisons with State-of-the-Art Methods: CSNet obtains the best PSNR and SSIM performance while also running fastest across the reported comparisons.The evaluation compares CSNet with DWT, TV, MH, CoS, and GSR on Set5 and Set14 across sampling ratios from 0.1 to 0.5.
- 4.3. Comparisons with State-of-the-Art Methods: On Set14, CSNet improves average PSNR over DWT, TV, MH, CoS, and GSR by approximately 3.91, 3.71, 2.60, 1.72, and 0.37 dB, respectively.Table 2 reports average PSNR, SSIM, and running-time comparisons on the 14-image Set14 benchmark.
- 4.3. Comparisons with State-of-the-Art Methods: On Set5, CSNet improves average PSNR over GSR by approximately 2.31, 1.46, 1.07, 1.08, and 0.31 dB at sampling ratios from 0.1 to 0.5.Table 1 covers PSNR, SSIM, and running-time comparisons on Set5.
5. CONCLUSION
The paper presents a deep network that learns compressed sampling and reconstructs images through sub-networks related to traditional CS reconstruction. CSNet reports significant performance improvements over state-of-the-art methods and real-time operation.
- 5. CONCLUSION: CSNet combines compressed sampling, initial reconstruction, and deep reconstruction sub-networks to address both major CS challenges.Its structure is related to the traditional block compressed sensing smooth projected Landweber algorithm.
- 5. CONCLUSION: The learned sampling sub-network avoids complicated manual sampling-operator design, while the reconstruction sub-network efficiently recovers the original image.The network is trained to learn the sampling operator automatically and maps measurements to reconstructed images.
- 5. CONCLUSION: CSNet achieves significant performance improvements over several state-of-the-art methods and runs in real time.The authors identify residual learning as a direction for further improving reconstruction performance and running time.