Source-linked AI summary
Rank Minimization for Snapshot Compressive Imaging
Yang Liu, Xin Yuan, Jinli Suo, David J. Brady, Qionghai Dai
TL;DR
SCI reconstruction quality remains too poor for wide application despite promising high-speed video and hyperspectral imaging results. This paper integrates SCI sensing with nonlocal self-similarity and rank minimization in a joint model solved by alternating minimization, achieving significant improvements over existing algorithms on simulation and real data.
Problem
Poor reconstructed-image quality limits SCI's wider applications, motivating algorithms that improve reconstruction quality.
Method
The paper formulates a joint SCI reconstruction model integrating compressive sampling with WNNM for video patch groups and solves it using alternating minimization.
Results
The proposed algorithm achieves significant improvements over existing algorithms across extensive simulation and real-data experiments.
Takeaways & Limitations
The algorithm demonstrates superiority on both simulation and real data for SCI reconstruction.
Takeaways & Limitations
Real hyperspectral reconstructions can exhibit oversmoothing, possibly due to system noise.
Abstract
from arXiv · showhide
Snapshot compressive imaging (SCI) refers to compressive imaging systems where multiple frames are mapped into a single measurement, with video compressive imaging and hyperspectral compressive imaging as two representative applications. Though exciting results of high-speed videos and hyperspectral images have been demonstrated, the poor reconstruction quality precludes SCI from wide applications.This paper aims to boost the reconstruction quality of SCI via exploiting the high-dimensional structure in the desired signal. We build a joint model to integrate the nonlocal self-similarity of video/hyperspectral frames and the rank minimization approach with the SCI sensing process. Following this, an alternating minimization algorithm is developed to solve this non-convex problem. We further investigate the special structure of the sampling process in SCI to tackle the computational workload and memory issues in SCI reconstruction. Both simulation and real data (captured by four different SCI cameras) results demonstrate that our proposed algorithm leads to significant improvements compared with current state-of-the-art algorithms. We hope our results will encourage the researchers and engineers to pursue further in compressive imaging for real applications.
1 INTRODUCTION
SCI maps high-dimensional video or hyperspectral data into single measurements, but reconstruction quality remains a barrier to wider application. The paper addresses this by integrating rank minimization, nonlocal self-similarity, and SCI-specific iterative reconstruction.
- Motivation: Poor reconstructed-image quality limits SCI's wider applications, motivating algorithms that improve reconstruction quality.
- Motivation: A single 256 × 256 measurement can encode a 148-frame 256 × 256 video clip, creating a highly underdetermined reconstruction problem.
- Motivation: The paper asks how to exploit high-dimensional structure and adapt image or video processing approaches to improve SCI reconstruction.
- Related work: Existing GMM-based SCI methods exploit video-patch sparsity but can produce blurry frames and artifacts, with Kobe reconstructions below 30dB PSNR.
- Contributions: DeSCI integrates SCI sampling with WNNM for video patch groups and solves the resulting joint model using an iterative projection-and-denoising framework.
- Contributions: DeSCI improves real-data reconstruction quality across CACTI, color-CACTI, CASSI, and high-speed stereo cameras, with reported robustness to noise.
2 REVIEW OF SNAPSHOT COMPRESSIVE IMAGING
SCI systems encode multiple temporal frames or spectral bands into one two-dimensional sensor measurement. The section reviews video and spectral implementations, their coding elements, and their broader compressive-imaging context.
- SCI systems: SCI encodes high-dimensional data onto a 2D sensor using temporal-variant masks for high-speed imaging and dispersion for spectral imaging.
- Validation scope: The proposed methods are validated on CACTI and CASSI, and adapting them to other coding strategies requires changing the sensing matrix.
- Video SCI: CACTI uses a temporal-variant mask and monochrome or color CCD to encode tens of temporal frames in one snapshot.
- Spectral SCI: CASSI spatially codes a spectral scene with a fixed mask, spectrally disperses it with a prism or grating, and detects tens of bands on a CCD.
3 MATHEMATICAL MODEL OF SNAPSHOT COM-
The SCI measurement model represents the desired signal through a structured sensing matrix rather than a dense random matrix. Its diagonal structure enables efficient computation, while the paper connects recovery performance to compression-code rate and distortion.
- Measurement model: SCI measurements are modeled as y = Φx + g, where Φ senses the desired signal x and g denotes measurement noise.
- Measurement model: Unlike traditional compressed sensing, SCI uses a structured sensing matrix built from diagonal matrices, as in CACTI and CASSI.
- Video SCI formulation: In CACTI, coded pixels from B high-speed frames are collapsed into one measurement pixel, giving a compressive sampling rate of 1/B.
- Computational structure: The SCI matrix satisfies ΦΦ^T being diagonal, a property used to derive an efficient algorithm for massive reconstruction data.
- Scope: Multiple-measurement hyperspectral systems are outside this paper's scope, although the algorithm could be adapted with minor modifications.
- Recovery theory: The recovery theory connects compression-code rate r and distortion δ with the number of frames B and reconstruction quality.
4 RANK MINIMIZATION FOR SIGNAL RECONSTRUC-
The paper integrates nonlocal patch self-similarity and weighted rank minimization into SCI reconstruction, then solves the resulting non-convex model with alternating updates. It exploits SCI’s structured sensing matrix to reduce computational and memory costs.
- 4.1 Rank minimization: Weighted nuclear norm minimization replaces nuclear norm minimization to reduce the degradation caused by convex relaxation under noise.For non-descending weights, the non-convex WNNM problem has a closed-form weighted soft-thresholding solution.
- 4.2 Integrating WNNM to SCI: Nonlocal similar patches are stacked into low-rank groups, providing the structural prior used for SCI reconstruction.Each group contains patches selected from a spatial-temporal search window, and its matrix is low rank because the patches have similar structures.
- 4.2 Integrating WNNM to SCI: The SCI-WNNM model combines a measurement-fidelity term with weighted nuclear-norm penalties on patch groups constructed from the signal.An auxiliary variable is introduced under ADMM, producing separate subproblems for measurement consistency and patch-group estimation.
- 4.3 Solving the SCI-WNNM problem: SCI’s structured sensing matrix makes ΦΦT diagonal, enabling element-wise updates and avoiding direct inversion of the large matrix ΦTΦ + γI.The matrix inversion formula converts the update into operations involving the measurement-space matrix I + Φγ^-1ΦT, whose structure can be handled efficiently.
- 4.3 Solving the SCI-WNNM problem: The patch-group update treats q as a noisy version of x and applies SVD-based weighted singular-value shrinkage before aggregating the recovered groups.The update uses q = x + e with zero-mean white Gaussian noise and computes ˆZi by weighted soft-thresholding of the singular values.
- 4.3 Solving the SCI-WNNM problem: The complete alternating reconstruction procedure, DeSCI, updates the measurement variable, patch groups, noise level, and ADMM multiplier across iterations.The algorithm initializes x = ΦTy, precomputes diagonal elements of ΦΦT, estimates patch groups through SVD and weighted shrinkage, and aggregates them into the recovered signal.
5 SIMULATION RESULTS
Simulation experiments evaluate DeSCI on video and hyperspectral SCI data, showing strong reconstruction quality, smoother frame-wise behavior, noise robustness, and the importance of WNNM denoising.
- Video compressive imaging results: DeSCI is evaluated on video SCI using Kobe, Traffic, Runner, and Drop datasets with eight 256 × 256 frames collapsed into one measurement.The simulations use shifting binary masks and compare DeSCI with GMM-TP, MMLE-GMM, MMLE-MFA, and GAP-TV.
- Video compressive imaging results: DeSCI exceeds the competing algorithms on all four datasets, with average gains up to 8.27dB in PSNR and 0.3211 in SSIM.The reported gains over GMM-TP, MMLE-GMM, MMLE-MFA, and GAP-TV are {7.47, 4.35, 7.54, 8.27}dB in PSNR and {0.3035, 0.2009, 0.3211, 0.0999} in SSIM, respectively.
- Video compressive imaging results: DeSCI smooths the usual PSNR and SSIM drop at the first and last reconstruction frames, although complex motion still lowers PSNR in Kobe’s final 16 frames.The smoother SSIM behavior is reported relative to the other algorithms in the Kobe and Traffic frame-wise comparisons.
- Robustness to noise: For noisy measurements, ADMM and GAP perform similarly without noise, while ADMM outperforms GAP in PSNR and SSIM as measurement SNR decreases.The experiments use a single Kobe measurement and recommend ADMM for realistic noisy systems.
- DeSCI with other denoising algorithms: DeSCI alternates projection through ADMM or GAP with WNNM denoising, and DeSCI-WNNM outperforms DeSCI-VBM4D by more than 2dB in PSNR and 0.025 in SSIM.DeSCI-VBM4D obtains PSNR values of {30.60, 26.60}dB and SSIM values of {0.9260, 0.8958} on Kobe and Traffic, respectively.
- Hyperspectral imaging results: DeSCI also reconstructs simulated hyperspectral data effectively, with reliable bird spectra correlations over 0.9999 and better toy-data spectra than GAP-TV.The bird data contain 24 spectral bands at 1021 × 703 pixels, while the toy data contain 31 bands at 512 × 512 pixels.
6 REAL DATA RESULTS
Real-data experiments across grayscale and color video plus hyperspectral imaging show that DeSCI improves reconstruction quality while reducing blur and measurement artifacts.
- DeSCI provides significantly better reconstruction results across real grayscale and color videos and hyperspectral images.The evaluation uses data from various SCI systems and applications.
- Grayscale high-speed video: DeSCI preserves fine details and sharp edges in chopper wheel reconstructions, whereas other leading algorithms suffer from motion blur artifacts.
- Grayscale high-speed video: DeSCI preserves sharp scene edges and background details in UCF video, while TwIST blurs details and obscures book characters.
- Color high-speed video: DeSCI produces smoother backgrounds and sharper hammer edges than the compared color-video methods, although some motion remains unresolved when aligned with the shifting mask.
- Hyperspectral image data: DeSCI preserves hyperspectral spectra and reduces compressed-measurement artifacts in real bird data, with 21 reconstructed bands shown after optical-calibration alignment.
- Hyperspectral image data: Some oversmoothing remains in DeSCI reconstructions of real bird hyperspectral data, possibly because of system noise.
- Hyperspectral image data: For real object hyperspectral data, DeSCI reconstructs clear, blur-free frames and avoids the fractional banana artifacts visible in TwIST and GAP-TV.
7 CONCLUDING REMARKS
The paper formulates DeSCI as a rank-minimization reconstruction framework for SCI and reports significant improvements on simulated and real data. It also identifies broader applications and deep-learning integration as future directions.
- DeSCI incorporates rank minimization into the SCI forward model and solves the resulting joint optimization problem with alternating minimization.
- The method exploits nonlocal self-similarity in video and hyperspectral frames and achieves significant improvements over existing algorithms.
- Extensive simulation and real-data experiments demonstrate the proposed algorithm’s reported superiority.
- The authors identify integrating the proposed framework with deep learning and applying it to other compressive imaging systems as future work.