Source-linked AI summary
Multispectral imaging using a single bucket detector
Liheng Bian, Jinli Suo, Guohai Situ, Ziwei Li, Feng Chen, Qionghai Dai
TL;DR
Existing multispectral imagers are photon-inefficient, spectrum-range limited, bulky, and expensive, while single-bucket approaches struggle to distinguish spectra without added projections. MSPI uses temporal sinusoidal spectral modulation within each spatial pattern and Fourier decomposition plus compressive sensing to reconstruct multispectral data without additional acquisition time, demonstrating 64×64×10-band visible imaging in about 1 minute.
Problem
Existing multispectral imagers detect spatial or spectral components separately with array detectors, limiting photon efficiency and spectrum range while increasing cost and bulk.
Method
MSPI multiplexes spectral information through temporal sinusoidal modulation during each spatial pattern and separates bands using Fourier decomposition before compressive-sensing reconstruction.
Results
64×64×10-band visible multispectral data were acquired with 3000 spatial patterns in approximately 1 minute, and Fourier peaks represented the response strengths of corresponding spectral bands.
Takeaways & Limitations
MSPI extends single-pixel imaging to multispectral capture without additional acquisition time or computational complexity, supporting compact, inexpensive, high-photon-efficient multispectral cameras.
Takeaways & Limitations
MSPI requires many projections and algorithmic reconstruction, trading temporal resolution for spatial and spectral resolution; its current spatial resolution is insufficient for practical applications.
Abstract
from arXiv · showhide
Current multispectral imagers suffer from low photon efficiency and limited spectrum range. These limitations are partially due to the technological limitations from array sensors (CCD or CMOS), and also caused by separative measurement of the entries/slices of a spatial-spectral data cube. Besides, they are mostly expensive and bulky. To address above issues, this paper proposes to image the 3D multispectral data with a single bucket detector in a multiplexing way. Under the single pixel imaging scheme, we project spatial-spectral modulated illumination onto the target scene to encode the scene's 3D information into a 1D measurement sequence. Conventional spatial modulation is used to resolve the scene's spatial information. To avoid increasing requisite acquisition time for 2D to 3D extension of the latent data, we conduct spectral modulation in a frequency-division multiplexing manner in the speed gap between slow spatial light modulation and fast detector response. Then the sequential reconstruction falls into a simple Fourier decomposition and standard compressive sensing problem. A proof-of-concept setup is built to capture the multispectral data (64 pixels $\times$ 64 pixels $\times$ 10 wavelength bands) in the visible wavelength range (450nm-650nm) with acquisition time being 1 minute. The imaging scheme is of high flexibility for different spectrum ranges and resolutions. It holds great potentials for various low light and airborne applications, and can be easily manufactured production-volume portable multispectral imagers.
1 Introduction
MSPI addresses photon inefficiency, limited spectral range, bulk, and cost in conventional multispectral imagers by encoding spatial and spectral information with one bucket detector. It uses temporal spectral modulation within the detector–patterning speed gap, followed by Fourier demultiplexing and compressive-sensing reconstruction.
- Motivation: Conventional multispectral imagers separately measure wavelengths with array detectors, causing photon inefficiency, limited spectrum range, bulk, and high cost.The paper notes that some near-infrared to short-wave-infrared multispectral imagers cost more than $50000.
- Motivation: Single-pixel imaging uses a bucket detector to collect all scene-interacted light, enabling compact, low-cost, photon-efficient, and spectrally flexible systems.SPI also imposes no requirement on the light path between the scene and detector, provided the interacted light is collected.
- Design challenge: Extending spatial modulation directly to three-dimensional spatial-spectral modulation would increase projections and reconstruction complexity.Alternative approaches include using a spectrometer, filters, or dispersive optics to separate wavelengths before detection.
- Proposed approach: MSPI encodes spectral information within each spatial pattern’s duration by exploiting the detector’s MHz response versus illumination patterning at no higher than KHz.This design avoids increasing requisite projections and capturing time compared with conventional SPI.
- Proposed approach: Spectrum-dependent sinusoidal modulation multiplexes bands into bucket-detector measurements, which are separated by Fourier decomposition before compressive-sensing reconstruction.Distinct dominant Fourier frequencies identify the response signals of different wavelength bands, while the decomposition can suppress system noise.
- Potential applications: The proposed system has potential for low-light, airborne, and portable applications because of its photon efficiency, noise robustness, compactness, broad spectral range, and low cost.Examples include fluorescence microscopy, Raman imaging, geologic mapping, mineral exploration, agricultural assessment, and environmental monitoring.
2 Results
The proof-of-concept MSPI system combines spatial modulation, rotating-film spectral modulation, and bucket detection to recover multispectral images. Experiments demonstrate 10-band visible imaging and reconstruction accuracy on a standard color checker.
- System and acquisition: MSPI adds spectral modulation to SPI so a bucket detector can resolve both spatial and spectral scene information.Spatial information is encoded by an SLM, while spectral information is encoded by wavelength-dependent temporal modulation.
- System and acquisition: A rotating sinusoidal film imposes different temporal intensity variations on different wavelengths after the spatial pattern is generated.The rainbow spectrum is spread along the film radius, and rotation at around 6000 r/min produces wavelength-dependent modulation.
- System and acquisition: 3000 random 64×64-pixel patterns were projected at 50 Hz while the bucket detector sampled at 100 kHz, completing acquisition in around 1 minute.A self-synchronization technique synchronized the DMD and detector.
- Color-scene imaging: The color-scene experiment covered 450nm-650nm and discretized the spectrum into 10 narrow bands using annular rings with sinusoidal periods from 2 to 20.Fourier coefficients at dominant frequencies represented the response strengths of the corresponding spectral bands.
- Accuracy analysis: On an X-Rite color checker, MSPI reconstructed 10 spectral bands from 450nm-650nm and compared recovered swatch spectra with ground truth using root mean square error.The reported small deviations, including for Orange and Yellow swatches, supported reconstruction accuracy.
- Color-scene imaging: Fourier decomposition separated multispectral response signals despite temporal-domain system noise, enabling reconstruction of the spectral-band sequences.Small Fourier-coefficient fluctuations were attributed to system noise, while dominant peaks corresponded to the sinusoidal spectral codes.
3 Discussion
MSPI provides a flexible single-bucket-detector approach to multispectral imaging, extending spatial coding to spatial-spectral coding without additional acquisition time or computational complexity relative to conventional 2D SPI. Its current trade-offs are temporal resolution and insufficient spatial resolution, though faster modulation, adaptive patterns, and parallel reconstruction offer improvement paths.
- Technique: MSPI extends conventional 2D spatial coding to 3D spatial-spectral coding through temporal sinusoidal spectral modulation within each spatial pattern.The approach uses the speed gap between slow spatial illumination patterning and fast detector response.
- Technique: MSPI resolves multispectral information without increasing acquisition time or computational complexity relative to conventional 2D SPI.
- Flexibility: Spectral-modulator specifications can be customized through annulus width, grating density, and multiplexing-film design to adjust resolution and multiplexing mode.
- Flexibility: The scheme can couple with microscopy or macroscopy, operate across spectrum ranges, and support passive illumination after light interacts with the scene.
- Limitations: MSPI trades temporal resolution for spatial and spectral resolution because its advantages require many projections and algorithmic reconstruction.Faster rotation, denser patterns, cross-channel priors, and parallel GPU reconstruction are proposed acceleration routes.
- Limitations: Current spatial resolution is insufficient for practical applications, although structural and adaptive patterns could improve resolution while reducing projections and computation.
4 Methods
MSPI reconstructs multispectral scenes by first demultiplexing wavelength-specific responses with Fourier analysis, then solving a compressive-sensing reconstruction independently for each band.
- Spectral demultiplexing: Fourier decomposition separates wavelength responses because each spectral band has a distinct sinusoidal modulation frequency.FFT converts each bucket-detector measurement sequence into the Fourier domain, where dominant frequencies identify spectral responses and help separate noise.
- Spectral demultiplexing: Each wavelength band is represented by a response-signal vector assembled from the Fourier coefficient obtained for every projected spatial pattern.For m projected patterns, the response signals for wavelength λ form a row vector bλ ∈ R^m.
- Multispectral reconstruction: The reconstructed image xλ contains the spatial scene at wavelength λ, with illumination patterns represented as rows of A ∈ R^(m×n).The scene and illumination patterns share the same spatial resolution, while n denotes the number of spatial pixels per pattern.
- Multispectral reconstruction: Compressive sensing reduces requisite projections by reconstructing each wavelength band through an optimization problem.The method formulates reconstruction separately for each wavelength under a compressive-sensing framework.
- Multispectral reconstruction: The objective minimizes the l1 norm of transformed image coefficients to enforce a sparsity prior on natural scene images.The transformation ψ maps xλ into a domain where the scene is assumed statistically sparse; linearized alternating direction method optimization obtains x*λ.
- Multispectral reconstruction: After solving the optimization for every wavelength band, the method produces the target scene’s multispectral images.Each band is reconstructed separately, yielding the final multispectral image set.