Source-linked AI summary
Single-shot multispectral imaging with a monochromatic camera
Sujit Kumar Sahoo, Dongliang Tang, Cuong Dang
TL;DR
The paper examines multispectral recovery and spectroscopy with scattering media under practical noise, precision, and dynamic-range constraints. It combines spectrally varying speckle PSFs with deconvolution and demonstrates recovery of complex multispectral objects, while reporting useful spectral resolution with ordinary diffusers.
Problem
Scattering-media spectroscopy must balance spectral resolution with practical imaging constraints such as finite precision, dynamic range, and noise.
Method
The technique uses wavelength-dependent speckle PSFs from a scattering medium as spectral filters and reconstructs images or spectra through cross-correlation and Wiener deconvolution.
Results
Complex multispectral objects were successfully recovered, including shared RGB spectral components and gradient or blended colours, although camera sensitivity distorted the composite colour appearance.
Takeaways & Limitations
A normal diffuser can provide 5~10 nm spectral resolution, which is reported as useful for applications such as characterizing LED spectra with a smartphone.
Abstract
from arXiv · showhide
Multispectral imaging plays an important role in many applications from astronomical imaging, earth observation to biomedical imaging. However, the current technologies are complex with multiple alignment-sensitive components, predetermined spatial and spectral parameters by manufactures. Here, we demonstrate a single-shot multispectral imaging technique that gives flexibility to end-users with a very simple optical setup, thank to spatial correlation and spectral decorrelation of speckle patterns. These seemingly random speckle patterns are point spreading functions (PSFs) generated by light from point sources propagating through a strongly scattering medium. The spatial correlation of PSFs allows image recovery with deconvolution techniques, while the spectral decorrelation allows them to play the role of tune-able spectral filters in the deconvolution process. Our demonstrations utilizing optical physics of strongly scattering media and computational imaging present the most cost-effective approach for multispectral imaging with great advantages.
1. Optical Setup and Data Processing
The system forms multispectral speckle measurements with a projector, diffuser, and monochromatic CMOS camera, then reconstructs spectral images computationally. Preprocessing and stored PSF transforms accelerate Wiener deconvolution while visualization colors remain illustrative.
- Optical acquisition: A projector generates 2D multispectral objects, which pass through a diffuser before speckle capture by a high-dynamic monochromatic CMOS camera.The diffuser and iris-2 are positioned about 210 mm from the object plane, while the camera is about 87.5 mm from the diffuser.
- Computational reconstruction: Wiener deconvolution reconstructs each spectral image in about 0.5 second on a normal PC.The implementation uses MATLAB on an Intel Core i7 computer with 16 GB memory.
- Computational reconstruction: Pre-calculating and storing spectral-PSF FFTs reduces the reconstruction process by 30%.The central 2048*2048-pixel sensor area is used for the experiments.
- Preprocessing: Dividing speckle patterns and PSFs by their low-frequency envelopes removes the halo effect and sharpens the speckles.
- Visualization: Captured measurements are grayscale; RGB colors and composite images are assigned afterward for illustration without additional color processing.Composite images combine three reconstructed red, green, and blue spectral images.
2. Measurement for the field of view (FOV)
The field of view is determined by the scattering medium’s memory-effect region, where point-spread functions remain sufficiently correlated and shift-invariant. In this setup, the correlation threshold gives a field of view about 3.2 mm in diameter, with estimated effective thickness varying by spectral band.
- FOV definition: The field of view is defined as the region where the PSF cross-correlation coefficient exceeds 0.5.It is characterized by comparing the central PSF with PSFs measured at different object-plane positions.
- FOV measurement: About 3.2 mm is the measured field-of-view diameter under the cross-correlation threshold.
- FOV definition: The memory-effect geometry relates the field of view to the incident-angle region through L = uφ.Here, u is the distance from the object plane to the scattering medium.
- Correlation model: The correlation model uses C, the wave vector k_0 = 2π/λ, and incident angle θ = x/u.These quantities connect PSF correlation with wavelength and transverse object position.
- Thickness estimation: 18, 22, 18, and 16 µm are the estimated effective scattering-medium thicknesses for white and RGB bands, respectively.
- Memory-effect evidence: Shift-invariant white speckle patterns persist across the illustrated positions x = 0, 0.4, 0.8, and 1.2 mm.The figure compares cross-correlation behavior across RGB and white spectral bands.
3. Effect of spectral overlap on the reconstructed multispectral images
Spectral overlap produces cross-talk between reconstructed channels, whereas non-overlapping narrow-band PSFs separate the RGB reconstructions. Narrow-band measurements nevertheless suffer low signal-to-noise, especially in the blue channel.
- Overlapping spectra: Spectral overlap causes cross-talk between the green and blue reconstructed spectral channels.
- Narrow-band spectra: Non-overlapping narrow-spectral PSFs separate the N, T, and U letters across the three RGB spectral images.
- Narrow-band limitations: Weak single-pixel projector intensities produce very low signal-to-noise measurements for narrow green and especially narrow blue PSFs.The reduced signal-to-noise degrades reconstructed image quality in the narrow green and narrow blue bands.
4. Demonstrations for mixed color components.
The technique reconstructs objects containing mixed CMY components and continuous rainbow gradients from monochromatic-camera speckle measurements. RGB deconvolution recovers shared spectral components, while composite images reproduce the complex object appearance with camera-sensitivity-dependent color distortion.
- Test objects: Mixed cyan, magenta, and yellow CDG letters and a gradient rainbow ring are reconstructed from raw speckle patterns.The corresponding spectral PSFs from Figure 2(c) are used for deconvolution.
- RGB reconstruction: RGB spectral PSFs recover pairs of CDG letters that share the same spectral element.
- Composite reconstruction: Composite RGB images resemble the original mixed-color letters and gradient rainbow object.The composites are formed by superposing the reconstructed red, green, and blue spectral images.
- Color fidelity: Camera wavelength sensitivity distorts the composite color quality, with highest sensitivity in green and lowest in blue.The paper states that calibration using camera and human-eye sensitivities would improve similarity to the original appearance.
4. A spectroscopy technique with strongly scattering media
The technique uses spectral decorrelation of scattering-medium PSFs to recover spectra from a broadband point source. Simulations demonstrate reconstruction at 2 nm and 10 nm spectral sampling.
- A broadband point source’s spectrum is reconstructed by cross-correlating its measured speckle image with multispectral PSFs.The PSFs can be measured at different wavelengths during a one-time calibration step.
- 2 nm and 10 nm spectral sampling both produce reconstructed spectra for a white LED in the simulated spectrometer demonstration.The demonstration uses independent simulated spectral PSFs and compares the recovered spectrum with the original spectrum.
- Increasing the effective thickness of a scattering medium reduces its spectral decorrelation bandwidth, while ordinary diffusers or tissue can exhibit bandwidths of 5–10 nm.
5. Deconvolution in presence of noise
The practical imaging model adds noise to the ideal multispectral convolution, so reconstruction requires a noise-aware deconvolution method. The paper uses Wiener deconvolution to approximate PSF inversion under finite precision and dynamic-range constraints.
- The observed image is modeled as the ideal multispectral convolution plus additive noise under finite precision and a fixed dynamic range.The ideal signal is the sum of wavelength-specific object–PSF convolutions.
- Wiener deconvolution is used as a fast method for solving the noisy PSF inversion problem.The method assumes white Gaussian noise with variance σ^2.
- The Wiener formulation provides an approximate solution resembling ideal deconvolution while accounting for measurement noise.
C FFT I FFT PSF O weiner I PSF FFT
The reconstruction uses a Wiener deconvolution parameter to suppress measurement artifacts. Compared with iterative alternatives, Wiener deconvolution avoids iterations and has logarithmic-scale execution time in the total pixel count.
- C>1 is chosen to keep the approximate Wiener solution clear from measurement deformation artifacts.The constant can be determined from the measured noise variance of the imaging system and environment.
- Iterative Richardson–Lucy deconvolution can produce heavily smoothed images with other artifacts despite not requiring a noise-variance parameter.
- Wiener deconvolution runs in logarithmic-scale time with total pixels and requires no iterations, whereas iterative methods grow in polynomial-scale time and need many iterations.The paper leaves systematic comparison of deconvolution methods beyond its scope.
6. Effect of imager’s dimensionality
The paper examines how image dimensionality affects reconstruction artifacts by varying the number of pixels in simulated speckle images and PSFs. Independent random signals become less correlated as dimensionality increases.
- Noise–signal correlation decreases as the total number of pixels N increases, under the paper’s independent-noise analysis.The relationship is illustrated by averaging cross-correlation values over 1000 trials.
- The cross-correlation coefficient of two independent random images is evaluated at different pixel counts to assess dimensionality effects.
- Reconstruction artifacts are visualized by cropping measured speckle images and PSFs to different sizes before deconvolving simulated RGB components.
- Figure S6 compares reconstructed 64x64-pixel objects using cropped raw speckle images and PSFs sized 128x128, 256x256, 512x512, and 1024x1024.The comparison shows the dimensionality settings used for the reconstruction visualization.
7. Effect of total spectral band numbers
The reconstruction assumes spectral PSFs are uncorrelated, so off-band components act as secondary noise. Increasing the number of spectral bands raises correlation and can produce artifacts.
- Uncorrelated PSFs allow off-band speckle components to be treated as secondary noise during reconstruction.
- As the number of spectral bands increases, the component spectrum V increases and causes a linear rise in correlation.
- Increased correlation produces reconstruction artifacts that can be suppressed by over-smoothing the reconstructed images.
8. Accuracy of Point Spreading Function
PSF accuracy is important because measurement noise enters reconstruction through the measured speckle image. The setup assumes PSF noise is uncorrelated and relies on one-time PSF measurements.
- Noise is assumed to occur during speckle-image measurement, while the PSF is measured once and reused for reconstruction.
- Small-projector measurements have low single-pixel intensity, and narrow band-pass filters further reduce the PSF signal-to-noise ratio.
- The reconstruction model further assumes that PSF noise V_λ is uncorrelated.
C FFT I FFT PSF O FFT
The reconstruction expression accounts for finite-dimensional correlation and PSF noise. Simulations show that increasing noise in the green PSF progressively degrades the reconstructed multispectral images.
- Finite-dimensional correlation produces smoothed noise in the reconstructed image.
- Reconstruction quality degrades as noise increases in the PSF used for reconstruction.
- The green-PSF experiment compares reconstruction at 5dB, 10dB, 15dB, and 20dB SNR.