Source-linked AI summary
Photon counting compressive depth mapping
Gregory A. Howland, Daniel J. Lum, Matthew R. Ware, John C. Howell
TL;DR
Ultra-low-light photon-counting lidar is difficult to scale to high transverse resolution with conventional scanning or detector arrays. This paper adapts a single-pixel camera to recover intensity and depth from linear compressive measurements, achieving 256×256 imaging in as little as 3 seconds and 32×32 tracking video at 14 frames per second.
Problem
Photon-counting lidar offers sensitive low-light detection, but scanning and detector-array approaches make high-resolution transverse imaging difficult.
Method
The method uses a photon-counting single-pixel camera to recover intensity and intensity-times-depth images from under-sampled linear projections, then divides them to obtain depth.
Results
The system demonstrates 256×256 intensity and depth imaging with acquisition times as short as 3 seconds, novelty filtering, and 32×32 video tracking at 14 frames per second.
Takeaways & Limitations
The protocol provides a simple way to add high-resolution depth mapping to low-light sensing applications using standard compressive-sensing techniques.
Takeaways & Limitations
The protocol does not exploit depth-map sparsity directly, so it may require more measurements than methods that recover depth itself.
Abstract
from arXiv · showhide
We demonstrate a compressed sensing, photon counting lidar system based on the single-pixel camera. Our technique recovers both depth and intensity maps from a single under-sampled set of incoherent, linear projections of a scene of interest at ultra-low light levels around 0.5 picowatts. Only two-dimensional reconstructions are required to image a three-dimensional scene. We demonstrate intensity imaging and depth mapping at 256 x 256 pixel transverse resolution with acquisition times as short as 3 seconds. We also show novelty filtering, reconstructing only the difference between two instances of a scene. Finally, we acquire 32 x 32 pixel real-time video for three-dimensional object tracking at 14 frames-per-second.
1. Introduction
Ultra-low-light TOF lidar is valuable but difficult to scale in transverse resolution because scanning slows acquisition and detector arrays face fabrication, noise, and readout challenges. The paper presents a single-pixel compressive-sensing alternative that recovers intensity and depth from under-sampled projections.
- Motivation: Photon-counting TOF lidar offers single-photon sensitivity and sub-nanosecond timing for ultra-low-light three-dimensional imaging.Applications include surface mapping, target recognition, tracking, machine vision, eye-safe ranging, and atmospheric sensing.
- Limitations of existing systems: Scanning single-element detectors require acquisition times proportional to spatial resolution, limiting real-time performance at higher resolutions.
- Limitations of existing systems: Photon-counting detector arrays are constrained by fabrication limits, dark counts, pixel cross-talk, readout noise, and per-pixel TOF correlation costs.The cited current-best and commercial resolutions are 32×128 and 32×32 pixels, respectively.
- Proposed direction: A single-element photon-counting detector in a compressive-sensing single-pixel camera can recover images from under-sampled incoherent projections.The scene is encoded with pseudorandom patterns displayed on an SLM or DMD.
- Paper contribution: The paper recovers intensity and depth at up to 256×256 pixels with acquisition times as short as 3 seconds, plus 32×32 video at 14 frames per second.It also demonstrates novelty filtering by reconstructing differences between scene instances.
2. Compressive Sensing
Compressive sensing reconstructs a sparsely represented signal from fewer linear measurements than its ambient dimension by combining an incoherent sensing matrix with sparsity-promoting optimization. The single-pixel camera implements this process optically using sequential DMD patterns and a bucket detector.
- Compressive sensing: Compressive sensing recovers an n-dimensional sparse signal from m<n measurements by optimization, exploiting compressibility below the Nyquist measurement count.
- Reconstruction: The reconstruction objective combines measurement fidelity with a sparsity-promoting penalty such as the ℓ1 norm or total variation.For k-sparse signals, exact reconstruction can require m proportional to klog(n/k) measurements.
- Sensing design: Random binary sensing vectors work well because the sensing matrix should be incoherent with the sparse representation used for reconstruction.
- Single-pixel camera: A single-pixel camera forms sensing-matrix rows with sequential pseudorandom binary DMD patterns and records each pattern’s measured intensity with a bucket detector.The recovered image is obtained by solving the compressive-sensing reconstruction problem.
3. Compressive Depth Mapping
The depth-mapping protocol avoids directly sampling the nonlinear depth map by reconstructing intensity and intensity-times-depth signals from photon counts and summed TOFs, then dividing the recovered signals. Noise reduction combines sparse recovery, thresholding, debiasing, and optional masking.
- 3.1. Adapting SPC for Depth Mapping: The protocol reconstructs a two-dimensional intensity image from photon counts and an intensity-times-depth image from summed photon TOFs using the same incoherent DMD measurements.A pulsed laser, DMD, photon-counting detector, and TCSPC module provide the measurements.
- 3.1. Adapting SPC for Depth Mapping: Direct depth recovery is nonlinear because each photon detection event contains both intensity and time-of-flight information.
- 3.1. Adapting SPC for Depth Mapping: Summed TOFs linearly sample XQ = XI.XD, allowing standard compressive-sensing recovery of the intensity-times-depth signal.The measurement equation has the same linear form as the standard compressive-sensing model.
- 3.1. Adapting SPC for Depth Mapping: The depth map is recovered by element-wise division of XQ by XI, with Nz(XI) preventing division by zero where no light is detected.
- 3.2. Protocol: The reconstruction protocol acquires YQ and YI, sparsely recovers XQ, thresholds significant coefficients, applies least-squares debiasing, and optionally masks noisy outputs.The same significant coefficients are assumed for XI and XQ before recovering XD.
- 3.2. Protocol: Optional masking improves spatial cleanup for very noisy measurements by balancing accurate outlines against accurate pixel values.
4. Experimental Setup
The system combines pulsed flood illumination, DMD-based Hadamard sensing, photon counting, and TCSPC timing to record photon counts and summed TOFs for each pattern. Narrow-band filtering keeps dark and ambient counts small in the reported laboratory setup, while reconstruction cost depends on resolution, scene complexity, and noise.
- 4. Experimental Setup: The setup uses 2 ns, 780 nm laser pulses at 10 MHz, a DMD displayed at 1440 Hz, and a photon-counting photomultiplier to record photon counts and summed TOFs.
- 4. Experimental Setup: Randomly selected rows of a zero-shifted, randomly permuted Hadamard matrix provide computationally efficient binary sensing patterns.The zero-shifted matrix replaces −1 entries with zero for the on-pixel detector.
- 4. Experimental Setup: The experiments use ℓ1 or total-variation regularization with gradient-projection or TVAL3 solvers, and Haar wavelets for sparse representation and debiasing.Higher-order Daubechies wavelets did not significantly improve the presented test scenes.
- 4. Experimental Setup: A 256×256 reconstruction takes less than 5 minutes, while a 32×32 reconstruction takes less than 5 seconds under the reported Matlab implementation.
- 4.1. Noise Considerations: Shot noise limits measured SNR to √η for η detected photons, and successful reconstruction generally requires measurement-vector variation to exceed shot noise.An exhaustive noise analysis is left for future work.
- 4.1. Noise Considerations: The reported laboratory system had less than 100 dark counts per second and approximately 1000 ambient-light counts per second against over 10^6 signal counts per second.The authors state that these unwanted counts negligibly affected the results, though mitigation may matter in real-world applications.
5. Results
The system recovers intensity and depth at 256×256 resolution under low-light, short-exposure conditions, calibrates depth accurately, filters scene changes, and tracks three-dimensional motion in real time.
- 5.1. Simple Scene: Less than 1 photon per significant pixel makes raster scanning ineffective, whereas incoherent projections combine flux from many pixels and measure half the available light on average.At the same dwell times, raster scans would require about 228 seconds and 46 seconds, compared with the compressive acquisitions.
- 5.2. Natural Scene: 256×256 natural-scene reconstructions recover recognizable objects and accurate target distances, with slightly more noise than the cardboard-cutout scene.The scene contains a cactus, shoe, and microscope and uses m = 0.3n measurements with Haar-wavelet sparsity.
- 5.3. Depth Calibration: Depth calibration agrees linearly with target distance, with slope 1.91, and resolves depth changes smaller than 2.54 cm despite 60 cm pulse lengths.The slope is slightly below the expected round-trip value of 2 because cable transit times are included.
- 5.4. Novelty Filtering: Novelty filtering isolates a moved object: short 37-second difference reconstructions remove poorly imaged static clutter and yield a recognizable ‘R’.The difference image contains negative and positive copies marking the former and new positions, respectively.
6. Protocol Trade-offs and Limitations
The protocol offers a simple compressive depth-mapping approach, but trades reduced complexity for limitations in depth-map sparsity, signal representation, pulse modeling, and very-low-photon reconstruction.
- Protocol trade-offs: The protocol adapts a single-pixel camera for ranging by adding a pulsed source while retaining standard linear compressive-sensing techniques.Photon-counting time of flight is measured and summed naturally within this framework.
- Depth-map sparsity: Depth-map sparsity is not directly exploited, so protocols that recover depth itself may require fewer measurements but use more complex measurement and reconstruction schemes.The current protocol instead reconstructs image-like quantities rather than optimizing directly for depth-map sparsity.
- Signal representation: Treating XQ as an image can make it less sparse than XI, limiting measurement reduction while leaving the best representation of XQ unresolved.Wavelet and total-variation representations are used, but a full complexity analysis remains open.
- Pulse modeling: Laser-pulse width and shape are omitted from the protocol, limiting very-high-range-resolution depth mapping when relative photon time of flight becomes ambiguous.Including pulse characteristics could improve range resolution but would make the objective function more complex.
- Photon-count regime: At photon counts approaching single digits per measurement, a logarithmic penalty may outperform the current least-squares penalty but requires different solvers.The manuscript’s datasets averaged thousands of photons per measurement, so this issue did not affect those experiments.
- Overall trade-off: The authors present the current scheme as a practical compressive-depth-mapping approach rather than a fundamentally optimal one, with added complexity weighed against diminishing returns.The stated trade-off is between protocol complexity and potentially limited additional benefit.
7. Conclusion
The system extends single-pixel cameras to photon-counting time-of-flight depth mapping using linear measurements and compressive sensing. It supports high-resolution, low-light imaging while enabling novelty filtering and real-time three-dimensional tracking.
- 7. Conclusion: 256×256-pixel depth and intensity imaging is achieved at practical acquisition times using linear photon-counting measurements.The approach operates at sub-picowatt light levels and supports standard compressive sensing techniques.
- 7. Conclusion: 32×32-pixel video at 14 frames per second enables real-time three-dimensional object tracking.
- 7. Conclusion: Differencing random projections of two scene instances directly reconstructs novelty images that remove static clutter and show scene changes.
- 7. Conclusion: Switching detectors enables adaptations to infrared, faster acquisition, and hyperspectral imaging while preserving the single-pixel architecture.The proof-of-principle system operates at 780 nm, with alternative detectors supporting other capabilities.
- 7. Conclusion: The protocol offers a simple way to add high-resolution depth mapping to low-light sensing applications without increased scaling cost or complexity.