PaperlayerView exact revision on arXiv

Source-linked AI summary

Photon-Efficient Computational 3D and Reflectivity Imaging with Single-Photon Detectors

Dongeek Shin, Ahmed Kirmani, Vivek K Goyal, Jeffrey H. Shapiro

arXiv:1406.1761v1stat.AP

TL;DR

Low-light active imaging traditionally requires many detections, whereas this paper develops fixed-dwell computational imaging for depth and reflectivity from about one detected photon per pixel averaged across the scene. Combining single-photon detection statistics with spatial priors, the method recovers robust images under background light and is similar to or slightly better than FPI at equal raster-scanned acquisition time.

  • Problem

    Low-light active imaging requires accurate depth and reflectivity estimates despite very little backreflected light reaching the detector.

  • Method

    The method combines physically accurate single-photon detection statistics with spatial correlations in natural scenes using a fixed dwell time per pixel and probabilistic modeling of individual detections.

  • Results

    Using on the order of 1 detected photon per pixel averaged over the scene, the method produces accurate depth and reflectivity images under significant background light and performs similarly to or slightly better than FPI at equal raster-scanned acquisition time.

  • Takeaways & Limitations

    Fixed dwell time makes the framework compatible with detector arrays and supports faster robust imaging through parallelization.

  • Takeaways & Limitations

    Reflectivity estimation fails when background light provides a detection during nearly every pulse repetition period, requiring sufficiently low average background probability.

Abstract

from arXiv · show

Capturing depth and reflectivity images at low light levels from active illumination of a scene has wide-ranging applications. Conventionally, even with single-photon detectors, hundreds of photon detections are needed at each pixel to mitigate Poisson noise. We develop a robust method for estimating depth and reflectivity using on the order of 1 detected photon per pixel averaged over the scene. Our computational imager combines physically accurate single-photon counting statistics with exploitation of the spatial correlations present in real-world reflectivity and 3D structure. Experiments conducted in the presence of strong background light demonstrate that our computational imager is able to accurately recover scene depth and reflectivity, while traditional maximum-likelihood based imaging methods lead to estimates that are highly noisy. Our framework increases photon efficiency 100-fold over traditional processing and also improves, somewhat, upon first-photon imaging under a total acquisition time constraint in raster-scanned operation. Thus our new imager will be useful for rapid, low-power, and noise-tolerant active optical imaging, and its fixed dwell time will facilitate parallelization through use of a detector array.

I. INTRODUCTION

The paper targets accurate active 3D and reflectivity imaging with roughly one detected photon per pixel by combining single-photon statistics with spatial correlations. Its fixed-dwell framework is designed for low-light operation and detector-array parallelization.

  • Motivation: 102 to 103 photons per pixel are conventionally needed for finely binned LIDAR histograms that support accurate depth and reflectivity imaging.Histogram timing and amplitude encode depth and reflectivity, respectively.
  • Contribution: The framework recovers reflectivity and 3D images simultaneously using on the order of 1 detected photon per pixel averaged over the scene.It avoids histograms and models individual detected photons probabilistically.
  • Contribution: Physically accurate photon statistics and spatial correlations support accurate imaging when very little backreflected light reaches the detector.The method uses deterministic dwell times, unlike first-photon imaging’s random per-pixel dwell times.
  • Prior processing: Traditional low-light processing estimates depth independently at each pixel before denoising, making Poisson-noise suppression more challenging.The paper compares its approach with denoising methods used after pixelwise maximum-likelihood estimation.
  • Comparison: The fixed-dwell framework performs similarly to or slightly better than first-photon imaging at equal total acquisition time in raster-scanned operation.Fixed dwell time also supports detector-array operation.

B. Main contributions

The paper introduces a physically accurate fixed-acquisition-time SPAD model and a reconstruction method that combines photon-counting statistics with spatial scene correlations. Experiments evaluate its low-light imaging performance.

  • Model: The method models arbitrary illumination pulses, ambient background light, dark counts, and inhomogeneous Poisson shot noise under fixed acquisition time.These components describe the SPAD signal in low-light conditions.
  • Reconstruction: The reconstruction estimates scene depth and reflectivity from noisy photon-detection data using single-photon statistics and spatial correlations.The spatial prior reflects correlations in real-world scenes.
  • Results: More than 100 times higher photon efficiency than traditional maximum-likelihood estimation was demonstrated experimentally.The method also achieved sub-pulse-width depth resolution when 54% of pixels had no detections.
  • Paper scope: The paper develops probabilistic models for measured data and uses them as the basis for a novel image-formation method.The manuscript also adds derivations, performance bounds, and experimental results beyond the abbreviated earlier presentation.

B. Detection

The detector records time-stamped SPAD clicks and total counts during a fixed number of laser pulses. The observation model combines signal, background, and dark-count processes under a low-flux assumption.

  • Acquisition: Each pixel is illuminated with N pulses, and the dwell time is Ta = NTr while total detections and their times are recorded.Background light is also directed onto the detector.
  • Signal model: Backreflected photon flux is modeled as a delayed, scaled pulse plus background: ri,j(t) = αi,js(t − 2zi,j/c) + bλ.Depth determines the delay and reflectivity scales the pulse.
  • Observation model: The SPAD output is an inhomogeneous Poisson process combining backreflected detections with independent homogeneous dark counts.Background and dark counts are grouped in the observation process.
  • Low-flux regime: Under low flux, the per-pulse signal-plus-background count is much less than one, allowing multiple detections per repetition interval to be neglected.The background count per period is B = (ηbλ + d)Tr.
  • Count distribution: With N pulses, the detected-photon count Ki,j is binomially distributed and converges to a Poisson random variable in the stated limiting regime.The binomial support is ki,j = 0, 1, ..., N.

C. Distributions of Single-Photon Detection Times

Detection-time statistics distinguish signal and noise within each repetition interval. Signal detections follow the shifted pulse shape, whereas background detections are uniformly distributed in time.

  • Timing observation: The SPAD records the first detected photon’s time within a repetition interval, localized to a bin of duration Δ.The timing density is derived from the probability of no earlier click followed by a click in the selected bin.
  • Density derivation: The detection-time density is obtained by taking the per-unit-time limit as Δ approaches zero.The derivation uses independent Poisson increments and the low-flux approximation.
  • Mixture model: Signal detection times follow the normalized time-shifted pulse shape, while background detection times are uniformly distributed on [0, Tr).The observed density is therefore a mixture distribution.
  • Mixture weights: The signal and noise mixture weights are determined by their respective expected counts, with signal probability ηαi,jS/(ηαi,jS + B).Background detections include ambient-light and dark-count contributions.

IV. CONVENTIONAL IMAGE FORMATION

Conventional image formation estimates reflectivity from photon counts and depth from photon detection times, with background light making depth estimation non-convex and dependent on unknown reflectivity.

  • A. Pixelwise ML Reflectivity Estimation: The constrained ML reflectivity estimate is computed from the total observed photon count k_i,j at each pixel.
  • A. Pixelwise ML Reflectivity Estimation: The normalized photon-count value is the traditional reflectivity estimate.
  • A. Pixelwise ML Reflectivity Estimation: When B = 0, the normalized count estimate equals the constrained ML reflectivity estimate under the Poisson approximation to the binomial distribution.
  • B. Pixelwise ML Depth Estimation: With B > 0, maximum-likelihood depth estimation requires solving a non-convex optimization problem.
  • B. Pixelwise ML Depth Estimation: When B > 0, maximum-likelihood depth estimation also requires the unavailable true reflectivity α_i,j, so the log-matched filter is traditionally used instead.
  • B. Pixelwise ML Depth Estimation: When B = 0, the log-matched filter solution equals the constrained ML depth estimate.

V. NOVEL IMAGE FORMATION

The novel formation method regularizes photon-count likelihoods using scene structure, rejects background detections before depth estimation, and uses convex optimization under suitable pulse-shape assumptions.

  • Low-data or low-SNR pixelwise ML estimates are inaccurate, motivating regularization that exploits spatial correlations in reflectivity and depth.
  • Step 1: Reflectivity estimation: The penalized ML reflectivity estimate solves a convex program because the reflectivity negative log-likelihood is strictly convex.
  • Step 1: Reflectivity estimation: The reflectivity penalty is convex and penalizes nonsmoothness, with β_α controlling its strength.
  • Step 2: Rejection of background detections: Background detections are censored before depth estimation because their likelihood contribution creates a non-convex cost with locally optimal solutions far from the global optimum.
  • Step 2: Rejection of background detections: Background detection times are spatially independent, whereas signal detections have lower conditional variance because pulse duration is much shorter than the repetition period and depths are spatially correlated.
  • Step 2: Rejection of background detections: The method computes each pixel’s rank-ordered mean from eight neighboring detection times and uses it to identify detections for censoring.
  • Step 2: Rejection of background detections: Pixels with no detected photons have empty uncensored sets, while missing rank-ordered means are assigned infinite time.
  • Step 3: Depth Estimation: For pulse shapes approximated by s(t) ∝ exp[−v(t)] with convex v(t), the depth likelihood is convex and supports penalized ML optimization.

VI. EXPERIMENTAL RESULTS

Experiments used raster-scanned SPAD measurements to evaluate reflectivity reconstruction, showing 16 resolved gray levels and substantial PSNR gains over pixelwise baselines.

  • The experiment raster-scanned 1000×1000 pixels using a 270 ps pulsed laser, 100 ns repetition period, and lensless SPAD with quantum efficiency η = 0.35.
  • A. Reflectivity Resolution Test: The proposed method resolved 16 gray levels, similarly to a ground-truth image requiring about 1000 photon detections per pixel.
  • A. Reflectivity Resolution Test: Reflectivity reconstruction quality was quantified using peak signal-to-noise ratio (PSNR).
  • A. Reflectivity Resolution Test: 16 dB: the proposed method’s PSNR exceeded pixelwise ML, and 3 dB: it exceeded bilateral-filtered pixelwise ML.

B. Depth Resolution Test

The method achieves millimeter-scale depth resolution and substantially reduces depth error relative to pixelwise ML estimates under sparse photon detections. Experiments also evaluate reflectivity and 3D recovery across natural scenes, background levels, dwell times, and comparisons with conventional LIDAR.

  • Depth resolution: 4 mm depth resolution was achieved, comparable to ground truth requiring 100 detections per pixel and superior to noisy pixelwise ML estimates.The test target used 5 cm × 5 cm squares of varying thickness mounted on a flat board.
  • Depth resolution: At the experimental background level, pixelwise ML estimates had an RMSE of at least 3 m because many pixels lacked photon-detection-time observations.Bicubic interpolation and median filtering were applied to denoise the pixelwise ML estimate.
  • Depth resolution: 760-fold depth error reduction was obtained relative to the denoised pixelwise ML estimate.The comparison uses the method’s 4 mm depth resolution and a denoised estimate formed after interpolation and median filtering.
  • Natural scenes: The framework produced 30.6 dB reflectivity PSNR and 0.8 cm 3D RMSE while using accurate photon statistics and spatial prior information.The method used a total variation semi-norm penalty, with parameters selected separately for reflectivity PSNR and 3D RMSE.
  • System parameters: RMSE decreased monotonically with increasing dwell time and signal-to-background ratio, demonstrating robustness under strong background noise and short acquisition times.Figure 8 reports mean photon counts of 1.4 and 0.6 for acquisition times of 100 µs and 50 µs, respectively.

F. Comparison with First-Photon Imaging

The fixed-dwell-time framework is compared with first-photon imaging and shown to preserve strong low-light performance while supporting detector-array operation. Its limitations include edge-related depth errors, multiple-reflection inaccuracies, and failure under sufficiently strong background.

  • Comparison with First-Photon Imaging: Matched-acquisition-time experiments across five scenes yielded performance similar to, or slightly better than, first-photon imaging.The comparison used reflectivity PSNR and depth RMSE, despite many method pixels having no detections.
  • Limitations: The method’s highest depth errors occur near object edges, where low SNR produces fewer detections and more background counts.Spatial correlations estimate missing depths but lose subtle depth details.
  • Limitations: Multiple reflections can cause estimation inaccuracies, although diffuse scattering makes them considerably weaker than direct reflections in quasi-Lambertian scenes.The probability of photon detection from multiple reflections diminishes exponentially.
  • Comparison with First-Photon Imaging: Our method fixes per-pixel dwell time, whereas first-photon imaging fixes detections per pixel and therefore has random dwell time.The fixed dwell time is compatible with detector arrays.
  • Limitations: Reflectivity estimation fails when background light is likely to produce a detection during every pulse repetition period.The experiments therefore used a narrowband spectral filter to keep ηα_i,jS ≈ B ≪ 1 on average.
  • Future Applications: The framework is proposed for other photon-counting applications, including spatially resolved FLIM and high-resolution LIDAR, and can extend across wavelengths.Improved background suppression and range gating could further improve accuracy.

APPENDIX

The appendix analyzes pixelwise estimation using Cramér–Rao bounds and asymptotic likelihood results. It establishes how photon accumulation affects reflectivity estimation while clarifying when the bound applies to maximum-likelihood estimates.

  • CRLB Analysis: The Cramér–Rao lower bound gives the MSE limit for an unbiased estimator, and an estimator is efficient when its MSE equals that bound.The appendix defines these concepts through Fisher information.
  • Reflectivity Estimation: Increasing the number of pulse repetitions N collects more photons and decreases the reflectivity-estimation CRLB.The result follows from the derived reflectivity bound under the low-flux approximation.
  • Reflectivity Estimation: The CRLB cannot be directly used to lower-bound the MSE of the unconstrained ML reflectivity estimate.The appendix attributes this distinction to the estimator and the bound’s conditions.
  • Asymptotic Analysis: In the limiting Poisson regime, the CRLB equals the ML reflectivity estimator’s MSE, making that estimator asymptotically efficient.The exact binomial-likelihood CRLB reduces to the first-order Taylor expansion of the Poisson-likelihood expression.

B. Mean-Square Error of 3D Estimation

The appendix derives pixelwise depth-estimation MSE expressions under low-flux assumptions and examines contributions from photon timing and missing detections. At high photon counts, pulse width becomes the dominant error source.

  • Depth CRLB: The depth CRLB is derived under the asymptotic low-flux regime, with the single-pulse rate λ_i,j(t) and its time derivative entering the expression.The assumptions include ηα_i,jS + B → 0+ and N → ∞ with a constant scaled detection probability.
  • Depth MSE: For Gaussian illumination pulses, the appendix exactly computes the depth MSE using the unconstrained log-matched-filter estimator.The resulting expression applies when detected photon count k_i,j is at least one.
  • Missing Detections: When k_i,j = 0, depth is imputed by a uniformly random guess over [0, cT_r/2), introducing a random-guess error term.This expression assumes zero background, B = 0.
  • Asymptotic Behavior: As C(α_i,j) → ∞, the pulse-width error term dominates the depth-estimation MSE.The appendix also states that the corresponding ML depth estimator becomes efficient.
Loading 1406.1761v1…