Source-linked AI summary
Non-line-of-sight Imaging
D. Faccio, A. Velten, G. Wetzstein
TL;DR
NLOS imaging seeks to reconstruct 3D scenes hidden from direct view using indirect light paths. This review models time-resolved image formation, surveys inverse methods and resolution limits, and finds multiple approaches whose advantages must be weighed for each application.
Problem
NLOS imaging seeks to reconstruct 3D scenes outside a camera’s direct line of sight, while many approaches require scanning a large visible relay surface.
Method
The review presents a general time-resolved image-formation model, surveys inverse methods, and discusses fundamental bounds on hidden-object reconstruction resolution.
Results
The review identifies multiple NLOS approaches with distinct advantages and hardware requirements for different applications.
Takeaways & Limitations
NLOS systems should be selected according to application needs, which may range from full 3D reconstruction to locating and identifying hidden objects.
Takeaways & Limitations
The best representation for general NLOS imaging remains unclear, requiring practical tradeoffs between reconstructed detail and memory use.
Abstract
from arXiv · showhide
Emerging single-photon-sensitive sensors combined with advanced inverse methods to process picosecond-accurate time-stamped photon counts have given rise to unprecedented imaging capabilities. Rather than imaging photons that travel along direct paths from a source to an object and back to the detector, non-line-of-sight (NLOS) imaging approaches analyse photons {scattered from multiple surfaces that travel} along indirect light paths to estimate 3D images of scenes outside the direct line of sight of a camera, hidden by a wall or other obstacles. Here we review recent advances in the field of NLOS imaging, discussing how to see around corners and future prospects for the field.
Inverse Light Transport for Time-resolved NLOS Imaging.
Time-resolved NLOS imaging models indirect light transport as a transient image and reconstructs hidden scenes through several inverse-method families. Resolution is fundamentally constrained by temporal separability, while wave-optics approaches, SPAD-enabled systems, and application-specific trade-offs shape future capabilities.
- Image Formation Model: The section presents a general time-resolved image-formation model, inverse methods, and fundamental bounds on hidden-object reconstruction resolution.The model represents measurements as a 3D space-time transient image containing directly reflected and indirect-path photons.
- Image Formation Model: The common NLOS model is nonlinear because attenuation depends on hidden-surface and visibility factors; setting g = 1 yields an easier linear approximation with added assumptions.The linearized model assumes isotropic scattering and no occlusions between scene points.
- Backprojection Methods: Backprojection is a popular reconstruction strategy, with confocal and non-confocal scanning related to spherical and elliptical Radon transforms, respectively.Filtered backprojection provides a standard solution for these inverse problems.
- Wave Optics: Wave-optics NLOS methods can handle glossy, specular, diffuse, and retro-reflective hidden materials with the same method, unlike geometric-optics approaches requiring reflectance modeling or estimation.The reported robustness applies across different hidden-surface reflectance properties.
- Resolution Limits: NLOS resolution is defined by the minimum separable distance between two scatterers, which requires their indirect reflections to be temporally resolvable.For confocal scanning, transverse and axial limits depend on temporal impulse-response FWHM, scatterer distance z, and scan size 2w × 2w m2; non-confocal transverse resolution theoretically decreases by a factor of two.
- Applications and Future Prospects: SPADs are well suited to extending LiDAR toward NLOS imaging, and NLOS capability could eventually be delivered as a software upgrade for LiDAR systems used in applications such as autonomous driving.The section emphasizes that competing NLOS approaches have distinct advantages that must be weighed for each application.