Source-linked AI summary
Optically lightweight tracking of objects around a corner
Jonathan Klein, Christoph Peters, Jaime Martín, Martin Laurenzis, Matthias B. Hullin
TL;DR
The paper addresses around-corner tracking without specialized ultrafast detectors or narrowband sources. It uses analysis-by-synthesis light-transport simulation with a standard 2D camera and laser pointer, demonstrating real-time tracking of unknown-object translation and known-object translation and orientation.
Problem
Around-corner object observation is difficult, while prior diffuse-reflection methods require ultrafast detectors or narrowband light sources.
Method
The approach repeatedly simulates three-bounce light transport and optimizes object pose parameters to match measured camera intensity distributions.
Results
The system tracked unknown-object translation and known-object translation and orientation in real time using a standard 2D camera and laser pointer.
Takeaways & Limitations
Around-corner tracking can be performed with temporally unresolved 2D images and a laser pointer, including room-sized scenes with sub-cm repeatability.
Takeaways & Limitations
The method assumes diffuse Lambertian surfaces and neglects mutual shadowing or inter-reflection between surfels, making the approximation most suitable for flat or mostly convex objects.
Abstract
from arXiv · showhide
The observation of objects located in inaccessible regions is a recurring challenge in a wide variety of important applications. Recent work has shown that indirect diffuse light reflections can be used to reconstruct objects and two-dimensional (2D) patterns around a corner. However, these prior methods always require some specialized setup involving either ultrafast detectors or narrowband light sources. Here we show that occluded objects can be tracked in real time using a standard 2D camera and a laser pointer. Unlike previous methods based on the backprojection approach, we formulate the problem in an analysis-by-synthesis sense. By repeatedly simulating light transport through the scene, we determine the set of object parameters that most closely fits the measured intensity distribution. We experimentally demonstrate that this approach is capable of following the translation of unknown objects, and translation and orientation of a known object, in real time.
Results
The system tracks occluded objects by matching measured wall intensity patterns to simulated three-bounce light transport, using a standard 2D camera and laser pointer. Experiments show real-time translation tracking for unknown objects and joint translation–rotation tracking for known objects, with practical accuracy limited by model bias and rotational noise.
- Results: Real-time tracking used a standard 2D camera and laser pointer, optimizing simulated three-bounce light transport against measured intensity images.The method represents objects with surfels and minimizes a least-squares cost over translation and rotation parameters.
- Results: Translation recovery errors were 4.8 cm, 2.9 cm, and 2.4 cm along the X, Y, and Z axes, respectively.The reported systematic bias was attributed to unmodeled occlusion of ambient light by the object.
- Results: Known-object rotation tracking had higher uncertainty because rotation changes the signal less and is ambiguous with translational motion.The ambiguity involved X translation versus Z rotation and Z translation versus X rotation; Y rotation was tracked robustly for the asymmetric car object.
- Results: Unknown-object tracking remained robust to noise and detected relative motion despite a systematic shift from the simplified object model.The simulation used a single oriented surface element instead of the detailed object model.
- Results: Linear background subtraction reduced bias and enabled robust motion tracking without a measured background in unknown rooms.Omitting background compensation degraded tracking, while fitting and subtracting a linear background model restored feasibility.
- Results: 10.2 frames per second for translation and 3.7 frames per second for translation plus rotation demonstrated real-time operation.Translation-only processing was limited by camera–laser capture, whereas joint translation–rotation processing was limited by computation.
Methods
The method simulates three-bounce diffuse light transport and compares processed camera measurements with scene hypotheses under simplified reflectance and geometry assumptions.
- Light transport simulation: Three-bounce light transport is simulated by summing surfel contributions, using radiative-transfer view factors and assumed diffuse reflectance.The renderer models laser-to-surfel-to-wall paths and aggregates each surfel’s contribution at camera pixels.
- Assumptions: The renderer neglects mutual shadowing and inter-reflection between surfels, an approximation justified for flat or mostly convex objects.More accurate BRDFs and textures can be incorporated when available.
- Pre-processing: Images are downsampled to 160×128 pixels, and averaging 300 background difference images reduces noise in the background estimate.Downsampling is considered safe because diffuse reflections contain little high-frequency information beyond noise.
- Measurement model: Measured images combine ambient, static-background, dynamic-object, photon, and read-noise contributions before reconstruction.The input image is formed from rapid laser-on/laser-off differences and a background calibration subtraction.