Source-linked AI summary
Single-pixel 3D imaging with time-based depth resolution
Ming-Jie Sun, Matthew. P. Edgar, Graham M. Gibson, Baoqing Sun, Neal Radwell, Robert Lamb, Miles J. Padgett
TL;DR
Real-time 3D imaging requires video-rate acquisition, while existing algorithms can prohibit real-time applications. This paper presents a pulsed structured-illumination single-pixel system that achieves 128 × 128 resolution, ~3 mm ranging accuracy, and continuous real-time 3D video.
Problem
Real-time 3D imaging applications demand video frame-rates, while reconstruction algorithms can prohibit real-time operation.
Method
The system combines pulsed structured illumination with a high-speed photodiode that records the temporal form of back-scattered light.
Results
~3 mm ranging accuracy is achieved at 128 × 128-pixel resolution, alongside continuous real-time 3D video.
Takeaways & Limitations
The system demonstrates millimetric 3D ranging using modest hardware and real-time video rates.
Takeaways & Limitations
System performance was mainly limited by the employed laser’s 7.4 kHz repetition rate.
Abstract
from arXiv · showhide
Time-of-flight three dimensional imaging is an important tool for many applications, such as object recognition and remote sensing. Unlike conventional imaging approach using pixelated detector array, single-pixel imaging based on projected patterns, such as Hadamard patterns, utilises an alternative strategy to acquire information with sampling basis. Here we show a modified single-pixel camera using a pulsed illumination source and a high-speed photodiode, capable of reconstructing 128x128 pixel resolution 3D scenes to an accuracy of ~3 mm at a range of ~5 m. Furthermore, we demonstrate continuous real-time 3D video with a frame-rate up to 12 Hz. The simplicity of the system hardware could enable low-cost 3D imaging devices for precision ranging at wavelengths beyond the visible spectrum.
Introduction
The paper presents a single-pixel 3D imaging system using pulsed structured illumination, a high-speed analogue photodiode, and an original reconstruction algorithm. It achieves ∼3 mm range-profile accuracy at ∼5 m with 128 × 128-pixel resolution and continuous real-time 3D video up to 12 Hz.
- Limitations of photon counting: Photon-counting detectors have electronic dead-times often 10’s of nanoseconds, preventing short-range timing from a single illumination pulse.Accurate temporal responses therefore require data summed over many back-scattered photons and usually several hundreds or thousands of illumination pulses.
- Advantages of analogue detection: A high-speed photodiode can retrieve the temporal response from a single illumination pulse, which is advantageous when reflected light intensity is comparatively large.Photon counting cannot operate under such conditions because nearer or more reflective objects trigger detection and render distant objects invisible.
- Proposed system: The system combines pulsed structured illumination with a short-response photodiode to sample the time-varying intensity of back-scattered light.Analogue recording of the full temporal waveform is paired with an original 3D reconstruction algorithm.
- Demonstrated performance: ∼3 mm range-profile accuracy is demonstrated at ∼5 m with 128 × 128-pixel image resolution while simultaneously recovering object reflectivity.This performance exceeds that implied by the detector and digitisation limits, corresponding to distances of 150 mm and 60 mm respectively.
- Demonstrated performance: 12 Hz frame-rate is achieved for continuous real-time 3D video using a compressive sampling scheme.The system demonstrates continuous real-time 3D video rather than only static scene reconstruction.
Experimental setup
The system combines pulsed structured illumination with high-speed photodiode detection and digitisation for single-pixel 3D reconstruction. Time-resolved measurements form an image cube from which depth and reflectivity are estimated, using cubic-spline interpolation to support efficient range estimation.
- Hardware: A pulsed laser and DMD provide time-varying structured illumination, while a high-speed photodiode with condenser optics measures back-scattered intensity.The analogue output is amplified and sampled by a high-speed digitiser.
- Structured illumination: Hadamard patterns are displayed with their inverse patterns, and differential measured intensities reduce noise from ambient-light fluctuations.The difference is taken between the measured intensities from each pattern pair.
- Reconstruction: M discretely sampled time-varying intensity points reconstruct M 2D images that form an x, y, z image cube for 3D imaging.Each transverse pixel has an intensity distribution along z related to pulse shape, detector response, digitisation, depth, and reflectivity.
- Depth and reflectivity estimation: Cubic spline interpolation adds minimal computational overhead, after which depth is found from each interpolated signal’s maximum and reflectivity from longitudinal averaging.The resulting depth and reflectivity information are combined to produce a 3D scene image.
- Limitations: Depth estimation assumes one surface per transverse pixel and works well for smooth features such as a mannequin head and ball.Amplitude noise limits depth accuracy; over-interpolation can increase precision without necessarily improving accuracy and also increases processing time.
Results
The system reconstructed distinguishable 128 × 128 pixel 3D scenes at approximately 5.5 m, achieving an RMSE of 2.62 mm against stereophotogrammetric reference data. Time-gating isolated a mannequin head behind obscuring netting, while evolutionary compressive sensing enabled continuous real-time 3D video at 5 Hz and up to 12 Hz with fewer patterns.
- Static 3D reconstruction: 128 × 128 pixel 3D reconstruction resolved distinguishable features, including the mannequin head profile and soccer ball, from a scene approximately 5.5 m away.Acquisition, data transfer, and image processing took approximately 130 seconds.
- Quantitative accuracy: 2.62 mm RMSE was measured for the reconstructed mannequin head relative to reference 3D data from a stereophotogrammetric camera system.The head was located 5.5 m from the imaging system.
- Imaging through obscuring material: Time-gated reconstruction isolated characteristic mannequin-head features despite black netting obscuring the line of sight.Artificial gating excluded reflected signals from the netting from the 3D reconstruction.
- Real-time reconstruction: Evolutionary compressive sensing selected informative Hadamard patterns from previous-frame intensities and reconstructed images through linear iteration.This scheme was introduced to reduce reconstruction time relative to conventional compressive sensing, whose algorithm often prohibits real-time application.
Discussion
The system achieved millimetric ranging accuracy and real-time video using modest hardware and a modified compressive sensing scheme without lengthy post-processing. Performance was limited by the 7.4 kHz laser repetition rate, while faster video and non-visible-wavelength extensions are possible.
- Performance: Millimetric ranging accuracy and real-time video rates were achieved using modest hardware and modified compressive sensing without lengthy post-processing.The modified scheme enabled real-time operation by avoiding lengthy post-processing.
- Limitations: 7.4 kHz laser repetition rate mainly limited system performance.The limitation arose from the repetition rate of the employed laser.
- Future improvements: A laser with repetition rate greater than or equal to the DMD modulation rate could increase 3D video rates by a factor of three and/or improve reconstruction accuracy through increased averaging.These improvements depend on replacing the limiting laser with one meeting or exceeding the DMD modulation rate.
- Wavelength extension: 400 nm −2500 nm DMD operational spectrum could support extension to non-visible wavelengths such as infrared using modified source and detection optics.Infrared 3D imaging could provide enhanced long-range visibility due to reduced atmospheric effects.
Methods
The experimental setup combines pulsed laser illumination, DMD pattern projection, photodiode detection, and high-speed digitization. The system operates at a 7.4 kHz laser repetition rate with a matched 2.6 ◦ field of view and a multi-step depth-estimation pipeline.
- Experimental setup: The setup uses a 532 nm pulsed laser, Texas Instruments Discovery 4100 DMD, Nikon ED projection lens, Fresnel collection lens, Si photodiode, and PicoScope digitizer.The digitizer provides 2.5 GSs−1 for 2 channels acquisition.
- Modulation and acquisition: 7.4 kHz determines the DMD modulation rate because the DMD operates in slave-mode, despite reaching up to 22.7 kHz.The modulation rate follows the laser repetition rate in this experiment.
- Optical alignment: 2.6 ◦ field-of-view is produced by the 0.8 mm2 photodiode active area and 20 mm focal length Fresnel lens system, matching the projection system.The photodiode active area is 0.8 mm2, and the Fresnel lens focal length is 20 mm.
- Depth estimation: Depth estimation applies Gaussian smoothing, intensity calibration, cubic spline interpolation, and depth determination.These processing steps form the stated depth-estimation procedure.