Source-linked AI summary
Micro Fourier Transform Profilometry ($μ$FTP): 3D shape measurement at 10,000 frames per second
Chao Zuo, Tianyang Tao, Shijie Feng, Lei Huang, Anand Asundi, Qian Chen
TL;DR
Existing high-speed 3D methods are limited by the number of projected patterns needed for phase recovery and disambiguation. The paper introduces μFTP, combining a high-frame-rate DLP system with a two-pattern reconstruction framework that encodes phase in a single high-frequency fringe image. It achieves dense, unambiguous 3D imaging at 10,000 fps, while the reconstruction implementation remains computationally slow.
Problem
High-speed 3D reconstruction is limited because reliable phase recovery and disambiguation require multiple projected patterns, while transient scenes need depth information at high frame rates.
Method
μFTP combines high-speed DLP fringe projection with phase recovery, unwrapping, error compensation, calibration, and temporal frequency variations to reconstruct 3D shape from two projected patterns.
Results
10,000 fps, depth accuracy better than 80 µm, and temporal uncertainties below 75 µm were achieved over a 400 mm × 275 mm × 400 mm measurement volume.
Takeaways & Limitations
μFTP provides dense, unambiguous, motion-artifact-free 3D imaging with high-quality 2D texture for transient scenes.
Takeaways & Limitations
Reconstruction is not yet fully optimized; one 3D frame takes approximately 870 ms on a desktop computer.
Abstract
from arXiv · showhide
Recent advances in imaging sensors and digital light projection technology have facilitated a rapid progress in 3D optical sensing, enabling 3D surfaces of complex-shaped objects to be captured with improved resolution and accuracy. However, due to the large number of projection patterns required for phase recovery and disambiguation, the maximum fame rates of current 3D shape measurement techniques are still limited to the range of hundreds of frames per second (fps). Here, we demonstrate a new 3D dynamic imaging technique, Micro Fourier Transform Profilometry ($μ$FTP), which can capture 3D surfaces of transient events at up to 10,000 fps based on our newly developed high-speed fringe projection system. Compared with existing techniques, $μ$FTP has the prominent advantage of recovering an accurate, unambiguous, and dense 3D point cloud with only two projected patterns. Furthermore, the phase information is encoded within a single high-frequency fringe image, thereby allowing motion-artifact-free reconstruction of transient events with temporal resolution of 50 microseconds. To show $μ$FTP's broad utility, we use it to reconstruct 3D videos of 4 transient scenes: vibrating cantilevers, rotating fan blades, bullet fired from a toy gun, and balloon's explosion triggered by a flying dart, which were previously difficult or even unable to be captured with conventional approaches.
Introduction
High-speed 3D sensing is constrained by the patterns needed for accurate phase recovery and ambiguity resolution. μFTP combines high-speed DLP projection with a two-pattern computational framework to reconstruct dense, unambiguous 3D shape at 10,000 fps.
- Motivation: High-speed cameras provide fast 2D recordings, but transient-scene analysis often requires high-frame-rate 3D information.The motivation spans biomechanics, industrial inspection, solid mechanics, and vehicle impact testing.
- Limitations of existing methods: Multi-frame profilometry achieves dense, accurate reconstruction but requires several patterns, limiting speed and causing motion-induced phase distortion.Phase shifting profilometry requires at least three fringe images, while motion between images can create artifacts.
- Limitations of existing methods: Stereo and related approaches can require more than nine images and at least two high-speed cameras, leaving 3D rates near hundreds of hertz.Examples include 330 Hz with array projection and 1,333 Hz with GOBO projection.
- μFTP approach: μFTP encodes phase in a single high-frequency sinusoidal pattern and resolves ambiguity using small temporal frequency variations.The method is designed for dense 3D reconstruction without restrictions on surface texture, scene complexity, or object motion.
- μFTP approach: The developed DLP fringe-projection system synchronizes binary pattern switching and image capture at up to 20,000 Hz.It combines a high-speed CMOS camera with a DLP-based projection system.
- μFTP approach: The computational framework recovers accurate, unambiguous, distortion-free 3D point clouds from every two projected patterns.Its stages include phase recovery, phase unwrapping, error compensation, and system calibration.
- Results: μFTP achieves 10,000 fps with depth accuracy better than 80 µm and temporal uncertainties below 75 µm in a 400 mm × 275 mm × 400 mm volume.The paper demonstrates textured 3D imaging of vibrating cantilevers, rotating fan blades, a flying bullet, and a bursting balloon.
Materials and methods
The materials and methods combine a synchronized high-speed binary fringe projector with μFTP reconstruction to recover dense 3D shape from few projected patterns. The system uses optimized timing, multi-wavelength patterns, background-normalized phase retrieval, phase unwrapping, and calibrated phase-to-coordinate mapping.
- Hardware system: 20,000 Hz: The custom DLP system synchronizes binary pattern projection with high-speed CMOS image capture.The camera operates at reduced resolution with a 46 µs exposure, while custom hardware provides synchronization.
- Hardware system: 42.72 µs: DMD pattern output combines 30.72 µs loading, 4 µs mirror transition, and 8 µs settling times.The system uses a 50 µs switching period and timing designed to maximize exposure while avoiding mirror-transition crosstalk.
- Acquisition: 2n patterns: μFTP sequentially projects n ≥ 2 high-frequency sinusoidal patterns with slightly different wavelengths, inserting a white pattern between sinusoids.For the demonstrated configuration, wavelengths {14, 16, 18} pixels provide a tradeoff between fringe contrast and unambiguous phase range.
- Acquisition: The wavelength set uses small fringe pitches for phase retrieval, while its least common multiple must span the varying-intensity axis to exclude phase ambiguities.These constraints jointly determine the selected multi-wavelength pattern set.
- Reconstruction: BNFTP uses each sinusoidal fringe image with a corresponding white image to recover wrapped phases despite zero-order background and reflectivity variation.The reconstruction processes a 2n-frame sliding window and obtains n wrapped phase maps.
- Reconstruction: The refined absolute phase establishes projector-camera correspondences and reconstructs 3D coordinates using calibrated triangulation with lens-distortion correction.The pipeline uses phase unwrapping and spatial refinement before the final mapping to 3D coordinates.
Results
µFTP achieves accurate, repeatable high-speed 3D measurement and reconstructs transient scenes including vibrating, rotating, ballistic, and explosive events. Comparisons show reduced motion artifacts and improved robustness under rapid motion and reflectivity variation.
- 3D reconstruction accuracy: Better than 80 µm accuracy and below 75 µm temporal STD are achieved at 10,000 3D fps in a 400 mm × 275 mm × 400 mm measurement volume.The center-to-center distance had 22.433 µm temporal STD, sphere points typically around 60 µm, and the moving ball radius 72.815 µm.
- 3D reconstruction accuracy: 820 measurements over 41 ms demonstrate repeatability for sphere points, inter-sphere distance, and the radius of a free-falling table tennis ball.The measurements were acquired at a pseudo frame rate of 20,000 fps.
- Vibrating cantilevers: µFTP reconstructs vibrating cantilevers in 3D and quantitatively tracks surface-point displacements, including amplitudes decreasing from about 50 mm to 10 mm at approximately 8 Hz.The two cantilevers vibrate out of phase because their hand releases differ by about 20 ms.
- Comparison with phase shifting profilometry: Compared with three-wavelength PSP, µFTP avoids notable motion ripples and produces distortion-free 3D reconstruction with high-quality 2D texture during fast, large-amplitude motion.The comparison used 6 projected patterns for µFTP and 9 for three-wavelength PSP, with a 400 µs timing difference between reconstructions.
- Robustness to reflectivity variation: With large reflectivity variations, conventional FTP shows artifacts and unwrapping errors, modified FTP retains severe fluctuations, whereas µFTP produces a smoother reconstruction without notable artifacts.The comparison used the same raw image data and the same phase-unwrapping and compensation algorithms.
Discussion
µFTP demonstrates dense, precise 3D measurement at 10,000 fps while acknowledging that its post-processing speed remains insufficiently optimized. GPU-parallelizable algorithms could enable low-latency real-time reconstruction with suitable synchronous camera acquisition.
- 10,000 fps measurement achieves depth accuracy better than 80 µm and temporal uncertainties below 75 µm across a 400 mm × 275 mm × 400 mm volume.
- Two images provide high-resolution, unambiguous depth information alongside high-quality 2D textures from the white patterns.
- µFTP extends unambiguous, motion-artifact-free 3D imaging toward tens of kilohertz, supported by experiments on rapidly moving and suddenly deforming objects.
- 870 ms is required to reconstruct one 3D frame on the reported desktop implementation, so processing speed is not yet fully optimized.
- GPU execution could substantially improve reconstruction because the 2D fast Fourier transform and pixel-wise PDM phase unwrapping are highly parallelizable.