Source-linked AI summary
Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement
Jiaming Qian, Shijie Feng, Tianyang Tao, Yan Hu, Yixuan Li, Qian Chen, Chao Zuo
TL;DR
Efficient absolute-phase recovery is a central challenge in high-speed fringe projection profilometry, where conventional methods face auxiliary-pattern, robustness, and measurement-volume constraints. The paper combines deep neural networks with stereo geometric constraints to learn phase retrieval and unwrapping from single-frame projection. Experiments report high-quality reconstruction, robust ambiguity removal over a larger depth range, motion-artifact-free operation, and quantitative sphere-measurement errors in the tens of micrometres.
Problem
Efficiently recovering absolute phase for high-speed, single-frame FPP is challenging because conventional SPU has robustness, measurement-range, and implementation limitations.
Method
Two convolutional neural networks combine phase acquisition, geometric constraints, and phase unwrapping, using two-camera fringe data and one pre-obtained reference-plane information set.
Results
The method accurately removes phase ambiguity over a large depth range and achieves sphere-radius deviations of 52.7 µm and 61.0 µm, with a center-distance error of 65.3 µm.
Takeaways & Limitations
Single-frame projection with fewer cameras and simpler algorithms supports high-quality, motion-artifact-free absolute 3D measurement in high-speed scenarios.
Abstract
from arXiv · showhide
Fringe projection profilometry (FPP) is one of the most popular three-dimensional (3D) shape measurement techniques, and has becoming more prevalently adopted in intelligent manufacturing, defect detection and some other important applications. In FPP, how to efficiently recover the absolute phase has always been a great challenge. The stereo phase unwrapping (SPU) technologies based on geometric constraints can eliminate phase ambiguity without projecting any additional fringe patterns, which maximizes the efficiency of the retrieval of absolute phase. Inspired by the recent success of deep learning technologies for phase analysis, we demonstrate that deep learning can be an effective tool that organically unifies the phase retrieval, geometric constraints, and phase unwrapping steps into a comprehensive framework. Driven by extensive training dataset, the neutral network can gradually "learn" how to transfer one high-frequency fringe pattern into the "physically meaningful", and "most likely" absolute phase, instead of "step by step" as in convention approaches. Based on the properly trained framework, high-quality phase retrieval and robust phase ambiguity removal can be achieved based on only single-frame projection. Experimental results demonstrate that compared with traditional SPU, our method can more efficiently and stably unwrap the phase of dense fringe images in a larger measurement volume with fewer camera views. Limitations about the proposed approach are also discussed. We believe the proposed approach represents an important step forward in high-speed, high-accuracy, motion-artifacts-free absolute 3D shape measurement for complicated object from a single fringe pattern.
I. INTRODUCTION
FPP enables accurate optical 3D measurement, but efficiently recovering absolute phase remains difficult in high-speed, single-frame scenarios. The paper addresses SPU’s limitations by integrating deep learning with geometric constraints for single-frame phase retrieval and unwrapping.
- FPP is widely used for optical 3D shape measurement because of its simple hardware, implementation flexibility, and high measurement accuracy.
- Single-frame acquisition is the ideal approach for improving FPP efficiency in high-speed 3D measurement.High-quality 3D information is increasingly important for online inspection, stress deformation analysis, and rapid reverse molding.
- TPU requires Gray-code or multi-wavelength auxiliary patterns, whereas SPU removes phase ambiguity through camera-projector geometric relationships without additional patterns.SPU therefore targets higher acquisition efficiency but relies on multiple camera views and suitable phase information.
- SPU remains limited by measurement volume, dense high-frequency fringe unwrapping, multi-frame phase acquisition, calibration demands, and algorithmic complexity.
- The proposed network implicitly incorporates geometric constraints and learns to obtain physically meaningful absolute phase from single-frame projection instead of following a conventional step-by-step process.
A. Phase retrieval and unwrapping with PS and SPU
Traditional PS and SPU retrieve wrapped phase and resolve fringe-order ambiguity using geometric constraints and phase matching. Their robustness is challenged by calibration errors, dense high-frequency fringes, narrow depth ranges, and multi-frame acquisition requirements.
- Phase retrieval with PS: An N-step PS system captures phase-shifted fringe patterns with two cameras before estimating the wrapped phase.The captured intensity depends on average intensity, amplitude, absolute phase, and the applied phase shift.
- Phase unwrapping with SPU: SPU reconstructs K 3D candidates for each camera point, projects them into another camera, and selects the matching point through wrapped-phase similarity.The selected fringe order k resolves the absolute phase ambiguity.
- Phase unwrapping with SPU: Calibration errors and ambient illumination can make an incorrect candidate appear more phase-similar than the correct match.Higher fringe frequency increases the number of candidates and the likelihood of such errors.
- Practical limitations: Multi-step PS improves measurement accuracy and robustness to ambient illumination, but high-frequency fringes are not recommended for conventional SPU.
- Practical limitations: Adding camera views or depth constraints can improve SPU stability, but increases hardware or algorithmic complexity and conventional depth constraints operate only over a narrow volume.
B. Phase retrieval and unwrapping with deep learning
The method combines deep neural networks with stereo phase unwrapping to retrieve high-quality wrapped phase and determine fringe orders from single-frame, dual-camera measurements.
- The proposed framework combines deep learning and stereo phase unwrapping to integrate phase retrieval, geometric constraints, and phase unwrapping.
- CNN1 processes single-frame fringe images from two cameras to learn high-quality phase information for wrapped-phase retrieval.Its outputs are the numerators and denominators of the arctangent function rather than directly linked wrapped phases.
- CNN2 uses two-camera fringe patterns and pre-obtained reference-plane information to estimate the measured object's fringe-order map.The reference information is obtained once for the setup and need not be repeatedly acquired in subsequent experiments.
- The resulting wrapped phases and fringe orders are combined to recover high-quality unwrapped phase, followed by 3D reconstruction using calibrated camera parameters.
III. EXPERIMENTS
Experiments use a dual-camera FPP system with 48-period patterns and a 240mm×200mm measurement field, trained on 1001 scenarios and evaluated on dynamic scenes.
- The experimental system uses one projector and two cameras, with 48-period phase-shifting patterns and an approximately 240mm×200mm measuring field.The projector resolution is 912 × 1140 and the camera resolution is 640 × 480.
- Training datasets contain 1001 different scenarios, and training and validation losses converge without overfitting after hundreds of epochs.Further training-data and process details are provided in Appendix C.
- Four continuously moving scenarios are used to evaluate dynamic-target measurement, although all training and validation datasets were collected in static scenes.
A. Qualitative evaluation
Qualitative tests compare the method with conventional approaches on static and dynamic scenes, showing broader phase disambiguation and motion-artifact-free dynamic reconstruction.
- Static scenes: In four static scenes, the method is compared with three alternatives, with the first method providing ground-truth data.The figure groups results by method across four scenes.
- Static scenes: The reference-plane approach unwraps phase only within a limited range, whereas the proposed method accurately removes ambiguity across a large depth range.The limited range is between −π and π of the reference plane's absolute phase.
- Static scenes: The proposed approach produces reconstruction quality almost equivalent to conventional PS, triple-camera SPU, and ADC methods.
- Dynamic scenes: For continuously moving scenes, multi-frame phase shifting produces motion-induced artifacts, while SPU results are sensitive to phase errors.
- Dynamic scenes: Because it is single-shot, the proposed approach can measure dynamic scenes uninterruptedly without motion artifacts.
B. Quantitative evaluation
Quantitative evaluation with two standard spheres reports micrometer-scale reconstruction errors and supports high-quality 3D measurement using fewer cameras and projection images.
- 52.7 µm and 61.0 µm are the radius deviations measured for reconstructed spheres with nominal radii of 25.3989 mm and 25.4038 mm.
- 65.3 µm is the error in the measured 99.9878 mm center distance between the reconstructed spheres.The standard-sphere center-to-center distance is 100.0532 mm with 1.1 µm uncertainty.
- The sphere experiment validates high-quality 3D measurements with fewer cameras, fewer projection images, and simpler algorithms.
A. Conclusions
The paper presents a deep-learning-enabled approach for single-shot absolute 3D shape measurement. It targets dense-fringe phase ambiguity in a larger measurement range with fewer perspectives and simpler algorithms, while acknowledging method limitations.
- The approach combines deep learning with geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement.
- It addresses phase ambiguity in dense fringes over a larger measurement range using less perspective information and simpler algorithms.
- Depth discontinuities can produce missing fringe order and continuity artifacts that limit the method.
- The method is intended to support high-accuracy, motion-artifacts-free absolute 3D shape measurement for complicated objects in high-speed scenarios.
B. Discussions
The discussion explains how the framework integrates traditionally sequential processing stages while using calibrated multi-camera geometry for reconstruction. It also describes the coordinate convention and calibration setup.
- Deep learning integrates phase acquisition, geometric constraints, and phase unwrapping into one comprehensive framework rather than reproducing them step by step.
- CNN1 uses convolutional and residual paths, while additional paths down-sample and upsample data for feature extraction.
- After absolute phase recovery, calibrated two-camera matching enables 3D reconstruction and automatically cancels nonlinearity errors.
- Figure 7 specifies the relative placement of standard spheres and the calibration board at the first calibration pose.
- The reconstructed coordinates use a world coordinate system whose 0-depth plane is the first calibration pose.
Appendix C. Training the neural networks
The networks are trained on diverse synthetic scenes assembled from simple and complex objects. Training and validation losses converge, with different training durations for CNN1 and CNN2 because their outputs use different scales.
- The training data comprise 1001 diverse scenes formed by arbitrarily combining and rotating different simple and complex objects.
- Each training set contains three camera views with 3-step phase-shifting fringe patterns, ground-truth numerator and denominator maps, and fringe-order maps.
- CNN1 loss curves converge after about 200 epochs, while CNN2 loss curves converge after 120 epochs.
- Training takes 25.56 hours for CNN1 over 400 epochs and 19.25 hours for CNN2 over 300 epochs.
- The loss scales differ because numerator M and denominator D can reach hundreds, whereas normalized fringe orders k are on a different scale.