PaperlayerView exact revision on arXiv

Source-linked AI summary

RACE-FPP: A Robust AI-assisted Characterisation Enhancement for Fringe Projection Profilometry

Osman Ali, Xiangjun Kong, Tibebe Yalew, Waiel Elmadih, Samanta Piano

arXiv:2610.08213v1cs.CVeess.IVphysics.optics

TL;DR

FPP characterization depends on checkerboard corner localization, yet challenging imaging conditions can impair detection, and existing corner-detector evaluations often omit broader system-level effects. RACE-FPP integrates learned corner localization with center-based projector characterization; as image quality deteriorated, it achieved lower camera, projector, and stereo reprojection errors than OpenCV.

  • Problem

    FPP characterization depends on accurate checkerboard corners, but challenging imaging conditions hinder localization and existing evaluations often omit effects on projector and stereo characterization.

    Source: Introduction · pp. 1–2

  • Method

    RACE-FPP integrates learned checkerboard-corner localization and white-square-center phase sampling into camera, projector, and stereo characterization while retaining the conventional geometric model.

    Source: Introduction · p. 2, Materials and methods · p. 3, Conclusion · p. 16

  • Results

    As image quality deteriorated, RACE-FPP achieved lower camera, projector, and stereo reprojection errors than OpenCV; on mixed images, camera global RMS error was 0.2591 px versus 1.2365 px.

    Source: Results and discussion · pp. 12–13

  • Takeaways & Limitations

    The approach maintains a more stable camera–projector relationship as image quality deteriorates.

    Source: Results and discussion · p. 14

    Evaluation covers a single FPP configuration, with transferability across camera resolutions, fields of view, optical geometries, and measurement systems left for future work.

    Source: Conclusion · p. 16

Abstract

from arXiv · show

Fringe Projection Profilometry (FPP) requires precise system characterisation to achieve reliable three-dimensional (3D) reconstructions; however, characterisation accuracy strongly depends on robust checkerboard feature localisation, which can deteriorate under challenging imaging conditions such as lens blur and characterisation target orientations. Existing deep learning-based corner detectors are typically assessed using detection metrics and camera reprojection error alone, without considering their wider impact on projector characterisation, camera-projector stereo characterisation consistency, or overall measurement accuracy. In this work, we introduce a complete FPP characterisation pipeline that incorporates deep learning-based corner detection into the standard camera characterisation workflow. We also characterise the projector by sampling phase values at the centres of the white squares in the characterisation target. Rather than treating corner detection as an isolated task, the proposed framework explicitly analyses how localisation errors propagate throughout the entire FPP characterisation chain. Performance is evaluated using detection metrics (e.g., precision and recall), camera and projector reprojection errors, and the camera and projector stereo characterisation. Across a mixed dataset of clean and degraded images, the camera reprojection error is reduced from 1.237 pixels to 0.259 pixels, while the projector reprojection error is reduced by roughly 50%. Dimensional evaluation of reconstructed artefacts shows improved geometric accuracy compared with those resulting from the conventional pipeline. Overall, the findings indicate increased robustness of system-level characterisation under challenging imaging conditions, thereby enabling more reliable industrial FPP measurements.

Introduction

The paper addresses how unreliable checkerboard localisation under non-ideal imaging conditions can affect the full FPP characterisation chain, proposing learned corner detection and centre-based projector correspondences. It evaluates characterisation through 3D reconstruction and dimensional measurements, not detection alone.

Introduction

  • Checkerboard localisation errors, including sub-pixel errors, can propagate through characterisation and stereo estimation, degrading reconstructed geometry and dimensional measurements.Lens defocus and environmental noise make localisation more difficult; biases in camera or projector parameters can also distort their estimated relative pose and triangulation.Source: Introduction · pp. 1–2
  • Conventional corner detectors can lose performance under practical FPP conditions, while learned methods predict corner coordinates by regression or heatmaps without explicit edge or intersection steps.Harris-style methods remain computationally demanding and limited in localisation precision, and FAST/SUSAN combinations can degrade under defocus, noise, and extreme board poses.Source: Introduction · p. 2
  • Existing evaluations largely emphasize detection metrics or camera reprojection error, leaving projector characterisation, camera-projector stereo consistency, and measurement accuracy less examined.Phase errors from reflectivity changes, low signal-to-noise ratio, underexposure, or saturation can impair world-to-projector correspondences.Source: Introduction · p. 2
  • The framework uses YOLOv11 to localise checkerboard corners for camera characterisation and samples absolute phase at derived white-square centres for projector correspondences.Sampling square centres avoids black-white reflectivity discontinuities and uses more uniform reflective regions.Source: Introduction · p. 2, Figure 1 · p. 3
  • The evaluation traces performance from camera, projector, and stereo characterisation through 3D reconstruction to local and global dimensional accuracy against CMM measurements.Clean images provide a reference baseline, while the primary focus is degraded imaging, including uniform and spatially varying degradation.Source: Introduction · pp. 2–3

Materials and methods

RACE-FPP integrates learned checkerboard corner localisation with camera and projector characterisation, stereo calibration, and dense 3D reconstruction. Its methods use white-square-centre phase sampling and evaluate characterisation across clear, mixed, and noisy images, with reconstructed geometry compared against CMM measurements.

Overview of the proposed characterisation framework

  • RACE-FPP links camera characterisation, projector characterisation, and camera-projector stereo characterisation before dense 3D reconstruction.The complete pipeline is evaluated using checkerboard localisation, camera and projector reprojection errors, stereo consistency, and CMM-referenced dimensional accuracy.Source: Materials and methods · pp. 3–4

Camera characterisation

  • YOLOv11 localises checkerboard corners, which are reordered into a consistent grid and passed to the unchanged Zhang camera-characterisation procedure.The study compares this pipeline with OpenCV corner coordinates using the same geometric estimation procedure.Source: Materials and methods · pp. 3–4
  • Camera pipelines are compared on noise-free, mixed, and noisy images using manually annotated keypoints and camera reprojection error.Mixed images include experimentally observed and synthetic degradation, including partial, directional, or full-image defocus and combinations of blur and noise.Source: Materials and methods · p. 4

Centre-based projector characterisation

  • Two orthogonal multi-frequency fringe sequences recover absolute phase maps, with vertical fringes encoding projector u_p and horizontal fringes encoding v_p coordinates.Phase-shifting demodulation and unwrapping recover the absolute phase used to compute projector coordinates.Source: Materials and methods · pp. 4–5
  • White-square centres replace checkerboard corners for projector phase sampling, using bilinear interpolation to estimate phase at sub-pixel centre locations.White interiors provide more uniform regions and stronger fringe modulation than black-white transitions, where reflectivity discontinuities can increase phase uncertainty; projector reprojection errors are compared across clear, mixed, and noisy datasets.Source: Materials and methods · pp. 5–6, Figure 2 · p. 5

Stereo characterisation and reconstruction

  • Camera, projector, and stereo parameters combine with absolute phase maps for 3D reconstruction, whose geometry is compared with CMM reference measurements.Stereo characterisation estimates the projector’s rotation and translation relative to the camera, the reference coordinate system for reconstruction.Source: Materials and methods · p. 6

Deep learning model

The section describes a YOLOv11 pose-estimation approach that detects checkerboard corners independently, supported by a manually annotated and synthetically degraded dataset, and evaluated against ground-truth keypoints and conventional detection.

Deep learning model

  • Training images comprise 200 captures of a 12 × 13 checkerboard with 132 interior corners, augmented with photometric and blur degradations to represent varied imaging conditions.The original high-resolution images measure 3000 × 2000 pixels; augmentations include brightness, contrast, gamma, Poisson and Gaussian noise, and full-frame and directional defocus blur (Figure 3).Source: Deep learning model · p. 6, Figure 3 · p. 7
  • Corners were manually annotated at high magnification because no automatic detector could be assumed to provide error-free labels under the challenging imaging conditions.Each image has 132 annotation lines with bounding boxes, keypoint coordinates, and visibility flags; a few out-of-frame corners are marked invisible, preserving correspondence across the full pattern.Source: Deep learning model · pp. 6–7
  • Treating each checkerboard corner as an independent target lets the YOLOv11 pose model retain detections in unaffected regions when local degradation disrupts other corners.Confidence scores can help filter uncertain predictions; the approach adapts the target representation without modifying the network architecture.Source: Deep learning model · pp. 6–7
  • The YOLOv11 model uses an 80/20 training-validation split and is evaluated on real characterisation images against manual keypoints and OpenCV checkerboard detections.Training used 1440 × 960 inputs, 200 epochs, batch size four, learning rate 0.001, AdamW, and early stopping; performance monitoring included precision, recall, and mAP.Source: Deep learning model · pp. 7–8

Results and discussion

The section compares the pipelines across image-quality conditions, considering localisation, characterisation and reconstruction outcomes. Performance was comparable under noise-free conditions, with OpenCV performing better on clean images; as image quality deteriorated, RACE-FPP maintained more reliable localisation and characterisation, and its reconstructions retained dimensional accuracy.

Keypoint detection performance

  • Training ended after 110 epochs (25.231 hours); final precision and recall were approximately 99.8% and 98.7%, and both mAP measures converged to approximately 99.5%.Training and validation losses converged without evident divergence.Source: Results and discussion · pp. 8–9
  • PCK@5 px and PCK@10 px reached 100% on noise-free and mixed datasets and exceeded 99% on noisy images; RACE-FPP also detected all 132 corners in the tested image qualities.At 1 px, its PCK was comparable to or higher than OpenCV’s, while both methods had relatively low PCK at 0.5 px.Source: Results and discussion · p. 9, Table 1 · p. 9

Corner localisation accuracy and robustness

  • Under partial/full defocus, mean Euclidean error was 1.22/2.00 px for RACE-FPP versus 2.27/3.30 px for OpenCV; standard deviations were 0.94/0.86 px versus 4.47/4.43 px.OpenCV errors reached almost 18 px in blurred conditions, while RACE-FPP showed tighter distributions without extreme outliers; both methods had similar noise-free mean errors of 0.63 px and 0.64 px, respectively.Source: Results and discussion · pp. 10–11, Table 2 · p. 11

Camera characterisation

  • Camera RMS reprojection error was 0.2591 px (mixed) and 0.4500 px (noisy) for RACE-FPP versus 1.2365 px and 2.8514 px for OpenCV.OpenCV had lower error on noise-free images (0.0806 px versus 0.2281 px), but processed only 10/16 mixed and 5/16 noisy images, compared with 16/16 for RACE-FPP; degraded-set errors use different pose subsets.Source: Results and discussion · pp. 11–13, Table 3 · p. 12
  • Camera intrinsic parameters estimated with RACE-FPP varied only slightly across datasets, while OpenCV parameters fluctuated more under degraded imaging conditions.RACE-FPP retained all 16 characterisation images per dataset, without dataset-specific detection-parameter adjustment.Source: Results and discussion · pp. 13–14

Projector and stereo characterisation

  • Under noisy conditions, projector and stereo errors were 0.6305 px and 0.1518 px for RACE-FPP, versus 1.6849 px and 3.1638 px for OpenCV.RACE-FPP retained sub-pixel stereo errors across all three conditions. Its centre-based strategy averages four detected corners to locate white-square centres and samples phase there by bilinear interpolation.Source: Table 5 · p. 13, Results and discussion · pp. 13–14

3D reconstruction and dimensional validation

  • Under noisy imaging, RACE-FPP kept sphere-radius deviations within approximately 0.04 mm and SD deviation at -0.115 mm, while Traditional FPP reached 0.764 mm and 4.385 mm.For the multi-feature artefact, RACE-FPP measured step heights of 4.91, 4.91, and 4.92 mm against the 4.917 mm CMM reference, with deviations within 0.01 mm; Traditional FPP deviations ranged from -0.02 to +0.15 mm.Source: Results and discussion · pp. 15–16, Table 6 · p. 15, Table 7 · p. 15

Results and discussion

  • Under degraded conditions, the proposed framework maintains geometrically consistent reconstructions and bounded dimensional deviations across both artefacts, while the conventional pipeline shows greater distortion and larger deviations.The findings link the proposed camera–projector characterisation's improved stability to dimensional fidelity in reconstructed geometry.Source: Results and discussion · p. 16

Conclusion

The framework combines deep learning-based checkerboard corner localisation with centre-based projector correspondences while retaining the conventional geometric camera–projector model. It improves robustness under degraded imaging and dimensional fidelity in reconstructed geometry.

Conclusion

  • The framework combines deep learning-based corner localisation with white-square-centre phase sampling for projector correspondences, modifying localisation and correspondence stages without replacing the physical characterisation model.White-square centres are derived from detected checkerboard geometry, while the conventional geometric camera–projector model is retained.Source: Conclusion · p. 16
  • On noisy images, camera reprojection error fell from 2.85 px (conventional) to 0.45 px (proposed), and projector reprojection error from 1.68 px to 0.63 px.The proposed method processed all characterisation images and maintained sub-pixel stereo reprojection errors across tested imaging conditions, although the conventional pipeline performed slightly better on clean, high-contrast images.Source: Conclusion · p. 16
  • For a precision dumbbell artefact under noisy conditions, centre-to-centre spacing deviation was -0.115 mm with the proposed framework versus 4.385 mm with the conventional pipeline.For a multi-feature artefact, step-height deviations from the CMM reference remained within 0.01 mm across characterisation conditions with the proposed framework.Source: Conclusion · p. 16
  • The study evaluated one FPP configuration; future work will test transfer across camera resolutions, fields of view, optical geometries, and measurement systems.It will also examine how training-data composition can reduce the small clean-image accuracy gap without sacrificing robustness to degraded acquisition conditions.Source: Conclusion · p. 16

Related papers

Loading 2610.08213v1…