PaperlayerView exact revision on arXiv

Source-linked AI summary

Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups

Jorge Beltrán, Carlos Guindel, Arturo de la Escalera, Fernando García

arXiv:2101.04431v2cs.ROcs.CV

TL;DR

Accurate calibration is needed to combine complementary LiDAR and camera data, but existing procedures can be burdensome and difficult to evaluate objectively. The paper extracts reference points from a custom target and registers them to estimate rigid extrinsic transformations across broad sensor combinations. Experiments report that the method notably outperforms existing approaches, with real-data tests confirming the simulation results.

  • Problem

    Existing extrinsic-calibration methods can be burdensome, configuration-specific, and difficult to assess because accurate ground truth is unavailable in practice.

  • Method

    The method extracts target reference points with modality-specific branches and estimates the sensor transformation by registering the resulting point sets.

  • Results

    The algorithm notably outperforms existing approaches, and real-data tests confirm the accuracy observed in simulation.

  • Takeaways & Limitations

    The approach supports LiDAR, monocular-camera, and stereo-camera pairs across same- or mixed-modality setups with minimal user intervention.

Abstract

from arXiv · show

Most sensor setups for onboard autonomous perception are composed of LiDARs and vision systems, as they provide complementary information that improves the reliability of the different algorithms necessary to obtain a robust scene understanding. However, the effective use of information from different sources requires an accurate calibration between the sensors involved, which usually implies a tedious and burdensome process. We present a method to calibrate the extrinsic parameters of any pair of sensors involving LiDARs, monocular or stereo cameras, of the same or different modalities. The procedure is composed of two stages: first, reference points belonging to a custom calibration target are extracted from the data provided by the sensors to be calibrated, and second, the optimal rigid transformation is found through the registration of both point sets. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups. In order to assess the performance of the proposed method, a novel evaluation suite built on top of a popular simulation framework is introduced. Experiments on the synthetic environment show that our calibration algorithm significantly outperforms existing methods, whereas real data tests corroborate the results obtained in the evaluation suite. Open-source code is available at https://github.com/beltransen/velo2cam_calibration

I. INTRODUCTION

The paper addresses difficult, accuracy-critical extrinsic calibration for heterogeneous automotive sensor setups. It introduces a general target-based method and simulation benchmark intended to support diverse configurations with minimal intervention.

  • Motivation: Accurate extrinsic calibration is required to combine complementary sensor data and establish correspondences for multimodal perception.Automotive systems need calibration accurate enough to preserve data association at long distances.
  • Research gap: Existing methods may require burdensome ad-hoc environments, target manual annotation, or specific configurations, while objective evaluation remains difficult.The lack of exact ground truth and standard metrics limits reliable comparison between approaches.
  • Scope: The method is designed for diverse cameras and multi-layer LiDARs, including lower-resolution devices, with large relative translations and rotations allowed when fields of view overlap.An overlapping field of view is the stated pose-related requirement.
  • Contribution: The proposed framework calibrates camera-camera, camera-LiDAR, and LiDAR-LiDAR pairs using a custom fiducial target designed for fast, minimally supervised operation in cluttered scenes.The target is intended to avoid ambiguity across sensor data representations.
  • Evaluation: Experiments report higher calibration accuracy than existing approaches, while the accompanying simulation framework supplies ground truth and supports fair benchmarking across sensor devices and poses.The synthetic suite and released implementation are intended to improve reproducibility and evaluation.

III. METHOD OVERVIEW

The method estimates the rigid transformation between any pair of LiDAR and camera sensors by extracting corresponding target points and registering them. It is designed for substantial sensor pose differences, subject to visibility and static-scene constraints.

  • Transformation model: The method estimates a six-parameter rigid-body transformation describing one sensor’s position and orientation relative to another.The parameters comprise three translations and three rotations: roll, pitch, and yaw.
  • Transformation model: A transformation matrix maps 3D points between the sensors’ coordinate systems once the relative-pose parameters are determined.For a LiDAR-monocular pair, the matrix transforms monocular coordinates into LiDAR space.
  • Target-based correspondence: A custom planar target provides geometrical and visual features that can be detected and paired across LiDAR, stereo, and monocular data.The target includes circular holes that exploit geometrical discontinuities in range and stereo point-cloud data.
  • Operating constraints: The approach permits substantial translations and rotations, but requires overlapping sensor fields of view and sufficient visibility of the target holes.For range data, each circle must be represented by at least three points; multiple target poses can improve accuracy and robustness in static scenes.
  • Processing stages: The two-stage procedure first localizes target reference points in each sensor frame, then computes transformation parameters that register those points.The registration seeks the best alignment, defined as minimum distance between corresponding reference points.

IV. TARGET SEGMENTATION

Target segmentation localizes four hole centers in each sensor’s coordinate system using modality-specific processing. LiDAR and stereo share a 3D-processing trunk, whereas monocular cameras use ArUco markers.

  • Reference-point output: The first stage outputs four 3D points representing calibration-target hole centers in each sensor’s local coordinates.These points provide the correspondences later used for sensor registration.
  • Modality-specific processing: LiDAR and stereo processing share a common point-cloud pipeline after initial segmentation, while monocular processing relies on ArUco markers.The monocular branch uses marker cues to retrieve the target geometry.
  • Frame processing: The procedure processes every sensor frame in parallel and does not require a common trigger or identical refresh rates when the scene is static.Static-scene operation permits asynchronous sensor data processing.
  • LiDAR preprocessing: LiDAR preprocessing applies Cartesian pass-through filters to retain the target region and points visible behind it through the holes.The behind-target points are needed for the target’s range-based segmentation.
  • LiDAR preprocessing: LiDAR target-edge extraction uses neighboring scan-plane ranges to compute depth discontinuities and filter points below a modality-specific threshold.The discontinuity compares a point’s range with adjacent points in the same scan plane.

B. Stereo Data Preprocessing

Stereo preprocessing converts the image pair into a 3D cloud, filters the search region, and extracts target edges using image appearance. The resulting cloud is prepared for common hole segmentation.

  • B. Stereo Data Preprocessing: Stereo matching converts the raw image pair into a 3D point cloud for depth-based calibration processing.The experiments use OpenCV’s Semi-Global Block Matching variant.
  • B. Stereo Data Preprocessing: Pass-through filters restrict the stereo cloud to the region containing the calibration target.
  • B. Stereo Data Preprocessing: A Sobel filter removes cloud points mapped to low-gradient image pixels, producing an edge-focused cloud less affected by stereo border-localization errors.
  • B. Stereo Data Preprocessing: The preprocessed cloud undergoes RANSAC plane segmentation, plane-based filtering, and projection onto the target plane before circle extraction.Frames without a suitable roughly vertical plane are discarded.

3) Circle Segmentation:

Circle segmentation identifies calibration-target holes from planar point data and validates their geometry before returning exactly four 3D reference centers. Repeated frames and clustering consolidate estimates against measurement and detection errors.

  • 3) Circle Segmentation:: Iterative 2D circle segmentation extracts supported pattern holes and accepts a frame only when at least four circles are found.Each detected circle’s inliers are removed before searching for the next one.
  • 3) Circle Segmentation:: Geometric consistency checks compare groups of four circle centers against the target rectangle’s theoretical dimensions using a percentage tolerance.The checks use diagonal, height, width, and perimeter.
  • 3) Circle Segmentation:: Validated hole centers are converted from the target-plane representation back into sensor coordinates as the cloud Pp, containing exactly four points.
  • 3) Circle Segmentation:: For monocular cameras, ArUco markers provide the cues needed to retrieve the calibration-target geometry.Four corner markers are handled jointly as an ArUco board, whose pose is optimized by minimizing reprojection error.
  • 3) Circle Segmentation:: Accumulating reference points across frames and clustering their positions increases robustness to sensor noise, sparse data, and nondeterministic procedures.More than four detected clusters causes the data to be rejected; otherwise, cluster centroids form Pc.

2) Accumulation over Several Target Poses:

Multiple target poses provide additional non-coplanar reference points for registration. Their points are labeled and paired across sensor clouds so the relative transformation can be optimized consistently.

  • 2) Accumulation over Several Target Poses:: Repeating segmentation for M target poses accumulates 4×M reference points for the registration stage.The sensor and target remain static during segmentation of each pose.
  • 2) Accumulation over Several Target Poses:: Selecting non-coplanar poses across varied sensor distances resolves possible ambiguities and improves final calibration quality.
  • 2) Accumulation over Several Target Poses:: The accumulated clouds contain estimated circle-center locations expressed in each sensor’s coordinate frame.
  • 2) Accumulation over Several Target Poses:: Registration minimizes alignment distance across 4×M objectives after corresponding reference points are correctly associated.
  • 2) Accumulation over Several Target Poses:: Spherical-coordinate ordering and inter-point distances label each center by hole position and target pose, without assuming identical cloud ordering.The labels distinguish top-left, top-right, bottom-left, and bottom-right holes.

B. Solution

The solution registers corresponding reference-point clouds from two sensor modalities by estimating a rigid transformation. It is evaluated with synthetic ground truth and real data under largely fixed, automatic settings.

  • B. Solution: Clouds from arbitrary modalities X and Y are registered with Umeyama’s procedure to minimize distances between corresponding points.
  • B. Solution: SVD solves the least-squares registration problem in closed form and yields the relative sensor-pose parameters.The method also handles coplanar points when only one target pose is used.
  • B. Solution: The validation combines a realistic synthetic suite with perfect ground truth and real-environment tests.
  • B. Solution: Experiments run without user intervention except coarse pass-through-filter tuning, with remaining parameters fixed across tests.

A. Synthetic Test Environment

The synthetic suite uses realistic simulated sensor models and exact transformation ground truth to evaluate reference-point extraction under varied noise and poses. Results show robustness to noise, with accuracy driven mainly by target pose and improved by clustering.

  • Test environment: The Gazebo suite replicates LiDAR, stereo-camera, and monocular-camera operation while providing exact sensor-to-sensor transformation ground truth.Sensor specifications include field of view, resolution, and accuracy representative of real devices.
  • Experimental design: The evaluation varies sensor-target poses, noise levels, and sensor modalities, including representative LiDAR devices with different scanning-layer distributions.Noise factor K models noise-free, realistic, and noisy conditions, with K = 1 as the default.
  • Limitations: Extreme configurations can prevent reference-point extraction: low-resolution LiDAR fails at far distances, while stereo depth estimation degrades as distance increases.Restricting target locations to a reasonable distance range can avoid these situations in typical use cases.
  • Reference-point extraction: Reference-point localization remains below 1 cm for P1 and P2 across noise levels, while P3 and especially P4 produce larger errors across modalities.Noise primarily increases error dispersion, particularly under K = 2.
  • Reference-point extraction: Under realistic noise, LiDAR remains robust in P3 and P4, whereas monocular cameras perform best in P1 and P2 but struggle in challenging poses.Relative target pose is a stronger factor than modality for localization accuracy.
  • Reference-point extraction: Cluster centroids consistently outperform single-frame estimates, with error reductions of 25.22% for stereo in P3 and 19.02% for HDL-64 in P3.The improvement is most pronounced when single-frame estimates have high dispersion.

2) Single-Pose Experiments:

Single-pose experiments evaluate calibration across same- and cross-modality sensor pairs using linear and angular transformation errors. The method is reliable with few aggregation iterations, while performance varies by sensor setup and strongly favors monocular-camera pairs.

  • Experimental design: The experiments cover LiDAR-LiDAR, camera-camera, and camera-LiDAR setups spanning diverse translations and rotations between sensors.The scenarios represent common configurations for field-of-view, resolution, and multimodal fusion applications.
  • Evaluation metrics: Calibration accuracy is measured using linear and angular errors between the estimated rigid transformation and ground truth.The translation vector and 3×3 rotation matrix define the evaluated transformation.
  • Iteration count: N = 30 offers a fair accuracy-time tradeoff, with reliable estimation across a wide range of iteration counts and extremely rare outliers.The analysis considers N from 1 to 40 data frames for reference-point extraction.
  • Comparison with prior methods: Stereo and monocular alternatives achieve similar accuracy and significantly outperform the compared methods, with the proposed angular accuracy supporting far-distance data fusion.The single-pose baseline provides a foundation for the full multi-pose calibration procedure.

3) Multi-Pose Experiments:

Using multiple calibration-target poses substantially reduces calibration error, with benefits leveling off after three poses. The approach also performs competitively against a state-of-the-art checkerboard method and reaches subcentimeter accuracy.

  • Multi-pose aggregation: 85.42% linear and 87.01% angular average error reductions are achieved with five target poses.The largest reductions occur for HDL-32/HDL-64, reaching 6.5 mm and 0.002 rad deviation from ground truth.
  • Multi-pose aggregation: 61.2% linear and 68.15% angular average reductions result from adding one target pose.Increasing poses is beneficial up to M = 3; additional poses produce mixed effects from nearly neutral to slightly positive.
  • Multi-pose aggregation: Calibration errors follow an almost-exponential decay as the number of poses increases.The trend is shown for four sensor setups using linear and angular RMSE in meters and radians.
  • Comparison with prior work: The proposed method achieves consistent improvements over the compared approach, exceeding 50% for angular error at M = 3.The comparison uses monocular/HDL-64 data with two and three calibration poses.
  • Multi-pose aggregation: Aggregating reference points across target locations provides a calibration solution with subcentimeter accuracy.The result follows from registering points collected across multiple target poses.

B. Real Test Environment

Real experiments evaluate the method across diverse camera and LiDAR combinations mounted on an experimental vehicle. The results show low dispersion, improved stability with multiple poses, and accurate alignment across traffic scenes, although real-data ground truth was unavailable.

  • Experimental setups: Real tests cover monocular, stereo, camera-camera, camera-LiDAR, and LiDAR-LiDAR calibration across varied sensor configurations.The experiments use narrow- and wide-angle cameras, regular and high-resolution 16-layer LiDARs, and a 32-layer LiDAR.
  • Evaluation scope: Real-sensor evaluation lacks ground truth for the relative sensor pose and therefore presents illustrative performance statistics.The reported plots show dispersion, means, standard deviations, and standard errors across calibration procedures.
  • Calibration stability: LiDAR and stereo reference-point dispersion is higher than monocular dispersion, but remains small enough for accurate registration.Some target poses produce outliers, without relevant impact on multi-pose calibration.
  • Calibration stability: Several target poses cause significant calibration changes up to three poses, after which the result plateaus.The comparison measures calibrations with M ∈ [1, 4] against the final M = 5 result.
  • Calibration results: 2.73 mm average coordinate error is obtained for narrow-angle camera-to-camera calibration against the manufacturer’s rectified-stereo baseline.This comparison provides a reference for one of the real calibration procedures.
  • Traffic-scene validation: Projected LiDAR points achieve perfect alignment with camera data across the tested traffic scenarios.Examples include narrow-angle camera/16-layer LiDAR, LiDAR/LiDAR, stereo/Hi-Res LiDAR, and wide-angle camera/32-layer LiDAR setups.

VII. CONCLUSION

The paper presents a general, low-intervention extrinsic-calibration method and a simulation suite with exact ground truth for evaluating it. Experiments show improved performance over existing approaches and confirmation on real data, while several automation and deployment limitations remain.

  • Contributions: The method calibrates arbitrary sensor pairs involving LiDARs, monocular cameras, or stereo cameras, including same- and cross-modality combinations.It is designed for autonomous-vehicle sensing setups and requires minimal user intervention.
  • Evaluation framework: The simulation suite provides exact ground truth for reliable evaluation of extrinsic-calibration methods.It addresses imprecision in performance assessment and supports quantitative comparison.
  • Conclusions: The algorithm notably outperforms existing approaches, and real-data tests confirm the simulation accuracy.The conclusion reports this outcome without specifying a single aggregate metric.
  • Limitations and future work: Manual pass-through filters remain necessary for target segmentation in cluttered scenes.Automated target isolation is identified as a way to remove this human intervention.
  • Limitations and future work: Outlier rejection, sensor-noise modeling, and guided target placement are identified as open improvements.These changes could address spurious samples, device-specific parameter settings, and insufficient target-location variability.
  • Limitations and future work: The method estimates a fixed extrinsic transform before deployment and would require early miscalibration detection for movable platforms.Recalibration would then be prompted when operational miscalibration is detected.
  • Conclusions: The approach is presented as a practical step toward deploying autonomous-driving and robotics solutions.The authors frame this consequence within the scope of addressing a common calibration problem.
Loading 2101.04431v2…