Source-linked AI summary
Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups
Carlos Guindel, Jorge Beltrán, David Martín, Fernando García
TL;DR
Combining lidar with stereo vision requires accurate extrinsic calibration, yet existing methods can demand complex setups or depend on sensor and scene structure. The paper presents an automated calibration method for varied automotive sensor configurations and evaluates it with exact-ground-truth simulation and real devices, where it outperforms existing approaches.
Problem
Existing calibration methods can require complex setups or lack generalization, while practical evaluation lacks accurate ground truth for sensor pose.
Method
The paper combines segmentation and registration with a simulation-based test suite that provides exact ground truth, then validates the approach using real sensor devices.
Results
The proposed algorithm outperforms existing approaches by a large margin, and real tests corroborate the simulation results.
Takeaways & Limitations
The method removes nearly all calibration burden while targeting accurate results with close-to-production sensors without user intervention.
Abstract
from arXiv · showhide
Sensor setups consisting of a combination of 3D range scanner lasers and stereo vision systems are becoming a popular choice for on-board perception systems in vehicles; however, the combined use of both sources of information implies a tedious calibration process. We present a method for extrinsic calibration of lidar-stereo camera pairs without user intervention. Our calibration approach is aimed to cope with the constraints commonly found in automotive setups, such as low-resolution and specific sensor poses. To demonstrate the performance of our method, we also introduce a novel approach for the quantitative assessment of the calibration results, based on a simulation environment. Tests using real devices have been conducted as well, proving the usability of the system and the improvement over the existing approaches. Code is available at http://wiki.ros.org/velo2cam_calibration
I. INTRODUCTION
Vehicle perception systems combine stereo vision and lidar because their complementary data improve environmental modeling, but accurate extrinsic calibration is required for fusion. Existing methods can require complex setups and may depend strongly on sensor parameters or environmental structure, motivating an automated method for varied automotive configurations.
- Stereo vision provides appearance and dense 3D information, while lidar offers accurate range measurements and a 360° field of view.Their complementary sensing makes them suitable for joint automotive perception systems.
- Sensor fusion requires accurately determining the relative pose, represented by the sensors’ extrinsic parameters.
- Existing calibration methods require complex setups or generalize poorly across sensor parameters and environmental structure.
- The proposed method targets stereo rigs and 360° multi-layer lidar scanners without strong assumptions about resolution or relative sensor pose.It is intended to support medium-resolution scanners, including 16-layer devices, and uses a simple setup exploiting cross-sensor correspondences.
II. RELATED WORK
Prior camera-to-range calibration work spans manual and automatic target-based methods, dense-cloud approaches, and methods using environmental features. Evaluation remains difficult because accurate ground truth for the six pose parameters is generally unavailable in practice.
- Camera-to-range calibration is a frequently studied problem, often constrained by mobile platforms that provide only one or four scan planes.These restrictions yield sparse range information compared with dense 3D scans.
- Traditional calibration methods use manual annotation, whereas automatic methods employ identifiable targets such as triangular boards, polygonal boards, or spheres.
- Dense-range approaches estimate extrinsic parameters from planar checkerboards or circular features in calibration patterns.
- Other methods avoid artificial targets by using linear features, the ground plane, or obstacles in the environment.
- Calibration assessment lacks accurate practical ground truth for the six pose parameters, encouraging custom metrics and sometimes inaccurate manual annotations.
III. CALIBRATION ALGORITHM
The calibration algorithm estimates the rigid transform between the stereo camera and lidar frames using synchronized stereo images, lidar scans, and one shared planar target. It segments reference points in both point clouds before registering them to recover the transform.
- The method estimates six extrinsic parameters: three translations and three rotations relating the camera and lidar coordinate frames.The resulting transformation maps 3D points from camera coordinates into lidar coordinates.
- It uses a short synchronized series of stereo images and lidar scans, assuming known sensor intrinsics, stereo baseline, and rectified images.
- A single planar target with four symmetrically arranged circular holes provides features visible to both sensors from one viewpoint.The sensors must overlap sufficiently for at least two lidar beams to intersect the holes.
- The method permits large relative sensor displacements and arbitrary target orientation beyond the required field-of-view overlap.
- Calibration has two stages: segment reference points in both clouds, then register them to estimate the transform.
A. Data Representation
The algorithm represents lidar and stereo observations as sensor-centered 3D point clouds, while retaining stereo intensity information for feature extraction. Spatial filtering and edge-based processing restrict both modalities to data relevant to the shared calibration target.
- Lidar data form a 3D range point cloud, while stereo data provide grayscale intensity and depth-derived 3D coordinates.Stereo matching maps image pixels into 3D points, producing a point cloud analogous to the lidar representation.
- The similarity between the lidar and stereo point clouds is used to determine the transformation between their sensor coordinate systems.
- Semi-Global Matching estimates stereo depth, while intensity information helps address border localization effects during calibration-target processing.The target is assumed to have sufficient texture to resolve stereo correspondence.
- Pass-through filters based on local-axis distances limit processing to the region likely shared by both sensors’ fields of view.
B. Target segmentation
The method segments the calibration target separately in lidar and stereo-camera point clouds, isolating planar target regions and their discontinuities. Lidar depth discontinuities and stereo image edges provide sensor-specific target features.
- Segmentation extracts subsets of each cloud representing geometrical shapes such as planes.
- RANSAC estimates lidar and stereo plane models while thresholding and orientation constraints help select the target plane.The plane must remain near the sensor frame’s vertical axis.
- Points sufficiently distant from each plane model are removed, producing the lidar and stereo cloud segments used downstream.
- Lidar points are filtered by depth discontinuities computed from neighboring range measurements.Each point receives a magnitude representing its depth difference relative to adjacent points.
- Stereo points are retained when their projections correspond to image edges detected with a Sobel filter.Points with Sobel values below the stereo threshold are removed.
C. Circle segmentation
Circle segmentation identifies the four calibration-target holes in both sensor clouds and estimates their centers as registration keypoints. Plane-based 2D processing, geometric constraints, and temporal accumulation improve the correspondence extraction.
- The algorithm segments the four target circles because their centers provide correspondence keypoints between the lidar and stereo clouds.
- Filtering removes non-circle points, including target-boundary outliers, before circle detection.Lidar processing retains rings with point counts compatible with circles and removes their outer points.
- Both clouds are rotated into plane-aligned 2D coordinates, then circles are detected by sample consensus using the known circle radius.
- Circle centers are accumulated across N frames and clustered, with cluster centroids used for registration to improve robustness against noise.
D. Registration
Registration estimates the lidar-camera transform from corresponding circle-center centroids in two stages. It first solves translation, then refines the complete six-parameter transformation with ICP.
- Registration minimizes distances between corresponding lidar and stereo centroids after applying the estimated transformation.
- The first stage assumes zero rotation and computes a pure translation from the four reference-point correspondences.
- Three equations per centroid pair form an overdetermined system whose least-squares solution is obtained using column-pivoting QR decomposition.
- The second stage estimates translation and rotation jointly by applying Iterative Closest Points to the clustered centroid clouds.The final transform composes the translation and full registration stages.
IV. EXPERIMENTAL RESULTS
The evaluation combines synthetic experiments with exact transformation ground truth and real-device tests. Its simulator-based protocol models sensor specifications and includes ordinary and challenging relative-pose settings.
- The method is evaluated with a synthetic test suite and real sensing devices from the IVVI 2.0 research platform.
- The simulation environment provides exact ground truth for the relative sensor transformation, enabling quantitative calibration assessment.
- Nine simulator setups independently evaluate transformation parameters in seven cases and test generalization under challenging conditions in two others.
- Sensor models reproduce real field of view, resolution, and accuracy, while the calibration target mimics the physical wooden target.
B. Performance Evaluation
The method is evaluated through simulated noise, frame accumulation, and comparisons with existing approaches, using linear and angular calibration errors. Errors converge quickly, and the proposed method is reported to outperform existing methods across tested lidar resolutions.
- Evaluation protocol: Calibration performance is measured as linear and angular differences between the estimated transform and ground truth.The translation vector and rotation matrix provide the respective components of the transform error.
- Frame accumulation: Both linear and angular errors converge quickly as more sensor frames are accumulated, becoming steady by the 30th iteration.Not every frame is necessarily used because segmentation may fail to produce an outcome.
- Noise robustness: Robustness is tested by modeling input noise as N(0, (Kσ0)^2) for three noise-factor values.The resulting calibration errors are summarized in Fig. 6, whose box red line denotes the median.
- Comparison methods: Comparisons include Velas et al. and Geiger et al., although both reference methods target monocular cameras rather than stereo rigs.Velas et al. produced valid results for most settings only with smaller transformation magnitudes, while Geiger et al. was evaluated on selected settings.
- Lidar resolution: Evaluation uses lidar devices with 16, 32, and 64 scan planes because prior comparison methods were not designed for mid-resolution laser scanners.Results considering only valid outcomes are presented in Fig. 8.
C. Tests in Real Scenarios
Real-scenario tests were conducted on the IVVI 2.0 platform. The results demonstrate the algorithm’s usability for automotive applications requiring registration of sensor data.
- Real-world evaluation: Tests in real scenarios used the IVVI 2.0 platform setup.The real-scene results are illustrated through cloud registration and range projections.
- Real-world evaluation: The real-scenario results proved the algorithm’s usability for automotive applications requiring data registration.Figure 9 presents cloud registration and range projections from the real scenes.
V. CONCLUSION
The paper presents an automatic lidar-stereo calibration methodology designed for accurate real setups and introduces simulation-based validation with exact measurements. Experiments report large-margin improvements over existing approaches, while real tests corroborate the simulation results.
- Contribution: The methodology aims to remove nearly all calibration burden and obtain accurate results in real setups without user intervention.It targets close-to-production sensor devices and differs from previous approaches in avoiding user intervention.
- Evaluation framework: Simulation software recreates sensor models and calibration scenes, providing exact measurements for quantitative validation.This addresses the difficulty of obtaining precise ground truth for sensor setups in practice.
- Results: Experiments show that the proposed algorithm outperforms existing approaches by a large margin.The conclusion reports this as the principal comparative outcome.
- Results: Real tests corroborate the results obtained with the simulation software.The paper also releases the calibration algorithm and test suite as open-source ROS implementations for reproducibility.