PaperlayerView exact revision on arXiv

Source-linked AI summary

Solving the Robot-World Hand-Eye(s) Calibration Problem with Iterative Methods

Amy Tabb, Khalil M. Ahmad Yousef

arXiv:1907.12425v1cs.ROcs.CV

TL;DR

Robot-world hand-eye calibration estimates the transforms linking a robot, camera, and world, with applications including robot-mounted-camera vision and multi-camera setups. The paper proposes iterative methods spanning two cost-function classes, rotation parameterizations, and separable or simultaneous formulations. Its methods show strong performance on several reported metrics, and the authors extend the approach to multiple cameras on one robot.

  • Problem

    Robot-world hand-eye calibration must estimate the transforms linking the robot end-effector to a camera and the robot base to the world, including for robot-mounted-camera vision and multi-camera setups.

    Source: §1.2 · p. 3

  • Method

    The paper proposes iterative methods using AX = ZB residual or camera reprojection cost functions, three rotation parameterizations, and separable or simultaneous formulations.

    Source: §1.2 · p. 3, §2 · p. 4

  • Results

    The authors report that c1 simultaneous gives the lowest combined rotation and translation error eC, c2 simultaneous is best for rrmse and rae without reprojection cost, and reconstruction accuracy error is rae ≤2mm for several methods.

    Source: §5.5 · p. 12, §6 · p. 15

  • Takeaways & Limitations

    The method collection extends to multiple cameras on one robot and is demonstrated on datasets with one to three cameras.

    Source: §6 · p. 15

    The erratum reports incorrect et computation in the original results tables and slight numerical differences in refreshed results after supporting libraries were revised.

    Source: §7 · p. 21

Abstract

from arXiv · show

Robot-world, hand-eye calibration is the problem of determining the transformation between the robot end-effector and a camera, as well as the transformation between the robot base and the world coordinate system. This relationship has been modeled as $\mathbf{AX}=\mathbf{ZB}$, where $\mathbf{X}$ and $\mathbf{Z}$ are unknown homogeneous transformation matrices. The successful execution of many robot manipulation tasks depends on determining these matrices accurately, and we are particularly interested in the use of calibration for use in vision tasks. In this work, we describe a collection of methods consisting of two cost function classes, three different parameterizations of rotation components, and separable versus simultaneous formulations. We explore the behavior of this collection of methods on real datasets and simulated datasets, and compare to seven other state-of-the-art methods. Our collection of methods return greater accuracy on many metrics as compared to the state-of-the-art. The collection of methods is extended to the problem of robot-world hand-multiple-eye calibration, and results are shown with two and three cameras mounted on the same robot.

1 Introduction

Robot-world hand-eye calibration estimates the transforms between the robot end-effector and camera, and between the robot base and world. This paper develops iterative methods and evaluates their variants for that problem, including an extension to multiple cameras on one robot.

1 Introduction

  • Robot-world hand-eye calibration estimates the hand-to-camera and robot-base-to-world homogeneous transformation matrices.The problem is written AX = ZB, with multiple robot poses providing multiple relationships.Source: §1 · pp. 1–2

1.1 Recent work

  • Prior methods include separable approaches that estimate rotation before translation, and simultaneous approaches that estimate both without decoupling their errors.Prior work also includes methods incorporating camera calibration parameters or reprojection error.Source: §1.1 · pp. 2–3

1.2 Our approach and contributions

  • The authors motivate calibration for robot-mounted-camera reconstruction, remote data collection, and setups with multiple cameras that may have different fields of view and no overlap.They seek methods robust to non-ideal calibration situations and motion sequences, and simultaneous multi-camera calibration avoids propagating error from a separate calibration step.Source: §1.2 · p. 3
  • The paper proposes iterative methods that minimize either AX = ZB residuals or camera reprojection error, with separable and simultaneous formulations and three rotation parameterizations.The methods extend to multiple cameras mounted on one robot.Source: §1.2 · pp. 3–4
  • The study compares cost functions, parameter choices, and separable versus simultaneous solutions against state-of-the-art methods on real and simulated datasets.The expanded work adds three rotation parameterizations, separable formulations, a comparison with Shah [10], four real datasets, and simulated datasets.Source: §1.2 · p. 4
  • Multiple-camera calibration estimates one robot-to-world transform and a camera transform for each camera, and the paper demonstrates the extension with one to three cameras.The authors describe it as, to their knowledge, the first attempt at robot-world hand-eye calibration for multiple cameras.Source: §1.2 · p. 4

2 Collection of methods description

The methods combine alternative rotation parameterizations with two cost-function classes and separable or simultaneous formulations. The reprojection-error class and its extension to multiple cameras add camera-specific calibration and accommodate missing observations.

2 Collection of methods description

  • The collection combines three rotation parameterizations with two cost-function classes, each supporting separable or simultaneous formulations.Euler angles use x, y, z rotation order; axis-angle uses a vector whose norm is the rotation angle, and quaternions use a four-element unit vector.Source: §2 · p. 4, §2.1 · p. 5
  • The first cost-function class estimates X and Z from world-to-camera and base-to-end-effector transformations, solving rotation and translation together or separately.The separable formulation estimates rotation first and translation afterward.Source: §2.1 · pp. 4–5

2.2 Second class: camera reprojection error

  • The reprojection-error class minimizes camera reprojection sum of squared error to estimate X and Z, using the first class’s result as an initial solution because it is sensitive to initialization.In rp2, intrinsic camera parameters are estimated rather than held constant; rsse is used because its minimizers also minimize reprojection root mean squared error.Source: §2.2 · pp. 5–6

2.3 Extending the two proposed classes of methods to multiple cameras

  • For multiple cameras, the formulation estimates one X and a separate Z for each camera, extending the proposed cost functions and their separable versions.The extension includes rp2 estimation of intrinsic parameters for all cameras.Source: §2.3 · pp. 6–7
  • The multiple-camera formulation handles cameras that do not see the calibration object at every robot position, with weights intended to give each camera equal influence.The weights can be adjusted for other needs.Source: §2.3 · pp. 6–7

3 Performance Evaluation on real datasets

The real-data evaluation uses eight laboratory and field datasets spanning different robot, camera, and lens combinations, and assesses the methods with calibration and reconstruction metrics. It also documents implementation choices and defines the errors used for comparison.

3 Performance Evaluation on real datasets

  • Eight real datasets span laboratory and field settings, varied robot-camera-lens combinations, and include multi-camera setups.Datasets 7 and 8 include a third RGB-D camera, but only its color camera was calibrated.Source: §3 · p. 7

3.1 Implementation details

  • The first cost-function class starts from identity rotations and zero translations; the second class uses a staged initialization based on c2, then rp1.For the experiments covered, the first class was not sensitive to its initial solutions.Source: §3.1 · p. 8
  • Seven comparison methods are implemented, including approaches by Zhuang et al., Dornaika and Horaud, Hirsh et al., Li et al., and Shah.The Li et al. dual-quaternion coefficients were obtained with Levenberg-Marquardt because their computation was not specified clearly.Source: §3.1 · p. 8

3.2 Error metrics

  • The evaluation uses rotation, translation, combined rotation-translation, reprojection, and reconstruction accuracy errors.The reconstruction metric was added for the paper’s reconstruction and vision-task motivation.Source: §3.2 · p. 8, §3.2.5 · p. 9

3.2.1 The Mean Rotation Error

  • The evaluation reports two rotation errors: eR1 has no units, while eR2 measures the relative rotation angle in degrees.The authors caution that eR1 may not be particularly meaningful for evaluating the methods.Source: §3.2.1 · p. 8

3.2.2 The Mean Translation Error

  • Mean translation error is derived from Equation 3 and reported in millimeters.Source: §3.2.2 · p. 9

3.2.3 The Mean Combined Rotation and Translation Error

  • The mean combined rotation and translation error is unitless.Source: §3.2.3 · p. 9

3.2.4 The Reprojection Root Mean Squared Error (rrmse) / 3.2.5 The Reconstruction Accuracy Error

  • Reprojection root mean squared error is reported in pixels, while reconstruction accuracy averages the Euclidean distance between estimated and true calibration-object points.Reconstruction first estimates 3D points from corresponding image points across robot positions.Source: §3.2.4 · p. 9, §3.2.5 · p. 9

4 Performance Evaluation on simulated datasets

The simulated evaluation generates robot and camera transformations, then tests behavior under controlled rotational noise and two translation scales. Errors are computed against known ground-truth transformations.

4 Performance Evaluation on simulated datasets

  • Simulated datasets use 25 robot poses with random rotations and translations, and exclude the second proposed method class because it would require simulated camera models.The protocol follows Shah, with translation components sampled from (0,1) for Simulated Dataset I and (0,1000) for Dataset II.Source: §4 · p. 9
  • Rotation-only noise is added to B_i through quaternion perturbation, across 19 evenly spaced noise magnitudes and ten trials.The noise parameter η ranges over (0,0.25].Source: §4 · p. 9

4.1 Error metrics for simulated datasets

  • Simulated errors compare estimated and ground-truth transformations for X and Z.Source: §4.1 · p. 10

4.1.1 Rotation error

  • Rotation errors are computed separately for the estimated X and Z rotation components against their ground truth.Source: §4.1.1 · p. 10

4.1.2 Translation error

  • Translation errors are computed separately for the estimated X and Z translation components against their ground truth.Source: §4.1.2 · p. 10

5 Experimental Results

Across real and simulated evaluations, the proposed methods show metric-dependent strengths, and recommendations vary with the error measure and camera-calibration quality. The reported comparisons also show that dataset and translation scaling can change method behavior.

5 Experimental Results

  • The experiments compare methods on eight real datasets and simulated datasets, including seven literature methods; simulated comparisons include the seven methods and the first proposed class.The real-dataset tables report five error metrics and implementation running times; the simulated results are shown graphically.Source: §5 · p. 10

5.1 Discussion of comparison methods on real datasets

  • The Dornaika–Horaud iterative method did not converge on any dataset with Ceres, while large orthonormality penalties produced little change from its initial solution or non-convergence.With levmar, it converged after 2–3 iterations after termination conditions were met, with results similar to its closed-form initial solution.Source: §5.1 · p. 10
  • On single-camera real datasets, comparison-method performance is dataset-dependent: Zhuang’s eC is 233.3 in Dataset 1 but exceeds 170,000 in Datasets 3–5, while Li’s Kronecker method leads the comparison group only in Dataset 5.Li’s Dataset 5 eC is 329.24, compared with eC ≥13,000 for the other comparison methods.Source: §5.1 · p. 11

5.2.1 First class of cost functions

  • In Dataset 5, separable first-class methods have rrmse above 48 pixels, while simultaneous versions have a maximum of 3.89 pixels.More broadly, c1 methods tend toward lower eC, while c2 methods tend toward lower rrmse; Datasets 7 and 8 are exceptions with similar c1 and c2 values.Source: §5.2.1 · p. 11

5.2.2 Second class of cost functions

  • Second-class methods consistently produce lower rrmse than first-class and comparison methods, but have higher rotation, translation, and combined errors than first-class methods and some comparisons.The rrmse difference between rp1, which holds intrinsic camera parameters constant, and rp2, which estimates them, is generally small.Source: §5.2.2 · p. 11

5.2.3 Effect of rotation representation

  • Rotation representation usually has small metric effects, but exceptions occur on Datasets 5 and 6 for separable c1 and c2 methods.On Dataset 1, Ceres produced rrmse values of 12.13, 3.62686, and 3.62641 pixels for Euler, axis-angle, and quaternion representations, respectively.Source: §5.2.3 · p. 11

5.5 Recommendations and Summary

  • Across datasets, proposed rp1/rp2 methods outperform comparison methods on reconstruction accuracy; c1 simultaneous minimizes eC, while c2 simultaneous gives the best rrmse and rae without reprojection error in the cost function.With reprojection error in the cost function, rp1 is generally better on rae with higher-quality camera calibration, while rp2 is better with poorer-quality intrinsic calibration.Source: §5.3 · p. 12, §5.5 · p. 12

5.4 Discussion of the simulated experiments

  • Translation scaling changes simulated outcomes: in Dataset II, Li’s Kronecker method and first-class simultaneous methods have translation errors ≤0.061, while other methods vary considerably.The experiments identify Li’s Kronecker method as a robust choice in both simulated settings, Shah’s method for Dataset I, and first-class methods for Dataset II.Source: §5.4 · p. 12

6 Conclusions

The methods perform consistently well on reconstruction accuracy, and the collection is extended to robot-world hand-eye calibration with multiple cameras mounted on one robot. The paper also introduces a metric for assessing suitability to reconstruction problems.

6 Conclusions

  • The collection includes two cost-function classes, simultaneous and separable formulations, camera reprojection error, and three rotation representations.The methods were evaluated relative to existing robot-world hand-eye calibration methods.Source: §6 · pp. 12, 15
  • The methods extend to calibrating multiple cameras mounted on one robot, demonstrated with datasets using one to three cameras.The authors describe this as the first attempt to solve the multiple-camera problem, to the best of their knowledge.Source: §6 · p. 15
  • The paper introduces Reconstruction Accuracy Error (rae) to assess a method’s suitability for reconstruction problems.The reported rae is ≤2mm for the highlighted methods.Source: §6 · p. 15
  • rae ≤2mm for the c2 simultaneous method from the first class and both methods in the second class on real datasets spanning mounting configurations and robot positions.rae is the average distance between a calibration pattern point and the point generated using the calibration information.Source: §6 · p. 16

7 Erratum

The erratum corrects an error in the software used to compute mean translation error and supplies refreshed results in Tables 10–17. It also corrects the metric’s unit label and notes slight numerical differences after supporting libraries were revised.

7 Erratum

  • The erratum says the software computed mean translation error et incorrectly in the original results (Tables 2–9); corrected, refreshed results appear in Tables 10–17, with et units corrected to mm2.Source: §7 · p. 21
  • The translation-error unit label should be mm2.The original tables had the unit label wrong.Source: §7 · p. 21
  • Because supporting libraries had been revised since the original results were prepared, numerical results differ slightly despite the same input; the erratum therefore provides completely refreshed results in Tables 10–17.Source: §7 · p. 21
Loading 1907.12425v1…