PaperlayerView exact revision on arXiv

Source-linked AI summary

Camera Calibration Using Inaccurate and Asynchronous Discrete GPS Trajectory from Drones

R. Yang, Y. Bar-Shalom, H. A. J. Huang

arXiv:2608.26548v1eess.SYcs.CV

TL;DR

The paper addresses stationary-camera calibration from inaccurate, asynchronous, discrete GPS drone trajectories. It jointly estimates camera orientation, GPS altitude bias, and time offset with a maximum-likelihood ILS procedure, achieving statistically acceptable CRLB consistency and a residual bias RMSE of 14% of measurement-error standard deviation on a recommended trajectory.

  • Problem

    Camera calibration from GPS drone trajectories must handle unknown GPS altitude bias, camera–GPS time offset, and discrete trajectory samples requiring interpolation and velocity estimation.

  • Method

    A special maximum-likelihood ILS algorithm jointly estimates yaw, pitch, roll, GPS altitude bias, and time offset while fitting discrete GPS positions and velocities.

  • Results

    The recommended trajectory achieved residual calibration bias RMSE of 14% of the measurement-error standard deviation, while estimates met CRLB and NEES requirements.

  • Takeaways & Limitations

    The recommended drone trajectory is suitable for practical camera calibration, with residual bias negligible relative to measurement error and time-offset error of 0.27ms versus 100ms GPS discretization.

  • Takeaways & Limitations

    The real setup requires further development to estimate focal length and handle GPS latitude–longitude quantization errors.

Abstract

from arXiv · show

This paper considers a stationary camera calibration problem, which estimates the camera orientation angles yaw, pitch and roll, using a drone trajectory recorded by a GPS. There are three challenges in using a GPS trajectory as ground truth for camera calibration. One, the altitude of GPS data is inaccurate with an unknown bias. Two, the GPS receiver and camera are not time synchronized, and there is an unknown time offset between the two systems. Three, the GPS trajectory is time-discrete and accurate interpolation is needed. This is actually an estimation problem since velocity is also needed. To address the first two challenges, we formulate the problem as a parameter estimation problem to estimate a vector consisting of the GPS altitude bias and time offset in addition to the camera yaw, pitch and roll biases. We then develop a special maximum likelihood estimator using the Iterated Least Squares algorithm which can work with a non-synchronized time-discrete GPS trajectory for the third challenge. Since the camera measurement errors are usually small, this requires a high calibration accuracy so that the residual bias error following the calibration should not be significant compared to the measurement error standard deviation. The calibration accuracy depends highly on the drone trajectory. This paper also recommends an appropriate drone trajectory which can yield a good calibration accuracy, namely, 14\% of the measurement error standard deviation. Simulation tests are conducted to demonstrate the algorithm performance. The estimation results meet the Cramer-Rao Lower Bound (CRLB) since the Normalized Estimation Error Squared w.r.t.\ the CRLB is statistically acceptable.

I. Introduction

The paper formulates stationary-camera orientation calibration from a GPS-recorded drone trajectory despite altitude bias, timing offset, and discrete trajectory data. It uses estimation theory to jointly estimate calibration and GPS-related parameters.

  • The calibration estimates stationary-camera yaw, pitch, and roll from a GPS-instrumented drone trajectory.The camera is modeled as a pinhole camera with known position and no radial or tangential distortion.
  • GPS-based calibration is complicated by altitude bias, an unknown camera–GPS time offset, and discrete trajectory samples lacking an analytical trajectory expression.GPS altitude is less accurate than horizontal position because altitude relies on fewer high-orbit satellites.
  • Point-to-point PnP matching is impractical because the 3D GPS trajectory and 2D camera trajectory have unknown altitude and timing discrepancies.The paper therefore turns to estimation theory rather than direct 3D–2D matching.
  • The estimation-theory formulation represents camera calibration as parameter estimation using stochastic measurement models and optimization algorithms such as ILS.The approach requires observability, meaning a unique solution.
  • The proposed approach jointly estimates camera orientation, GPS altitude bias, and time offset using an enhanced ILS algorithm for discrete GPS trajectories.The paper recommends a practical drone path because residual calibration bias must remain small relative to camera measurement error.

III. Problem Formulation

The problem formulation models camera measurements as noisy projections of a time-shifted, altitude-biased GPS trajectory. Its parameter vector jointly contains three orientation angles, altitude bias, and camera–GPS time offset.

  • The unknown parameter vector contains camera angles α, ϵ, and ρ, GPS altitude bias ℏ, and time offset τ.These five parameters are estimated simultaneously.
  • The measurement model relates the camera measurement vector Z to the trajectory through a nonlinear measurement function H.Z contains n discrete-time image-coordinate points and additive zero-mean Gaussian measurement noise.
  • The trajectory X is a discrete-time ENU GPS drone trajectory corrected for the unknown altitude bias and time offset.The trajectory points correspond to camera measurement times after applying τ.
  • The available GPS trajectory is not aligned with the camera-time trajectory, so the relationship between the two discrete trajectories must be estimated for the stochastic model.Their sampling times and intervals can differ.
  • The projection pipeline shifts and altitude-corrects each trajectory position, transforms it into camera coordinates, and then maps it into image space.The function h_i uses the rotation transformation and camera focal length in this conversion.

IV. Estimation Algorithm

The paper uses a unique Iterated Least Squares algorithm to estimate the calibration parameters from camera measurements and GPS trajectory data. Each iteration first estimates trajectory positions and velocities, then updates the parameter estimate until convergence.

  • The ILS algorithm estimates the parameter vector θ using camera measurements Z, GPS trajectory X, and an initial estimate ˆθ0.Its distinctive structure uses Z and X to estimate X and ˙X before estimating θ.
  • Each iteration estimates Xj and its velocity ˙Xj from X and ˆθj before updating ˆθj.
  • The parameter estimate is updated with an optimization algorithm based on the nonlinear model and the estimated trajectory quantities.
  • The iteration stops when a satisfactory estimate ˆθ is obtained.

A. Estimate X and its Velocities ˙X

The method estimates trajectory positions and velocities at camera times from discrete GPS points. A two-step least-squares fit combines estimates from neighboring points before and after each target point to reduce maneuver-related error.

  • Estimate X and its Velocities ˙X: Trajectory positions and velocities are estimated at GPS times [t1 + τ, …, tn + τ] corresponding to camera measurement times [t1, …, tn].The unknown time offset τ makes this more than ordinary interpolation because velocities are also estimated.
  • Estimate X and its Velocities ˙X: The one-step LS fitting approach can produce large errors when a drone maneuver occurs within its sliding window.
  • Estimate X and its Velocities ˙X: The two-step LS fitting approach applies separate estimators to neighboring points before and after the target point, producing estimates b and a.
  • Estimate X and its Velocities ˙X: The fitting procedure estimates coordinate-specific velocities and accelerations from neighboring GPS points using least squares.The x, y, and z coordinates are treated separately, with δ=3 neighboring points used in the stated LS model.
  • Estimate X and its Velocities ˙X: The final estimate c is a weighted combination of the two neighboring estimates, reducing estimation error.The likelihood of each fitted point is computed from its measurement residual before combining them.

B. Update the estimate of θ

The parameter vector θ is updated by solving the optimization problem with Iterated Least Squares under the Gaussian measurement model. Iteration continues until the estimate converges according to a stopping criterion.

  • Update the estimate of θ: The parameter θ is estimated by applying Iterated Least Squares to the optimization problem defined by the stochastic model.The ILS algorithm numerically solves the maximum-likelihood estimate under the Gaussian assumption.
  • Update the estimate of θ: The final estimate ˆθ is the value to which the iterative update converges under the stopping criterion.The derivatives required for the update are provided in Appendix B.

C. Stopping Criterion

The algorithm uses a tight, element-wise stopping criterion based on normalized measurement residuals. It also limits the iteration count to prevent runs from continuing indefinitely.

  • Stopping Criterion: The squared measurement residual is normalized element by element during iteration j.
  • Stopping Criterion: Every normalized residual element vj,i must remain below the 3 sigma limit.The test covers all 2n measurement elements.
  • Stopping Criterion: Element-wise checking prevents a few large residuals from being hidden by many small residuals.
  • Stopping Criterion: The maximum number of iterations is set to 20 to prevent runs that cannot satisfy the stopping criterion.

V. Simulation Results

The simulations evaluate calibration accuracy across two drone trajectories, comparing estimation errors with CRLB benchmarks and examining observability. Scenario 1 performs well, while scenario 2 is marginally observable, especially when pitch and GPS altitude bias are both unknown.

  • Estimation Accuracy: 100 Monte Carlo runs evaluate RMSE against CRLB benchmarks for the simulated scenarios.The study also assesses statistical efficiency using NEES with respect to the CRLB.
  • Trajectory Observability: Scenario 2 is marginally observable because its similar-range flight legs allow pitch and GPS altitude bias to compensate for each other.Its image-space trajectories are nearly parallel, unlike scenario 1, whose differing ranges distinguish the effects of pitch and altitude bias.
  • Trajectory Observability: Including GPS altitude bias significantly increases estimation error for the previously recommended scenario 2 trajectory.The path is therefore not practical for calibration when GPS altitude bias is included as an unknown.
  • Time-Offset Estimation: 0.27ms is the best time-offset RMSE in scenario 1, below the GPS time discretization of 100ms.This indicates that the trajectory estimation algorithm overcomes the discrete GPS sampling limitation.

B. Statistical Efficiency

The paper evaluates statistical efficiency using NEES relative to the CRLB and examines how residual calibration errors affect image-space bias. Scenario 1 is efficient, while scenario 2 is borderline and the recommended calibration achieves negligible residual bias.

  • Statistical Efficiency: NEES evaluates statistical efficiency against the CRLB using 100 runs and a 5-degree-of-freedom chi-square probability region of [0.8, 12.8].Estimates are statistically efficient when 95% of NEES values fall within this interval.
  • Statistical Efficiency: Scenario 1 had 0 NEES values outside [0.8, 12.8], versus an expected 5, indicating statistically efficient estimates.The result is consistent with equality in the CRLB.
  • Statistical Efficiency: Scenario 2 had 9 NEES values outside [0.8, 12.8], indicating marginal statistical efficiency for a marginally observable scenario.The reported borderline behavior is attributed to the standard deviation of exceedance counts being approximately 2.
  • Statistical Efficiency: Scenario 1 had an average NEES of 4.69, while scenario 2 had 5.86, supporting efficiency for scenario 1 and borderline efficiency for scenario 2.The 95% average-NEES probability region is [4.1, 5.63].
  • Impact of the Residual Biases: 0.20pixel was the highest residual-bias RMSE, compared with 1.41pixel total measurement-error standard deviation, making the bias 7.2 times smaller.The result is based on residual yaw, pitch, and roll errors evaluated over 5×5 image grids.
  • Impact of the Residual Biases: 0.29pixels was the maximum combined yaw, pitch, and roll bias, still negligible compared with the 1.41pixel measurement RMSE.The largest combined bias occurs at one corner of the image grid.

VI. Conclusions

The paper develops a GPS-based stationary-camera calibration algorithm that jointly estimates orientation, GPS altitude bias, and camera–GPS time offset. Simulations support the recommended trajectory, whose residual bias is negligible, while real setups require additional modeling work.

  • VI. Conclusions: The algorithm jointly estimates camera yaw, pitch, roll, GPS altitude bias, and GPS–camera time offset from discretized GPS trajectories.The GPS trajectories use a 0.1s time interval.
  • VI. Conclusions: The recommended drone trajectory produced estimation results meeting the CRLB and NEES requirements and was considered suitable for practical use.These conclusions are based on simulation tests.
  • VI. Conclusions: 0.27ms was the best time-offset estimation RMSE, below the GPS trajectory discretization of 100ms.The result indicates that the trajectory estimation algorithm overcomes the GPS discretization problem effectively.
  • VI. Conclusions: 14% of the measurement-error standard deviation was the residual calibration-bias RMSE for the recommended trajectory, which was negligible.The residual bias concerns the calibration result obtained with the recommended trajectory.
  • VI. Conclusions: Real camera setups require further work because focal length may need estimation and GPS latitude–longitude quantization can matter at near range.The proposed ILS algorithm needs further development to handle these effects.

Appendix A The Importance of Being Earnest about Radians

The appendix explains that Taylor expansions of trigonometric functions require angles in radians. Using degrees can produce an incorrect approximation even when sine and cosine themselves are unit-invariant.

  • Appendix A The Importance of Being Earnest about Radians: Taylor expansion of sin(30.01°) must use radians to obtain the correct first-order approximation of 0.50015.Using degrees directly produces a wrong result.
  • Appendix A The Importance of Being Earnest about Radians: A small 0.01° increment before cos(·) can cause an incorrect Taylor-expanded result when degrees are used.The appendix therefore requires angle conversion to radians for series expansions.

Appendix B Derivatives for (40)

The derivative appendix supplies gradients for the estimation equations and clarifies that trajectory position and velocity are estimated for use in those derivatives. The orientation angles are represented in radians.

  • Appendix B Derivatives for (40): The iteration index j is omitted while the gradients needed for the estimation equations are provided.These gradients support the algorithm’s iterative calculations.
  • Appendix B Derivatives for (40): The derivative with respect to τ uses estimated drone velocity components evaluated at the time-shifted trajectory point.The expression combines x-, y-, and z-velocity terms with the orientation-angle trigonometric factors.
  • Appendix B Derivatives for (40): The trajectory point and velocity used in the derivatives are estimated in Section IV-A.The point is evaluated at t_k + τ.
  • Appendix B Derivatives for (40): The angles α, ϵ, and ρ must be expressed in radians in the derivative calculations.The appendix refers to Appendix A for the unit requirement.
Loading 2608.26548v1…