PaperlayerView exact revision on arXiv

Source-linked AI summary

Automatic Reproducible Camera Intrinsic Calibration

Xiangcheng Hu

arXiv:2609.10082v1cs.ROcs.CV

TL;DR

The paper addresses automatic selection of calibration images and radial distortion order, which existing workflows often leave to practitioners. It combines model-conditioned iterative rejection with held-out pose-only validation and an interactive tool. Across seven datasets, image filtering reduces held-out error by up to 25%, with order selection providing further improvement.

  • Problem

    Calibration workflows must determine both which images to use and which radial distortion order to adopt, but calibration-set residuals cannot reliably resolve either choice.

  • Method

    The pipeline iteratively rejects high-residual views separately for each candidate order, selects the order on held-out images with intrinsics and distortion fixed, and integrates both steps into an interactive tool.

  • Results

    The pipeline achieves the lowest held-out mean across five public datasets, with image filtering reducing held-out error by up to 25% and order selection providing further improvement.

  • Takeaways & Limitations

    Automatic image-set and distortion-order selection can improve held-out intrinsic-calibration performance without relying on manual image selection or calibration-set residuals.

Abstract

from arXiv · show

Accurate camera intrinsic calibration is fundamental to robot perception, and the accuracy depends on the quality of the collected images. However, existing target-based calibration methods often require the practitioner to manually filter out high-quality images and to specify an appropriate radial distortion order. This paper presents a fully automatic intrinsic calibration pipeline that determines both from the collected data. We adopt an iterative rejection scheme that estimates parameters on a candidate image set and removes views whose mean residual exceeds a multiple of the median. Crucially, this process runs independently under each candidate distortion order, so that the retained image set is consistent with the residual scale of that order. Further, the distortion order is selected on held-out images, with the intrinsics and distortion fixed and only the board pose re-estimated, ensuring that an added coefficient is supported by independent observations. Finally, we integrate both steps into an interactive calibration tool that supports full-pipeline data inspection and parameter estimation. Experiments on our own camera data and five public real-world datasets show that image filtering reduces the held-out reprojection error by 25\%, the order selection further by 5\%, achieving the lowest held-out mean among four compared configurations without manual image selection. We will release the code and data to facilitate future research.

I. INTRODUCTION

Accurate intrinsics matter because calibration errors recur as systematic geometric bias across downstream vision tasks. Existing practice leaves image filtering and radial distortion-order selection unresolved, while residual-based decisions are coupled to the fitted model.

  • Calibration errors recur as systematic geometric bias in depth estimation, pose estimation, visual odometry, and multi-sensor fusion.
  • Field calibration must assess image quality from collected data because hand-held board acquisition lacks production fixtures and prescribed orientations.
  • Per-view residuals cannot independently identify images to remove because they are available only after estimating the calibration parameters.
  • Higher-order models always reduce calibration-set residuals through fitting, so that residual cannot establish improved prediction on unseen images.
  • Residual thresholds and retained image sets vary with distortion order, making a fixed image set inconsistent across candidate models.

B. Contributions

The paper contributes automatic image-set and distortion-order selection, then combines both into an interactive calibration tool. The pipeline evaluates image rejection separately for each model and assesses model adequacy on held-out views.

  • The paper introduces iterative image rejection using a median-scaled residual threshold under each distortion order.
  • Distortion order is selected on held-out images with intrinsics and distortion fixed, avoiding calibration-set residuals for model assessment.
  • Both selection steps are integrated into an interactive tool supporting full-pipeline data inspection and parameter estimation.
  • Calibration minimizes reprojection error over a selected image subset, while held-out error re-estimates only board poses with intrinsics and distortion fixed.

B. Stage 1: Detection-Based Acceptance

The pipeline uses detection completeness for initial acceptance, then iteratively removes geometrically inconsistent views using residuals. Recomputing the median tightens the threshold and adapts rejection to each candidate distortion order.

  • Stage 1 admits an image only when the detector recovers the complete inner corner grid.
  • Stage 2 targets views with successful corner extraction that remain geometrically inconsistent with the rest of the image set.
  • At each iteration, images whose mean residual exceeds a threshold proportional to the current median residual are rejected and calibration is re-solved.
  • Recomputing the median tightens the threshold after outlier removal and adapts it to the residual scale of each candidate order.
  • The rejection procedure runs independently under each candidate order, producing a model-conditioned estimation set.

III. DISTORTION-MODEL SELECTION

The order-selection comparison is restricted to two radial distortion models. A deterministic held-out split is used only when enough accepted images are available, and validation views are drawn from the accepted pool.

  • The comparison considers M2 with k1, k2, p1, p2 and M3 with the additional radial coefficient k3.
  • The accepted Stage 1 images are split deterministically by assigning every fifth image in capture order to validation.
  • The order-selection split is applied only when at least Nmin = 15 accepted images are available; otherwise M2 is retained.
  • Validation views come from the Stage 1 accepted pool rather than rejected images, preventing extraction noise from biasing the distortion-model comparison.

B. Pose-Only Validation and Criterion

Each distortion candidate is evaluated on validation views after fixing its intrinsics and distortion and re-estimating only board poses. The higher-order model is adopted only when it does not worsen either mean or worst-case validation error, using independently held-out observations.

  • For each candidate distortion model, intrinsic parameters are estimated with that model’s own rejection process, producing candidate-specific intrinsics and distortion.
  • Validation fixes intrinsics and distortion, then re-estimates only the six-degree-of-freedom board pose for each held-out image.This makes evaluation a perspective-n-point problem with 2N_i measurements per view.
  • Each distortion order is compared using the estimation set induced by its own residual scale and a common validation subset.The comparison is therefore controlled while accounting for candidate-specific image rejection.
  • With α = 1 and β = 1, the higher-order model is adopted only when it worsens neither mean nor worst-case validation error.Parity avoids penalizing differences within the validation-error noise level, and the added coefficients must be supported by independent observations.
  • After selecting M⋆, rejection is rerun on the full accepted pool, and the resulting final estimation set produces the final intrinsic and distortion parameters.Validation views can enter the final estimate at this stage, while rejected views are recorded.

IV. INTERACTIVE CALIBRATION TOOL

The interactive offline tool makes data-dependent image filtering and distortion-order decisions inspectable and reproducible. It supports folder import, target specification, calibration, and provenance recording, while the algorithmic core remains scriptable.

  • The tool workflow consists of importing an image folder, specifying target geometry, and running calibration.
  • The interface displays every image’s filtering status and the distribution of calibration-point residuals.
  • The algorithmic core is independent of the interface and can be invoked from scripts.The reported Section V results are produced through this scriptable core.
  • Calibration provenance records retained and rejected images, residuals, the C and V subsets, selected-order validation quantities, and corner-zone observation counts.

A. Experimental Setup

The evaluation spans own-camera and public data across four camera models, with held-out reprojection error as the main criterion. Image filtering improves Rig-A validation performance substantially, while the setup separates calibration fit from unseen-view evaluation.

  • The evaluation covers own-camera images and five public datasets spanning four camera models with 62°–111° diagonal fields of view and 0.3–2.3 Mpx resolution.Rig-A contains 39 hand-held images; Rig-B uses two independent sessions, while the public data include OpenCalib-F, ROS, and OpenCV stereo datasets.
  • The comparison includes ROS-calib, mrcal, and an ablation disabling image filtering and fixing the distortion order at M2.
  • Held-out reprojection error is the evaluation criterion, whereas calibration-set residual is treated as a fit diagnostic.For Rig-A, all experiments use the same six held-out images.
  • On Rig-A, Stage 1 rejects 12 of 39 images and Stage 2 rejects seven additional images under the final selected order.OpenCalib-F retains all 22 images, while the remaining datasets reject one to two images.
  • The filtering targets views affected by systematic corner-extraction error, whose rejected-image median per-view error on Rig-A is 4.7 times that of accepted images.
  • 25%: applying both filtering stages reduces Rig-A held-out error from 0.419 to 0.316 px under M2.The same comparison reduces calibration-set error from 0.467 to 0.342 px, and the held-out evaluation uses the same six images.

C. Distortion-Model Selection Analysis

The analysis selects between M2 and M3 using held-out views rather than calibration-set error, whose reduction can conflate added flexibility with image rejection. M3 is favored when independent validation supports it, particularly for wider-field data.

  • Order-selection evidence: The calibration-set error falls from 0.342 px under M2 to 0.143 px under M3, but this combines coefficient flexibility with six additional rejected images.On the 26 M2-retained images, the added coefficient alone reduces error to 0.252 px.
  • Order-selection evidence: Held-out error decreases from 0.316 px under M2 to 0.299 px under M3, satisfying both conditions of the order criterion.The intrinsics and distortion are evaluated on held-out views rather than selected by calibration-set fit.
  • Robustness: Across five split offsets, M3 lowers held-out mean error by 0.073 ± 0.034 px and is selected at four offsets.Using α = 0.98 instead of parity would reject M3 on two OpenCalib-F folds with only 1.2% and 1.3% improvement.
  • Dataset scope: The order criterion applies only when each estimation fold has at least Nmin images, which occurs only for the two wider-field datasets.On OpenCalib-F, M3 is selected on three of four folds and reduces held-out error from 0.230 to 0.226 px; narrower-field datasets use filtering alone.
  • Model behavior: Under M2, residuals rise from 0.23 to 0.74 px toward the image border, whereas M3 remains near 0.15 px over the observed range.The calibration points reach r = 841 px versus r = 1126 px at the corner, where radial scale factors diverge by 21.2%.

D. Comparison with Existing Methods

The proposed pipeline achieves the lowest held-out mean on all five public datasets and improves over configurations that rely on fixed or overly flexible distortion models. Its scope includes seven datasets and four camera models, with broader model families left for future work.

  • Public-dataset comparison: The proposed pipeline achieves the lowest held-out mean on all five public datasets.On Rig-A, it yields 0.143 px on retained images, compared with 0.151 px for mrcal and 0.299 px for ROS-calib.
  • Public-dataset comparison: At eight coefficients, mrcal has the lowest calibration-set residual but 7.6 to 63.3% higher held-out error than at four coefficients across all five datasets.This comparison directly illustrates the risk of selecting distortion order from calibration-set fit.
  • Contribution: The pipeline determines both the image set and radial distortion order from collected data without relying on calibration-set residuals for either decision.The conclusion summarizes the method across seven datasets and four camera models.
  • Contribution: Image filtering reduces held-out error by up to 25%, with order selection providing further improvement.The reported comparison covers experiments on seven datasets across four camera models.
  • Scope: The evaluation is limited to radial distortion order, while rational and fisheye families are identified as future extensions.The paper also proposes adding geometric coverage constraints to the filtering criterion.

Supplementary Material

The supplementary material gives the complete selection algorithm, its fallback behavior, reproducibility resources, and Rig-A residual diagnostics. It also records that the repository is withheld during review despite a planned release.

  • Algorithm: Algorithm 1 initializes detected images, evaluates M2 and M3 on estimation and held-out subsets, then selects M3 only when criterion (6) is satisfied.Otherwise, or when the accepted set is too small to split, the pipeline retains M2.
  • Algorithm: The selection function repeatedly solves calibration under a candidate model and removes views until the set stabilizes or falls below Nstop.The final selected model is then used to calibrate the full candidate image set and produce the calibration record.
  • Diagnostics: Fig. 5 plots per-view residuals for all 39 Rig-A images, marking the calibration median and rejection threshold τ = 2˜e.The figure colors images by selection result under the final model.
  • Reproducibility: The tool, data, and scripts intended to reproduce every reported number will be released, while the repository is withheld during review.The supplementary material presents both the complete pipeline and per-view filtering statistics.

A. Complete Pipeline

The complete pipeline applies staged residual-based filtering and records the resulting diagnostics in an interactive tool. The implementation exposes image status, point coverage, model settings, and software-layer outputs for inspection.

  • A. Complete Pipeline: The pipeline’s fallback retains M2 when the accepted set is too small to split, while the final model determines Rig-A filtering outcomes.Replacing the median-scaled threshold with ˜e+λˆσ leaves the retained set unchanged for λ ≥3 but worsens held-out error to 0.409 px for λ ≤2.
  • A. Complete Pipeline: Stage 1 rejects twelve images with a median residual 4.7 times that of accepted images, and Stage 2 rejects seven views exceeding τ = κ˜e after rescaling.The residual scale is recomputed on the Stage 1 retained set before Stage 2 rejection.
  • B. Software Implementation: The interface shows each image’s selection status, calibration-point distribution, corner-zone counts, board and model settings, residual, and views exceeding κ˜e.Stage 1 rejections are greyed out and Stage 2 rejections are marked in orange.
  • B. Software Implementation: The software separates a calibration core from session and interface layers that manage datasets, diagnostics, and displayed results.The core implements extraction, both selection stages, estimation, and order selection; the session delegates computation to a worker thread.
Loading 2609.10082v1…