Source-linked AI summary
Towards Initialization-free Calibrated Bundle Adjustment
Carl Olsson, Amanda Nilsson
TL;DR
Initialization-free pOSE methods reconstruct without calibration and then upgrade a projective solution, a process that can require more data and be problematic under noise. This paper incorporates pairwise relative rotations into pOSE to use calibration information, and reports convergence to the global optimum from random starts with high probability.
Problem
Existing initialization-free pOSE methods first optimize a projectively invariant uncalibrated reconstruction and then upgrade it; under noise the upgrade may not fit all intrinsics, and the uncalibrated stage requires more data.
Source: §1 · pp. 1–2Method
The method combines pOSE with pairwise relative rotation estimates, making the objective similarity-invariant and encouraging near-metric reconstructions using calibration information.
Source: §1 · p. 2Results
Takeaways & Limitations
The combined formulation produces near-metric reconstructions that do not need a separate upgrade to conform to the real scene geometry.
Source: §6 · p. 7The prior stratified upgrade has no guarantee of fitting all camera intrinsics under noise, and its uncalibrated reconstruction requires more data because cameras have more degrees of freedom.
Source: §1 · p. 2
Abstract
from arXiv · showhide
A recent series of works has shown that initialization-free BA can be achieved using pseudo Object Space Error (pOSE) as a surrogate objective. The initial reconstruction-step optimizes an objective where all terms are projectively invariant and it cannot incorporate knowledge of the camera calibration. As a result, the solution is only determined up to a projective transformation of the scene and the process requires more data for successful reconstruction. In contrast, we present a method that is able to use the known camera calibration thereby producing near metric solutions, that is, reconstructions that are accurate up to a similarity transformation. To achieve this we introduce pairwise relative rotation estimates that carry information about camera calibration. These are only invariant to similarity transformations, thus encouraging solutions that preserve metric features of the real scene. Our method can be seen as integrating rotation averaging into the pOSE framework striving towards initialization-free calibrated SfM. Our experimental evaluation shows that we are able to reliably optimize our objective, achieving convergence to the global minimum with high probability from random starting solutions, resulting in accurate near metric reconstructions.
1. Introduction
Bundle adjustment is non-convex and typically needs a suitable initialization, while existing initialization-free pOSE methods first reconstruct without calibration and then upgrade the result. This paper combines pOSE with relative rotation estimates to incorporate calibration information and produce near-metric reconstructions, reporting reliable convergence from random starts.
1. Introduction
- Bundle adjustment is non-convex with numerous local minima, while incremental reconstruction can accumulate drift and require intermediate bundle-adjustment stages.Incremental reconstruction builds from a two-view base through consecutive resection and triangulation problems.Source: §1 · p. 1
- Rotation averaging reduces parameters by optimizing camera orientations, but independently estimated two-view poses can be inconsistent, so bundle-adjustment refinement may still be needed.A sufficiently dense camera graph has been reported to leave no local minima other than the global one.Source: §1 · p. 1
- Earlier pOSE methods use an uncalibrated reconstruction followed by a projective upgrade, which can fail under noise and requires more data because uncalibrated cameras have more degrees of freedom.The passage contrasts 11 degrees of freedom for uncalibrated cameras with 6 for calibrated cameras.Source: §1 · pp. 1–2
- Relative rotation estimates reduce projective invariance to similarity invariance, allowing the objective to encourage near-metric reconstructions while incorporating camera calibration information.The method does not explicitly enforce the given intrinsics because doing so makes optimization sensitive to local minima.Source: §1 · p. 2
- The combined pOSE and relative-rotation formulation reportedly converges to the global optimum from random starting solutions with high probability.The contributions also state that the optimization is solved using VarPro.Source: §1 · p. 2
2. Projective Ambiguity and Upgrades
Uncalibrated reconstruction admits projectively equivalent solutions, while traditional upgrades use camera calibration to seek a metric reconstruction. The section introduces relative-rotation constraints as an alternative way to reduce this ambiguity and discusses a projective-invariant measure of upgradability.
2.1. Uncalibrated Reconstruction
- Uncalibrated reconstruction has projectively equivalent solutions related by a 3D projective transformation, with reprojection costs invariant to that transformation.The camera-point graph represents cameras and 3D points as nodes, with reprojection errors on its edges.Source: §2.1 · pp. 2–3
2.2. Upgrades
- Traditional upgrades use knowledge of camera internal parameters to seek calibrated cameras, but noisy constraints and inexact upgrades can degrade reconstruction quality.Upgrade methods solve an overdetermined system for Ω and then extract H; under noise, exact solutions are unlikely, and extraction may require modification.Source: §2.2 · p. 3, §2.3 · p. 3
2.3. Using Relative Rotation Measurements
- Pairwise relative-rotation penalties add camera-graph constraints that are not projectively invariant and encourage solutions close to calibrated without a separate upgrade.Their weighting is designed to penalize deviations from the estimated rotation’s tangent plane and approximate changes in two-view reprojection error.Source: §2.3 · p. 3
2.4. Invariance of the Fundamental Matrix
- The normalized difference between a fundamental matrix’s two nonzero singular values measures how close it is to being essential and thus how close a solution is to upgradable.The stated converse requires camera centers that are not collinear.Source: §2.4 · p. 4
3. pOSE with Relative Rotation Estimates
The section combines pOSE with pairwise relative rotation estimates to seek near-metric reconstructions. Its formulation retains pOSE's bilinear least-squares structure while adding rotation information.
3.1. pOSE
- pOSE replaces nonlinear perspective error with object-space error and yields a bilinear least-squares objective in camera matrices and 3D points.This structure permits closed-form point solutions for given camera parameters, enabling VarPro.Source: §3.1 · p. 4
- A small η allows larger projective-depth variations, while weak depth terms prevent trivial zero-penalty solutions.The depth term favors projective depth approximately 1.Source: §3.1 · p. 4
3.2. pOSE with Rotation Averaging
- Pairwise relative rotation estimates from two-view relative pose add calibration information and reduce projective ambiguity to similarity ambiguity.The method penalizes deviations between relative camera rotations and their estimates.Source: §3.2 · pp. 4–5
- The rotation penalty locally approximates the best reprojection error for a chosen rotation, then lifts that penalty to general 3 × 3 matrices.The lift distinguishes tangent-space rotations from normal-space deviations; β controls the tradeoff between pOSE and rotation averaging.Source: §3.2 · p. 5
4. Optimization Method
The optimization method uses VarPro to reduce the problem to camera parameters, then updates the remaining variables exactly at each step.
4. Optimization Method
- Writing the objective as squared residuals enables a damped update of B after projecting out the linearized influence of v.The projected least-squares step uses J and K, the Jacobians with respect to B and v.Source: §4 · pp. 5–6
- The objective is linear least squares in C and t, so their optimal values can be computed in closed form for any fixed B.VarPro substitutes these optimal values back into the objective, leaving an optimization problem over B.Source: §4 · p. 5
- After each proposed B update, VarPro recomputes v exactly and accepts the step only if the residual error decreases.The algorithm adjusts its damping parameter after acceptance or rejection.Source: §4 · p. 6
5. Experiments
Across the experiments, adding pairwise rotation penalties improves near-metric reconstruction and increases the chance of reaching the global minimum from random starts. The rotation-averaging optimization also reaches its minimum in fewer iterations than pOSE.
5.1. Effects of different rotation constraints
- Our rotation-averaging approach reaches the global minimum from random starts more often than the other methods.Figure 3 summarizes objective-value histograms across datasets and methods; direct rotation parameterization and orthogonality-only penalties have low success rates.Source: §5.1 · p. 6, Figure 3 · p. 7
- 35 versus 137 iterations were needed on average for Ahlstromer by our model and pOSE, respectively.The reported runtimes were 5 seconds and 17 seconds, with roughly equal time per iteration.Source: §5.2 · p. 6
5.2. Near Metric Reconstructions
- Adding the relative-rotation penalty generally increases pOSE error because the proposed objective also optimizes that rotation component.The comparison evaluates projection quality using pOSE error for both methods.Source: §5.2 · pp. 6–7
- The proposed formulation yields solutions closer to metric than pOSE-only, with the difference generally growing with dataset size.For Urban, the range of normalized fundamental-matrix singular-value differences is almost 1 under pOSE-only, meaning at least one matrix is far from essential.Source: §5.2 · p. 7
- 6 to over 50 times larger are the fundamental-matrix measures for pOSE-only than for rotation averaging on larger datasets.For the smallest datasets with 25 to 35 cameras, pOSE-only measures are only two to four times larger.Source: §5.2 · p. 7
6. Conclusions
The method combines pOSE with relative rotation estimates to produce near-metric reconstructions whose residuals are invariant only to similarity transformations. It converges to the right solution from random non-metric starts with high probability, while simple rotation-parameterization and orthogonality-penalty modifications can get stuck in local minima.
6. Conclusions
- The method combines pOSE with relative rotation estimates, producing near-metric reconstructions that do not need a projective upgrade.Its new residuals are invariant only to similarity transformations.Source: §6 · p. 7
- Simple pOSE modifications, including rotation parameterization and orthogonality-constraint penalties, can get stuck in local minima.Source: §6 · p. 8
- The proposed formulation converges to the right solution from random non-metric starting solutions with high probability.The evaluation contrasts this with simple pOSE modifications that can get stuck in local minima.Source: §6 · pp. 8–9