Source-linked AI summary
Global Structure-from-Motion Revisited
Linfei Pan, Dániel Baráth, Marc Pollefeys, Johannes L. Schönberger
TL;DR
SfM must recover 3D structure and camera motion, while existing incremental and global approaches trade robustness and accuracy against scalability and efficiency. GLOMAP addresses this trade-off through joint global estimation of camera and point positions, achieving comparable or superior accuracy and robustness to incremental methods while being orders of magnitude faster. The method remains vulnerable to failures in rotation averaging and image correspondence.
Problem
SfM seeks to recover 3D structure and camera motion, but global methods have faced an accuracy and robustness gap relative to incremental methods.
Method
GLOMAP jointly estimates camera and point positions in a global positioning step instead of separately performing translation averaging and triangulation.
Results
GLOMAP achieves comparable or superior accuracy and robustness to incremental methods while being orders of magnitude faster.
Takeaways & Limitations
GLOMAP provides a general-purpose global SfM pipeline that closes the robustness and accuracy gap to incremental systems while preserving global efficiency.
Takeaways & Limitations
Failures in rotation averaging or image correspondence can degrade results or cause catastrophic failures, especially with symmetric structures or drastic appearance and viewpoint changes.
Abstract
from arXiv · showhide
Recovering 3D structure and camera motion from images has been a long-standing focus of computer vision research and is known as Structure-from-Motion (SfM). Solutions to this problem are categorized into incremental and global approaches. Until now, the most popular systems follow the incremental paradigm due to its superior accuracy and robustness, while global approaches are drastically more scalable and efficient. With this work, we revisit the problem of global SfM and propose GLOMAP as a new general-purpose system that outperforms the state of the art in global SfM. In terms of accuracy and robustness, we achieve results on-par or superior to COLMAP, the most widely used incremental SfM, while being orders of magnitude faster. We share our system as an open-source implementation at {https://github.com/colmap/glomap}.
1 Introduction
Structure-from-Motion recovers 3D structure and camera motion from images, but the field has long balanced incremental robustness and accuracy against global scalability and efficiency. GLOMAP revisits global SfM by jointly estimating camera and point positions, aiming to close this gap while supporting practical data settings.
- SfM recovers 3D structure and camera motion from image collections for downstream tasks including novel-view synthesis and cloud-based mapping and localization.
- Incremental SfM sequentially registers cameras and structure with repeated bundle adjustment, achieving high accuracy and robustness but limiting scalability.Global SfM instead estimates camera geometry for all images at once through rotation and translation averaging.
- GLOMAP replaces translation averaging followed by separate triangulation with joint camera-and-point position estimation in one global positioning step.The system is designed as a general-purpose global SfM pipeline and can handle unknown camera intrinsics and sequential image data.
- GLOMAP targets the robustness and accuracy of incremental SfM while retaining the efficiency of global pipelines, and is released as open source.
- Translation averaging is ill-posed and sensitive to noisy measurements, co-linear motion, weak view-graph connectivity, and inaccurate camera intrinsics.Relative translations are recoverable from two-view geometry only when camera intrinsics are known.
3 Technical Contributions
GLOMAP improves global SfM by jointly estimating camera and point positions, using robust camera-ray constraints instead of ill-posed translation averaging followed by triangulation. Its pipeline combines correspondence filtering, global estimation, refinement, and optional camera clustering for challenging imagery.
- Global positioning: GLOMAP jointly estimates camera and point positions instead of sequentially performing translation averaging and global triangulation.
- Global positioning: Randomly initialized cameras and points are optimized by adjusting positions while keeping camera orientations fixed, minimizing angles between measurements and image rays.
- Robust objective: The method uses bounded normalized direction differences with Huber robustification, reducing outlier influence compared with unbounded reprojection errors.The angular error is strictly bounded to [0, 1].
- Robust objective: Discarding relative translation constraints improves applicability when camera intrinsics are inaccurate or unknown and when cameras depart from the expected pinhole model.The formulation relies on filtered two-view geometry and camera-ray errors rather than relative translation estimates.
- Evaluation: On ETH3D SLAM, the proposed system outperforms global SfM baselines by a large margin and achieves better results than COLMAP.
- Proposed pipeline: The pipeline estimates global rotations, filters inconsistent poses, jointly estimates cameras and points, then applies global bundle adjustment and optional structure refinement.Camera clustering can additionally produce coherent reconstructions for internet imagery with wrongly matched non-overlapping images.
4 Experiments
Experiments across calibrated and uncalibrated, unordered and sequential datasets show that GLOMAP combines strong reconstruction accuracy and robustness with global-SfM efficiency. Ablations indicate that joint camera-and-point positioning is important, while failures remain under rotation-averaging or correspondence errors.
- Experimental setup: GLOMAP is evaluated against OpenMVG, Theia, and COLMAP across ETH3D, LaMAR, IMC 2023, and MIP360 datasets.The evaluation spans calibrated to uncalibrated and unordered to sequential image collections.
- ETH3D SLAM: 8% higher recall and 9 and 8 additional AUC points at the 0.1m and 0.5m thresholds, respectively, are obtained over COLMAP on ETH3D SLAM, while COLMAP is one order of magnitude slower.Against other global pipelines, GLOMAP improves recall by 18% and 4% and raises AUC by around 11 points at 0.1m.
- ETH3D MVS (rig): GLOMAP reconstructs all ETH3D MVS (rig) scenes, matches or exceeds COLMAP’s accuracy where COLMAP succeeds, and runs about 3.5 times faster than COLMAP.It is a little slower than global SfM baselines on this dataset.
- ETH3D MVS (DSLR): On ETH3D MVS (DSLR), GLOMAP outperforms OpenMVG and Theia and achieves accuracy comparable to COLMAP, but rotational symmetry causes inaccurate reconstruction for exhibition_hall.All methods have comparable runtimes because the scenes are small.
- LaMAR and IMC 2023: GLOMAP substantially outperforms competing baselines on LaMAR and IMC 2023, including COLMAP on LaMAR HGE and LIN and global baselines at IMC 2023’s 3°, 5°, and 10° AUC thresholds.For LIN, structure refinement is omitted because of a memory limitation; IMC 2023 ground truth is generated by COLMAP with held-out imagery.
- Ablations: Ablations show that relative translation constraints deteriorate convergence and overall performance, supporting the joint global-positioning strategy using camera and point information.The ablations compare camera-only, camera-plus-point, and LUD-based positioning alternatives.
- Limitations: Failure cases primarily arise from rotation-averaging errors on symmetric structures or from incorrect or missing image correspondences.The paper notes that drastic appearance or viewpoint changes can produce degraded or catastrophic failures.
5 Conclusion
The paper introduces GLOMAP, a global SfM pipeline that merges camera-position estimation and 3D structure estimation into one global positioning step. Across extensive experiments, it achieves accuracy and robustness comparable or superior to incremental methods while being orders of magnitude faster.
- Conclusion: GLOMAP merges camera positions and 3D structure into a single global positioning step instead of separate translation averaging and point triangulation.The conclusion identifies the use of points in optimization as the key design choice.
- Conclusion: Extensive experiments show accuracy and robustness comparable or superior to incremental methods while retaining global-SfM efficiency.The paper reports that GLOMAP is orders of magnitude faster.
- Conclusion: The GLOMAP code is released as open source under a commercially friendly license.
S1 Additional Comparisons
Additional comparisons show that HSfM can match GLOMAP on some calibrated datasets but fails on sparse or sequential datasets and when intrinsics are unavailable. The experiments compare GLOMAP with HSfM and LiGT across datasets with and without camera calibrations.
- The experiments additionally compare GLOMAP with HSfM and LiGT, using Theia and OpenMVG implementations with stated pipeline adjustments.Theia receives extra bundle-adjustment steps for a fair HSfM comparison, while LiGT uses the authors’ OpenMVG implementation and pipeline.
- HSfM achieves comparable results to GLOMAP on the ETH3D MVS rig and DSLR datasets with known camera intrinsics.
- HSfM fails on ETH3D SLAM and LaMAR, producing significantly less accurate reconstructions than the other tested pipelines.The authors attribute these failures to the sparsity and sequential nature of the datasets.
- HSfM substantially falls behind all tested methods on IMC 2023 and MIP360, where camera calibrations are unavailable.HSfM assumes known intrinsics and keeps them fixed until the final bundle-adjustment step, whereas coarse priors are insufficient without further refinement.
S2 Additional Reconstruction Results
Additional reconstruction results indicate that GLOMAP accurately and robustly recovers both general scene structure and fine details. Table S4 reports that it largely outperforms other baselines while achieving results similar to COLMAP.
- GLOMAP accurately and robustly reconstructs scenes, recovering both their general structure and fine details.
- GLOMAP largely outperforms other baselines on MIP360 while obtaining results similar to COLMAP.For scenes with unregistered testing images, the dataset’s reference camera pose is used.
S3 Novel View Synthesis
The novel-view-synthesis evaluation uses Instant-NGP on MIP360 and measures PSNR and SSIM. GLOMAP reconstructions achieve synthesis scores similar to COLMAP, while OpenMVG and Theia fail on several scenes and can produce blurrier results.
- The evaluation tests reconstruction quality for novel view synthesis using Instant-NGP on the MIP360 dataset.
- PSNR and SSIM are used as the quantitative metrics for comparing synthesized images.
- GLOMAP and COLMAP achieve similar PSNR and SSIM scores, whereas OpenMVG and Theia fail on several scenes.Theia and OpenMVG produce more blurred synthesized results in several scenes, indicating poorer camera-pose quality.
S4 Effect of Camera Clustering
GLOMAP uses camera clustering to improve reconstruction quality for unordered internet image collections. The qualitative comparison shows that the mechanism is effective at pruning floating structures.
- Camera clustering is proposed for obtaining clean and coherent reconstructions from unordered internet image collections.
- The qualitative comparison shows that camera clustering effectively prunes floating structures.
S5 Robustness of Global Positioning.
The experiment tests global positioning robustness under synthetic Gaussian reprojection noise while isolating global positioning from global bundle adjustment. With perfect observations and 0px noise, the method reliably converges.
- Synthetic Gaussian reprojection noise is added to perfect image observations generated from COLMAP triangulations and ground-truth cameras.Each observation is perturbed after projecting the triangulations to the ground-truth cameras.
- Global bundle adjustment is omitted to isolate the performance of global positioning.
- 0px noise: perfect image observations reliably converge.
S6 Detailed Results of ETH3D SLAM
The ETH3D SLAM section reports per-sequence results whose relative performance is consistent with averaged results across sequences sharing a prefix. Supplementary figures also show reconstructions, novel-view synthesis, and camera-clustering results.
- S6 Detailed Results of ETH3D SLAM: Per-sequence ETH3D SLAM results are reported in Table S6, with relative performance consistent across sequences sharing the same prefix.
- Qualitative Results: Fig. S1 presents example GLOMAP reconstructions across various datasets.
- Qualitative Results: Fig. S2 shows qualitative novel-view synthesis results using Instant-NGP for bicycle, bonsai, garden, room, and stump scenes.The caption notes visually evident differences among these scenes.
- Qualitative Results: Fig. S3 reports qualitative camera-clustering results on 1DSfM datasets.