PaperlayerView exact revision on arXiv

Source-linked AI summary

HoloRecon: Incremental Reconstruction of Long-Range Panoramas with Global Consistency

Zhuoxiao Li, Xinyi Liu, Hongye Hou, Jing OU, Kang Du, Shuai Zhang, Dongli Wu, Taoyu Wu, Siyuan Feng, Xiaohui Zhu, Wufan Zhao

arXiv:2610.36876v1cs.CV

TL;DR

Repeated structures and limited overlap make global connections difficult in long-sequence panoramic reconstruction, while feed-forward models’ computational costs limit the number of jointly processed views. HoloRecon combines sparse mapping with feed-forward geometry to verify distant matches and correct the global pose graph; experiments show improved mean ATE and high-fidelity dense reconstruction.

  • Problem

    Repeated, similar structures and small partial overlap make reliable long-range connections difficult, while feed-forward models’ computational costs limit jointly processed views and leave separate windows needing global constraints.

    Source: §1 · p. 2

  • Method

    HoloRecon couples ORB-based sparse mapping with a fine-tuned feed-forward model, calibrating predicted geometry with sparse landmarks and using verified distant matches to constrain pose-graph correction.

    Source: §1 · p. 2, §3.1 · p. 4, §3.3 · p. 5

  • Results

    Experiments across 360LOC, YUTO MMS, and PanoRecon show improved mean ATE and high-fidelity dense reconstruction results.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Sparse mapping anchors local feed-forward geometry, while that geometry strengthens constraints used to correct the sparse map.

    Source: §6 · p. 9

    Severe motion blur or prolonged low-texture segments can reduce reliable feature observations for tracking and depth calibration; the system does not explicitly handle dynamic objects and does not yet achieve real-time reconstruction.

    Source: §6 · p. 10

Abstract

from arXiv · show

Long-sequence panoramic reconstruction requires detailed scene geometry and globally consistent camera poses across distant revisits. Panoramic SLAM and incremental SfM rely on feature correspondences, but repeated, similar structures and small partial overlap make reliable long-range connections difficult to establish, leaving accumulated drift unresolved. Feed-forward models offer detailed local geometry, but their high computational costs limit inference window size, necessitating additional constraints for global consistency. This motivates combining continuous feature-based tracking and sparse mapping with rapid, high-quality local reconstruction from feed-forward models to achieve accurate localization and globally consistent dense mapping over long sequences. We therefore propose a panoramic reconstruction framework that couples an ORB-based sparse panoramic mapping frontend with a fine-tuned feed-forward model. Our key insight is that predicted local geometry enables 3D verification of feature matches across distant revisits, yielding long-range camera constraints even between regions. We calibrate predicted depths with sparse landmarks and use verified connections to optimize the global pose graph, updating dense geometry with the corrected camera poses. We also introduce PanoRecon, a seven-scene benchmark with repetitive architecture, large and small loops, and reference camera poses and LiDAR geometry for evaluating global consistency. Our method achieves the lowest mean absolute trajectory error and high F-1 Score among the evaluated methods. Project Page:https://zhuoxiaoli.github.io/HoloRecon_page/

1 Introduction

Long-range panoramic reconstruction must recover detailed geometry while keeping camera poses consistent across distant revisits, where repeated structures and limited overlap make reliable connections difficult. HoloRecon combines sparse panoramic mapping with feed-forward geometry to verify long-range matches and introduces PanoRecon to evaluate global consistency.

1 Introduction

  • Repeated facade textures, symmetric layouts, and limited overlap make it difficult to distinguish true revisits from similar places, so false loops distort maps while missed loops leave drift uncorrected.The introduction identifies reliable evidence for distinguishing revisits as necessary for global correction.Source: §1 · p. 2
  • HoloRecon couples ORB-based sparse mapping with a fine-tuned feed-forward model, using predicted geometry to verify distant feature matches and establish long-range camera constraints.The framework calibrates predicted geometry with sparse landmarks; accepted connections drive pose-graph correction and subsequent dense-geometry updates.Source: §1 · p. 2
  • PanoRecon comprises seven real-world scenes with repeated architectural textures and structures, multiple revisits separated by long travel distances, and LiDAR reference poses and dense geometry.These references support evaluation of trajectory accuracy and reconstruction consistency across distant revisits.Source: §1 · p. 2
  • The framework’s stated contributions are globally consistent panoramic mapping, long-range connections verified without shared sparse landmarks, and a benchmark for trajectory and dense-reconstruction evaluation.The connection mechanism combines feed-forward panoramic geometry with feature matching.Source: §1 · p. 2

2 Related Work

Prior work extends SfM and SLAM to panoramic imagery and uses feed-forward models for geometry and mapping. Distant partial-overlap regions still require reliable associations for global refinement.

2.1 Structure from Motion

  • Distant regions with partial overlap still require reliable associations for global refinement, motivating predicted depths to support feature correspondences across views.Prior spherical-imagery work includes pose estimation, bundle adjustment, dense matching, and structural constraints from lines and architectural layouts.Source: §2.1 · p. 2

2.2 Visual Odometry and Panoramic SLAM

  • Panoramic and spherical visual-odometry and SLAM systems use visual, feature-based, direct, learned, or visual–inertial approaches, but global accuracy depends on distinguishing true revisits from repeated appearance.The related work also notes equirectangular mapping and LiDAR-depth integration.Source: §2.2 · p. 3

2.3 Feed-Forward Reconstruction

  • Feed-forward reconstruction work spans persistent-state, local-to-world, causal-memory, and graph-optimization approaches, while panoramic systems predict geometry or use attention for revisit verification.VGGT-Long processes overlapping chunks; VGGT-SLAM 2.0 uses attention to verify revisits.Source: §2.3 · p. 3

3 Method

The method anchors local feed-forward geometry to an ORB sparse map, uses calibrated geometry to verify long-range feature matches, and feeds accepted connections into global refinement. It then updates cached dense geometry using the corrected map without rerunning feed-forward inference.

3 Method

  • The framework alternates between placing local dense predictions in the sparse map, verifying long-range connections, and propagating global corrections back to dense geometry.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Anchoring Feed-Forward Geometry to the Sparse Map

  • Sparse landmarks calibrate Pi3 depth scales, placing each submap’s predicted geometry in the ORB map’s coordinates.A shared window scale and frame-specific corrections are robustly fit from landmark observations.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 Verifying Long-Range Connections with Predicted Geometry

  • Candidate historical submaps are selected by voxel overlap and visibility, then mutual ORB matches across keyframe pairs combine overlap distributed across views.Source: §3.2 · p. 4
  • Depth-supported matches estimate a submap similarity, which is accepted only after multi-view geometric checks and agreement across neighboring historical submaps.The checks include bidirectional distance and bearing agreement, nondegenerate geometry, and distinct feature support from multiple keyframes on both sides.Source: §3.2 · pp. 4–5

3.3 Propagating Global Corrections to Sparse and Dense Geometry

  • Verified submap transforms become camera constraints; pose-graph optimization updates cameras and landmarks, then observation fusion creates shared cross-region observations for global bundle adjustment.Source: §3.3 · p. 5, §3.2 · p. 5
  • Panoramic bundle adjustment jointly refines graph-corrected poses and landmarks, while GPU preconditioned conjugate gradients solve the reduced camera system.Source: §3.3 · p. 5
  • After refinement, the method recalibrates and repositions cached dense points using optimized camera poses and depths without repeating Pi3 inference.Source: §3.3 · pp. 5–6

4 The PanoRecon Benchmark

PanoRecon comprises seven scenes with repeated architecture, low-texture surfaces, large loops, and distant revisits with limited overlap. It provides LiDAR-based reference poses and geometry for trajectory and reconstruction evaluation.

4 The PanoRecon Benchmark

  • Seven scenes include low-texture surfaces, repeated architecture, large loops, and distant revisits with limited overlap to test accumulated-drift correction.Source: §4 · p. 6
  • LiDAR trajectories are optimized with RTK corrections, and calibrated camera poses and point clouds define reference images in a shared metric coordinate system.ERP frames are globally matched to fixed-pose LiDAR reference images; the mean reprojection RMSE is 0.1917 pixels.Source: §4 · p. 6
  • Reconstruction evaluation aligns trajectories to reference poses, refines geometry alignment with ICP, and uses bidirectional nearest-neighbor distances for precision, recall, and F-score.Source: §4 · p. 6

5 Experiments

The experiments use 16-keyframe submaps with four shared keyframes and run on a single NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU.

5 Experiments

  • Submaps contain 16 panoramic keyframes, with four shared keyframes between consecutive submaps.Source: §5 · p. 6
  • Depth calibration uses a log-ratio Huber threshold of 0.05; connection verification uses 0.3 map-unit and 1° bidirectional thresholds with at least three keyframes on each side.Source: §5 · p. 6
  • GPU bundle adjustment uses double precision and at most 250 PCG iterations with a 10^-5 tolerance, on one NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU.Source: §5 · p. 6

Datasets.

The evaluation spans 360LOC, YUTO MMS, and PanoRecon, comparing localization, dense reconstruction, runtime, and component ablations. Results highlight trajectory accuracy, dense-map quality, and the roles of global constraints and refinement.

Datasets.

  • On the evaluated datasets, the baselines include feed-forward reconstruction, panoramic visual odometry and SLAM, and SfM; localization and dense reconstruction are assessed with trajectory and point-cloud metrics.Reported pose metrics are RMSE after one global Umeyama Sim(3) alignment; dense precision, recall, and F-score are measured at 20 cm.Source: Datasets · pp. 6–7

5.1 Trajectory Comparison

  • Our method achieves the lowest mean ATE on all three datasets, with its largest advantage on YUTO MMS.Stella-vSLAM trajectories are distorted by false loop closures on Residence2 and Sequence C; Scal3R and COLMAP show scale drift in the described settings.Source: §5.1 · p. 7, Table 1 · p. 7

5.2 Dense Reconstruction Comparison

  • The largest F-score gains occur on long-range YUTO MMS sequences, while feed-forward geometry recovers more continuous surfaces on weakly textured walls and reflective areas.PatchMatch-based COLMAP and Stella-vSLAM-Dense exhibit holes and noisy geometry in those regions.Source: §5.2 · p. 8, Table 2 · p. 8, Figure 6 · p. 8

5.3 Runtime.

  • The GPU backend reduces global-refinement cost while preserving comparable final reprojection error, and matrix-free Schur optimization makes growing observation graphs more efficient to solve.The analysis concerns repeated correction as long-range connections accumulate.Source: §5.3 · p. 8, Table 3 · p. 8

5.4 Ablation Studies

  • In the Residence2 submap experiment, a single camera constraint with BA provides most of the ATE reduction, additional constraints improve ATE further, and BA restores local motion consistency.BA alone changes little; graph correction improves global alignment.Source: §5.4 · pp. 8–9, Table 4 · p. 9
  • Global pose-graph correction reduces ATE while slightly increasing reprojection error; observation fusion then adds long-range support, and subsequent BA reduces residuals and further improves trajectory accuracy.The reported behavior is consistent with optimizing camera constraints before pixel-level refinement.Source: §5.4 · p. 9, Table 5 · p. 9
  • Frame-specific depth-scale calibration improves the precision–recall balance after registration, while whole-submap alignment retains higher recall.The reported advantage of per-frame calibration is in registered-geometry fidelity and F-score.Source: §5.4 · p. 9, Table 6 · p. 9
  • For selected small-overlap Residence2 pairs, all-frame matching increases support and balanced groups pass verification at unchanged thresholds; predicted depth expands 3D support and view pooling improves correct recall.The comparison concerns long-range connections between submaps.Source: §5.4 · p. 9, Table 7 · p. 9

6 Conclusion

The conclusion frames local feed-forward geometry and sparse mapping as mutually supportive for global consistency in long panoramic sequences. Across three datasets, the authors report improved mean ATE and high-fidelity dense reconstruction, while noting limits in challenging or dynamic settings and real-time operation.

6 Conclusion

  • Sparse mapping anchors local feed-forward geometry, which in turn strengthens constraints used to correct the sparse map.The paper describes local geometry as 3D evidence for verifying distant feature matches and establishing camera constraints.Source: §6 · p. 9
  • PanoRecon comprises seven scenes with repetitive architecture and long-range revisits, using reference poses and LiDAR geometry to evaluate trajectory accuracy and reconstruction consistency..Source: §6 · p. 9
  • Experiments across 360LOC, YUTO MMS, and PanoRecon show improved mean ATE and high-fidelity dense reconstruction..Source: §6 · p. 9
  • Severe motion blur or prolonged low-texture segments can reduce reliable feature observations needed for tracking and depth calibration; the system also lacks explicit dynamic-object handling and real-time reconstruction.Future work targets more robust correspondences and efficient feed-forward models for streaming reconstruction from 360° video in dynamic scenes.Source: §6 · p. 10

A.1 Datasets.

The appendix describes three dataset sources, including PanoRecon's repeated-architecture routes with distant, limited-overlap revisits. It also specifies how reference geometry is prepared and how methods' inputs and evaluation frames are sampled.

A.1 Datasets.

  • The 360LOC split provides four daytime mapping sequences with reference poses and LiDAR maps, while YUTO MMS supplies four long vehicle sequences with loops, turns, and dynamic traffic.YUTO MMS covers 20.1 km; Sequence C spans 9.137 km over 25 minutes and contains 10,778 images.Source: §A.1 · p. 13
  • Reference point clouds are formed by transforming and accumulating LiDAR data in a common world frame; 360LOC's dense reference points are voxel-averaged from valid mapped-frame points.YUTO MMS uses released scans, ground-truth poses, and sensor calibration.Source: §A.1 · p. 13
  • PanoRecon contains seven low-texture scenes with repeated architectural appearance and geometry, large and small loops, and distant revisits with limited overlap.At revisits, buildings may appear from farther away or oblique angles and occupy only a small ERP-image region.Source: §A.1 · p. 13, Figure 8 · p. 13

A.2 Baselines Settings.

  • SLAM methods process full sequences, with PanoRecon evaluation keyframes retained at one-second intervals and YUTO MMS evaluation poses sampled every three images; SfM inputs follow corresponding sampling intervals.Scal3R uses every third image on YUTO MMS Sequences C and D because full-sequence inputs cause GPU out-of-memory errors; PanoVGGT also encounters out-of-memory errors under the stated setup.Source: §A.2 · p. 14

B Additional Method Implementation Details

The appendix details how the system builds and calibrates submaps, verifies long-range connections, and uses them to refine camera poses and dense geometry.

B.1.1 Sparse mapping and prediction windows

  • The panoramic frontend tracks ERP images, adds keyframes and triangulated landmarks, and locally refines them before depth calibration.Overlapping inference windows provide context and cached depths, while their interior frames supply scale fitting and point-cloud output.Source: §B.1.1 · p. 14

B.1.2 Landmark observations and depth scale calibration

  • Landmark observations provide geometric depths for robustly fitting shared window scales and supported frame corrections to predicted depths.Overlapping windows receive an additional scale adjustment from shared source keyframes, views, and pixels.Source: §B.1.2 · p. 16

B.2.1 Candidate submap selection

  • Calibrated submap points, voxel overlap, and line-of-sight visibility select candidate historical submaps for feature matching.Expanded spatial bounds and nearby historical windows add candidates that may be displaced by drift.Source: §B.2.1 · p. 17

B.2.2 Feature matching and geometric verification

  • Matched features with valid calibrated depths are lifted into 3D, and RANSAC with least-squares refitting estimates a similarity transform between submaps.Bidirectional distance and bearing checks test correspondence consistency.Source: §B.2.2 · pp. 17–18

B.2.3 Validation across views and camera constraints

  • Connections require inliers from multiple views, distinct feature support, and nondegenerate geometry; large corrections receive additional cross-validation between frame groups.Selected camera constraints are limited per submap pair, whose total weight is normalized across its edges.Source: §B.2.3 · pp. 19–20

B.2.4 Descriptor matching and cache reuse

  • The CPU and CUDA implementations reuse cached descriptors and feature matches, then recompute 3D coordinates and geometric verification after global correction.Source: §B.2.4 · p. 20

B.3.1 Pose graph optimization and landmark fusion

  • Pose-graph optimization combines spanning-tree, covisibility, and verified camera-constraint edges; landmark fusion and covisibility updates prepare shared observations for global bundle adjustment.Inconsistent new edges are pruned before graph re-optimization.Source: §B.3.1 · pp. 21–22

B.3.2 Panoramic bundle adjustment; B.3.3 Dense reconstruction after global refinement

  • Panoramic bundle adjustment refines poses and landmarks, while matrix-free Schur optimization eliminates landmark increments to solve the camera system.After global refinement, affected windows are recalibrated and cached dense predictions are repositioned with updated poses and scales.Source: §B.3.2 · p. 22, §B.3.3 · p. 23

C.1 Runtime Analysis

On Residence2, geometry-based submap alignment improves the adapted PanoVGGT reconstruction metrics. The runtime analysis also reports the time spent in major reconstruction stages on YUTO MMS Sequence C.

C.1 Runtime Analysis

  • On YUTO MMS Sequence C, Pi3 inference takes 38.96 minutes, global BA solves take 0.87 minutes, and connection matching and verification take 56.60 minutes.COLMAP incremental reconstruction and BA take 618.52 minutes, while photometric and geometric MVS take 609.85 minutes; separate preparation, conversion, and export are excluded.Source: §C.1 · p. 23, Table 9 · p. 23

C.2 Applying Submap Alignment to PanoVGGT

  • ATE falls from 26.831 to 4.203 m and F-score rises from 4.40% to 18.67% with geometry-based submap alignment on Residence2.Removing loop factors alone yields the lowest rotational RPE but substantially worse trajectory and reconstruction accuracy.Source: §C.2 · p. 24

D Per-Scene Localization Results

The section directs readers to per-scene localization results for 360LOC, YUTO MMS, and PanoRecon.

D Per-Scene Localization Results

E Dense reconstruction evaluation

The section presents dense-reconstruction evaluations across 360LOC, YUTO MMS, and PanoRecon, with additional point-cloud comparisons for selected scenes.

E Dense reconstruction evaluation

Related papers

Loading 2610.36876v1…