PaperlayerView exact revision on arXiv

Source-linked AI summary

GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction

Youmin Zhang, Fabio Tosi, Stefano Mattoccia, Matteo Poggi

arXiv:2309.02436v1cs.CVcs.RO

TL;DR

Local camera tracking alone cannot fully prevent drift in complex scenes, particularly with monocular input. GO-SLAM combines global pose optimization and instant mapping, achieving state-of-the-art performance across datasets in robust tracking and dense mapping.

  • Problem

    Local camera tracking alone cannot fully prevent drift in complex scenes, particularly with monocular input.

    Source: §1 · p. 2

  • Method

    GO-SLAM combines drift-corrected pose tracking, loop closing, full bundle adjustment, and instant mapping that adapts reconstruction to optimized poses and depths.

    Source: §3 · p. 3

  • Results

    GO-SLAM achieves state-of-the-art performance on various datasets, robustly tracking and densely mapping large-scale scenes, especially long monocular trajectories without depth.

    Source: §5 · p. 9

  • Takeaways & Limitations

    The framework supports monocular, stereo, and RGB-D input.

    Source: §5 · p. 9

    Point-cloud, surfel-based, and volumetric representations lack flexibility at shape extraction, inhibiting high-fidelity reconstruction.

    Source: §1 · p. 2

Abstract

from arXiv · show

Neural implicit representations have recently demonstrated compelling results on dense Simultaneous Localization And Mapping (SLAM) but suffer from the accumulation of errors in camera tracking and distortion in the reconstruction. Purposely, we present GO-SLAM, a deep-learning-based dense visual SLAM framework globally optimizing poses and 3D reconstruction in real-time. Robust pose estimation is at its core, supported by efficient loop closing and online full bundle adjustment, which optimize per frame by utilizing the learned global geometry of the complete history of input frames. Simultaneously, we update the implicit and continuous surface representation on-the-fly to ensure global consistency of 3D reconstruction. Results on various synthetic and real-world datasets demonstrate that GO-SLAM outperforms state-of-the-art approaches at tracking robustness and reconstruction accuracy. Furthermore, GO-SLAM is versatile and can run with monocular, stereo, and RGB-D input.

1. Introduction

GO-SLAM addresses drift and inconsistent dense reconstruction through global pose optimization and on-the-fly implicit mapping. It combines robust tracking, real-time loop closing, and online full bundle adjustment, and supports monocular, stereo, and RGB-D input.

1. Introduction

  • Prior NeRF-based SLAM achieves precise reconstruction and pose estimation in small scenes, but without global loop closure and bundle adjustment, drift grows with processed frames and reconstruction can collapse.The introduction cites recent [35] [19] [53] and concurrent [20] [31] [9] [52] work in describing this limitation.Source: §1 · p. 2
  • GO-SLAM continuously aligns poses across the complete input-frame history with real-time global pose optimization.Its end-to-end global optimization uses learned richer features and geometry cues, such as pixel-wise flow, to remain robust in challenging regions.Source: §1 · p. 2
  • Its memory- and time-efficient alignment strategy enables instantaneous loop closures to correct the global structure.Loop closing is paired with online full bundle adjustment that optimizes the geometry of the complete keyframe history.Source: §1 · p. 2
  • GO-SLAM continuously updates a neural implicit 3D reconstruction using the latest globally optimized poses, enabling real-time mapping and local-detail capture.The compact multiscale representation uses multi-resolution hash encoding and supports high-frequency updates as poses and depths are optimized.Source: §1 · p. 2
  • The proposed deep-learning architecture jointly performs robust pose estimation and dense 3D reconstruction with monocular, stereo, or RGB-D cameras.Source: §1 · p. 2

2. Related Work

Prior work spans discrete mapping and neural implicit SLAM, but the cited systems expose a gap in online global correction: discrete representations can drift, while several NeRF-SLAM systems lack loop closing and full bundle adjustment.

Online 3D Reconstruction and SLAM

  • Discrete depth-point, height-map, surfel, and volumetric methods pursue global consistency with bundle adjustment and loop closing but remain prone to tracking drift and reconstruction distortion; DROID-SLAM [41] performs global bundle adjustment only offline.The passage argues that offline refinement alone can fail to eliminate drift in challenging cases, motivating online bundle adjustment for on-the-fly correction.Source: §2 · p. 2

NeRF-based Visual SLAM

  • Neural implicit fields support novel-view synthesis and reconstruction, while NeRF-based visual SLAM offers greater scene completeness, especially in unobserved regions, and continuous modeling at arbitrary resolution.iMAP [35] and NICE-SLAM [53] learn pose tracking and room-scale mapping from RGB-D; concurrent methods [20] [52] use RGB sequences without depth sensors.Source: §2 · p. 2
  • Orbeez-SLAM [9] uses ORB-SLAM2 [26] visual odometry for accurate pose estimation, but missing loop closing and global bundle adjustment limits large-scale reconstruction across these methods.NeRF-SLAM [31] integrates DROID-SLAM [41] for tracking but inherits the absence of online loop closing and full bundle adjustment, restricting globally consistent reconstruction.Source: §2 · pp. 2–3
  • Some methods relax the posed-image requirement by starting from imperfect camera poses, but only on small objects [22] [47].Source: §2 · p. 2

3. Method

GO-SLAM uses keyframe-based tracking with online drift correction, loop closing, and full bundle adjustment, while instant mapping updates the reconstruction from optimized global poses and depths.

3. Method

  • GO-SLAM combines keyframe-based tracking, online drift correction, full bundle adjustment, loop closing, and instant mapping that adapts to globally optimized poses and depths.Source: §3 · p. 3

3.1. Tracking with Global Optimization

  • A recurrent optical-flow update compares each incoming frame with the last keyframe, creating a new keyframe when average flow exceeds a threshold.Source: §3.1 · p. 3
  • The keyframe graph links temporally adjacent and high-co-visibility frames, detects historical loop closures, and supports asynchronous full bundle adjustment for online global refinement.Loop candidates are validated after three consecutive detections when their mean flow is below the co-visibility threshold.Source: §3.1 · pp. 3–4

3.2. Instant Mapping

  • Mapping snapshots poses and depths, then selects the latest two and unmapped keyframes, the top 10 by pose change, and 10 stratified samples.This selection prioritizes recent optimization updates while retaining older geometry.Source: §3.2 · pp. 4–5
  • Multi-resolution hash encodings feed SDF and color networks; rendering predicts pixel color and depth, optimized with RGB, depth, Eikonal, and SDF losses.The mapping thread optimizes sampled pixels from selected keyframes for a fixed number of iterations.Source: §3.2 · pp. 5–6

4. Experimental Results

GO-SLAM is evaluated across synthetic and real-world datasets with monocular, stereo, and RGB-D input. The experiments compare tracking and reconstruction against other SLAM systems and test the effects of its design choices and hardware requirements.

4.1. Implementation Details

  • The system uses pre-trained DROID-SLAM weights for tracking and trains rendering networks from scratch, with default settings specified for each input mode.The local window size is 25 for RGB-D and stereo, and 50 for monocular input.Source: §4.1 · p. 6

4.2. Datasets

  • The evaluation covers TUM RGB-D, EuRoC, ETH3D-SLAM, ScanNet, and Replica, using RGB-D, stereo, or monocular input as supported by each dataset.Images are resized to 384 × 512 for TUM RGB-D, EuRoC, and ETH3D-SLAM; 240 × 320 for ScanNet; and 320 × 640 for Replica.Source: §4.2 · p. 6

4.3. Evaluation Metrics

  • Tracking is evaluated with aligned Absolute Trajectory Error RMSE; reconstruction uses accuracy, completion, completion ratio, and F-score, while depth rendering uses Depth L1 error.Unobserved regions are removed for reconstruction evaluation, and depth rendering is assessed using the ground-truth trajectory.Source: §4.3 · pp. 6–7

4.4. Comparison with state-of-the-art SLAM

  • GO-SLAM produces stereo trajectories comparable to state-of-the-art stereo SLAM on EuRoC, while outperforming published RGB-D systems on ETH3D-SLAM.On EuRoC monocular sequences, it predicts accurately in all sequences, unlike NeRF-based SLAM without online bundle adjustment; its reconstructions are more complete and smoother than DROID-SLAM’s.Source: §4.4 · pp. 7–8, Table 2 · p. 7, Figure 5 · p. 7
  • On ScanNet, GO-SLAM consistently yields the best results, with online loop closing and full bundle adjustment helping maintain accuracy on long monocular sequences.On TUM RGB-D, it achieves the lowest average trajectory error among deep-learning methods and improves pose estimation over recent NeRF-based RGB-D systems.Source: §4.4 · pp. 7–8, Table 3 · p. 8
  • On Replica, GO-SLAM achieves pose and reconstruction accuracy comparable to existing RGB-D and concurrent monocular methods, while its depth rendering is better than methods with comparable results.GO-SLAM runs at 8 FPS with maximum GPU memory use of 18 GB; the compared concurrent methods do not run in real time.Source: §4.4 · p. 8, Table 4 · p. 8

4.5. Ablation Study

  • Ablations show that change-guided keyframe selection, geometric losses, frame skipping, loop closing, and full bundle adjustment support reconstruction quality, real-time operation, and improved global pose estimation.Latest-keyframe or stratified sampling alone is insufficient; RGB loss alone is unsatisfactory, while combining losses performs best. Skipping frames up to 8× causes only minimal degradation in mapping and tracking, and the full system achieves the best pose and reconstruction results while enabling real-time performance.Source: §4.5 · p. 9

4.6. CPU/GPU Requirements.

  • On Replica with RGB-D input, GO-SLAM achieves an effective balance of CPU/GPU requirements and SLAM performance.The reported hardware evaluation includes CPU requirements, maximum GPU memory consumption, average FPS, and final accuracy; experiments use an NVIDIA RTX 3090 GPU.Source: §4.6 · p. 9, Table 9 · p. 9

5. Conclusions

GO-SLAM is a real-time deep-learning SLAM system that produces globally consistent reconstructions from monocular, stereo, or RGB-D input. It combines loop closure, online full bundle adjustment, and continuously updated NeRF-based reconstruction, with reported state-of-the-art performance across datasets.

5. Conclusions

  • GO-SLAM achieves state-of-the-art performance across datasets, reliably tracking and densely mapping large-scale scenes, especially on long monocular trajectories without depth.The conclusion reports robustness and state-of-the-art performance across various datasets.Source: §5 · p. 9
  • The system detects loop closures and performs online full bundle adjustment to minimize trajectory error.Source: §5 · p. 9
  • Its NeRF-based reconstruction is efficient, compact, and multi-resolution, and is continuously updated at high frequency to adapt to optimized global geometry.Source: §5 · p. 9
  • The real-time deep-learning SLAM algorithm supports monocular, stereo, and RGB-D input while producing globally consistent reconstruction.Source: §5 · p. 9

A. SLAM Configuration · A.1. Rendering Network Details.

GO-SLAM’s rendering setup uses separate SDF and color networks with multi-resolution hash encodings. The appendix specifies their architecture and training optimizer and learning rates.

A.1. Rendering Network Details.

  • The rendering architecture has separate SDF and color networks, with 16 hash-encoding levels spanning resolutions 16 to 4096.Figure A presents the rendering-network details.Source: §A.1 · p. 12
  • The hash encodings use feature dimension 2 and maximum entries per level of 219.Source: §A.1 · p. 12
  • The SDF network uses 32 hidden units, while the color network uses 64.Source: §A.1 · p. 12
  • Training uses AdamW, with learning rates of 10^-2 for learnable hash encodings and 10^-3 for SDF and color network parameters.Source: §A.1 · p. 12

B. More Qualitative Results

The section presents detailed qualitative results on ScanNet, EuRoC, and Replica.

B. More Qualitative Results

RGB-D Ours · GT Mesh NICE-SLAM

The qualitative comparisons show GO-SLAM producing globally consistent, clean, and complete reconstructions in real time, while NICE-SLAM and DROID-SLAM baselines exhibit limitations in trajectory, distortion, or reconstruction quality.

GT Mesh NICE-SLAM

  • On ScanNet, GO-SLAM produces globally consistent reconstructions with monocular or RGB-D input in real time, while NICE-SLAM shows accumulated trajectory errors and 3D distortion.Source: GT Mesh NICE-SLAM · p. 13
  • On EuRoC, GO-SLAM reconstructions are cleaner and more complete than DROID-SLAM [6] reconstructions obtained with TSDF-Fusion [2].Source: Figure C · p. 13
  • On Replica, GO-SLAM produces dense reconstructions in real time with monocular or RGB-D input, whereas depth-only NICE-SLAM [7] takes hours (<1 FPS).Source: Figure D · p. 14

Related papers

Loading 2309.02436v1…