PaperlayerView exact revision on arXiv

Source-linked AI summary

RGBD GS-ICP SLAM

Seongbo Ha, Jiung Yeon, Hyeonwoo Yu

arXiv:2403.12550v2cs.CV

TL;DR

Dense SLAM seeks high-fidelity spatial representation for robotics, VR, and AR, while neural implicit methods incur rendering overhead. This paper couples G-ICP tracking and 3DGS mapping through a shared Gaussian map, reporting state-of-the-art spatial representation, pose estimation, and total system speed.

  • Problem

    Dense SLAM needs high-fidelity spatial representation, rendering performance, and accurate trajectories, while INR-based SLAM incurs rendering overhead.

    Source: §1 · p. 1

  • Method

    The system couples G-ICP tracking and 3DGS mapping in one Gaussian map, exchanging covariances with scale alignment and using dynamic keyframe selection.

    Source: §1 · p. 3, Keyframe Selection · p. 7, Scale Aligning · p. 8

  • Results

    The approach reports state-of-the-art spatial representation, camera pose estimation, and total system speed, reaching up to 107 FPS overall.

    Source: §1 · p. 3, §5 · p. 14

  • Takeaways & Limitations

    The shared Gaussian representation minimizes redundant computation, while dynamic keyframe selection enhances tracking and mapping performance.

    Source: §5 · p. 14

    The system relies on depth for 3D structure, so RGB-D sensor depth noise can limit reconstructed-map quality in real-world environments.

    Source: §5 · p. 15

Abstract

from arXiv · show

Simultaneous Localization and Mapping (SLAM) with dense representation plays a key role in robotics, Virtual Reality (VR), and Augmented Reality (AR) applications. Recent advancements in dense representation SLAM have highlighted the potential of leveraging neural scene representation and 3D Gaussian representation for high-fidelity spatial representation. In this paper, we propose a novel dense representation SLAM approach with a fusion of Generalized Iterative Closest Point (G-ICP) and 3D Gaussian Splatting (3DGS). In contrast to existing methods, we utilize a single Gaussian map for both tracking and mapping, resulting in mutual benefits. Through the exchange of covariances between tracking and mapping processes with scale alignment techniques, we minimize redundant computations and achieve an efficient system. Additionally, we enhance tracking accuracy and mapping quality through our keyframe selection methods. Experimental results demonstrate the effectiveness of our approach, showing an incredibly fast speed up to 107 FPS (for the entire system) and superior quality of the reconstructed map.

1 Introduction

Dense SLAM needs high rendering performance and accurate pose estimates, while existing approaches trade tracking speed against computation or use 2D image-rendering errors. The paper combines G-ICP tracking and 3DGS mapping through shared Gaussian representations and scale alignment.

1 Introduction

  • The system reaches up to 107 FPS overall with superior map quality; Fig. 1 compares PSNR against whole-system FPS.The figure reports averages across eight Replica scenes.Source: Figure 1 · p. 2, §1 · p. 3
  • Existing 3DGS methods use indirect, image-based tracking, while decoupled tracking and mapping require extra computation and separate 3D features.Photometric-error methods estimate poses by minimizing differences between rendered and observed images.Source: §1 · p. 2
  • The proposed coupled framework uses G-ICP to track directly against the 3DGS map, sharing one Gaussian map between tracking and mapping.Tracking covariances initialize 3DGS mapping, and map Gaussians provide points and covariances for G-ICP.Source: §1 · pp. 2–3
  • The paper presents its G-ICP and 3DGS fusion as a real-time dense SLAM system that actively uses 3D information for tracking.The claimed speed and map-quality outcomes are reported for the overall system.Source: §1 · p. 3
  • Sharing G-ICP and 3DGS covariances with scale alignment reduces redundant computations and supports rapid convergence of map primitives.The approach reuses covariances computed during tracking and avoids recalculating map covariances.Source: §1 · p. 3

2 Related Work

Related work spans classical dense mapping, neural implicit representations, and 3D Gaussian Splatting. The paper contrasts their mapping and tracking strategies, including the extra resources required by decoupled approaches.

2 Related Work

  • Classical dense SLAM represents maps with SDF or TSDF rather than sparse point clouds or grids.These representations are described as supporting dense maps for interaction in applications such as AR and robotics.Source: §2 · p. 4
  • Neural implicit methods track by reducing rendered-versus-observed image error, while Orbeez-SLAM and vMAP separate tracking from mapping.Orbeez-SLAM uses ORB-SLAM2 with instant-ngp; vMAP uses ORB-SLAM3 and separate MLPs for objects.Source: §2 · p. 4
  • 3DGS offers high-fidelity representation and faster rendering than NeRF, but dense photometric-error tracking still suffers from speed limitations.GS-SLAM, SplaTAM, and Gaussian Splatting SLAM use dense photometric error; Photo-SLAM instead integrates ORB-SLAM3 for tracking.Source: §2 · p. 4

3 Method

The method couples G-ICP tracking with 3D Gaussian Splatting mapping through a shared Gaussian map and covariance information. It combines scale handling and separate tracking and mapping keyframe strategies to support tracking and mapping quality.

3 Method

  • G-ICP aligns current-frame source Gaussians with map Gaussians to estimate pose, while GS optimizes the map’s Gaussian coordinates, color, and opacity using rendered RGB and depth losses.Both processes use Gaussians represented by coordinates and covariances.Source: §3 · pp. 4–5
  • Tracking covariances are reused when adding keyframes, avoiding map covariance recomputation and GS densification or opacity resets.The shared Gaussian source reduces redundant computation across tracking and mapping.Source: §3 · pp. 5–6

3.1 G-ICP Tracking

  • Ellipse regularization preserves target-map Gaussian shapes, which can encode lines, corners, and other features, rather than forcing them into a plane-like scale.Conventional point-to-plane ICP regularizes scale to [1, 1, ϵ].Source: §3.1 · p. 6
  • Correspondence proportions from G-ICP trigger tracking keyframes when they fall below thresholds, supporting consistent tracking and map Gaussian density.The correspondence distances are available from G-ICP without additional calculations.Source: Keyframe Selection · p. 7
  • Only keyframe Gaussians that do not overlap the current map are added, preventing tracking discrepancies from being carried into the map and accumulating.The paper links this filtering strategy to more accurate tracking and greater system robustness.Source: Keyframe Selection · p. 7

3.2 GS Mapping

  • Scale normalization addresses oversized covariances from sparse, distant RGB-D points; the paper reports improved GS mapping and G-ICP tracking after normalized keyframes are added.The normalized scale matrix is Λ′′ = Λ′/z^p, with p determined empirically.Source: Scale Aligning · pp. 7–8
  • Mapping-only keyframes provide additional viewpoints for GS training without using those frames for tracking, balancing mapping performance against accumulated scan-matching errors.The paper states that G-ICP already computes covariance for all frames, so this selection adds no overall computational load.Source: Additional Keyframe Selection for… · p. 8
  • Randomly training on selected keyframes and pruning Gaussians that fall into local minima address viewpoint-driven mapping degradation while preserving geometric structure for tracking.The paper describes the risk of Gaussian scales elongating when training repeatedly from a specific viewpoint.Source: Additional Keyframe Selection for… · p. 9

4 Experiments

Experiments evaluate tracking, system speed, map quality, and component ablations on synthetic Replica and real-world TUM-RGBD data. Results favor the coupled G-ICP/3DGS approach overall, with depth noise qualifying its TUM map quality.

4.1 Experimental Setup

  • Replica and TUM-RGBD evaluations use synthetic and real-world data, respectively, and report ATE RMSE for tracking and PSNR, SSIM, and LPIPS for map quality.The TUM-RGBD images have substantial noise and blur, with information loss in many depth-image regions.Source: §4.1 · p. 9

4.2 Camera Tracking Accuracy

  • On Replica, the method achieves state-of-the-art tracking across scenes, reducing trajectory error by more than 50% versus the previous best; on TUM-RGBD, it beats coupled methods but trails decoupled methods.The authors note that decoupled approaches use separate map information and costly tracking-feature extraction.Source: §4.2 · p. 9, Table 1 · p. 10

4.3 System Speed and Quality of Reconstructed Map

  • 107 FPS overall; under the 30-FPS cap, the method achieves state-of-the-art Replica map quality, while uncapped runs retain superior quality and exceed Orbeez-SLAM by more than four times.Table 3 compares system speed and map quality on Replica; tracking speed is limited to 30 FPS in one evaluation case and unrestricted in the other.Source: §4.3 · p. 10, Table 3 · p. 11
  • On TUM-RGBD, compared with SplaTAM [14], PSNR falls by approximately 11.7%, while speed improves about 91.6 times under limited conditions and up to 227 times at maximum speed.The authors attribute the map-quality trade-off to noisy, information-poor depth images; Fig. 5 reports more ground-truth-like renderings than SplaTAM [14] and Point-SLAM [32].Source: §4.3 · p. 12, Figure 5 · p. 12

4.4 Ablation study

  • Ellipse regularization achieves the best tracking accuracy because it accounts for map Gaussians’ characteristics, whereas plane regularization’s planar assumption leaves accuracy improvements inadequate.The ablation reports average ATE RMSE across three TUM-RGBD scenes.Source: Scale Regularization · pp. 12–13, Table 5 · p. 12
  • Using G-ICP-computed covariances improves tracking and mapping, and dividing scale by z1.5 gives the best performance among the tested scale-alignment settings.Scale alignment reduces scale differences between existing and newly added Gaussians, smoothing the transition from G-ICP to 3DGS.Source: Scale Regularization · p. 13, Table 6 · p. 13
  • Mapping-only keyframes yield the best tracking accuracy and map quality by selectively adding keyframes that limit scan-matching error while supplying training images and primitives.The ablation averages results across eight Replica scenes.Source: Scale Regularization · pp. 13–14, Table 7 · p. 13
  • Random-keyframe training improves map quality through viewpoint diversity, while pruning improves tracking and mapping by removing overfit or unnecessary Gaussians; densifying is unnecessary.The mapping ablation compares random, covisible, and most-recent-keyframe training, as well as Gaussian densifying and pruning.Source: Avoiding Local Minima while Mapping · p. 14, Table 8 · p. 14

5 Conclusion

RGBD GS-ICP-SLAM combines G-ICP tracking and 3D Gaussian mapping in a shared map, with exchanged covariances, scale alignment, and dynamic keyframe selection. The authors report strong spatial representation and pose estimation alongside fast system speed, while noting depth-noise limits on real-world map quality.

5 Conclusion

  • The shared Gaussian map supports both tracking and mapping; exchanging Gaussians with scale alignment reduces redundant computation, and dynamic keyframe selection improves tracking and mapping performance.The authors report state-of-the-art performance in spatial representation, camera pose estimation, and total system speed.Source: §5 · p. 14
  • Depth noise in RGB-D sensors limits reconstructed-map quality in real-world environments, and the authors suggest trading some speed for robust RGB information.Source: §5 · p. 15
Loading 2403.12550v2…