PaperlayerView exact revision on arXiv

Source-linked AI summary

WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments

Jianhao Zheng, Zihan Zhu, Valentin Bieri, Marc Pollefeys, Songyou Peng, Iro Armeni

arXiv:2504.03886v1cs.CVcs.RO

TL;DR

Dynamic objects challenge SLAM systems built on static-scene assumptions and can cause tracking and reconstruction errors. WildGS-SLAM uses DINOv2 features and an uncertainty MLP within a monocular 3D Gaussian SLAM pipeline, weighting tracking and mapping against unreliable regions. Evaluations report state-of-the-art dynamic SLAM performance in tracking and novel view synthesis, while failure cases remain when views insufficiently reveal static backgrounds.

  • Problem

    Traditional SLAM systems assume static scenes, while dynamic objects, occlusions, and lighting variation create tracking ambiguity and generalization challenges.

  • Method

    WildGS-SLAM predicts per-pixel uncertainty from pretrained DINOv2 features with a shallow MLP and uses it in dense bundle adjustment and Gaussian map optimization.

  • Results

    WildGS-SLAM achieves state-of-the-art performance in dynamic SLAM tasks, excelling in tracking and novel view synthesis across evaluated datasets.

  • Takeaways & Limitations

    The purely geometric monocular framework removes dynamic distractors without explicit semantic segmentation or prior class information while supporting high-fidelity scene reconstruction.

  • Takeaways & Limitations

    The on-the-fly uncertainty predictor can miss distractors when few views observe the same regions or dynamic objects remain stationary from similar directions.

Abstract

from arXiv · show

We present WildGS-SLAM, a robust and efficient monocular RGB SLAM system designed to handle dynamic environments by leveraging uncertainty-aware geometric mapping. Unlike traditional SLAM systems, which assume static scenes, our approach integrates depth and uncertainty information to enhance tracking, mapping, and rendering performance in the presence of moving objects. We introduce an uncertainty map, predicted by a shallow multi-layer perceptron and DINOv2 features, to guide dynamic object removal during both tracking and mapping. This uncertainty map enhances dense bundle adjustment and Gaussian map optimization, improving reconstruction accuracy. Our system is evaluated on multiple datasets and demonstrates artifact-free view synthesis. Results showcase WildGS-SLAM's superior performance in dynamic environments compared to state-of-the-art methods.

1. Introduction

Dynamic environments challenge SLAM because moving objects, occlusions, and lighting variation undermine static-scene assumptions and generalization. WildGS-SLAM addresses these issues with uncertainty-aware geometric tracking and mapping for monocular RGB input, reporting robust reconstruction and rendering across indoor and outdoor scenarios.

  • Traditional SLAM assumes rigid scenes, making dynamic objects a source of tracking errors.
  • Recent motion, semantic, and depth-based methods often struggle to generalize across varied and unpredictable motion patterns.Dynamic distractors, occlusions, and changing lighting introduce ambiguity in real-world scenes.
  • WildGS-SLAM uses a 3D Gaussian representation and uncertainty-aware tracking and mapping with monocular RGB input.The approach is purely geometric and removes dynamic distractors without explicit depth or semantic labels.
  • WildGS-SLAM reports improved performance over prior work across indoor and outdoor scenes, with artifact-free rendering and high-fidelity novel view synthesis.
  • A shallow MLP decodes pretrained DINOv2 features into per-pixel uncertainty for tracking and Gaussian map optimization.Uncertainty weights dense bundle adjustment and informs the rendering loss during mapping.
  • The framework includes a Wild-SLAM Dataset spanning diverse indoor and outdoor scenes, object motions, and occlusions for unconstrained evaluation.

2. Related Work

Prior visual SLAM systems use depth, semantic, motion, or sparse geometric cues to handle dynamic content, while neural and Gaussian representations improve dense reconstruction and view synthesis. WildGS-SLAM extends uncertainty-based distractor removal to sequential monocular SLAM with a dense 3D Gaussian map.

  • 2.1. Traditional Visual SLAM: Traditional dynamic SLAM methods commonly filter dynamic regions using RGB-D or stereo input, semantic classes, motion masks, or depth residuals.
  • 2.1. Traditional Visual SLAM: Existing traditional SLAM methods do not support monocular input without prior class information, according to the authors.The authors attribute this limitation to sparse monocular representations limiting purely geometric identification of dynamic regions.
  • 2.2. Neural Implicit and 3DGS SLAM: Neural implicit representations and 3D Gaussian Splatting support dense reconstruction and novel view synthesis in SLAM research.
  • 2.2. Neural Implicit and 3DGS SLAM: WildGS-SLAM simultaneously estimates camera poses and builds a static-scene 3D Gaussian map from RGB image sequences.Its uncertainty module guides tracking and mapping to mitigate dynamic distractors.
  • 2.2. Neural Implicit and 3DGS SLAM: WildGS-SLAM uses DINOv2 features and an MLP to produce an uncertainty map for purely geometric monocular dynamic-scene SLAM.It extends uncertainty-based distractor removal from sparse-view settings with known camera poses to sequential SLAM.
  • 2.2. Neural Implicit and 3DGS SLAM: MonST3R detects moving objects from optical-flow differences but is limited to short sequences and does not support view synthesis.

3. Method

WildGS-SLAM reconstructs dynamic scenes by combining uncertainty-aware monocular tracking with incremental 3D Gaussian mapping. DINOv2 features and an online MLP predict uncertainty that downweights moving-object regions in bundle adjustment and rendering optimization.

  • System overview: WildGS-SLAM tracks camera poses while reconstructing the static scene as a 3D Gaussian map from streamed RGB frames.The system progressively expands and optimizes the map as new keyframes arrive.
  • 3.1. Preliminary on 3D Gaussian Splatting: The scene is represented by anisotropic Gaussians containing color, opacity, means, and covariance matrices, which are projected and splatted onto the image plane.Rendered color and depth are produced by depth-ordered alpha blending of projected Gaussians.
  • 3.2. Uncertainty Prediction: A shallow MLP decodes DINOv2 features into per-pixel uncertainty and trains incrementally as frames stream into the system.The uncertainty objective combines modified SSIM, depth uncertainty, and regularization terms; the depth loss compares rendered and metric depth.
  • 3.3. Tracking: Uncertainty-aware dense bundle adjustment weights reprojection errors by flow confidence and predicted uncertainty, minimizing the influence of moving-object pixels.Metric-depth disparity regularization stabilizes pose estimation before the online uncertainty predictor converges, while consistency masks deactivate unreliable regions.
  • 3.4. Mapping: During mapping, uncertainty weights color and depth rendering losses so distractors contribute less, while isotropic regularization limits excessive Gaussian elongation.The map is optimized over a local covisibility-based keyframe window using rendered color and depth images.

4. Experiments

WildGS-SLAM is evaluated across dynamic indoor and outdoor datasets using tracking, input-view rendering, novel-view synthesis, and ablation studies. It generally outperforms competing methods, producing artifact-free renderings while remaining effective with monocular input.

  • Tracking Results: On the Wild-SLAM MoCap dataset, WildGS-SLAM significantly outperforms baselines on average, except for a slight tracking-error increase in the Person sequence.Refusion and DynaSLAM (N+G) use raw depth for dynamic-object identification, while MonST3R-SW still shows significant tracking errors on longer sequences.
  • Rendering Results: WildGS-SLAM produces artifact-free, realistic input-view renderings and achieves the best novel-view synthesis performance on Wild-SLAM MoCap.The reported novel-view advantage is attributed to uncertainty-aware mapping, with PSNR metrics shown in the qualitative comparisons.
  • Rendering Results: On the Wild-SLAM iPhone dataset, WildGS-SLAM achieves the best rendering results and assigns higher uncertainty to distractor shadows, whereas competing methods can miss complete dynamic objects.The displayed uncertainty map is blurry because DINOv2 features and mapping operate at reduced resolution; sharper maps trade efficiency for resolution.
  • Rendering Results: On Bonn RGB-D, WildGS-SLAM renders dynamic scenes realistically under motion blur, while ReFusion and DynaSLAM (N+G) exhibit object-removal or dynamic-region-detection failures.ReFusion retains a person in the TSDF, and DynaSLAM (N+G) produces partial black masks or defaults to the original image as the inpainted result.
  • Tracking and Ablation Results: WildGS-SLAM achieves the best tracking performance on all reported TUM RGB-D sequences, and the ablation study reports that the complete design outperforms all tested variants.The ablation compares uncertainty-aware design choices against alternatives, including predefined distractor types with YOLOv8 and SAM segmentation.

5. Conclusion

WildGS-SLAM is presented as a purely geometric dynamic-environment SLAM method that achieves state-of-the-art tracking and novel view synthesis. Its uncertainty predictor remains limited when few views capture the same regions.

  • WildGS-SLAM uses a purely geometric framework with a shallow MLP predicting per-pixel uncertainty from pre-trained 3D-aware features.The predictor separates static and dynamic scene elements for tracking and rendering.
  • The method achieves state-of-the-art performance in dynamic SLAM, including tracking and novel view synthesis, across newly collected and existing datasets.
  • Limitation: The uncertainty predictor is trained on-the-fly with input frames, limiting distractor recognition when only a few views capture the same regions.The authors suggest motion priors as a possible way to improve dynamic-scene handling and tracking robustness.
  • Acknowledgements: The authors acknowledge contributors who assisted with Wild-SLAM data collection and the release and curation of exhibition data.

Supplementary Material

Figure 7 documents the camera, calibration setup, and two static scenes used in the supplementary material.

  • Figure 7a shows the Intel RealSense D455 camera.
  • Figure 7b shows the calibration board used to align the camera reference frame with OptiTrack’s rigid body frame.
  • Figures 7c and 7d show Static scene 1 and Static scene 2.

Abstract

The supplementary material provides dataset information, implementation details, baseline descriptions, and additional experimental analyses.

  • The supplement includes more information about the Wild-SLAM dataset in Section 6.
  • Section 7 provides implementation details for WildGS-SLAM and the baseline methods.
  • Section 8 contains additional results and ablations.

6. Wild-SLAM Dataset

The Wild-SLAM MoCap Dataset contains RGB-D sequences designed to benchmark dynamic SLAM with moving distractors, while supporting monocular and RGB-D evaluation. Its acquisition includes camera synchronization and pose calibration with OptiTrack, and the dataset documentation covers sequence properties and recording conditions.

  • Dataset: The Wild-SLAM MoCap Dataset contains 10 RGB-D sequences with moving objects as distractors for dynamic SLAM benchmarking.Aligned depth images support evaluation of RGB-D baselines and future research, although WildGS-SLAM uses monocular inputs.
  • Dataset: Sequences were captured with an Intel RealSense D455 at 720 × 1280 resolution and 30 fps.
  • Synchronization: An iPhone flashlight state change provided corresponding timestamps for synchronizing the OptiTrack system with the RealSense camera.The D455 frame rate was increased to 60 fps to improve synchronization accuracy.
  • Calibration: Four reflective markers attached to the D455 defined a rigid body, and a calibration board established the transformation between MoCap and camera coordinates.
  • Dataset overview: The dataset overview organizes sequences by name, distractors, static environment, number of frames, and trajectory length.
  • Discussion: Both datasets capture humans performing activities, with the Wild-SLAM MoCap Dataset recorded in controlled environments and participant consent obtained for data sharing.

7. Implementation Details

WildGS-SLAM initializes tracking with early keyframes, then uses multiview depth consistency and uncertainty-aware optimization to refine poses and maps. Its experiments compare the method with diverse baseline systems and implementation variants.

  • Initialization: The first 12 keyframes initialize tracking with dense bundle adjustment while uncertainty weighting is disabled before the uncertainty MLP is trained.The resulting coarse poses initialize the map and train the uncertainty predictor.
  • Depth regularization: A disparity regularization mask suppresses unreliable depth regions identified through multiview consistency and DINO feature similarity.The mask uses relative depth agreement and a cosine-similarity threshold to reject incorrect correspondences.
  • Global refinement: After processing all frames, global bundle adjustment refines keyframe poses using the frame-graph construction adopted from DROID-SLAM.The final tracking objective retains the bundle-adjustment term while omitting disparity regularization once sufficient multiview information is available.
  • Baseline evaluation: The evaluation compares WildGS-SLAM with baselines differing in dynamic-scene handling, open-source availability, and use of object or semantic priors.Baseline results come from cited papers or open-source implementations, depending on the method.
  • Map refinement: Final map refinement uses all keyframes and an L1 RGB re-rendering loss weighted by the uncertainty map.This refinement follows the optimization strategy used in MonoGS.

8. Additional Experiments

Additional experiments evaluate rendering, tracking, runtime, uncertainty prediction, pretrained features, and failure cases across dynamic and static datasets. WildGS-SLAM generally performs strongly, while failures arise from complex backgrounds and insufficient views of unobstructed static content.

  • Rendering results: WildGS-SLAM produces substantially better input-view rendering on the TUM RGB-D walking sequences.The comparison covers freiburg3_walking_static and freiburg3_walking_xyz.
  • Runtime: The fast version retains a clear margin over baselines with comparable runtime after disabling or reducing several low-impact processes.Changes include less frequent optimization, skipped non-keyframe refinement, and fewer final map-refinement iterations.
  • Rendering results: The method effectively removes distractors on TUM RGB-D, whereas competing systems exhibit ghosting, black holes, whitish, blurry, or floating artifacts.The reported artifacts reflect incomplete distractor removal, insufficient multiview inpainting, or no explicit dynamic-scene handling.
  • Tracking results: WildGS-SLAM performs best on average for full dynamic TUM RGB-D tracking and remains on par with monocular Gaussian-Splatting methods on static sequences.The static evaluation uses ATE RMSE, while the full dynamic evaluation reports tracking performance across dynamic sequences.
  • Uncertainty prediction: The online uncertainty module rapidly identifies a moving distractor and stabilizes its estimate shortly afterward.The reported example identifies the distractor at frame 215, with stabilization at frame 451.
  • Ablations: The disparity-regularization ablation favors applying the mask during capture and excluding the term during final global bundle adjustment.The reported improvement is attributed to stronger multiview depth refinement during final global adjustment.
  • Failure cases: The method can misclassify complex static textures as uncertain and reconstruct stationary dynamic objects when no unobstructed static observation is available.These failure cases are reported for the shopping and wandering sequences, respectively.
Loading 2504.03886v1…