Source-linked AI summary
FLoTS-SLAM: Optical Flow-Derived Uncertainty for Monocular Dynamic SLAM with Two-Stage Optimization
Haorui Du, Lipu Zhou
TL;DR
Dynamic objects create erroneous correspondences that degrade monocular SLAM's camera poses and 3D reconstruction, while uncertainty-based methods lack direct motion evidence and may leave residual influence in bundle adjustment. FLoTS-SLAM derives motion uncertainty from optical-flow residuals and uses two-stage pose and depth refinement with dynamic-pixel re-anchoring. It reports higher dynamic-mask IoU on DAVIS and lower average ATE on Bonn and TUM than the stated baselines.
Problem
Dynamic objects disrupt cross-view geometric consistency, degrading camera pose estimation and 3D reconstruction; recent uncertainty methods lack direct motion evidence and may be inaccurate.
Source: §I · pp. 1–2Method
FLoTS-SLAM derives uncertainty from optical-flow residuals against camera-induced flow and re-anchors high-uncertainty pixels during two-stage pose and depth optimization.
Source: §I · p. 2, D. Two-Stage Optimization with… · p. 5Results
35% higher mean DM-IoU on DAVIS than a state-of-the-art baseline, and 59% and 19% lower average ATE on Bonn and TUM, respectively, than the DROID-SLAM backbone.
Source: §V · p. 8Takeaways & Limitations
Abstract
from arXiv · showhide
Monocular simultaneous localization and mapping (SLAM) remains challenging in dynamic environments. Real-time dynamic object identification in such environments is a critical problem. Recent approaches estimate per-pixel uncertainty from rendering residuals or multiview feature inconsistency to identify dynamic regions. These methods lack direct motion evidence, and their accuracy may degrade when reconstruction is inaccurate or features are ambiguous. In addition, such uncertainty merely downweights pixels in bundle adjustment (BA), so inaccurate uncertainty estimates can still impair camera pose estimation and 3D reconstruction. Accordingly, we present a monocular RGB SLAM system for dynamic scenes. Specifically, a motion uncertainty estimation method is proposed to derive uncertainty from the residual between optical flow and camera motion induced flow, directly reflecting object motion. The estimation varies with frame alignment accuracy, reducing false detections and yielding accurate pixel-level masks of dynamic regions. To make BA more robust, we estimate camera poses from high confidence pixels. These poses are then used to reproject dynamic pixels during joint refinement of poses and depth, reducing the influence of object motion on both pose and depth updates. Moreover, we provide a quantitative evaluation of dynamic masks and analyze how these masks affect the downstream tracking and reconstruction of dynamic objects. Our approach improves the mean dynamic mask IoU by 35\% over a state-of-the-art baseline on DAVIS, and reduces the average ATE by 59\% on Bonn and 19\% on TUM relative to the SLAM backbone used in our system. The source code will be made publicly available.
I. INTRODUCTION
Dynamic objects disrupt cross-view geometric consistency, harming camera poses and 3D reconstruction. FLoTS-SLAM estimates motion uncertainty from optical-flow residuals and uses two-stage optimization; it reports improved mask IoU and lower ATE on the evaluated datasets.
Motivation
- Dynamic objects disrupt geometric consistency across views, introducing erroneous correspondences that degrade camera pose estimation and 3D reconstruction.The paper frames reliable monocular SLAM in dynamic environments as a continuing challenge.Source: §I · p. 1
- Existing uncertainty-weighted bundle adjustment can leave dynamic-pixel residuals affecting both depth estimates and keyframe poses.All pixels remain in bundle adjustment, so inaccurate uncertainty at any pixel can affect estimation.Source: §I · p. 2
Contributions
- Its two-stage optimization estimates poses from high-confidence pixels, then jointly refines poses and depth using reprojected dynamic pixels.This is designed to reduce object motion's influence on pose and depth updates and improve tracking robustness.Source: §I · p. 2
- FLoTS-SLAM derives motion uncertainty from optical-flow residuals against camera-induced flow and adapts it to frame-alignment accuracy.The optical flow comes from the system tracker's existing correspondence predictions without additional computation.Source: §I · p. 2
Results
- On DAVIS, FLoTS-SLAM improves mean DM-IoU by 35% over a state-of-the-art baseline, and reduces average ATE versus its DROID-SLAM backbone by 59% on Bonn and 19% on TUM.Source: §I · p. 2
II. RELATED WORK
The related work covers moving-object segmentation and dynamic SLAM, including category-based and uncertainty-based approaches. The paper positions direct optical-flow motion evidence against these approaches' real-time costs, category limits, and indirect appearance cues.
A. Moving Object Segmentation
- Dedicated moving-object segmentation methods use multi-frame windows, but their inference costs are prohibitive for real-time systems.Methods span coherent-motion grouping, complementary motion and appearance cues, and foundation-model or pretrained-geometry priors.Source: A. Moving Object Segmentation · p. 2
B. Traditional Dynamic SLAM
- Traditional dynamic SLAM identifies regions through geometric consistency, semantic priors, or both, and often excludes them from optimization.DynaSLAM, for example, segments movable instances and verifies them with multiview reprojection consistency.Source: B. Traditional Dynamic SLAM · p. 2
C. Dynamic NeRF- and 3DGS-based SLAM
- Dynamic NeRF- and 3DGS-based SLAM methods use segmentation-network masks, which follow object categories and cover only classes seen by the network.FLoTS-SLAM instead derives uncertainty from optical-flow residuals against camera-induced flow, using direct motion evidence.Source: C. Dynamic NeRF- and 3DGS-based SLAM · p. 2
D. Optimization in Dynamic SLAM
Dynamic SLAM optimization can be compromised by motion-induced residuals that uncertainty weighting does not fully suppress. FLoTS-SLAM uses dense tracking, motion uncertainty, and two-stage optimization to reduce object-motion influence on pose and depth updates.
D. Optimization in Dynamic SLAM
- Inaccurate uncertainty weights may leave motion-induced residuals insufficiently suppressed during joint pose and depth refinement, compromising estimation accuracy.The proposed two-stage optimization is intended to improve robustness and accuracy of pose and depth estimation.Source: D. Optimization in Dynamic SLAM · p. 2
A. System Overview
- FLoTS-SLAM derives motion uncertainty from optical-flow residuals against camera-induced flow, then uses low-uncertainty pixels with fixed depths for pose estimation.It jointly refines poses and depths afterward, replacing high-uncertainty pixels' correspondence targets with reprojections from the first-stage poses; the branches alternate within each Gauss–Newton iteration.Source: A. System Overview · p. 3, Figure 2 · p. 3
B. Dense Tracking with Per-Pixel Uncertainty
- Dense tracking maintains a frame graph of co-visible keyframes, with camera poses and inverse-depth maps as state variables.A fixed pretrained recurrent update operator predicts dense correspondence targets and confidence for each connected keyframe pair.Source: B. Dense Tracking with Per-Pixel… · p. 3
- The dense correspondence objective assumes residuals arise from pose and inverse-depth errors, an assumption moving-object displacements violate.The solver may then incorrectly adjust camera poses and inverse depths to reduce motion-induced residuals.Source: B. Dense Tracking with Per-Pixel… · p. 3
- High-uncertainty pixels contribute little under uncertainty-aware downweighting but remain in the frame graph.The confidence matrix is attenuated by a monotonically decreasing function of per-pixel uncertainty.Source: B. Dense Tracking with Per-Pixel… · p. 4
C. Motion Uncertainty Estimation
FLoTS-SLAM estimates motion uncertainty from the residual between observed optical flow and camera-induced flow. It adapts the residual penalty to alignment accuracy and combines flow and feature-consistency cues in a learned uncertainty prediction.
C. Motion Uncertainty Estimation
- The residual between optical flow and camera-induced flow isolates pixel displacement unexplained by camera motion and scene depth; it decreases on static background as optimization converges but persists on moving objects.The uncertainty loss is motivated by a Gaussian negative log-likelihood, with a regularizer that prevents uncertainty from diverging and drives it toward the pixel residual level.Source: C. Motion Uncertainty Estimation · p. 4
- The residual transform compresses large residuals when alignment is poor, then approaches the squared residual as alignment improves, reducing false dynamic classifications from misalignment.Its transition uses the median residual in the optimization window and an alignment threshold; the median can indicate background alignment when static pixels form a substantial share of samples.Source: C. Motion Uncertainty Estimation · p. 4
- A DINOv2-feature head predicts per-pixel uncertainty, combining flow-residual and feature-consistency costs to produce spatially consistent estimates.The head is an affine map followed by offset softplus, and its parameters are shared across the sequence; geometry and uncertainty are alternately updated at each iteration.Source: C. Motion Uncertainty Estimation · p. 4
D. Two-Stage Optimization with Dynamic-Pixel Re-Anchor
The method separates pose estimation from joint pose-and-depth refinement because residual dynamic motion can bias both shared poses and per-pixel depths. It estimates a reference pose from low-uncertainty pixels, then re-anchors high-uncertainty pixels during refinement.
D. Two-Stage Optimization with Dynamic-Pixel Re-Anchor
- Dynamic-pixel residuals can bias inverse depth and shared keyframe poses, so the method first estimates poses from the lowest-uncertainty pixels while holding inverse depths fixed.Selected pixels retain their original confidence weights without uncertainty attenuation; the reference pose is not the final pose and need only exclude dynamic pixels from the lowest-uncertainty quantile.Source: D. Two-Stage Optimization with… · p. 4
- In Stage II, poses and inverse depths are jointly refined over all pixels using uncertainty weights, with high-uncertainty pixels’ correspondence targets replaced by predictions from the reference pose.The replacement prediction is computed once using the reference pose and the depths held fixed in Stage I, and remains fixed during Stage II.Source: D. Two-Stage Optimization with… · p. 5
- Re-anchoring removes the object-motion gradient from pose and inverse-depth updates at the reference state while retaining a local quadratic constraint; it is applied only when motion contrast and tracking consistency support it.The method compares target discrepancies for high- and low-uncertainty pixels and waits for directionally consistent tracking corrections before applying re-anchoring in the final global adjustment.Source: D. Two-Stage Optimization with… · p. 5
IV. EXPERIMENTS
Experiments evaluate dynamic-mask accuracy, camera localization, and processing speed across Bonn, TUM RGB-D, and DAVIS. The paper compares against methods from several dynamic-SLAM and general-purpose SLAM categories.
IV. EXPERIMENTS
- Bonn RGB-D and dynamic TUM RGB-D sequences assess camera localization under moving foreground interference, while DAVIS supplies pixel-level annotations for dynamic-mask evaluation.The evaluation uses three public benchmarks.Source: §IV · p. 5
- The comparison includes semantic- or geometry-based dynamic SLAM, general-purpose deep SLAM, dynamic Gaussian-splatting SLAM, robust dynamic-scene methods, and a feedforward baseline.Listed baselines include DynaSLAM, DROID-SLAM, DG-SLAM, UP-SLAM, ADD-SLAM, Flow4DGS-SLAM, WildGS-SLAM, DynaMoN, DROID-W, and MonST3R.Source: §IV · p. 5
- Evaluation reports ATE RMSE after Sim(3) alignment for localization and FPS for efficiency; DM-IoU measures spatial agreement between predicted and ground-truth dynamic masks.A high DM-IoU requires complete coverage and low background contamination.Source: §IV · p. 5
A. Evaluation of Dynamic Masks
On DAVIS, FLoTS-SLAM achieves the best overall dynamic-mask IoU and leads in most scene categories. Qualitative comparisons report more complete, coherent masks with sharper boundaries and fewer false background responses across indoor scenes and DAVIS sequences.
A. Evaluation of Dynamic Masks
- Across Bonn, TUM-RGBD, and DAVIS examples, FLoTS-SLAM produces more complete object regions and sharper boundaries while suppressing background responses.The figure compares input frames and predictions from MonST3R [40], WildGS-SLAM [13], DROID-W [12], and FLoTS-SLAM; DROID-W can respond to distinctive static structures and provide incomplete coverage.Source: Figure 3 · p. 6, A. Evaluation of Dynamic Masks · p. 6
- FLoTS-SLAM achieves the best overall DM-IoU on DAVIS and leads in most categories, particularly human-motion and vehicle scenes.DAVIS sequences are divided into Animals, Pedestrians, Human Motion, Wheeled and Board Devices, and Vehicles; the results support generalization of uncertainty estimation across diverse dynamic scenes.Source: A. Evaluation of Dynamic Masks · p. 6
B. Impact of Dynamic-Mask Accuracy on Downstream Tasks
Dynamic-mask accuracy affects both object tracking and 3D reconstruction. False positives and missed dynamic regions produce distinct downstream errors.
B. Impact of Dynamic-Mask Accuracy on Downstream Tasks
- All methods use the same downstream pipeline to isolate mask quality; Mask R-CNN identifies instances and Hungarian matching associates them across frames.Source: B. Impact of Dynamic-Mask Accuracy on… · p. 6
- False positives create spurious tracks on static objects, while incomplete masks break tracks of moving objects.The same mask errors also assign static geometry to the dynamic layer or leave object geometry incomplete; more complete masks reduce background contamination.Source: B. Impact of Dynamic-Mask Accuracy on… · p. 6
C. Evaluation of Camera Tracking Performance
FLoTS-SLAM achieves the lowest reported average ATE on both Bonn and TUM dynamic sequences. The reported Bonn gains are largest when a moving foreground and sufficient static structure support the two-stage optimization.
C. Evaluation of Camera Tracking Performance
- 2.06 cm average ATE on Bonn is lower than DROID-W (2.23), WildGS-SLAM (2.44), and depth-aided DynaSLAM (N+G) (6.04 cm).FLoTS-SLAM ranks first or tied-first on seven of nine sequences.Source: C. Evaluation of Camera Tracking… · p. 6
- On Bonn, gains are largest with a prominent moving foreground and sufficient static structure; otherwise re-anchoring is disabled without degrading two-stage accuracy.Source: C. Evaluation of Camera Tracking… · p. 6
- 1.32 cm average ATE on TUM is lower than DROID-W (1.37) and WildGS-SLAM (1.52).The passage attributes the smaller margins to abundant static constraints despite uncertainty down-weighting alone.Source: C. Evaluation of Camera Tracking… · p. 6
D. Ablation Study
The ablation results support the contribution of each component to camera tracking, while the mask examples show downstream differences in tracking and reconstruction.
D. Ablation Study
- Removing any component raises camera-tracking error; disabling uncertainty estimation in BA causes the largest degradation.Removing re-anchoring conditions, two-stage optimization, or dynamic-pixel re-anchoring also consistently degrades tracking, with the components jointly providing an accurate reference pose and reducing object-motion influence on pose optimization.Source: D. Ablation Study · pp. 6–7
- Our masks preserve stable identities for moving people, whereas DROID-W tracks stationary furniture and loses continuity on people.Same-color regions denote the same track ID.Source: Figure 4 · p. 7
- More complete masks reduce spurious background geometry and yield more accurate dynamic-object reconstructions than the partial or contaminated reconstructions described for DROID-W.Source: Figure 5 · p. 7
E. Runtime Analysis
FLoTS-SLAM improves dynamic-mask and camera-tracking results relative to the stated baselines. It maintains a processing rate above 10 FPS despite modest additional optimization costs.
E. Runtime Analysis
- Above 10 FPS on both datasets, with modest additional cost from the first-stage pose-only solve and two re-anchoring tests.Runtime experiments used a single NVIDIA RTX 4090 GPU and a 96-core CPU.Source: E. Runtime Analysis · p. 7
V. CONCLUSION
- Approximately 35% higher mean DM-IoU on DAVIS and average ATE reductions of approximately 59% on Bonn and 19% on TUM versus the DROID-SLAM backbone.The conclusion attributes robustness to motion uncertainty adapted to frame alignment and two-stage optimization with dynamic-pixel re-anchoring.Source: §V · pp. 7–8