Source-linked AI summary
DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments
Chao Yu, Zuxin Liu, Xinjun Liu, Fugui Xie, Yi Yang, Qi Wei, Qiao Fei
TL;DR
DS-SLAM targets the unresolved problems of dynamic-object interference and limited semantic understanding in visual SLAM. It combines semantic segmentation with moving-consistency checking and builds a dense semantic octo-tree map; on high-dynamic sequences, reported ATE improvements reach nearly 98% versus ORB-SLAM2.
Problem
Visual SLAM remains vulnerable to dynamic objects and geometric-only maps do not provide the semantic information needed for robots to understand scenes.
Method
DS-SLAM runs tracking, semantic segmentation, mapping, loop closing, and dense-map creation in parallel while combining segmentation with moving-consistency checking and log-odds semantic mapping.
Results
ATE RMSE and S.D. improvement values reach 97.91% and 97.94%, respectively, on most high-dynamic sequences compared with ORB-SLAM2.
Takeaways & Limitations
DS-SLAM improves robustness and stability in high-dynamic environments while providing a semantic octo-tree map for potential high-level robot tasks.
Takeaways & Limitations
Recognizable object types are restricted, limiting DS-SLAM’s scope of application, and the octo-tree map must be rebuilt after loop closure.
Abstract
from arXiv · showhide
Simultaneous Localization and Mapping (SLAM) is considered to be a fundamental capability for intelligent mobile robots. Over the past decades, many impressed SLAM systems have been developed and achieved good performance under certain circumstances. However, some problems are still not well solved, for example, how to tackle the moving objects in the dynamic environments, how to make the robots truly understand the surroundings and accomplish advanced tasks. In this paper, a robust semantic visual SLAM towards dynamic environments named DS-SLAM is proposed. Five threads run in parallel in DS-SLAM: tracking, semantic segmentation, local mapping, loop closing, and dense semantic map creation. DS-SLAM combines semantic segmentation network with moving consistency check method to reduce the impact of dynamic objects, and thus the localization accuracy is highly improved in dynamic environments. Meanwhile, a dense semantic octo-tree map is produced, which could be employed for high-level tasks. We conduct experiments both on TUM RGB-D dataset and in the real-world environment. The results demonstrate the absolute trajectory accuracy in DS-SLAM can be improved by one order of magnitude compared with ORB-SLAM2. It is one of the state-of-the-art SLAM systems in high-dynamic environments. Now the code is available at our github: https://github.com/ivipsourcecode/DS-SLAM
I. INTRODUCTION
Visual SLAM is mature but remains vulnerable to dynamic environments and lacks semantic understanding needed for higher-level robot tasks. DS-SLAM addresses both issues by filtering dynamic scene portions and producing a semantic octo-tree map.
- Geometric-only maps lack semantic attributes, limiting a robot’s understanding of its surroundings and support for advanced tasks.
- Dynamic objects such as walking people can corrupt feature association, state estimation, and system reliability in pure vision-based SLAM.
- On TUM RGB-D and a real robot, DS-SLAM is reported to improve accuracy and robustness over ORB-SLAM2 in dynamic environments.
- DS-SLAM combines semantic segmentation with moving consistency checking to filter dynamic portions before pose estimation.
- DS-SLAM provides a semantic octo-tree map through a separate dense-map thread, using semantic information for potential high-level robot tasks.
A. Semantic SLAM
Semantic SLAM extends geometric mapping with conceptual scene information, while DS-SLAM further uses semantic information during tracking to address dynamic objects.
- Geometric-only maps do not provide conceptual knowledge of surroundings needed for complex robot tasks.
- Earlier semantic-mapping approaches associated labels with reconstructed objects but focused primarily on mapping and recognition.
- DS-SLAM uses semantic information both to generate an octo-tree environment representation and to filter tracking outliers in dynamic scenes.
- Most SLAM approaches assume a static environment, making them unsuitable for severe dynamic scenarios involving active objects such as humans.
- Optical-flow inconsistency can distinguish moving targets from static background before dynamic features are discarded for pose estimation.
III. SYSTEM INTRODUCTION
DS-SLAM builds on ORB-SLAM2 as a global feature-based visual SLAM solution, adding mechanisms for dynamic-object detection and semantic octo-tree mapping. Its architecture distributes processing across parallel threads.
- DS-SLAM adopts ORB-SLAM2 to provide a global feature-based SLAM solution for dynamic-object detection and semantic octo-tree mapping.
- Five parallel threads handle tracking, semantic segmentation, local mapping, loop closing, and dense map creation.
- Tracking extracts ORB features, performs an initial moving-consistency check, stores potential outliers, and then uses pixel-wise semantic labels.
- The framework processes raw RGB images in tracking and semantic-segmentation threads simultaneously before dynamic-feature rejection and pose estimation.
B. Semantic Segmentation
DS-SLAM uses SegNet for real-time pixel-wise semantic segmentation, with people treated as likely dynamic outliers in practical applications.
- SegNet provides pixel-wise semantic segmentation in real time using a model trained on PASCAL VOC.
- The trained SegNet model segments 20 classes in total.
- Because people are commonly dynamic in real applications, DS-SLAM assumes feature points located on people are likely outliers.
C. Moving Consistency Check
The moving consistency check uses feature correspondences, epipolar geometry, and a preset distance threshold to identify points exhibiting motion.
- Optical flow provides matched feature points between the current and previous frames for motion analysis.
- The fundamental matrix maps a previous-frame feature to its corresponding epipolar-line search domain in the current frame.
- The distance from a matched point to its corresponding epipolar line is used to assess moving consistency.
- A matched point is classified as moving when its distance exceeds a threshold.The threshold is represented by the preset value ε.
D. Outliers Rejection
DS-SLAM rejects dynamic-scene outliers by combining semantic object regions with moving-consistency results before feature matching.
- Semantic segmentation supplies complete object outlines that complement moving-consistency checks, whose contours may be incomplete for deformable objects.
- Tracking can perform moving-consistency checks while waiting for semantic segmentation results from its independent thread.The tracking and semantic-segmentation thread times are reported as approximately equal in the experiments.
- DS-SLAM treats humans as the representative dynamic-object category because human activity strongly interferes with robot localization.The paper states that, in theory, the approach applies to multiple recognized and segmented dynamic objects.
- If detected people are moving, ORB features inside their outlines are removed before matching.
- If no people are detected, or detected people are static, ORB features are matched directly with the last frame to predict pose.
E. Dense semantic 3D octo-tree map building
The dense semantic map converts keyframes and segmentation results into a probabilistic global octo-tree whose voxels encode occupancy and semantic labels.
- Keyframe transforms, depth images, and segmentation results are used to construct and maintain a global octo-tree map from local point clouds.
- The octo-tree representation is selected because it is flexible, compact, updatable, efficiently stored, and suitable for navigation.
- Each voxel receives a color representing a semantic label, and probabilistic semantic fusion supports attribute updates.Examples include red for sofa and pink for person.
- Occupancy probability is recovered from log odds through logit and inverse-logit transforms.
- A voxel’s log-odds score increases with repeated occupied observations and decreases otherwise, supporting dynamic-environment map building.
- A voxel is treated as occupied and visualized only when its occupancy probability exceeds a predefined threshold.
IV. EXPERIMENTAL RESULTS
The experiments evaluate module timing and DS-SLAM performance in dynamic environments using challenging TUM RGB-D sequences containing moving people.
- The evaluation measures the time required by each module and tests DS-SLAM on dynamic sequences from the public TUM RGB-D dataset.
- Moving people make the TUM RGB-D sequences challenging because they can reduce SLAM robustness and accuracy.In an extreme case, moving people occupy more than half of the image.
A. Evaluation using TUM RGB-D dataset
On TUM RGB-D dynamic sequences, DS-SLAM is evaluated against ORB-SLAM2 using trajectory and drift metrics, with additional assessment of runtime and mapping results.
- The experiments use walking sequences as high-dynamic cases and sitting sequences as low-dynamic cases, with ground truth from external motion capture.
- ATE measures global trajectory consistency, while RPE measures translational and rotational drift in the comparison with ORB-SLAM2.
- DS-SLAM achieves up to 97.91% improvement in ATE RMSE and 97.94% improvement in ATE S.D. in high-dynamic sequences.
- Improvements are not obvious in low-dynamic sequences such as fr3_sitting_static because ORB-SLAM2 already performs well there.
- ATE and RPE errors are significantly reduced by DS-SLAM in the high-dynamic fr3_walking_xyz sequence.
- The main thread processes each frame in 59.4ms, including semantic segmentation, visual odometry, pose graph optimization, and dense semantic octo-tree map creation.
V. CONCLUSIONS
DS-SLAM is a real-time semantic SLAM system designed to reduce dynamic objects’ influence on pose estimation while producing a semantic octo-tree map. Its limitations include restricted semantic recognition, octo-tree rebuilding after loop closure, and ongoing work to improve real-time performance.
- DS-SLAM combines tracking, semantic segmentation, local mapping, loop closing, and dense map creation in parallel.
- A SegNet-based semantic segmentation network and moving consistency check filter dynamic scene portions before matched feature points are removed.This design targets dynamic regions such as walking people.
- The dense semantic 3D octo-tree map uses log-odds scoring to filter unstable voxels and supports navigation and complex robot tasks.
- DS-SLAM outperforms ORB-SLAM2 in accuracy and robustness on challenging high-dynamic TUM RGB-D sequences.
- Semantic recognition is limited to the object types supported by the segmentation network, restricting the system’s application scope.
- The octo-tree map must be rebuilt after loop closure, while future work targets moving-consistency-check optimization and mobile-robot high-level tasks.