PaperlayerView exact revision on arXiv

Source-linked AI summary

DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments

Chao Yu, Zuxin Liu, Xinjun Liu, Fugui Xie, Yi Yang, Qi Wei, Qiao Fei

arXiv:1809.08379v2cs.RO

TL;DR

DS-SLAM targets the unresolved problems of dynamic-object interference and limited semantic understanding in visual SLAM. It combines semantic segmentation with moving-consistency checking and builds a dense semantic octo-tree map; on high-dynamic sequences, reported ATE improvements reach nearly 98% versus ORB-SLAM2.

  • Problem

    Visual SLAM remains vulnerable to dynamic objects and geometric-only maps do not provide the semantic information needed for robots to understand scenes.

    Source: §I · p. 1

  • Method

    DS-SLAM runs tracking, semantic segmentation, mapping, loop closing, and dense-map creation in parallel while combining segmentation with moving-consistency checking and log-odds semantic mapping.

    Source: E. Dense semantic 3D octo-tree map… · p. 4, §V · p. 7

  • Results

    ATE RMSE and S.D. improvement values reach 97.91% and 97.94%, respectively, on most high-dynamic sequences compared with ORB-SLAM2.

    Source: A. Evaluation using TUM RGB-D dataset · p. 5

  • Takeaways & Limitations

    DS-SLAM improves robustness and stability in high-dynamic environments while providing a semantic octo-tree map for potential high-level robot tasks.

    Source: §I · p. 1, A. Evaluation using TUM RGB-D dataset · p. 5

    Recognizable object types are restricted, limiting DS-SLAM’s scope of application, and the octo-tree map must be rebuilt after loop closure.

    Source: §V · p. 7

Abstract

from arXiv · show

Simultaneous Localization and Mapping (SLAM) is considered to be a fundamental capability for intelligent mobile robots. Over the past decades, many impressed SLAM systems have been developed and achieved good performance under certain circumstances. However, some problems are still not well solved, for example, how to tackle the moving objects in the dynamic environments, how to make the robots truly understand the surroundings and accomplish advanced tasks. In this paper, a robust semantic visual SLAM towards dynamic environments named DS-SLAM is proposed. Five threads run in parallel in DS-SLAM: tracking, semantic segmentation, local mapping, loop closing, and dense semantic map creation. DS-SLAM combines semantic segmentation network with moving consistency check method to reduce the impact of dynamic objects, and thus the localization accuracy is highly improved in dynamic environments. Meanwhile, a dense semantic octo-tree map is produced, which could be employed for high-level tasks. We conduct experiments both on TUM RGB-D dataset and in the real-world environment. The results demonstrate the absolute trajectory accuracy in DS-SLAM can be improved by one order of magnitude compared with ORB-SLAM2. It is one of the state-of-the-art SLAM systems in high-dynamic environments. Now the code is available at our github: https://github.com/ivipsourcecode/DS-SLAM

I. INTRODUCTION

Visual SLAM is mature but remains vulnerable to dynamic environments and lacks semantic understanding needed for higher-level robot tasks. DS-SLAM addresses both issues by filtering dynamic scene portions and producing a semantic octo-tree map.

  • Geometric-only maps lack semantic attributes, limiting a robot’s understanding of its surroundings and support for advanced tasks.Source: §I · p. 1
  • Dynamic objects such as walking people can corrupt feature association, state estimation, and system reliability in pure vision-based SLAM.Source: §I · p. 1
  • On TUM RGB-D and a real robot, DS-SLAM is reported to improve accuracy and robustness over ORB-SLAM2 in dynamic environments.Source: §I · p. 1
  • DS-SLAM combines semantic segmentation with moving consistency checking to filter dynamic portions before pose estimation.Source: §I · pp. 1–2
  • DS-SLAM provides a semantic octo-tree map through a separate dense-map thread, using semantic information for potential high-level robot tasks.Source: §I · pp. 1–2

A. Semantic SLAM

Semantic SLAM extends geometric mapping with conceptual scene information, while DS-SLAM further uses semantic information during tracking to address dynamic objects.

  • Geometric-only maps do not provide conceptual knowledge of surroundings needed for complex robot tasks.Source: A. Semantic SLAM · p. 2
  • Earlier semantic-mapping approaches associated labels with reconstructed objects but focused primarily on mapping and recognition.Source: A. Semantic SLAM · p. 3
  • DS-SLAM uses semantic information both to generate an octo-tree environment representation and to filter tracking outliers in dynamic scenes.Source: A. Semantic SLAM · p. 3
  • Most SLAM approaches assume a static environment, making them unsuitable for severe dynamic scenarios involving active objects such as humans.Source: B. SLAM in dynamic environments · p. 3
  • Optical-flow inconsistency can distinguish moving targets from static background before dynamic features are discarded for pose estimation.Source: B. SLAM in dynamic environments · p. 3

III. SYSTEM INTRODUCTION

DS-SLAM builds on ORB-SLAM2 as a global feature-based visual SLAM solution, adding mechanisms for dynamic-object detection and semantic octo-tree mapping. Its architecture distributes processing across parallel threads.

  • DS-SLAM adopts ORB-SLAM2 to provide a global feature-based SLAM solution for dynamic-object detection and semantic octo-tree mapping.Source: A. Framework of DS-SLAM · p. 3
  • Five parallel threads handle tracking, semantic segmentation, local mapping, loop closing, and dense map creation.Source: A. Framework of DS-SLAM · p. 3
  • Tracking extracts ORB features, performs an initial moving-consistency check, stores potential outliers, and then uses pixel-wise semantic labels.Source: A. Framework of DS-SLAM · p. 3
  • The framework processes raw RGB images in tracking and semantic-segmentation threads simultaneously before dynamic-feature rejection and pose estimation.Source: A. Framework of DS-SLAM · p. 3, §III · p. 3

B. Semantic Segmentation

DS-SLAM uses SegNet for real-time pixel-wise semantic segmentation, with people treated as likely dynamic outliers in practical applications.

C. Moving Consistency Check

The moving consistency check uses feature correspondences, epipolar geometry, and a preset distance threshold to identify points exhibiting motion.

D. Outliers Rejection

DS-SLAM rejects dynamic-scene outliers by combining semantic object regions with moving-consistency results before feature matching.

  • Semantic segmentation supplies complete object outlines that complement moving-consistency checks, whose contours may be incomplete for deformable objects.Source: D. Outliers Rejection · p. 4
  • Tracking can perform moving-consistency checks while waiting for semantic segmentation results from its independent thread.The tracking and semantic-segmentation thread times are reported as approximately equal in the experiments.Source: D. Outliers Rejection · p. 4
  • DS-SLAM treats humans as the representative dynamic-object category because human activity strongly interferes with robot localization.The paper states that, in theory, the approach applies to multiple recognized and segmented dynamic objects.Source: D. Outliers Rejection · p. 4
  • If detected people are moving, ORB features inside their outlines are removed before matching.Source: D. Outliers Rejection · p. 4
  • If no people are detected, or detected people are static, ORB features are matched directly with the last frame to predict pose.Source: D. Outliers Rejection · p. 4

E. Dense semantic 3D octo-tree map building

The dense semantic map converts keyframes and segmentation results into a probabilistic global octo-tree whose voxels encode occupancy and semantic labels.

IV. EXPERIMENTAL RESULTS

The experiments evaluate module timing and DS-SLAM performance in dynamic environments using challenging TUM RGB-D sequences containing moving people.

  • The evaluation measures the time required by each module and tests DS-SLAM on dynamic sequences from the public TUM RGB-D dataset.Source: §IV · p. 5
  • Moving people make the TUM RGB-D sequences challenging because they can reduce SLAM robustness and accuracy.In an extreme case, moving people occupy more than half of the image.Source: §IV · p. 5

A. Evaluation using TUM RGB-D dataset

On TUM RGB-D dynamic sequences, DS-SLAM is evaluated against ORB-SLAM2 using trajectory and drift metrics, with additional assessment of runtime and mapping results.

V. CONCLUSIONS

DS-SLAM is a real-time semantic SLAM system designed to reduce dynamic objects’ influence on pose estimation while producing a semantic octo-tree map. Its limitations include restricted semantic recognition, octo-tree rebuilding after loop closure, and ongoing work to improve real-time performance.

  • DS-SLAM combines tracking, semantic segmentation, local mapping, loop closing, and dense map creation in parallel.Source: §V · p. 7
  • A SegNet-based semantic segmentation network and moving consistency check filter dynamic scene portions before matched feature points are removed.This design targets dynamic regions such as walking people.Source: §V · p. 7
  • The dense semantic 3D octo-tree map uses log-odds scoring to filter unstable voxels and supports navigation and complex robot tasks.Source: §V · p. 7
  • DS-SLAM outperforms ORB-SLAM2 in accuracy and robustness on challenging high-dynamic TUM RGB-D sequences.Source: §V · p. 7
  • Semantic recognition is limited to the object types supported by the segmentation network, restricting the system’s application scope.Source: §V · p. 7
  • The octo-tree map must be rebuilt after loop closure, while future work targets moving-consistency-check optimization and mobile-robot high-level tasks.Source: §V · p. 7

Related papers

Loading 1809.08379v2…