PaperlayerView exact revision on arXiv

Source-linked AI summary

SuMa++: Efficient LiDAR-based Semantic SLAM

Xieyuanli Chen, Andres Milioto, Emanuele Palazzolo, Philippe Giguère, Jens Behley, Cyrill Stachniss

arXiv:2105.11320v1cs.RO

TL;DR

Dynamic objects can corrupt LiDAR SLAM maps and localization, while semantic information is needed for intelligent navigation. SuMa++ extends surfel-based LiDAR mapping with learned point labels, semantic dynamic-object filtering, and semantic ICP constraints, achieving better pose-estimation accuracy than purely geometric mapping on KITTI. Its main limitation is that it cannot filter dynamics in the first observation without a special initialization procedure.

  • Problem

    Dynamic objects can corrupt mapping and derail localization, complicating reliable semantic mapping in realistic autonomous-vehicle environments.

  • Method

    SuMa++ integrates RangeNet++ point-wise semantic labels into a surfel-based LiDAR map to filter dynamic objects and add semantic constraints during ICP.

  • Results

    SuMa++ achieves considerably better pose-estimation accuracy than the pure geometric approach on challenging KITTI sequences.

  • Takeaways & Limitations

    Semantic consistency can combine with geometric information to improve robustness to outliers and support semantic mapping using only three-dimensional laser scans.

  • Takeaways & Limitations

    The method cannot filter dynamic objects in the first observation and removes all potentially movable classes during initialization instead.

Abstract

from arXiv · show

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfel-based mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

I. INTRODUCTION

SuMa++ addresses semantic mapping and localization in dynamic environments using only LiDAR data, integrating learned point labels into surfel mapping. The approach targets more reliable mapping and pose estimation than purely geometric or camera-dependent alternatives.

  • Motivation: Dynamic driving environments make consistent mapping difficult because moving objects cause wrong associations between observations and maps.Semantic information is also needed for navigation behaviors such as legally parking or safely pulling over in previously unseen locations.
  • Approach: SuMa++ integrates semantic information from an FCN with a modern LiDAR SLAM pipeline to generate semantic maps and improve map geometry and odometry.The system uses three-dimensional laser range scans rather than camera data.
  • Approach: Point-wise semantic labels are obtained by spherical projection, refined to reduce reprojection artifacts, and integrated into labeled surfels.The refinement uses erosion followed by depth-based flood-fill before semantic labels are added to the map.
  • Approach: Semantic consistency is used to filter moving objects and add semantic constraints to scan registration, reducing the risk of dynamic objects entering the map.These operations combine semantic and geometric information during mapping and alignment.
  • Evaluation: The paper evaluates SuMa++ on challenging KITTI sequences and reports superior performance to purely geometric surfel mapping and removing all potentially moving objects.The evaluation includes urban, countryside, and highway scenes.
  • Related Work: Prior semantic SLAM methods commonly use cameras, RGB-D sensors, or camera-LiDAR combinations, while some LiDAR-only methods target different semantic mapping or retrieval objectives.Some semantic registration methods also cannot operate online because of long processing times.

III. OUR APPROACH

SuMa++ extends surfel-based LiDAR mapping with point-wise semantic labels, dynamic-object filtering, and semantic constraints for projective ICP. Its pipeline preserves the geometric mapping structure while adding semantic checks during map updating and registration.

  • Semantic Integration: The approach extends the SuMa surfel-based mapping pipeline by integrating semantic information from RangeNet++.RangeNet++ produces point-wise labels from spherical projections of point clouds.
  • Semantic Integration: Semantic labels are used both to filter dynamic objects and to add semantic constraints that make pose estimation more robust and accurate.These constraints complement the geometric information used by scan registration.
  • Notation: Each observed point cloud is transformed between coordinate frames using a homogeneous 4×4 transformation matrix.The notation distinguishes rotational and translational components of the transformation.
  • Surfel-based Mapping: SuMa generates spherical vertex and normal maps, then uses projective ICP against a rendered map view to update the pose.Pose increments are chained to obtain the world-frame pose.
  • Surfel-based Mapping: The map stores surfels with positions, normals, radii, timestamps, and stability estimates used to distinguish stable from unstable elements.Loop-closure detection and pose-graph optimization are also part of the underlying SuMa system.

C. Semantic Segmentation

SuMa++ obtains point-wise semantic labels with RangeNet++ from spherical laser-scan projections and stores each inferred label and its probability with the corresponding surfel.

  • Semantic Segmentation: RangeNet++ segments a spherical-projection range image for each laser scan and predicts a semantic label for every point.The network uses a SqueezeSeg-based architecture with a DarkNet53 backbone while remaining real-time capable.
  • Semantic Segmentation: The inferred semantic label y and its probability are stored as additional attributes for each surfel.This makes point-wise semantic information available for integration into the map.

D. Refined Semantic Map

The refined semantic-map module reduces projection artifacts by eroding uncertain boundaries and filling them using depth-consistent neighboring labels. The resulting mask contains fewer artifacts than raw predictions.

  • Refinement Motivation: RangeNet++ reprojection can create label errors because projective inputs and downsampling produce blob-like outputs.The flood-fill refinement operates inside preprocessing using depth information from the vertex map.
  • Inputs and Output: The flood-fill takes the raw semantic mask Sraw and corresponding vertex map VD as input and outputs the refined mask SD.Mask pixels contain semantic labels, while vertex-map pixels contain nearest-point 3D coordinates.
  • Erosion: Erosion removes boundary pixels and small wrongly labeled regions when neighboring labels disagree within a kernel of size d.The method relies on higher uncertainty at object boundaries than near object centers.
  • Depth-based Flood-fill: Empty boundary pixels are filled from neighboring labels when the corresponding 3D-point distances are below the threshold θ.Depth consistency prevents filling across incompatible surfaces.
  • Result: The refined semantic map contains fewer artifacts than raw predictions, with most wrong labels on a building wall corrected in the illustrated example.The figure presents raw, eroded, and filled semantic masks alongside depth information.

E. Filtering Dynamics using Semantics

The method uses semantic consistency to identify and remove dynamic surfels while retaining static objects that remain useful for scan registration.

  • Filtering Dynamics using Semantics: Semantic consistency between the new observation and world model identifies surfels likely belonging to objects that moved between scans.Inconsistent labels receive a stability penalty, enabling unstable surfels to be removed after repeated observations.
  • Filtering Dynamics using Semantics: Figure 4 compares surfels from SuMa, the proposed method, and removal of all potentially moving objects.SuMa is shown with label colors but does not use semantic information.
  • Filtering Dynamics using Semantics: The stability log odds ratio is penalized using penalty odds, while measurement noise is modeled through angle- and distance-dependent terms.The formulation uses surfel and measurement normals together with their distance.
  • Filtering Dynamics using Semantics: Unlike motion-history approaches requiring a global map and accurate pose estimates, this method exploits semantic labels to handle moving objects.The comparison concerns the information and pose assumptions required by the alternative approach.
  • Filtering Dynamics using Semantics: Naively removing all movable-class surfels also removes parked cars, whereas semantic filtering removes dynamic outliers while retaining static parked cars for ICP registration.Removing static features can cause registration failures because of missing correspondences.

F. Semantic ICP

Semantic ICP augments geometric frame-to-model registration with label-based constraints and confidence weighting, reducing the influence of inconsistent or outlier correspondences.

  • Semantic ICP: The ICP objective adds semantic constraints to the geometric optimization to reduce the influence of outliers.The semantic term combines higher-level scene understanding with geometric information.
  • Semantic ICP: Each current-scan vertex is projectively associated with a reference model vertex and its normal, producing a residual and weight for ICP.The residual and weight are denoted ru and wu, respectively.
  • Semantic ICP: Gauss-Newton iteratively solves for pose increments using the weighted residual system and its Jacobian.W contains residual weights, r is the stacked residual vector, and J is the Jacobian with respect to the six-dimensional increment.
  • Semantic ICP: The residual weight combines robust Huber weighting with semantic compatibility between the observation and rendered model labels.The semantic compatibility term uses predicted-label certainty and distinguishes matching from nonmatching labels.
  • Semantic ICP: In a highway scene with two moving cars, label disagreement after filtering produces low weights in the Semantic ICP weight map.Darker pixels represent lower weights, showing reduced influence for those correspondences.

IV. EXPERIMENTAL EVALUATION

The evaluation compares SuMa++ with the original SuMa and a naive movable-object removal variant on KITTI sequences using LiDAR odometry and semantic-labeling experiments.

  • Experimental Evaluation: The evaluation tests whether SuMa++ maps scenes with many moving objects accurately and outperforms naive removal across urban, countryside, and highway environments.These are the paper’s stated experimental claims.
  • Experimental Evaluation: KITTI point clouds come from a Velodyne HDL-64E S2 recorded at 10 Hz, with odometry evaluated using relative translation and rotation errors over multiple pose distances.Ground-truth poses are generated from an inertial navigation system in most sequences.
  • Experimental Evaluation: SuMa++ is compared against SuMa and SuMa nomovable, which removes all movable semantic classes.The movable classes include vehicles, bicycles, motorcycles, and persons.
  • Experimental Evaluation: RangeNet++ was trained on KITTI Odometry training sequences 00–10, excluding sequence 08 for validation.The training used available point-wise annotations.
  • Experimental Evaluation: On the reported hardware, RangeNet++ labels a scan in 75 ms on average and surfel mapping takes 48 ms on average.Loop-closure integration can take at most 190 ms in some sequence-00 situations.

A. KITTI Road Sequences

On challenging KITTI Road sequences dominated by moving cars and sparse static features, SuMa++ produces more consistent maps and more accurate pose estimates than purely geometric SuMa.

  • A. KITTI Road Sequences: SuMa without semantics locks ICP to consistently moving cars, causing incorrect sensor motion estimates and map inconsistencies.The qualitative example highlights inconsistent traffic-sign locations and map errors.
  • A. KITTI Road Sequences: Highway sequences contain many moving cars but few distinctive static roadside features, making scan registration difficult.These raw road sequences are not part of the odometry benchmark and have no provided semantic-segmentation labels.
  • A. KITTI Road Sequences: SuMa++ filters moving cars using semantics and consequently generates a more consistent map with more accurate pose estimates.The relative translational-error plot compares per-timestamp errors for SuMa and SuMa++ in the example.
  • A. KITTI Road Sequences: SuMa++ generally produces more consistent trajectories and lower translational error than SuMa across the KITTI Road evaluation.Its performance is very similar to SuMa nomovable, supporting moving-object inconsistency as the main reason for SuMa’s worse performance in these sequences.

B. KITTI Odometry Benchmark

On the KITTI Odometry Benchmark, SuMa++ generally matches state-of-the-art performance, while removing all potentially moving objects can diverge or degrade pose estimation, especially in urban scenes.

  • B. KITTI Odometry Benchmark: SuMa++ shows similar performance to state-of-the-art LiDAR-based SLAM approaches on most KITTI Odometry sequences.The comparison reports relative translational and rotational errors.
  • B. KITTI Odometry Benchmark: The SuMa nomovable baseline diverges particularly in urban scenes, despite their man-made structures and distinctive features.The reported behavior contrasts with the expectation that urban structure should make registration easier.
  • B. KITTI Odometry Benchmark: Removing semantic classes can delete static surfels and parked cars, making the surfel map sparser and pose estimation worse.Parked cars can provide distinctive or reliable alignment features in these scenes.
  • B. KITTI Odometry Benchmark: On unseen KITTI test trajectories, SuMa++ achieved 0.0032 deg/m rotational error and 1.06% translational error, versus 0.0032 deg/m and 1.39% for SuMa.The server-side evaluation prevented parameter tuning on the test set.

C. Discussion

The method preserves semantically static objects as landmarks while addressing dynamic-object contamination, but initialization and indiscriminate removal remain important constraints.

  • Discussion: Semantically static objects, including vegetation, are not penalized during map updates because features such as tree trunks can support pose estimation.The geometric Huber-based outlier rejection may still down-weight these parts.
  • Discussion: Dynamic objects cannot be filtered from the first observation when their presence prevents estimating a proper initial velocity or pose.The initialization procedure therefore removes all potentially movable object classes.
  • Discussion: A more robust alternative would retrospectively backtrack map changes caused by objects changing their observed motion state.
  • Discussion: Blindly removing selected classes can deteriorate localization accuracy, although retaining potentially movable objects in long-term maps may represent otherwise occluded environmental parts.

V. CONCLUSION

The paper presents camera-free semantic mapping from three-dimensional laser scans, using semantics alongside geometry to improve pose estimation and filter dynamic objects. Future work targets semantic loop closure and finer-grained scene information.

  • V. CONCLUSION: The approach builds semantic maps from laser-based point-cloud segmentation without requiring camera data.It uses semantic information from three-dimensional laser range scans.
  • V. CONCLUSION: Semantic consistencies between scans and the map filter dynamic objects and provide higher-level constraints during ICP.The method combines semantic and geometric information for pose estimation.
  • V. CONCLUSION: The method achieves considerably better pose estimation accuracy than the pure geometric approach on the KITTI Vision Benchmark dataset.
  • V. CONCLUSION: Future research will investigate semantics for loop closure detection and finer-grained information such as lane structure or road type.
Loading 2105.11320v1…