PaperlayerView exact revision on arXiv

Source-linked AI summary

LCDNet: Deep Loop Closure Detection and Point Cloud Registration for LiDAR SLAM

Daniele Cattaneo, Matteo Vaghi, Abhinav Valada

arXiv:2103.05056v4cs.ROcs.CVcs.LG

TL;DR

LiDAR loop closure detection must address weak reverse-loop performance and difficult point-cloud alignment. LCDNet jointly recognizes places and estimates 6-DoF transformations with differentiable unbalanced optimal transport, achieving strong results across datasets and challenging conditions.

  • Problem

    Existing deep LiDAR loop detectors underperform handcrafted methods on reverse loops, while registration methods are limited by large initial rotations and inconsistent evaluation protocols.

  • Method

    LCDNet combines a shared point-feature encoder, global place descriptors, and a differentiable unbalanced-optimal-transport relative-pose head for joint loop detection and registration.

  • Results

    LCDNet sets the reported state of the art, reaching 0.96 AP on KITTI sequence 08 versus 0.65 AP for the previous best method and retaining 100% success after removing a 90° sector.

  • Takeaways & Limitations

    The approach supports reverse-loop detection, partial-overlap registration, ICP refinement, complete LIO-SAM integration, and evaluation across an unseen city and sensor setup.

Abstract

from arXiv · show

Loop closure detection is an essential component of Simultaneous Localization and Mapping (SLAM) systems, which reduces the drift accumulated over time. Over the years, several deep learning approaches have been proposed to address this task, however their performance has been subpar compared to handcrafted techniques, especially while dealing with reverse loops. In this paper, we introduce the novel LCDNet that effectively detects loop closures in LiDAR point clouds by simultaneously identifying previously visited places and estimating the 6-DoF relative transformation between the current scan and the map. LCDNet is composed of a shared encoder, a place recognition head that extracts global descriptors, and a relative pose head that estimates the transformation between two point clouds. We introduce a novel relative pose head based on the unbalanced optimal transport theory that we implement in a differentiable manner to allow for end-to-end training. Extensive evaluations of LCDNet on multiple real-world autonomous driving datasets show that our approach outperforms state-of-the-art loop closure detection and point cloud registration techniques by a large margin, especially while dealing with reverse loops. Moreover, we integrate our proposed loop closure detection approach into a LiDAR SLAM library to provide a complete mapping system and demonstrate the generalization ability using different sensor setup in an unseen city.

I. INTRODUCTION

LCDNet addresses loop closure detection and point-cloud registration within LiDAR SLAM, targeting reverse loops and large initial misalignments where existing approaches are limited.

  • SLAM builds an environment map while simultaneously localizing the robot using sensors including LiDAR, cameras, GPS, and IMUs.
  • Loop detection identifies revisited places, while loop closure aligns the current scan and adds a pose-graph constraint to reduce accumulated drift.
  • LiDAR offers geometric reconstruction robust to illumination changes, but handcrafted descriptors require ad hoc comparisons whose runtime grows with past scans.
  • Deep LiDAR loop detectors are faster than handcrafted methods but remain weaker, especially for reverse loops, while ICP depends on sufficiently similar initial alignment.
  • Existing learned registration benchmarks generally allow rotations up to 45°, whereas loop closures can involve 180° rotations and failure beyond 120°.
  • LCDNet jointly detects loops and estimates full 6-DoF relative transformations using shared features, global descriptors, and differentiable unbalanced optimal transport.

II. RELATED WORKS

Prior work spans visual and LiDAR place recognition and point-cloud registration, while LCDNet combines learned descriptors with registration designed for difficult misalignments.

  • Loop Closure Detection: Vision-based loop closure methods use handcrafted or learned image representations but are vulnerable to illumination and appearance variation.
  • Loop Closure Detection: LiDAR methods exploit accurate 3D geometry through keypoints, bag-of-words models, global descriptors, and semantic graph matching.
  • Loop Closure Detection: Deep LiDAR methods compute compact descriptors or estimate yaw discrepancy, extending learned place recognition beyond handcrafted representations.
  • Point Cloud Registration: ICP requires a rough initial alignment and becomes more complex with point count, motivating global and learned registration methods.
  • Point Cloud Registration: Existing learned registration methods commonly handle rotations up to 45° and may fail beyond 120°, whereas LCDNet handles arbitrary initial rotation without an initial guess.
  • Proposed Approach: LCDNet uses a shared feature extractor, a global-descriptor head, and a relative-pose head to support loop detection and registration together.

A. Feature Extraction

LCDNet adapts PV-RCNN into a LiDAR feature extractor that combines sparse voxel pyramids, BEV features, raw points, and sampled keypoints.

  • The feature extractor is built on PV-RCNN, combining voxel-based high-level features with fine-grained PointNet-type features.
  • The input is a point cloud with x, y, z, and intensity values, and the extractor outputs N keypoint feature vectors.
  • Voxelization averages points within cells, sparse 3D convolutions form a four-level pyramid, and the coarsest map is converted into a 2D BEV feature map.
  • Voxel Set Abstraction aggregates pyramid, BEV, and input-point features around N uniformly distributed keypoints selected by Farthest Point Sampling.
  • For each keypoint and pyramid level, neighboring voxel features are sampled and aggregated before later MLP processing produces final feature vectors.
  • The adapted extractor uses the full 360° surrounding view rather than only the camera field of view.

B. Global Descriptor

The global descriptor head converts keypoint features into a compact representation using NetVLAD, dimensionality reduction, and context gating.

  • NetVLAD transforms the N × D keypoint feature set into a K × D representation by learning K cluster centers and differentiable assignments.
  • A multilayer perceptron compresses the NetVLAD representation into a G-dimensional descriptor, which Context Gating then reweights by feature context.
  • Context Gating uses sigmoid-based element-wise weighting to emphasize more discriminative features across the complete descriptor.

C. Relative Pose Estimation

LCDNet estimates a 6-DoF transformation between partially overlapping point clouds using differentiable unbalanced optimal transport and weighted SVD. The approach supports end-to-end training and robust inference-time registration.

  • Relative pose estimation: LCDNet estimates the 6-DoF transformation aligning source point cloud P with target point cloud S under driving conditions.It matches keypoint features and uses their 3D coordinates to estimate the transformation.
  • Relative pose estimation: Because sparse sampling can place a source point between multiple target points, one-to-one correspondence is unsuitable for this task.The matching formulation therefore needs soft correspondences rather than strict point pairing.
  • Unbalanced optimal transport: The Sinkhorn algorithm approximates optimal transport in a fast, parallelizable, and differentiable manner for feature matching.The mapping sparsity parameter λ is learned through backpropagation rather than manually fixed.
  • Unbalanced optimal transport: Unbalanced optimal transport relaxes mass preservation, allowing unmatched points caused by occlusions or objects present in only one cloud.Its parameter ρ controls how much mass is preserved, while UOT reduces the impact of incorrect matches and keypoint-sampling stochasticity.
  • Transformation estimation: Weighted SVD converts soft correspondences and 3D keypoint coordinates into a rigid-body transformation between the original point cloud and its projection.Because the UOT procedure and SVD are differentiable, the relative pose head can be trained end-to-end against ground-truth transformations.
  • Transformation estimation: At inference, LCDNet replaces the UOT pose head with feature-based RANSAC to estimate accurate relative poses using robust correspondence selection.This preserves end-to-end training while using a robust estimator during inference.

D. Loss Function

LCDNet jointly learns place-recognition descriptors, relative poses, and UOT correspondences through a combined training objective. Triplet learning separates same-place and different-place descriptors, while pose supervision is applied to positive pairs.

  • Descriptor learning: Triplet loss trains global descriptors to place same-location point clouds closer than point clouds depicting different places.The loss uses an anchor, a positive sample from the same place, and a negative sample from a different place.
  • Training pairs: Negative samples are selected online from different-place samples within the batch, while relative pose transformations are computed only for positive pairs.The predicted transformation is compared with the ground-truth transformation after transforming the anchor point cloud.
  • Correspondence supervision: An auxiliary loss supervises the matches estimated by the unbalanced optimal transport module.This adds correspondence learning alongside descriptor and pose objectives.
  • Joint objective: The final objective linearly combines triplet, pose, and optimal-transport losses.The loss-balancing coefficient β is empirically set to 0.05.
  • Joint objective: Combining these losses with data augmentation yields distinctive keypoint features that are invariant to rotation and translation.The shared feature extractor learns these properties through backpropagation.

E. SLAM System

LCDNet is integrated into LIO-SAM as a loop-closure component that combines descriptor-based candidate selection, relative pose estimation, ICP refinement, and geometric verification.

  • System integration: LCDNet is integrated into LIO-SAM, a tightly coupled LiDAR–inertial odometry framework built on a factor graph.LIO-SAM accepts LiDAR point clouds and IMU measurements and supports IMU, LiDAR odometry, GPS, and loop-closure constraints.
  • Loop-closure pipeline: A descriptor distance below threshold th selects a loop candidate, after which LCDNet estimates the 6-DoF relative transformation.The candidate decision uses the place-recognition descriptors, while the transformation comes from the relative pose head.
  • Loop-closure pipeline: ICP further refines the estimated transformation and supplies the geometric check before adding a loop-closure factor.The factor is added only when the ICP fitness score exceeds threshold th_icp.
  • Loop-closure pipeline: The loop-detection step uses neither IMU nor GPS measurements, and ICP verification discards remaining false-positive detections.This separates descriptor and geometric loop verification from the other factor-graph measurements.

IV. EXPERIMENTAL EVALUATION

The evaluation tests LCDNet on three autonomous-driving datasets designed to cover same-direction and reverse loops, different environments, dynamic objects, and sensor setups. Experiments target loop detection, registration without an initial guess, partial overlap, and generalization.

  • Evaluation goals: The experiments are designed to test reverse-loop detection, point-cloud alignment without an initial guess, and robust registration under partial overlap.The Freiburg data additionally evaluates generalization to a different city and sensor setup.
  • Datasets: Experiments evaluate LCDNet on three autonomous-driving datasets and exclude sequences without loop closures.Training and testing sequences, loop counts, and revisited-place directions are reported in Table I.
  • Datasets: KITTI provides six loop-containing sequences, mostly with same-direction loops, while sequence 08 includes reverse-loop closures.The evaluation uses SemanticKITTI ground truth because some original KITTI ground truth is not aligned with nearby loop closures.
  • Datasets: KITTI-360 contains more loops and reverse loops than standard KITTI, and evaluation focuses on sequences 02 and 09.These sequences contain the highest number of loop closures among the selected KITTI-360 sequences.
  • Datasets: The Freiburg dataset contains same- and reverse-direction loops collected across different days in a different city using a Velodyne HDL-64E and Applanix POS LV.It also includes many dynamic objects, supporting tests across city, sensor, and environmental differences.

B. Implementation and Training Details

LCDNet is trained and evaluated under explicit loop-closure criteria and two precision-recall protocols, with comparisons against handcrafted and deep-learning baselines. The reported results show strong performance across same-direction and reverse loops, including the best performance for LCDNet†.

  • Evaluation setup: Real loops are defined by ground-truth pose distances below four meters, while the previous 50 scans are excluded from candidate search.
  • Evaluation protocols: LCDNet is evaluated using precision-recall curves and Average Precision under two protocols that select either the highest-similarity candidate or all above-threshold pairs.Varying the similarity threshold produces precision-recall points and Average Precision values.
  • Baselines: Comparisons include M2DP, Scan-Context, Intensity Scan-Context, LiDAR-IRIS, OverlapNet, and Semantic Graph Place Recognition.Official implementations and author-provided pretrained models are used where available.
  • Results: LCDNet† achieves the best performance in all KITTI and KITTI-360 experiments, with LCDNet reported as the second-best method.The advantage over previous state-of-the-art methods is especially prominent for reverse-direction loops.
  • Results: LCDNet maintains similar performance for reverse and same-direction loops, achieving AP values of 0.94 and 0.97 respectively.Under protocol 2, LCDNet's reverse-loop AP is even better than its same-direction result.

D. Evaluation of Relative Pose Estimation

LCDNet is evaluated for full 6-DoF relative pose estimation against handcrafted and DNN-based registration methods on KITTI and KITTI-360, including challenging reverse-loop and partial-overlap settings. It achieves the highest reported success rates and lowest rotation errors, while its PV-RCNN backbone, optimal transport loss, and keypoint design materially affect performance.

  • Relative pose comparison: LCDNet and LCDNet† achieve the highest success rates and lowest rotation errors among compared methods, reaching 100% success in three of four sequences.PCAM often has the lowest translation errors but is not robust to registration under partial overlap.
  • Evaluation protocol: The evaluation measures success rate, translation error, and rotation error, defining successful alignment as errors below 5° rotation and 2 m translation.Errors are averaged over successful pairs and over all positive pairs.
  • Partial overlap: On MulRan, approximately 70° of rear LiDAR field of view is obstructed, leaving very limited overlap in reverse-direction scenarios where all considered approaches failed in preliminary evaluations.The authors identify this as a limitation of scan-to-scan methods and suggest scan-to-map approaches for such scenarios.
  • Backbone ablation: PV-RCNN achieves average precision 0.94, mean rotation error 3.13°, and mean translation error 1.62 m, outperforming alternative backbones.The second-best backbone reaches average precision 0.67, while EdgeConv and RandLA-Net report 16.85° and 3.55 m on the cited pose metrics.
  • Architectural ablations: Lowering point-feature dimensionality decreases performance, while removing the auxiliary optimal transport loss significantly reduces average precision and relative transformation performance.The loss is reported to promote more distinctive features benefiting both loop detection and relative transformation estimation.
  • Architectural ablations: Performance generally improves with more keypoints, but average precision does not improve at 8192, leading the final model to use 4096 keypoints.The choice reflects the higher memory and computation required by the larger setting.

G. ICP with Initial Guess

LCDNet’s relative pose estimate is used to initialize ICP, addressing failures under large reverse-loop rotation misalignment. The initialization substantially improves ICP’s runtime and alignment error in those cases, while LCDNet also provides fast loop-map querying and joint loop detection with 6-DoF registration.

  • ICP refinement: When reverse loops are present, LCDNet initialization greatly reduces ICP runtime and final errors, unlike same-direction loops where ICP already performs satisfactorily.The reported comparison uses ICP with and without the LCDNet relative pose estimate as the initial guess.
  • ICP refinement: ICP with LCDNet initialization is 4 times faster and achieves an RMSE 22 times lower than ICP without an initial guess.The comparison is reported on average in the Figure 5 and Figure 6 experiment.
  • ICP refinement: ICP fails under significant rotation misalignment, whereas LCDNet aligns the point clouds and further improves the result when its prediction initializes ICP.The qualitative comparison covers ICP alone, LCDNet alignment, and subsequent ICP refinement.
  • Runtime analysis: LCDNet queries more than 18000 scans in five milliseconds, while ISC, LiDAR-Iris, and OverlapNet can require up to 100 seconds for a single query.The runtime experiment includes descriptor extraction and map-query comparisons on the stated hardware.
  • Runtime analysis: LCDNet is the only compared method reported to perform both loop closure detection and 6-DoF point-cloud registration under driving conditions.Several faster alternatives estimate only 1-DoF rotation or achieve unsatisfactory registration performance.

I. Qualitative Results

LCDNet reduces loop-detection errors on KITTI and KITTI-360, and integrating it into LIO-SAM improves loop closure under accumulated drift.

  • LCDNet detects same-direction and reverse-direction loops, but still produces some false positives and false negatives.
  • LCDNet† reduces false positives and false negatives while maintaining accurate true-positive detections across KITTI and KITTI-360 sequences.
  • On KITTI sequence 08, LCDNet’s false negatives are almost completely eliminated by LCDNet†, although a few false positives remain.
  • Replacing LIO-SAM’s original loop-closure pipeline with LCDNet detects closures missed under accumulated drift and improves overall SLAM performance.
  • The integrated LIO-SAM system with LCDNet is publicly released.

K. Generalization Analysis

LCDNet generalizes to an unseen city and different sensor setup without retraining, supporting loop detection, mapping, and point-cloud registration in new environments.

  • Generalization Experiments: LCDNet and LCDNet† are evaluated in Freiburg using a car equipped with a roof-mounted rack of LiDAR sensors, without retraining or fine-tuning on Freiburg data.
  • Generalization Experiments: The Freiburg evaluation expands the real-loop distance threshold from 4 to 10 meters because the city includes dual carriageways unlike the primarily narrow roads in training datasets.
  • Mapping Results: The resulting Freiburg map is well aligned with aerial imagery, with no evidence of drift, while LCDNet corrects accumulated drift in the integrated SLAM system.
  • Point Cloud Registration: 98.94% and 99.81% success rates are achieved by LCDNet and LCDNet†, respectively, compared with 92.49% for the second-best method on Freiburg registration.
  • Point Cloud Registration: LCDNet aligns opposite-direction point clouds with 0.34° average rotation error and 0.15 m translation error, versus 1.84° and 0.41 m for LiDAR-IRIS and PCAM.
  • Point Cloud Registration: LCDNet retains a 100% success rate after removing a 90° sector from each point cloud, while the second-best method drops from 95% to 55%.
Loading 2103.05056v4…