PaperlayerView exact revision on arXiv

Source-linked AI summary

LoGG3D-Net: Locally Guided Global Descriptor Learning for 3D Place Recognition

Kavisha Vidanapathirana, Milad Ramezani, Peyman Moghadam, Sridha Sridharan, Clinton Fookes

arXiv:2109.08336v3cs.CVcs.RO

TL;DR

LoGG3D-Net addresses limitations in enforcing desirable local-feature properties for 3D place recognition by jointly optimizing local and scene-level embeddings with a local consistency loss. Evaluated on two large-scale public datasets, the method operates in near real-time and encodes point clouds using second-order pooling.

  • Problem

    Existing global-descriptor learning methods have limited ability to enforce desirable properties in intermediate local features, although accurate place recognition supports data association and SLAM.

  • Method

    LoGG3D-Net jointly optimizes local and scene-level embeddings using a local consistency loss that brings corresponding-point features together and separates non-corresponding features.

  • Results

    LoGG3D-Net demonstrates superiority in an end-to-end setting across two large-scale public datasets while operating in near real-time.

  • Takeaways & Limitations

    The approach combines local consistency training with second-order pooling to produce a single global descriptor from local features.

Abstract

from arXiv · show

Retrieval-based place recognition is an efficient and effective solution for re-localization within a pre-built map, or global data association for Simultaneous Localization and Mapping (SLAM). The accuracy of such an approach is heavily dependent on the quality of the extracted scene-level representation. While end-to-end solutions - which learn a global descriptor from input point clouds - have demonstrated promising results, such approaches are limited in their ability to enforce desirable properties at the local feature level. In this paper, we introduce a local consistency loss to guide the network towards learning local features which are consistent across revisits, hence leading to more repeatable global descriptors resulting in an overall improvement in 3D place recognition performance. We formulate our approach in an end-to-end trainable architecture called LoGG3D-Net. Experiments on two large-scale public benchmarks (KITTI and MulRan) show that our method achieves mean $F1_{max}$ scores of $0.939$ and $0.968$ on KITTI and MulRan respectively, achieving state-of-the-art performance while operating in near real-time. The open-source implementation is available at: https://github.com/csiro-robotics/LoGG3D-Net.

I. INTRODUCTION

LoGG3D-Net addresses fragile large-scale 3D place recognition by jointly learning consistent local features and discriminative global descriptors. It combines local consistency training with second-order aggregation and demonstrates strong performance on two public datasets while operating near real time.

  • I. INTRODUCTION: 3D point-cloud place recognition supports global constraints that restrict cumulative errors in SLAM, but existing large-scale outdoor data-association methods remain fragile.LiDAR point clouds provide viewpoint and illumination invariance, while their sparsity and variable point distributions make useful representation challenging.
  • I. INTRODUCTION: LoGG3D-Net jointly optimizes local and scene-level embeddings through a local consistency loss for LiDAR place recognition.The loss increases similarity between corresponding-point features and decreases similarity between non-corresponding features.
  • I. INTRODUCTION: Second-order pooling with differentiable Eigen-value power normalization aggregates local features into a global descriptor in an end-to-end setting.The paper identifies this as a higher-order aggregation approach not previously explored in an end-to-end LiDAR place-recognition architecture.
  • I. INTRODUCTION: Using 2 large-scale public datasets, the method demonstrates superiority in an end-to-end setting while operating in near real-time.The introduction reports evaluation across KITTI and MulRan.

II. RELATED WORK

3D LiDAR place-recognition methods include local matching, handcrafted, hybrid, and end-to-end retrieval approaches. End-to-end methods scale through global descriptors but have relied mainly on global supervision and first-order aggregation.

  • II. RELATED WORK: Local feature-matching methods depend on accurate global pose estimates to constrain correspondences and therefore do not scale well to large-scale relocalization.These approaches commonly detect keypoints and describe their local neighborhoods.
  • II. RELATED WORK: Retrieval-based methods encode each point cloud as a single global descriptor, making database querying inexpensive and scalable to large environments.The related work groups descriptors into handcrafted, hybrid, and end-to-end learning-based categories.
  • II. RELATED WORK: Hybrid methods combine mathematical models with data-driven representations but can struggle when point-segment structure differs from training environments.Locus is cited as a hybrid LiDAR place-recognition example with this adaptation limitation.
  • II. RELATED WORK: End-to-end methods learn global descriptors contrastively, but existing systems apply supervision only at the global level and use first-order aggregation.Higher-order aggregation has been used for 3D place recognition but not previously in an end-to-end trainable architecture.

III. PROPOSED METHOD

The proposed architecture maps raw point clouds to fixed-size global descriptors using a sparse convolutional U-Net and second-order feature aggregation. Its training objective extends global metric learning with a local consistency signal.

  • III. PROPOSED METHOD: LoGG3D-Net uses a SparseConv U-Net to embed points, applies local consistency to nearby point-cloud pairs, and aggregates features into a global descriptor.The architecture then uses a quadruplet loss for scene-level learning.
  • III. PROPOSED METHOD: Second-order pooling followed by differentiable Eigen-value power normalization forms the global scene descriptor from local features.This aggregation is part of the proposed end-to-end architecture.
  • III. PROPOSED METHOD: Point-cloud retrieval learns a mapping from variable-sized point clouds to fixed-size descriptors suitable for metric-learning-based database queries.The formulation represents each point cloud with a descriptor g in a fixed-dimensional space.
  • III. PROPOSED METHOD: Global metric learning alone supervises scene-level descriptors and cannot fully enforce desirable properties of intermediate local features.The paper motivates adding a training signal directly to local representations.

B. Our Approach

The approach hypothesizes that consistent local features across overlapping revisits produce more repeatable global descriptors. It enforces this property with correspondence-aware training over sparse U-Net features.

  • B. Our Approach: Enforcing consistency between corresponding local features is hypothesized to produce more repeatable global descriptors after aggregation.Corresponding points are defined using proximity in a common global coordinate frame, and their features should be nearby in embedding space.
  • B. Our Approach: The local consistency loss adds a training signal that makes features of corresponding points consistent across point clouds from nearby locations.The paired clouds are assumed to have considerable overlap.
  • B. Our Approach: A Sparse U-Net backbone combines voxel-based multiscale neighborhood learning with point-based high-resolution information for local feature extraction.The two branches address complementary information in sparse point clouds.

1) Point Correspondences:

Point correspondences are established by aligning nearby point clouds, matching points within a radius, and applying a hardest-contrastive loss to their local features.

  • 1) Point Correspondences:: Nearby point clouds are aligned using geolocation and ICP, then corresponding points are found with radius-based nearest-neighbor search.FLANN makes the nearest-neighbor search efficient.
  • 1) Point Correspondences:: The local consistency loss pulls corresponding-point features together while separating features from non-corresponding points.The method uses hardest-contrastive loss for this positive-versus-negative feature comparison.
  • 1) Point Correspondences:: Hard-negative mining samples feature candidates and uses a hinge loss with an indicator to retain valid non-corresponding negatives.Margins m_p and m_n and the negative weight λ_n control the loss terms.
  • 1) Point Correspondences:: The local consistency loss acts on the feature-mapping parameters to produce well-formed, repeatable local features.

D. Global Descriptor

The global descriptor aggregates point features with second-order pooling, applies eigenvalue power normalization, and trains scene-level retrieval with a quadruplet loss.

  • D. Global Descriptor: Second-order pooling forms a matrix from point-feature outer products and takes the element-wise maximum across the point cloud.The resulting matrix captures second-order feature statistics before descriptor formation.
  • D. Global Descriptor: Eigen-value power normalization raises each singular value to a power, after which the matrix is flattened and normalized into the global descriptor.The implementation uses differentiable SVD and sets α = 0.5.
  • D. Global Descriptor: The scene-level loss applies a quadruplet objective to an anchor, positives, negatives, and an additional negative sampled outside the positive neighborhoods.Positive and negative samples are selected using distance thresholds, and the hardest positive is used within each tuple.

E. Joint Local and Global Loss

The network jointly optimizes global scene-level learning and local feature consistency through a weighted sum of their losses.

  • E. Joint Local and Global Loss: The joint objective combines the global scene-level loss Lg with the local consistency loss Llc.The scalar hyperparameter ω controls the contribution of Llc.

IV. EXPERIMENTAL SETUP

Experiments use two multi-sequence LiDAR datasets, controlled preprocessing and training settings, and an ablation of the local-consistency-loss weight.

  • IV. EXPERIMENTAL SETUP: Training removes the ground plane with RANSAC, voxel-downsamples points at 10 cm, caps inputs at 35K points, and uses augmentation to reduce overfitting.The implementation uses PyTorch, sparse convolutions, and distributed training on 12 Nvidia Tesla P100-16GB GPUs.
  • IV. EXPERIMENTAL SETUP: The descriptor dimension is fixed at 256 for comparison with PointNetVLAD, with specified local-feature, loss-margin, sampling, and optimizer settings.The local feature dimension is d = 16, and positive and negative pair thresholds are 3 m and 20 m.
  • IV. EXPERIMENTAL SETUP: The ablation varies ω on MulRan sequences and reports F1_max for each test sequence to assess the local consistency loss.The accompanying table compares the joint loss components Lg and Llc.
  • IV. EXPERIMENTAL SETUP: The evaluation uses KITTI and MulRan, collected with different LiDAR sensors in Germany and Korea across dynamic urban environments.Both datasets are collected from moving vehicles.
  • IV. EXPERIMENTAL SETUP: KITTI training uses leave-one-out cross-validation over 11 sequences and evaluates six sequences containing revisits.The evaluation sequences are 00, 02, 05, 06, 07, and 08.
  • IV. EXPERIMENTAL SETUP: MulRan training uses four sequences, while evaluation covers nine sequences including unseen KAIST test sets for generalization assessment.

C. Evaluation Criteria

The evaluation measures sequential place recognition with F1max and examines how local consistency loss affects performance and learned local features.

  • Evaluation criteria: F1max compares methods using precision-recall performance under 3m true-positive and 20m false-positive revisit thresholds.Queries are matched against previously seen global descriptors while nearby database entries are excluded using dataset-specific temporal thresholds.
  • Ablation study on point-wise loss: The local consistency loss improves place recognition performance across the evaluated MulRan ablation sequences.The ablation varies the loss weight ω while training on DCC1 and Riverside1 and testing on DCC2 and Riverside2.
  • Ablation study on point-wise loss: t-SNE visualizations show locally consistent point features, with colors indicating cross-cloud feature similarity for corresponding locations.The visualization compares features from two nearby point clouds and does not use point correspondence information during inference.
  • Ablation study on point-wise loss: 61% mean F1max improvement over the baseline is achieved with local consistency loss weight ω = 1.0, the best tested setting.A weight of ω = 0.1 yields a 26% improvement; subsequent experiments use ω = 1.0.

B. Comparison to State-of-the-Art

LoGG3D-Net is compared with handcrafted, hybrid, and end-to-end place-recognition methods on KITTI and MulRan using F1max and precision-recall curves.

  • Comparison to State-of-the-Art: LoGG3D-Net achieves the best MulRan mean F1max, 5% higher than ScanContext, while Locus scores 1% higher than LoGG3D-Net on KITTI.The comparison uses the F1max metric under 3m and 20m revisit criteria.
  • Comparison to State-of-the-Art: Locus's KITTI advantage is associated with training on KITTI sequences 05 and 06, while its MulRan performance is relatively low.The cited comparison identifies this training overlap as contributing to Locus's extremely high KITTI performance.
  • Comparison to State-of-the-Art: Precision-recall evaluation includes KITTI and MulRan sequences selected for challenging repetitive and reverse-direction revisits.KITTI 02 contains repetitive environments and an opposite-direction intersection revisit; MulRan DCC 03 contains long reverse-direction traversals.
  • Comparison to State-of-the-Art: LoGG3D-Net and PointNetVLAD use compact 256-dimensional descriptors, compared with 1200-dimensional ScanContext and 4096-dimensional Locus descriptors.The paper presents compact descriptors as scaling better to large databases for real-time robotic applications.

C. Runtime Analysis

Runtime analysis compares preprocessing, description, and retrieval costs on MulRan DCC1, showing that LoGG3D-Net supports real-time operation.

  • Runtime Analysis: The runtime table reports average preprocessing, description, and querying times in milliseconds on a database of 5541 MulRan DCC1 point clouds.Retrieval time increases with database size, so the reported values use the DCC1 evaluation setting.
  • Runtime Analysis: Locus has high preprocessing and description time because its feature extraction uses 30–80 sequential, non-parallelized forward passes.ScanContext also has high retrieval time because it computes cosine similarity across shifted descriptor variants.
  • Runtime Analysis: ∼10Hz real-time inference is enabled by LoGG3D-Net's lowest preprocessing and retrieval times, supporting SLAM loop-closure integration.The runtime comparison is conducted on MulRan DCC1 using an 8-core Intel i7-9700 CPU and an RTX 2080Ti GPU.
  • Runtime Analysis: LoGG3D-Net has lower preprocessing and retrieval costs than the compared methods, while PointNetVLAD has the most efficient descriptor extraction.PointNetVLAD's total time is increased by ground-plane removal and iterative downsampling preprocessing.
Loading 2109.08336v3…