PaperlayerView exact revision on arXiv

Source-linked AI summary

OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition

Junyi Ma, Jun Zhang, Jintao Xu, Rui Ai, Weihao Gu, Xieyuanli Chen

arXiv:2203.03397v4cs.RO

TL;DR

LiDAR-based place recognition must support navigation tasks such as SLAM loop closure and global localization under changing conditions. OverlapTransformer uses range-image depth with Transformer-based, yaw-angle-invariant descriptor generation, achieving fast execution and reported performance across multiple environments and challenges.

  • Problem

    LiDAR-based place recognition compares current scans with previously recorded observations to support loop closure and global localization in autonomous navigation.

  • Method

    OverlapTransformer uses only range-image depth with a lightweight Transformer and NetVLAD-based architecture to generate yaw-angle-invariant global descriptors.

  • Results

    The method reports effective loop-closure detection on KITTI and Ford Campus, long-term and viewpoint-changing evaluation on Haomo, and runtime below 2 ms per frame.

  • Takeaways & Limitations

    The reported results support LiDAR-only place recognition that generalizes across environments and handles changing viewpoints while remaining suitable for fast retrieval.

Abstract

from arXiv · show

Place recognition is an important capability for autonomously navigating vehicles operating in complex environments and under changing conditions. It is a key component for tasks such as loop closing in SLAM or global localization. In this paper, we address the problem of place recognition based on 3D LiDAR scans recorded by an autonomous vehicle. We propose a novel lightweight neural network exploiting the range image representation of LiDAR sensors to achieve fast execution with less than 2 ms per frame. We design a yaw-angle-invariant architecture exploiting a transformer network, which boosts the place recognition performance of our method. We evaluate our approach on the KITTI and Ford Campus datasets. The experimental results show that our method can effectively detect loop closures compared to the state-of-the-art methods and generalizes well across different environments. To evaluate long-term place recognition performance, we provide a novel dataset containing LiDAR sequences recorded by a mobile robot in repetitive places at different times. The implementation of our method and dataset are released here: https://github.com/haomo-ai/OverlapTransformer

I. INTRODUCTION

LiDAR-based place recognition supports loop closure and global localization, while OverlapTransformer uses range images and yaw-angle-invariant descriptors to recognize places across viewpoints and environments.

  • LiDAR-based place recognition compares current observations with previously recorded observations to support SLAM loop closure and global localization.
  • OverlapTransformer uses only depth information from LiDAR range images to extract yaw-angle-invariant global descriptors.The design targets robustness to changing viewpoints, including vehicles traveling in different directions.
  • The lightweight architecture combines Transformer attention with a NetVLAD head to compress range images into global descriptors.
  • The method is trained on part of KITTI and evaluated on KITTI, Ford Campus, and a newly released Haomo dataset.The Haomo dataset includes long-term, reverse-driving, and different-appearance challenges.
  • The authors claim loop-closure detection without additional information, cross-environment generalization without fine-tuning, viewpoint robustness, and faster execution than most state-of-the-art methods.

II. RELATED WORK

Prior LiDAR place-recognition methods use handcrafted signatures, local or global learned descriptors, semantic cues, and point-cloud representations; OverlapTransformer instead uses single-scan range images with Transformer-enhanced descriptors.

  • Earlier handcrafted methods generate global signatures by projecting point clouds, counting heights, or encoding maximum heights in spatial bins.
  • Learning-based approaches include local-feature pipelines and direct global-descriptor methods such as PointNetVLAD and sparse-convolution approaches.
  • OverlapTransformer directly generates global descriptors from single LiDAR range images rather than local point-cloud maps or NDT cells.This representation is described as suitable for fast computation and natural yaw-angle invariance.
  • Some recent methods exploit depth, normal, intensity, semantic, or topological information for loop closure and localization.
  • Its pipeline encodes range images, applies Transformer-based feature processing, and uses NetVLAD with MLPs to produce one-dimensional global descriptors.

A. Range Image Encoder

The range image encoder projects LiDAR point clouds into compact images and preserves yaw-angle equivariance while extracting features. Its architecture avoids width compression, padding, and dropout to limit discretization errors.

  • A. Range Image Encoder: A LiDAR point cloud is projected into a range image whose pixels contain 3D points and whose dimensions depend on sensor parameters.Each point p_i=(x,y,z) is mapped to image coordinates, while its range is r=||p||_2.
  • A. Range Image Encoder: Yaw rotation of the point cloud corresponds to a horizontal column shift of the resulting range image.The correspondence is expressed through the yaw rotation matrix Rθ and column-shift matrix C_s.
  • A. Range Image Encoder: The encoder receives an h×w×1 range image and outputs a 1×w×c feature volume, with one channel for range depth.The encoder compresses only the vertical dimension, preserving the width dimension.
  • A. Range Image Encoder: Avoiding width compression, padding, and dropout preserves yaw-angle equivariance through the encoder layers.These design choices are intended to avoid discretization errors associated with yaw transformations.

B. Transformer Module

The transformer module upgrades encoder features by modeling relationships among spatial features through self-attention. Its one-block design combines attention, feed-forward processing, and normalization while preserving yaw-angle equivariance.

  • B. Transformer Module: The attentional feature transformer uses one block containing multi-head self-attention, a feed-forward network, and layer normalization.The single-block design targets both recognition accuracy and execution efficiency.
  • B. Transformer Module: Multi-head self-attention learns relationships among feature splits represented as queries, keys, and values.The split dimension is denoted by d_k.
  • B. Transformer Module: The attention output is processed by the feed-forward network and layer normalization to produce the final attentional feature volume S.Channel-wise concatenation combines the coarse encoder feature F with the attention feature A.
  • B. Transformer Module: Channel splitting, elementwise feed-forward processing, and the remaining operations preserve yaw-angle equivariance for shifted encoder features.The equivariance argument applies to the attention inputs and the feed-forward network.
  • B. Transformer Module: The transformer exploits spatial feature relationships through attention to produce more distinctive features for LiDAR place recognition.The paper relates this mechanism to using distinctive landmarks and their relationships for determining places.

C. Global Descriptor Generator

The global descriptor generator combines yaw-angle-equivariant feature sequences with NetVLAD’s permutation invariance to produce yaw-angle-invariant global descriptors. Toy rotations illustrate that intermediate features shift while the final descriptor remains unchanged.

  • C. Global Descriptor Generator: Original PointNetVLAD does not directly produce yaw-angle-invariant descriptors because rotating point coordinates changes its input features.The proposed generator addresses this limitation by combining NetVLAD permutation invariance with yaw-angle-equivariant features.
  • C. Global Descriptor Generator: The feature volume S is represented as a set of one-dimensional channel vectors before global descriptor generation.The vectors are written as S=Z={z_1,z_2,...,z_w}, with each vector containing c channels.
  • C. Global Descriptor Generator: The global descriptor generator is yaw-angle-invariant because NetVLAD is permutation invariant.Yaw-induced shifts change the ordering of equivariant features without changing the generated descriptor.
  • C. Global Descriptor Generator: Under toy rotations, the range-image encoder and transformer outputs shift horizontally, while the global descriptor remains the same.The example compares 0°, 90°, and 180° yaw rotations.
  • C. Global Descriptor Generator: During online operation, Euclidean distances between yaw-angle-invariant descriptors identify nearest reference places.Descriptors represent LiDAR scans for place retrieval.

D. Network Training

Network training uses scan overlap rather than ground-truth distance to supervise descriptor learning. A lazy triplet loss pulls queries toward hard positives and separates them from sampled negatives.

  • D. Network Training: Scan overlap supervises training because it describes LiDAR-scan similarity and relates to subsequent registration quality.For a query and reference scan, the reference is reprojected into the query coordinate frame before overlap calculation.
  • D. Network Training: Each training tuple contains one query descriptor, k_p positive descriptors, and k_n negative descriptors for lazy triplet loss.The loss uses squared Euclidean distance and a margin α.
  • D. Network Training: Pairs with overlap greater than 0.3 are positives, while all other pairs are negatives.The threshold defines the positive and negative samples used by the triplet loss.
  • D. Network Training: The loss minimizes distance to the hardest positive descriptors and maximizes distance to all sampled negative descriptors.This directly shapes the learned global descriptor space for retrieval.

IV. HAOMO DATASET

The Haomo dataset is introduced to address the shortage of repetitive reverse-route sequences for long-term, large-scale LiDAR place recognition. It contains urban Beijing sequences collected with a 32-beam LiDAR and supporting sensors.

  • Dataset motivation: The dataset targets long-term large-scale place recognition with repetitive routes and opposite driving directions.Sequences 1-1 and 1-2 follow the same route in opposite directions, while sequences 2-1 and 2-2 cover a longer route on different dates.
  • Data collection: Haomo was collected in Beijing by a mobile robot equipped with a HESAI PandarXT 32-beam LiDAR, wide-angle camera, and RTK GNSS.
  • Evaluation sequences: Sequence 1-3 serves as an online query against sequences 1-1 and 1-2 for forward- and reverse-driving evaluation.

V. EXPERIMENTAL EVALUATION

The evaluation tests loop-closure detection, long-term place recognition, viewpoint changes, and runtime efficiency. These experiments are designed to assess generalization, yaw-angle invariance, and practical deployment potential.

  • Evaluation goals: The experiments evaluate LiDAR-only loop-closure detection, cross-environment generalization without fine-tuning, long-term recognition, viewpoint changes, and runtime.
  • Evaluation scope: The evaluation covers KITTI, Ford Campus, and the newly introduced Haomo dataset across loop-closure and place-recognition settings.

A. Implementation and Experimental Setup

The experiments use range images matched to the LiDAR beam counts and evaluate loop closure, generalization, and long-term place recognition across three datasets. Performance is measured with retrieval and classification-style metrics against non-semantic baselines.

  • Implementation and setup: KITTI and Ford Campus use 1×64×900 range images, while Haomo uses 1×32×900 range images.
  • Loop closure detection: KITTI sequences 03–10 train the model, sequence 02 validates it, and sequence 00 evaluates it; Ford Campus sequence 00 tests transfer without fine-tuning.
  • Loop closure detection: Loop closures are defined by overlap greater than 0.3, with nearby database scans excluded to avoid detecting recent observations.
  • Loop closure detection: The loop-closure comparison uses AUC, F1 max, recall@1, and recall@1% against methods that do not use semantic information.
  • Place recognition: Haomo evaluates short-term same-direction, short-term inverse-direction, and long-term large-scale challenges using database sequences for training and new query sequences for testing.
  • Place recognition: The method outperforms other methods on the first two Haomo challenges, particularly reverse driving, and retains superiority on long-term large-scale sequences.

D. Study on Yaw-Angle-Invariance

The yaw-angle study rotates KITTI query scans in 30-degree increments while retaining the original database. Results compare Recall@1 across baselines and support the proposed yaw-angle-invariant descriptor design.

  • Yaw-angle-invariance test: Query scans are rotated around the yaw axis in 30-degree steps and matched against the unchanged original database using Recall@1.
  • Yaw-angle-invariance results: OverlapTransformer significantly outperforms the baselines by exploiting yaw-angle-equivariant range images in its yaw-angle-invariant model.
  • Baseline comparison: LiDAR Iris achieves rotation invariance by repeatedly rotating features and selecting the best score, making it slower than the proposed approach.

E. Ablation Study on Transformer Module

The ablation compares alternative modules between the range image encoder and global descriptor generator, while runtime experiments assess execution speed against baseline methods.

  • Ablation setup: The ablation evaluates five configurations spanning no transformer, convolutional replacement, and one, three, or six transformer modules.The database is Haomo sequence 1-2 and the query is sequence 1-3.
  • Ablation conclusion: One transformer block is selected because deeper transformer stacks may require more training data and time to achieve good performance.
  • Runtime: 1.37 ms per scan makes OverlapTransformer the fastest descriptor-generation method among the compared learning-based state-of-the-art methods.Descriptor-generation timing includes LiDAR preprocessing; the comparison uses 2000 reference scans and averages ten experiments.
  • Runtime: 730 Hz operation is reported as much faster than the LiDAR scanning rate, supporting the method’s runtime claim.
Loading 2203.03397v4…