PaperlayerView exact revision on arXiv

Source-linked AI summary

Lightweight Unsupervised Deep Loop Closure

Nate Merrill, Guoquan Huang

arXiv:1805.07703v2cs.RO

TL;DR

Real-time visual SLAM needs loop closure that remains fast and reliable despite changing viewpoints, illumination, weather, and dynamic objects, while existing ConvNet approaches can be slow or require labeled data. The paper trains an unsupervised autoencoder with randomized projective transformations and HOG reconstruction targets; across datasets, the authors report generally better effectiveness and efficiency than benchmark methods.

  • Problem

    Real-time visual SLAM requires low-latency, high-fidelity loop closure despite major appearance changes, while ConvNet approaches can be slow or depend on labeled training data.

    Source: §I · p. 1

  • Method

    An unsupervised autoencoder learns compact image descriptors by training on raw images altered with projective transformations and reconstructing HOG descriptors.

    Source: §I · p. 1, §III · p. 2, §V · p. 8

  • Results

    Across different datasets, the proposed method generally outperforms state-of-the-art image-description methods in precision-recall effectiveness and runtime efficiency.

    Source: §V · p. 8

  • Takeaways & Limitations

    The compact, fast model is presented as suitable for real-time SLAM and as a candidate replacement for larger, slower classification networks in ConvNet-based place recognition.

    Source: §V · p. 8

Abstract

from arXiv · show

Robust efficient loop closure detection is essential for large-scale real-time SLAM. In this paper, we propose a novel unsupervised deep neural network architecture of a feature embedding for visual loop closure that is both reliable and compact. Our model is built upon the autoencoder architecture, tailored specifically to the problem at hand. To train our network, we inflict random noise on our input data as the denoising autoencoder does, but, instead of applying random dropout, we warp images with randomized projective transformations to emulate natural viewpoint changes due to robot motion. Moreover, we utilize the geometric information and illumination invariance provided by histogram of oriented gradients (HOG), forcing the encoder to reconstruct a HOG descriptor instead of the original image. As a result, our trained model extracts features robust to extreme variations in appearance directly from raw images, without the need for labeled training data or environment-specific training. We perform extensive experiments on various challenging datasets, showing that the proposed deep loop-closure model consistently outperforms the state-of-the-art methods in terms of effectiveness and efficiency. Our model is fast and reliable enough to close loops in real time with no dimensionality reduction, and capable of replacing generic off-the-shelf networks in state-of-the-art ConvNet-based loop closure systems.

I. INTRODUCTION

Real-time visual SLAM needs low-latency, high-fidelity loop closure despite major changes in image appearance, while existing approaches may not be fast or robust enough for challenging environments. The paper introduces an unsupervised, compact autoencoder-based ConvNet and evaluates it against state-of-the-art methods.

I. INTRODUCTION

  • Loop closure must be low-latency and high-fidelity to support bounded localization errors, yet viewpoint, illumination, weather, and dynamic objects can dramatically change a location’s appearance.Existing methods may not be sufficiently fast or robust for real-time use in challenging environments.Source: §I · p. 1
  • ConvNet-based place-recognition approaches can require slow feature extraction or querying, or large amounts of labeled training data.Off-the-shelf and specialized ConvNets are both used for place recognition, but these limitations remain across approaches.Source: §I · p. 1
  • The paper designs an unsupervised loop-closure autoencoder using HOG and projective transformations to learn compact features robust to appearance changes.The authors describe the model as requiring few parameters and suitable for real-time loop closing, including on resource-constrained systems.Source: §I · p. 1
  • The authors report extensive comparisons across datasets and provide code, a pretrained model, and a new dataset.The cited contribution passage states that these materials are open sourced.Source: §I · p. 1

II. RELATED WORK

Prior place-recognition methods trade off speed, robustness to appearance changes, and training-data demands. This paper builds on unsupervised autoencoders, using homographies and HOG to target viewpoint variation with a lightweight architecture.

II. RELATED WORK

  • Bag-of-words methods such as FAB-MAP and DBoW2 support fast querying but may fail under large appearance changes, while SeqSLAM can fail under large viewpoint changes.Other hand-crafted approaches formulate loop closure as sparse optimization.Source: §II · p. 2
  • Generic ConvNet features have supported place recognition, but descriptors can be too large, feature extraction or querying can be slow, and the networks are not trained specifically for loop closure.The passage describes methods combining ConvNet features with landmarks, bag-of-words querying, or sequence search.Source: §II · p. 2
  • Supervised place-recognition networks such as NetVLAD and the method of Lopez-Antequera et al. rely on labeled training images or triplets.The latter method reduces images to a 128-dimensional vector; the passage characterizes these methods as requiring substantial human labeling effort.Source: §II · p. 2
  • Gao and Zhang’s denoising autoencoder performs comparably to FAB-MAP 2.0 but has slow feature extraction; this work instead uses homographies and HOG in a lightweight unsupervised architecture.The authors motivate viewpoint-altered reconstruction as better matched to place recognition than reconstructing images with randomly altered pixels.Source: §II · p. 2

III. UNSUPERVISED DEEP LOOP CLOSURE

The proposed autoencoder maps raw images to compact descriptors intended to remain invariant to appearance differences without labeled images. Its training design replaces ordinary input dropout with viewpoint-altering projective transformations and uses HOG descriptors as reconstruction targets.

III. UNSUPERVISED DEEP LOOP CLOSURE

  • The model maps high-dimensional raw images into a low-dimensional descriptor space intended to be invariant to appearance differences, without requiring labeled images.The authors describe the network and training scheme as producing a compact, robust feature embedding.Source: §III · p. 2

A. Design Motivation

  • Standard denoising autoencoders randomly drop input values, but that does not teach the image variations encountered in place recognition, such as viewpoint and illumination changes.The passage cautions that direct deployment of such autoencoders may therefore be suboptimal for place recognition.Source: A. Design Motivation · p. 2
  • Randomized projective transformations impose training noise that models viewpoint changes due to robot motion, rather than using only simple translations.The authors state that raw image pairs alone were not enough for their approach.Source: A. Design Motivation · p. 2
  • An autoencoder trained to reconstruct the paired raw image learned zero vectors without extra optimization constraints, suggesting the mapping needed more information.That alternative shared the proposed model’s encoding layers and used deconvolution and unpooling to reconstruct the other image.Source: A. Design Motivation · p. 3
  • HOG supplies geometric information and some illumination robustness, while randomized projective transformations provide viewpoint variation; the authors did not add dynamic objects to training pairs.The authors note that HOG itself is not very robust to viewpoint changes and that adding dynamic-object occlusions could potentially improve robustness.Source: A. Design Motivation · p. 3

B. Network Architecture

The training network maps a raw image to a HOG descriptor through a compact convolutional autoencoder. Training data are generated from unlabeled images using randomized homographies, and deployment retains only the input and convolutional layers.

B. Network Architecture

  • Each training pair supplies one raw image as X1 and a HOG descriptor as X2, with X2 arranged as an N × D batch tensor.The raw-image tensor X1 has dimensions N × 120 × 160.Source: B. Network Architecture · p. 3
  • The training network reconstructs X2 from X1 using convolutional and fully connected layers with an ℓ2 loss; deployment retains X1 and the three convolutional layers.The model uses ReLU activations in convolutional layers and sigmoid activations in fully connected layers, and takes 139 MB of GPU memory.Source: B. Network Architecture · p. 3

C. Network Training

  • The model can be trained on unlabeled images from any scene, and training on the Places dataset’s more than 8 million images improved performance over smaller datasets.Places was originally designed for scene recognition, and its images were mostly unrelated to scenes a loop-closure algorithm might encounter.Source: C. Network Training · p. 3
  • Randomly selected corner points define a homography that warps each image into a different viewpoint, with point-selection bounds chosen to limit excessive distortion.The authors randomly choose which member of each pair enters the raw-image tensor to avoid bias from the warped images’ zoomed-in appearance.Source: C. Network Training · p. 4
  • The HOG target compresses a 19,200-pixel image to 3,648 dimensions, and the model was trained for roughly 42 epochs at a fixed learning rate of 9 × 10^-4.The authors note that this aggressively compressed HOG descriptor may not itself be very informative for place recognition, but helps the autoencoder learn an image encoding.Source: C. Network Training · p. 4

D. Online Use

Online use stores extracted descriptors in a database and queries for loop-closure candidates. The descriptors are compact enough for direct similarity comparisons without dimensionality reduction.

D. Online Use

  • The online system creates a database of extracted descriptors and uses linear search to retrieve loop-closure candidates.The paper reports no speedup from K-D trees for 1,064-dimensional vectors, even with approximate search.Source: D. Online Use · p. 4
  • Training pairs are formed by randomly selecting image points and warping the image so those points map to the warped image’s corners.Source: Figure 3 · p. 4
  • Compact descriptors allow direct similarity calculations without dimensionality reduction.Source: D. Online Use · p. 4
  • The descriptor database and querying method achieve faster-than-real-time querying with minimal memory usage.The authors suggest the lightweight model could speed up ConvNet-based place-recognition systems with competitive accuracy.Source: D. Online Use · p. 4

IV. EXPERIMENTAL RESULTS

The evaluation compares loop-closure descriptors using runtime for efficiency and precision-recall performance for effectiveness. On Alderley, the proposed method has the highest AUC and r value among the compared methods.

IV. EXPERIMENTAL RESULTS

  • Effectiveness is evaluated with precision-recall curves, using AUC and maximum recall at 100% precision (r), while runtime measures efficiency.Source: §IV · p. 4
  • On Alderley, the proposed method has the highest AUC and r value among the state-of-the-art algorithms.Source: Figure 5 · p. 5
  • Comparisons include an autoencoder, supervised LA, DBoW2, compressed AlexNet, and HOG; the proposed model uses training data separate from the test environments.The experiments use a common linear nearest-neighbor search to compare descriptors.Source: §IV · pp. 4–5

A. The Alderley Dataset

The proposed method performs strongly on Alderley and Gardens Point, with results spanning difficult weather, viewpoint, brightness, and nighttime conditions. The experiments also distinguish daytime performance from the sharper nighttime challenge.

A. The Alderley Dataset

  • Alderley pairs rainy-night and sunny-day sequences, and the evaluation uses their last 200 frames; Gardens Point comprises two daytime traversals and one nighttime traversal.Source: A. The Alderley Dataset · p. 5, B. The Gardens Point Dataset · p. 5
  • On Alderley, the proposed method achieves the highest AUC and r by large margins, despite having trained on another subset of the dataset.The other methods struggle with the substantial appearance differences.Source: A. The Alderley Dataset · p. 5

B. The Gardens Point Dataset

  • On Gardens Point at night, the method has the highest r and second-highest AUC, while DBoW2, HOG, and the autoencoder fail.The comparison avoids environment-specific training.Source: B. The Gardens Point Dataset · p. 5
  • In Gardens Point daytime sequences, the method, AlexNet, and LA perform comparably, the autoencoder is close, and DBoW2 falls short.HOG is less robust than the trained descriptor even on this easier dataset.Source: B. The Gardens Point Dataset · p. 5

C. The Nordland Dataset

The Nordland and Campus Loop experiments test seasonal change, weather, viewpoint, and dynamic-object variation. The proposed method outperforms competitors on Nordland and is comparatively most robust on the challenging Campus Loop dataset.

C. The Nordland Dataset

  • The Nordland test uses 5,357 frames from moving Winter and Spring sequences, which combine seasonal change with train-induced blur.The selected frames exclude tunnels and stopped trains to avoid unusable images and precision-recall biases.Source: C. The Nordland Dataset · pp. 5–6
  • On Nordland, the proposed model outperforms and the other methods, although trained on all but the final hour of each sequence, including the test data.Source: C. The Nordland Dataset · p. 6

D. Our Campus Loop Dataset

  • On Campus Loop, performance is worse overall than on the earlier datasets, but the proposed method is comparatively most robust; performs worse than the other deep methods and DBoW2 in AUC.Source: D. Our Campus Loop Dataset · p. 6
  • Because no existing dataset provided all the challenges, Campus Loop comprises two 100-image campus sequences captured nine days apart under contrasting snow, cloud, and sunshine conditions.Images also vary in perspective and contain dynamic objects.Source: D. Our Campus Loop Dataset · p. 6

E. Runtime Evaluation

Runtime tests find the proposed method fast for feature extraction and database querying, including on large databases. The comparisons cover KITTI and St. Lucia under specified hardware and benchmark conditions.

E. Runtime Evaluation

  • The timing comparison excludes Lopez-Antequera et al. because their matching library was not open-sourced, and excludes reduced HOG because its runtime was not relevant to the comparison.Source: E. Runtime Evaluation · p. 7
  • Runtime tests measure feature extraction from raw images and separate single-nearest-neighbor query times, using KITTI sequence 00 and DBoW2 and AlexNet as benchmarks.Table I reports times in milliseconds for a database of 4,541 images; tests used an i7-6700HQ CPU and GeForce GTX 960M GPU.Source: E. Runtime Evaluation · pp. 6–7, Table I · p. 7
  • Our method extracts features faster than the other methods with GPU forward passes, remains reasonably fast on CPU, and queries faster than DBoW2; query cost stays inexpensive at large database sizes.The variable-size database test uses St. Lucia, with more than 30,000 stereo pairs.Source: Figure 11 · p. 7, E. Runtime Evaluation · p. 7

F. Online Loop Closure

The authors test online loop closure on KITTI sequences 00 and 05 with a simple keyframe-query procedure. The method consistently closes loops using a threshold selected from unrelated ground-truth data.

F. Online Loop Closure

G. Integration into ConvNet-Based Place Recognition Systems

The paper reports broad gains in place-recognition effectiveness and runtime, and demonstrates integration into a landmark-based ConvNet system without further descriptor reduction.

G. Integration into ConvNet-Based Place Recognition Systems

V. CONCLUSIONS

  • The unsupervised model uses randomized projective transformations and HOG reconstruction, needs no labeled or specially curated training data, and can be fine-tuned or retrained as more data becomes available.Source: §V · p. 8
  • Across different datasets, the method generally outperforms state-of-the-art place-recognition benchmarks in precision-recall and runtime.Source: §V · p. 8
Loading 1805.07703v2…