PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast and Robust Teach-and-Repeat Navigation Using MixVPR Visual Place Recognition*

Václav Truhlařík, Tomáš Pivoňka, Libor Přeučil

arXiv:2610.09631v1cs.RO

TL;DR

Appearance-based teach-and-repeat systems using advanced visual place recognition can navigate changing environments, but two-stage learned systems have high computational requirements for real-time performance. MixTaR replaces the earlier visual place recognition method with MixVPR and a local-feature approach to horizontal-shift computation. Tests report comparable navigation accuracy to other systems and successful operation in indoor and outdoor experiments.

  • Problem

    Two-stage visual place recognition systems have high computational requirements for real-time performance, despite the value of robust localization in changing environments.

    Source: B. Teach-and-Repeat Using Visual Place… · p. 2

  • Method

    MixTaR modifies SSM-Nav2 [2] by using one-stage MixVPR and local features from its network to compute horizontal shifts.

    Source: A. System Architecture · p. 2

  • Results

    Real-world experiments found MixTaR’s path-following accuracy comparable to other systems, and it localized from an arbitrary trajectory point and completed the task.

    Source: D. Outdoor Experiments · p. 5, §V · p. 6

  • Takeaways & Limitations

    MixTaR’s speed and efficiency make robust teach-and-repeat navigation suitable for smaller mobile robots with limited hardware, such as UAVs.

    Source: §V · pp. 5–6

    Deployment on low-end hardware and extensive subsequent experiments remain future development steps for MixTaR.

    Source: §V · p. 6

Abstract

from arXiv · show

Teach-and-repeat navigation systems employing advanced visual place recognition techniques for localization exhibit key attributes for long-term mobile robot navigation, such as the ability to operate in unstructured and dynamic environments. However, existing solutions based on deep-learning techniques are computationally demanding, limiting their applicability. This work introduces a novel and efficient teach-and-repeat system built on the modern visual place recognition method MixVPR. Real-world testing demonstrated its ability to operate both indoors and outdoors, achieving robustness and navigation precision comparable to other state-of-the-art systems. In addition, its lower hardware requirements make it suitable for a wide range of robotic platforms and practical applications.

I. INTRODUCTION

The paper addresses computationally demanding visual place recognition in appearance-based teach-and-repeat navigation. It introduces MixTaR, which uses MixVPR and a new horizontal-shift computation approach, and evaluates the system in indoor and outdoor settings.

I. INTRODUCTION

  • Reliable navigation in unstructured, changing environments is important for long-term robot autonomy, while specialized sensors can be expensive and data processing computationally intensive.The introduction highlights efficient systems using a single forward-looking monocular camera as valuable for practical applications.Source: §I · p. 1
  • Teach-and-repeat systems first record a trajectory during a taught traversal, then autonomously follow it using current sensor data and information captured during teaching.Appearance-based systems use one-dimensional localization or tracking and steer reactively from the image shift between the current and corresponding stored image.Source: §I · p. 1
  • Deep-learning visual place recognition can operate in challenging lighting and low-structure environments, but its computational demands limit existing appearance-based systems.The paper identifies the long computation time of earlier systems as stemming from the high demands of their visual place recognition methods.Source: §I · p. 1
  • MixTaR replaces the earlier system’s visual place recognition method with one-stage MixVPR and introduces a new approach to horizontal-shift computation.The system is tested against state-of-the-art appearance-based teach-and-repeat systems indoors, outdoors, and on a dataset for testing appearance-based methods.Source: §I · p. 1
  • Robustness in marker-free monotonous environments and sharing taught visual information are described as useful for industrial and multi-robot applications.The paper gives autonomous warehouses as an example of an industrial setting.Source: §I · p. 1

II. RELATED WORK

The related-work section surveys position-based and appearance-based teach-and-repeat systems. It contrasts their sensing and localization approaches, then describes systems that use learned visual features to compute steering shifts.

A. Teach-and-Repeat Systems Overview

  • Position-based systems localize in space and require stereo cameras to create local 3D maps, including SURF-based systems that can follow multi-kilometer trajectories,.Later variants added multi-experience teaching or deep-learned local visual features [8].Source: A. Teach-and-Repeat Systems Overview · pp. 1–2
  • Appearance-based systems use stored images for trajectory tracking and reactive steering; BearNav mainly relies on odometry and uses SURF matching to correct drift.Earlier approaches included polygonal-trajectory systems and systems for general trajectories.Source: A. Teach-and-Repeat Systems Overview · p. 2
  • A Siamese-network method encodes compared images into 3D vectors and computes horizontal shift by cross-correlation.Related systems use it to compute shifts directly or support local-feature matching.Source: A. Teach-and-Repeat Systems Overview · p. 2

B. Teach-and-Repeat Using Visual Place Recognition

This section reviews teach-and-repeat systems using visual place recognition for localization. It explains the benefit of basic localization along a trajectory and the computational challenge posed by two-stage learned systems.

B. Teach-and-Repeat Using Visual Place Recognition

  • Tracking-based localization requires starting at the trajectory’s initial position, while basic localization can start mid-trajectory and recover from failures.This motivates using robust visual place recognition for localization along the taught path.Source: B. Teach-and-Repeat Using Visual Place… · p. 2
  • Deep-learning visual place recognition can work across day-night and seasonal changes, a property identified as important for long-term autonomy.Source: B. Teach-and-Repeat Using Visual Place… · p. 2
  • SSM-Nav combines two-stage SSM-VPR with particle-filter localization that fuses visual place recognition and wheel odometry.It computes steering shifts from matched local features and has operated indoors and outdoors, including challenging lighting and monolithic-surface environments.Source: B. Teach-and-Repeat Using Visual Place… · p. 2
  • Advanced two-stage visual place recognition systems have high computational requirements for real-time performance.The SSM-Nav2 re-ranking approach uses D2-Net features for re-ranking and horizontal-shift computation [2].Source: B. Teach-and-Repeat Using Visual Place… · p. 2

C. MixVPR

MixVPR is a one-stage learned visual place recognition system that creates global image descriptors for retrieval. MixTaR incorporates it into a teach-and-repeat architecture and uses local features for steering correction.

C. MixVPR

  • MixVPR performs visual place recognition in one stage rather than the usual filtering and re-ranking steps, with image representations suited to large databases and fast recognition.It is described as achieving performance close to current state-of-the-art systems.Source: C. MixVPR · p. 2
  • MixVPR uses a convolutional backbone and Feature Mixer to produce a normalized global image descriptor for similarity-based image search.The network processes feature maps with multi-layer perceptron blocks before reducing and flattening them into the descriptor.Source: C. MixVPR · p. 2

A. System Architecture

  • MixTaR modifies SSM-Nav2 [2] by replacing its two-stage visual place recognition with MixVPR and using local features from MixVPR’s inner layers to compute horizontal shifts.The system is primarily designed for a ground robot with a single forward-looking monocular camera.Source: A. System Architecture · p. 2
  • During repeating, global descriptors localize the robot along the taught trajectory, while a particle filter fuses visual localization with traveled distance to estimate its position.Local descriptors determine horizontal shift correction, which is converted into a reactive control command.Source: A. System Architecture · p. 2

B. Teaching Phase

During teaching, MixTaR samples and stores images and MixVPR descriptors along the desired path. During repeat, it combines visual localization with odometry to estimate the robot’s position along that path.

B. Teaching Phase

  • Image sampling becomes denser on turns and sparser on straight segments, reducing database images and improving localization speed.Sampling distance is adjusted according to angular velocity.Source: B. Teaching Phase · p. 3
  • MixVPR produces a 4096-dimensional global descriptor, while normalized columns of its backbone output tensor serve as local visual features.Images are resized to 320 x 320 px; the backbone output has shape 20x20x1024.Source: B. Teaching Phase · p. 3

C. Localization and Position Tracking

  • A particle filter combines visual localization with traveled distance from odometry, returning the average position of its five best particles.Visual matches increase the weights of nearby particles; odometry updates particle positions as the robot moves.Source: C. Localization and Position Tracking · p. 3

D. Horizontal Shift Computation

MixTaR computes horizontal image shifts by aggregating coordinate differences from mutually matched local features. On the offline dataset, its shift estimates were more often correct than those of previous systems, with comparable standard deviations.

D. Horizontal Shift Computation

  • The system crossmatches local features, builds a histogram of matched-feature horizontal coordinate differences, and returns the most frequent shift.Fig. 2 shows matched features and the shift histogram; it displays only a fourth of all real matches.Source: D. Horizontal Shift Computation · p. 3, Figure 2 · p. 4

A. Offline Testing

  • Offline testing compared MixTaR with SSM-Nav2, SSM-Nav, and two BearNav2 variants across outdoor daylight, outdoor night, and indoor sequences.A shift counted as correct when it fell within the ground-truth interval defined by projections of points at 5 meters and infinity.Source: A. Offline Testing · pp. 3–4
  • MixTaR outperformed all previous systems in shift-estimation correctness, with standard deviations comparable to other systems.Table I covers daytime, nighttime, and indoor sequences.Source: A. Offline Testing · p. 4

B. Hardware and System Setup

Real-world testing used a Clearpath Husky A200 with a monocular RGB camera and external computing hardware. The software setup supported running multiple models in parallel during teaching.

B. Hardware and System Setup

  • The real-world platform was a four-wheeled differential-drive Clearpath Husky A200 robot.It was shown during an outdoor experiment.Source: B. Hardware and System Setup · p. 4, Figure 3 · p. 4
  • Table I covers shift-computation testing on daytime, nighttime, and indoor dataset sequences for teach-and-repeat systems.Source: Table I · p. 4
  • A front-mounted RealSense D435 camera ran in single-RGB monocular mode, paired with an external computer containing a GTX 1070 GPU and Core i5-7300HQ processor.The computer had 32 GB RAM and a 500 GB SSD.Source: B. Hardware and System Setup · p. 4
  • The ROS2 Humble and Ubuntu 20.04 software platform ran multiple models in parallel during teaching so the user could select one for repeat navigation.Individual ROS nodes were developed in Python.Source: B. Hardware and System Setup · p. 4

C. Indoor Experiments

In indoor experiments, MixTaR was slightly less accurate than the two reference systems but completed its navigation tasks significantly faster than SSM-Nav2. It also followed shifted-start and low-light trajectories, and its ability to start from arbitrary points was tested.

C. Indoor Experiments

  • MixTaR had slightly worse navigation accuracy than both reference systems, but completed appearance-based T&R tasks significantly faster than SSM-Nav2.Computational times covered feature extraction, VPR candidate detection, and shift computation during repeat-phase driving on the advanced path with more than a hundred images.Source: C. Indoor Experiments · pp. 4–5, Figure 4 · p. 5, Table II · p. 5
  • All systems followed the taught trajectory from a starting position shifted 30 cm left; MixTaR was also tested for recovery after sudden lateral and rotational displacements.The displacement test simulated a kidnapping scenario, with the recorded path shown in Fig. 5.Source: C. Indoor Experiments · pp. 4–5, Figure 5 · p. 5
  • Under limited lighting from outside corridors, all systems traversed both the basic and advanced paths while maintaining mean-deviation performance.Source: C. Indoor Experiments · p. 5
  • BearNav’s higher precision primarily came from repeating taught velocities, but unlike SSM-Nav2 and MixTaR, it could not localize from an arbitrary point along the trajectory.That arbitrary-start capability was experimentally verified for SSM-Nav2 and MixTaR.Source: C. Indoor Experiments · p. 5

D. Outdoor Experiments

Outdoors, MixTaR and SSM-Nav2 both navigated a 150 m campus trajectory reliably, including while people crossed the camera view. MixTaR had the lower mean deviation at seven measurement points.

D. Outdoor Experiments

V. CONCLUSIONS

MixTaR combines MixVPR localization with a new horizontal-shift approach, and it outperformed other state-of-the-art methods on successful shift calculation. Real-world experiments showed comparable navigation accuracy, while the system ran faster than other appearance-based T&R systems of this type.

V. CONCLUSIONS

  • MixTaR reliably navigated challenging environments with moving people or variable lighting and ran faster than other appearance-based T&R systems of this type.Source: §V · p. 5
  • MixTaR outperformed other state-of-the-art approaches in the percentage of successfully calculated shifts on a novel horizontal-shift dataset.Source: §V · pp. 5–6
  • Real-world experiments showed navigation accuracy comparable to other systems, and MixTaR could localize from an arbitrary trajectory point and complete the task.Source: §V · p. 6
  • Its speed and efficiency make robust T&R navigation suitable for smaller robots with limited hardware, such as UAVs; low-end deployment and extensive experiments remain future steps.Source: §V · p. 6
Loading 2610.09631v1…