PaperlayerView exact revision on arXiv

Source-linked AI summary

ViNG: Learning Open-World Navigation with Visual Goals

Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, Sergey Levine

arXiv:2012.09812v2cs.ROcs.AIcs.LG

TL;DR

Visual navigation must cope with unstructured environments, high-dimensional observations, and goals that are difficult to specify without spatial information. ViNG combines learned dynamical distance with graph search over prior image observations to navigate toward a goal image using offline data. It outperforms several competitive offline RL and geometric baselines, while its reported comparisons also include parity with an online method that uses more data.

  • Problem

    Visual navigation must handle complex dynamics and high-dimensional observations, while real-world data collection has cost and safety constraints and location goals are difficult to specify without spatial information.

    Source: §I · p. 1

  • Method

    ViNG learns dynamical distance between observations, embeds past observations in a topological graph, and plans over it toward a visually specified goal using offline real-world data.

    Source: §I · pp. 1–2

  • Results

    ViNG outperforms several competitive offline RL and geometric baselines, reaches 86% success on the most challenging tasks, and performs on par with online SoRB using 40 hours of offline data versus SoRB's 200 hours online.

    Source: §I · p. 2, A. Goal-Conditioned Visual Navigation… · p. 5, C. Comparisons to Online Methods · p. 6

  • Takeaways & Limitations

    ViNG is demonstrated for last-mile delivery and warehouse inspection, and transfers to novel environments with as little as 20 minutes of data.

    Source: §I · p. 2

Abstract

from arXiv · show

We propose a learning-based navigation system for reaching visually indicated goals and demonstrate this system on a real mobile robot platform. Learning provides an appealing alternative to conventional methods for robotic navigation: instead of reasoning about environments in terms of geometry and maps, learning can enable a robot to learn about navigational affordances, understand what types of obstacles are traversable (e.g., tall grass) or not (e.g., walls), and generalize over patterns in the environment. However, unlike conventional planning algorithms, it is harder to change the goal for a learned policy during deployment. We propose a method for learning to navigate towards a goal image of the desired destination. By combining a learned policy with a topological graph constructed out of previously observed data, our system can determine how to reach this visually indicated goal even in the presence of variable appearance and lighting. Three key insights, waypoint proposal, graph pruning and negative mining, enable our method to learn to navigate in real-world environments using only offline data, a setting where prior methods struggle. We instantiate our method on a real outdoor ground robot and show that our system, which we call ViNG, outperforms previously-proposed methods for goal-conditioned reinforcement learning, including other methods that incorporate reinforcement learning and search. We also study how \sysName generalizes to unseen environments and evaluate its ability to adapt to such an environment with growing experience. Finally, we demonstrate ViNG on a number of real-world applications, such as last-mile delivery and warehouse inspection. We encourage the reader to visit the project website for videos of our experiments and demonstrations sites.google.com/view/ving-robot.

I. INTRODUCTION

Visual navigation must handle complex dynamics, high-dimensional observations, offline-data constraints, and goals specified without spatial position. ViNG addresses these challenges by planning over a learned graph of image observations, using offline real-world data and design choices intended to overcome assumptions in earlier planning-and-learning methods.

I. INTRODUCTION

  • Visual navigation is challenging because environments have complex dynamics, images are high-dimensional, data collection has cost and safety constraints, and goals are difficult to specify without spatial information.The introduction also notes the challenge of generalizing across settings and environments.Source: §I · p. 1
  • ViNG learns goal-directed navigation from offline data without spatial maps, combining learned dynamical distance with graph search over prior observations.The system targets visually specified goals in open-world settings, including urban, grassy, and rocky terrain.Source: §I · pp. 1–2

II. RELATED WORK

  • Prior planning-and-learning methods rely on assumptions such as exact simulation, simplified action spaces, or online data collection, which limit real-world applicability.ViNG instead uses offline real-world data and does not require a simulator or interactive access to an environment.Source: §II · p. 2
  • ViNG adds graph pruning and negative sampling to prior planning-and-learning approaches and operates on images with continuous robot actions.The paper describes it as a real-world ground-robot system for long-horizon visual goal navigation without simulated training or hand-designed localization and mapping.Source: §II · p. 2

III. PROBLEM STATEMENT AND SYSTEM OVERVIEW

The task is to reach a location from an image captured there, recognize arrival, and operate without a spatial map using previously collected trajectories. ViNG runs on an outdoor ground robot but bases its method on onboard camera images.

III. PROBLEM STATEMENT AND SYSTEM OVERVIEW

  • The robot navigates to a goal location specified by an image, recognizes when it arrives, and uses prior trajectories without a spatial map or state estimation.Trajectories may come from teleoperation, self-exploration, or random walks; the system commands continuous linear and angular velocities.Source: §III · p. 2

A. Mobile Robot Platform

  • ViNG uses a Clearpath Jackal UGV with an onboard camera; GPS and the laser scanner serve only as safety mechanisms during data collection.The robot also carries an IMU, wheel encoders, and an NVIDIA Jetson TX2 computer.Source: A. Mobile Robot Platform · pp. 2–3

B. Data Collection & Labeling

ViNG learns from previously collected experience and combines learned navigation functions with a topological graph to reach visual goals. Its design targets offline, real-world navigation, where graph pruning and negative mining are key technical insights.

B. Data Collection & Labeling

  • Core experiments use offline data collected more than 10 months before evaluation, despite differences in appearance, lighting, season, and time of day.The paper also collected limited additional data for environment-generalization experiments.Source: B. Data Collection & Labeling · p. 3

IV. VISUAL NAVIGATION WITH GOALS

  • ViNG trains an environment-independent traversability function and a relative pose predictor from previously collected experience, then deploys them with past observations.The method is organized into training learned functions and deploying them with prior experience for goal-directed behavior.Source: §IV · p. 3
  • At deployment, ViNG builds a topological graph whose observation nodes and traversability-and-proximity edges support graph search for the next controller waypoint.The robot localizes its current and goal observations in the graph and follows the selected path progressively.Source: §IV · p. 3
  • Graph pruning and negative mining are identified as key technical insights for learning goal-conditioned policies entirely from offline data, where prior methods struggle especially on long-horizon goals.The paper says these insights improve real-world performance and avoid simulators and online sampling.Source: §IV · p. 3

A. Learning Dynamical Distances

ViNG learns dynamical distance as the estimated number of controller steps between observations. The paper compares supervised regression with temporal-difference learning and reports little difference between them in its experiments.

A. Learning Dynamical Distances

  • Dynamical distance estimates the number of controller time steps needed to navigate between two observations, capturing traversability beyond geometry.The paper gives compliant grass versus bushes as an example of visually similar but differently traversable terrain.Source: A. Learning Dynamical Distances · p. 3
  • The supervised approach regresses elapsed steps between observations from the same trajectory, but may overestimate the true shortest-path distance.The labels use the difference in trajectory indices as the elapsed timestep count.Source: A. Learning Dynamical Distances · p. 3
  • Temporal-difference learning theoretically converges to shortest-path distance, but showed little experimental difference from supervised learning; the authors expect it to matter when shortest paths are much shorter than random walks.The temporal-difference approach adds complexity.Source: A. Learning Dynamical Distances · p. 3

1) Negative Mining (Key Idea 1):

Training the distance function only on same-trajectory observation pairs performed poorly, which the authors hypothesize reflects distribution shift. They add cross-trajectory negative examples, which they identify as critical to successful training and evaluation.

1) Negative Mining (Key Idea 1):

  • Training distance prediction only on same-trajectory observation pairs performed poorly, which the authors hypothesize was caused by distribution shift at graph construction.Graph construction evaluates pairs from different trajectories and potentially different times of day.Source: 1) Negative Mining (Key Idea 1) · p. 3
  • Negative sampling adds observation pairs from different trajectories and labels their distance as dmax, addressing the mismatch encountered during graph construction.The positive training pairs are drawn from the same trajectory.Source: 1) Negative Mining (Key Idea 1) · p. 3
  • The authors identify negative-sampling augmentation as critical to successful distance-function training and evaluation, with significant improvements over prior methods.The paper links this improvement to its offline-data setting.Source: 1) Negative Mining (Key Idea 1) · pp. 3–4

B. The Topological Graph

ViNG uses a learned-distance topological graph to plan toward visual goals and a relative-pose controller to follow waypoints. In real-world tests, only ViNG reaches goals over 40 meters away.

B. The Topological Graph

  • The topological graph uses previously observed observations as nodes and learned-distance predictions as weighted edges.The graph represents traversability between observations.Source: B. The Topological Graph · p. 4

1) Graph Pruning (Key Idea 2):

  • Graph pruning omits easily traversable edges below a threshold because the controller can traverse them with high probability, limiting quadratic graph growth.The method sparsifies trajectories by thresholding which edges are added.Source: 1) Graph Pruning (Key Idea 2) · p. 4

2) Planning with the Graph:

  • The planner localizes current and goal observations, uses weighted Dijkstra search, and passes the next node on the planned path to the controller.The current and goal observations are connected to their most-traversable graph neighbors.Source: 2) Planning with the Graph · p. 4

C. Designing the Controller

  • A learned relative-pose predictor estimates the waypoint's pose from two observations, and odometry with a PD controller steers the robot toward it.The predictor is supervised using relative poses estimated from onboard odometry for nearby trajectory pairs.Source: C. Designing the Controller · p. 4
  • Only ViNG reaches goals over 40 meters away, although all non-random methods reach nearby goals.Success rate is the average portion of the expert trajectory to the goal that each run completes.Source: Figure 3 · p. 4

D. Implementation Details

ViNG’s implementation uses shared visual encoders for traversability and relative-pose prediction, trained with separate objectives. The experiments assess offline navigation, adaptation to new environments, and controller design choices.

D. Implementation Details

  • Traversability and relative-pose models use a MobileNet encoder followed by three dense layers; the two models share the encoder.The traversability model projects 1024-dimensional latents to 50 class labels, while the pose predictor projects them to waypoint outputs.Source: D. Implementation Details · p. 4
  • Traversability is trained on positive and negative pairs with cross-entropy, while the pose predictor uses positive pairs and an ℓ2 regression loss.Training uses batches of 128 and Adam with a learning rate of 10^-4.Source: D. Implementation Details · p. 4

V. EXPERIMENTS

  • The experiments ask whether ViNG works from offline data, generalizes and adapts to novel environments, and benefits from its controller design.The controller alternatives are compared with the approach in Section IV-C.Source: §V · p. 4

A. Goal-Conditioned Visual Navigation from Offline Data

In the original outdoor environment, ViNG is evaluated against offline navigation baselines using previously collected data. The section also tests adaptation to new environments and compares ViNG-Finetune with alternative models.

A. Goal-Conditioned Visual Navigation from Offline Data

  • The evaluation uses 40 hours of previously collected data gathered over 10 months, with substantial seasonal appearance variation.The real-world evaluation compares ViNG with SPTM, off-SoRB, a state-estimator baseline, and random navigation; the state-estimator baseline has GPS access.Source: A. Goal-Conditioned Visual Navigation… · pp. 4–5
  • 86% success on the most challenging tasks; ViNG performs substantially better across goal distances, especially beyond 30 meters.The evaluation reports the average portion of the expert trajectory to the goal completed across runs.Source: A. Goal-Conditioned Visual Navigation… · p. 5

B. Generalization and Adaptation

  • For urban goals about 50 meters away, ViNG alone consistently reaches the goal across three runs while avoiding collisions or getting stuck.Source: Figure 4 · p. 5
  • ViNG-Finetune adapts in four new environments by combining initial human-controlled exploration with autonomous goal-reaching experience to finetune its models.The finetuning data includes both expert and self-collected trajectories.Source: B. Generalization and Adaptation · p. 5
  • After 60 minutes of practice, ViNG reaches the goal in all three attempts in the new domain; performance improves with more autonomous practice.Across the four new environments, the robot reaches the goal in most cases after 60 minutes of data collection.Source: B. Generalization and Adaptation · p. 5, Figure 5 · p. 6
  • ViNG-Finetune outperforms source-only, target-only, and SPTM-Finetune baselines on generalization success.Table I summarizes success in four new environments using 60 minutes of experience in each new environment.Source: B. Generalization and Adaptation · p. 5, Table I · p. 6

C. Comparisons to Online Methods

The simulated comparison finds ViNG competitive with online SoRB and more successful than offline baselines at distance goals. The section also presents adaptation in a novel environment using practice with self-proposed goals.

C. Comparisons to Online Methods

  • The online comparison uses a Unity-based photorealistic simulator because online reinforcement-learning sample requirements preclude testing those methods in the real world.Source: C. Comparisons to Online Methods · p. 6
  • With 60 minutes of experience in a novel environment, ViNG gains competence at distance goals through practice on self-proposed goals and controller finetuning.The figure shows example rollouts toward a goal 35 meters away.Source: Figure 6 · p. 6
  • ViNG performs competitively with SoRB and is substantially more successful than all offline baselines at reaching distance goals in simulation.SoRB uses 200 hours of online data, compared with ViNG’s 40 hours of offline data; SoRB must recollect its data for each experiment.Source: Figure 7 · p. 6, C. Comparisons to Online Methods · p. 6

D. Ablation Experiments

The ablations examine controller choices and the contribution of graph pruning and negative sampling. Qualitative demonstrations show ViNG performing visually specified delivery and inspection tasks.

D. Ablation Experiments

  • Waypoint-based control is key to ViNG’s performance, particularly for distant goals; graph pruning and negative sampling are essential to its real-world performance.The ablations also report similar performance with supervised or TD learning for the traversability model.Source: D. Ablation Experiments · p. 6, Table II · p. 7

E. Applications and Qualitative Results

VI. CONCLUSION

ViNG uses visual observations and goals with a pruned topological graph and relative-pose prediction to navigate using offline experience. The conclusion highlights quick adaptation to new environments and real-world applications, while noting dependence on a static offline dataset amid shifting observations.

VI. CONCLUSION

  • Graph pruning, relative-pose prediction, and negative mining support successful navigation using only offline experience.The conclusion contrasts this setting with prior methods that often fail when learning from offline experience.Source: §VI · p. 7
  • ViNG can be quickly adapted to navigate in new environments.The authors connect generalization and self-improvement to adaptation to novel environments.Source: §VI · p. 7
  • Demonstrations include contactless last-mile delivery and autonomous inspection in dense urban environments that may be unmapped or GPS-denied.Source: §VI · p. 7
  • ViNG requires a static offline observation dataset, while lighting and dynamic-object shifts motivate future work on more robust observation and goal representations.The authors identify distributional shifts as a constraint on the method's generalization capabilities.Source: §VI · p. 7
Loading 2012.09812v2…