Source-linked AI summary
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System
Gregory Kahn, Pieter Abbeel, Sergey Levine
TL;DR
Purely geometric navigation can fail when geometry does not reveal traversability or terrain preferences. BADGR learns these physical affordances from self-supervised off-policy experience, predicting future events to plan actions. In real-world urban and off-road environments, it navigates around geometrically distracting obstacles, incorporates terrain preferences, generalizes to novel environments, and improves with more data.
Problem
Purely geometric navigation may treat traversable terrain such as tall grass as an obstacle and cannot fully represent physical properties such as bumpiness or surface preference.
Method
BADGR trains an end-to-end predictive navigation system with self-supervised off-policy data gathered in real-world environments, using observations and future actions to predict relevant events for planning.
Results
BADGR navigated real-world urban and off-road environments with geometrically distracting obstacles, incorporated terrain preferences, generalized to novel environments, and improved as it gathered more data.
Takeaways & Limitations
Physical navigational affordances can be learned from autonomous experience, allowing one system to reason about both geometric and non-geometric objectives without human supervision.
Abstract
from arXiv · showhide
Mobile robot navigation is typically regarded as a geometric problem, in which the robot's objective is to perceive the geometry of the environment in order to plan collision-free paths towards a desired goal. However, a purely geometric view of the world can can be insufficient for many navigation problems. For example, a robot navigating based on geometry may avoid a field of tall grass because it believes it is untraversable, and will therefore fail to reach its desired goal. In this work, we investigate how to move beyond these purely geometric-based approaches using a method that learns about physical navigational affordances from experience. Our approach, which we call BADGR, is an end-to-end learning-based mobile robot navigation system that can be trained with self-supervised off-policy data gathered in real-world environments, without any simulation or human supervision. BADGR can navigate in real-world urban and off-road environments with geometrically distracting obstacles. It can also incorporate terrain preferences, generalize to novel environments, and continue to improve autonomously by gathering more data. Videos, code, and other supplemental material are available on our website https://sites.google.com/view/badgr
I. INTRODUCTION
BADGR addresses the limits of purely geometric navigation by learning physical navigational affordances from the robot’s own experience. It uses self-supervised off-policy data to predict relevant future events and plan navigation without human labeling or simulation.
- Motivation: Geometric navigation can misclassify traversable obstacles such as tall grass and overlook preferences between visually similar surfaces.Pure geometry may therefore fail in open-world environments where traversability and terrain quality matter.
- Motivation: The robot can retrospectively label experience using measurable outcomes such as collisions and terrain bumpiness, forming a self-supervised multi-task reinforcement learning problem.These labels train predictions of which future action sequences may lead to undesirable or preferred events.
- BADGR: BADGR is an end-to-end, fully autonomous navigation system that gathers real-world off-policy data and predicts future events from sensor readings and executed actions.The system uses these predictions to avoid events such as collisions and seek events such as smooth terrain.
- BADGR: BADGR combines geometric and non-geometric navigation objectives without human labeling, expensive sensors, or simulated data, while improving through autonomous data collection.The complete system is presented as the first to combine these capabilities using the described components.
- Results: Using 42 hours of autonomously collected data, BADGR navigates environments with geometrically distracting obstacles, incorporates terrain preferences, generalizes to novel environments, and outperforms a LIDAR policy in complex settings.The reported experiments also show continued improvement as more data are gathered.
II. RELATED WORK
Prior navigation systems largely rely on mapping and geometric planning, while learning-based approaches address geometry or semantics with important limitations. BADGR instead learns geometric and non-geometric navigational objectives from off-policy experience without human supervision.
- Geometric and semantic methods: The predominant navigation approach builds a map, localizes the robot, and plans actions through simultaneous localization and mapping.This approach has achieved strong results across indoor and outdoor scenarios.
- Geometric and semantic methods: Methods that estimate scene geometry remain limited because geometry provides only a partial description of the environment.A geometric model can incorrectly treat tall grass as untraversable.
- Geometric and semantic methods: Semantic approaches associate sensory data with labels such as traversable or bumpy terrain, but typically depend on SLAM systems or human-provided labels.These dependencies limit how such methods address purely geometric navigation.
- Comparison: Compared with GCG and CAPs, BADGR learns both geometric and non-geometric navigation without human supervision in complex real-world environments.GCG learned only collision avoidance, whereas CAPs required human supervision for noncollision behaviors.
III. BERKELEY AUTONOMOUS DRIVING GROUND ROBOT
BADGR autonomously collects and labels off-policy experience, learns to predict future navigational events, and plans actions according to a user-defined reward. Its self-supervision depends on measurable events the robot has experienced.
- System overview: BADGR gathers off-policy real-world data, labels events such as collisions and bumpy terrain, and adds those labels back into its dataset.The resulting dataset supports predictive learning without human annotation.
- System overview: Its predictive model takes current observations and future action sequences as input and predicts relevant future events.Observations can include camera images, while actions can include linear and angular velocity commands.
- Planning: A user-defined reward function expresses the navigation task through relevant events, and BADGR plans and executes actions that maximize that reward.Example objectives include reaching a goal while avoiding collisions and bumpy terrain.
- Assumptions: The method assumes the robot can learn only from events it has experienced and measured with onboard sensors, and that experiencing even undesirable events is acceptable.This retrospective supervision assumption includes collisions and other measurable outcomes.
- System overview: The system description covers the robot, data collection and labeling, model training, and planning components before summarizing the complete system.
A. Mobile Robot Platform
BADGR uses a robust Clearpath Jackal platform equipped for long-term autonomous operation in urban and off-road environments. Its onboard computing, storage, communication, and sensing support large-scale data collection.
- Mobile Robot Platform: The platform design prioritizes long-term autonomy with minimal human intervention.The robot was designed for robustness and large-scale data collection.
- Mobile Robot Platform: The Clearpath Jackal measures 508mm × 430mm × 250mm and weighs 17kg, supporting navigation in urban and off-road environments.It uses differential-drive control through desired linear and angular velocity setpoints.
- Sensors: The sensor suite includes a 6-DOF IMU, GPS, wheel encoders, two forward-facing 170° field-of-view 640 × 480 cameras, 2D LIDAR, and a compass.
- Compute and data: An NVIDIA Jetson TX2 runs deep-learning applications and interfaces with the sensors and low-level microcontrollers.Sensor data are stored on an external SSD, while a mounted 4G smartphone enables remote monitoring and possible teleoperation.
B. Data Collection
BADGR gathers diverse real-world off-policy data with a time-correlated random policy and automated recovery, while catastrophic failures can still require manual resets.
- Data collection policy: BADGR uses off-policy data collection so previously gathered data remains usable for training.The approach avoids repeatedly discarding older data during policy retraining.
- Data collection policy: A time-correlated random walk improves exploration and produces action sequences more realistic for test-time behavior than uniform random control.Uniform random commands would primarily drive the robot straight because of its linear and angular velocity interface.
- Automated recovery: The collection system detects collisions or stuck states and uses an automated backup-and-rotation reset to continue gathering data.Collision detection uses LIDAR in urban environments and IMU measurements in off-road settings.
- Automated recovery: Manual intervention remains necessary when automated recovery fails, such as after the robot flips over.The robot usually recovers automatically, but catastrophic failures periodically require human resets.
- Data storage: The robot saves raw sensory data onboard and later subsamples it to 4Hz for processing.Data is copied to a desktop machine after daily collection before subsampling.
C. Self-Supervised Data Labelling
BADGR retrospectively derives navigation-event labels from collected sensor data, then adds those labels to the dataset for predictive-model training.
- Event labeling: BADGR extracts self-supervised labels for navigation-relevant events from raw, subsampled sensory data.The events are selected because they can be obtained from the collected data without external annotation.
- Event labeling: The experiments label three events: collision, bumpiness, and position.Collision detection differs by environment, bumpiness uses IMU angular-velocity thresholds, and position comes from onboard measurements.
- Event labeling: After labeling each timestep, BADGR inserts the event labels into the dataset and trains a model to predict which actions lead to them.This creates the supervision used for action-conditioned future-event prediction.
D. Predictive Model
BADGR learns an image-based, action-conditioned predictive network that maps current observations and future action sequences to multiple future navigational events.
- Model inputs and outputs: The predictive model takes the current sensor observation and H future actions as input, producing predictions for K future events.The function is parameterized by θ and predicts events across the action horizon.
- Network architecture: The network processes image observations with convolutional and fully connected layers before recurrently processing each future action.The recurrent network outputs predicted events corresponding to the sequential future actions.
- Training: Training uses collected observations, actions, and event labels to penalize discrepancies between predicted and ground-truth events.The model is optimized using a loss function over the collected dataset.
- Training: Individual event losses use cross entropy for discrete events and mean squared error for continuous events.The network parameters are trained with minibatch gradient descent.
E. Planning
BADGR plans by scoring predicted future events under a user-defined reward, optimizing sampled action sequences, and executing the first action in a model-predictive-control loop.
- Reward and objective: A reward function translates desired and undesirable predicted events into the task objective, such as reaching a goal while avoiding collisions and bumps.The reward is evaluated on predicted future events from the learned model.
- Reward and objective: At each timestep, BADGR solves for the future action sequence that maximizes the reward predicted by its learned model.The resulting action sequence is used within a receding-horizon planning procedure.
- Stochastic optimization: The planner samples N time-correlated action sequences centered on a running estimate of the optimal sequence.The sampling distribution is controlled by parameters governing noise scale and temporal correlation.
- Stochastic optimization: It propagates each candidate through the predictive model, computes rewards, and updates the running estimate with a reward-weighted average.The parameter γ controls how strongly high-reward sequences are weighted.
- Execution: The robot executes only the first action before replanning, while the optimizer warm-starts from the previous solution and exploits temporal correlations.The authors report that this optimizer was necessary for good planning in their experiments.
F. Algorithm Summary
BADGR trains a predictive model from autonomously collected data and uses it for receding-horizon planning. Training retrospectively adds self-supervised event labels, while deployment alternates planning with executing the first planned action.
- Training: During training, BADGR records sensor observations and executed actions, retrospectively computes self-supervised event labels, and adds them to the dataset.The predictive model is then trained on the augmented dataset by minimizing the training objective.
- Deployment: At deployment, a user-defined reward function specifies the navigation task, and BADGR plans action sequences that maximize predicted reward.The plan uses the trained predictive model and the robot’s current sensor observation.
- Deployment: BADGR repeatedly executes only the first action of each planned sequence, then replans until the task is complete.This produces an alternating planning-and-execution procedure.
- Training: BADGR gathers training data by executing actions from a data-collection policy and resetting after collisions when necessary.
IV. EXPERIMENTS
Experiments evaluate BADGR against geometric and naïve baselines in urban and off-road settings, including terrain preference, collision avoidance, adaptation, and generalization. BADGR learns physically relevant navigation behavior, improves with additional data, and navigates unseen environments.
- Experimental setup: BADGR was evaluated in real-world urban and off-road terrain using 34 hours of urban data and 8 hours of off-road data.The dataset contained 720,000 off-policy datapoints.
- Baselines: BADGR was compared with a 2D-LIDAR policy and a naïve policy that drives straight toward the specified goal.The LIDAR policy represents a geometric baseline, while the naïve policy provides a lower-bound baseline.
- Urban environment: BADGR learned to predict collision and bumpiness outcomes for candidate action sequences, including collisions near buildings or bushes and smoother travel on concrete than grass.
- Urban environment: In the urban environment, BADGR always reached the goal while avoiding bumpy terrain by learning that concrete paths are smoother than grass.The LIDAR policy reached the goal but failed to avoid bumpy grass terrain, while the naïve policy almost always crashed.
- Off-road environment: In the off-road environment, BADGR almost always reached the goal while avoiding collisions and getting stuck, without falsely treating all grass as an obstacle.The LIDAR policy rarely crashed or became stuck but sometimes refused to move because it labeled grass as untraversable.
- Off-road environment: BADGR reached the goal 1.2× faster on average than the LIDAR policy by treating some grass as traversable rather than taking a roundabout path.
- Adaptation: After three additional hours of self-supervised target-domain data, the finetuned policy succeeded on every trial, whereas the zero-shot policy failed on every trial due to collision.A target-domain-only model performed better than zero-shot but failed more frequently than the finetuned model using both data sources.
- Generalization: With the full 42-hour dataset, BADGR successfully navigated three novel environments ranging from forest to urban buildings.The authors state that generalization requires a sufficiently large and diverse dataset.
V. DISCUSSION
BADGR navigates complex real-world environments, incorporates terrain preferences, generalizes to novel environments, and improves with additional data. However, deployment still requires periodic human assistance, substantial new data and retraining for improvement, and has only been evaluated in static environments.
- BADGR can outperform a LIDAR policy in complex real-world settings and generalize to novel environments.
- Using 42 hours of data, BADGR can navigate geometrically distracting obstacles and incorporate preferences such as avoiding bumpy terrain.
- BADGR improves as it gathers more data, supporting adaptation beyond its initial training experience.
- Periodic human assistance remains necessary, for example when the robot flips over, creating a deployment burden in remote locations.
- Further improvement requires a non-negligible amount of additional data and retraining from scratch, while experiments covered only static environments.