Source-linked AI summary
Combining Optimal Control and Learning for Visual Navigation in Novel Environments
Somil Bansal, Varun Tolani, Saurabh Gupta, Jitendra Malik, Claire Tomlin
TL;DR
Robot navigation in unknown environments is difficult when model-based control lacks a known map and onboard sensors provide only partial observations. The paper couples RGB-based waypoint prediction with model-based planning and feedback control, reporting more reliable and efficient navigation than geometric mapping-based or end-to-end alternatives, including transfer from simulation to the real world.
Problem
Navigation in a priori unknown environments is difficult because model-based methods must operate with partial onboard observations, while mapping and end-to-end alternatives have important limitations.
Method
LB-WayPtNav predicts collision-free waypoints from RGB observations and uses model-based optimal control to generate smooth, dynamically feasible trajectories executed with feedback control.
Results
The approach reaches goals more reliably and efficiently than geometric mapping-based and end-to-end alternatives, and transfers from simulation to unseen real-world environments without real-world finetuning or data collection.
Takeaways & Limitations
Factorizing high-level navigation decisions and low-level control supports navigation in novel cluttered environments without relying on detailed explicit 3D maps.
Takeaways & Limitations
The method assumes perfect state estimation and a static environment, and appearance differences between simulation and reality can produce poor waypoint predictions.
Abstract
from arXiv · showhide
Model-based control is a popular paradigm for robot navigation because it can leverage a known dynamics model to efficiently plan robust robot trajectories. However, it is challenging to use model-based methods in settings where the environment is a priori unknown and can only be observed partially through on-board sensors on the robot. In this work, we address this short-coming by coupling model-based control with learning-based perception. The learning-based perception module produces a series of waypoints that guide the robot to the goal via a collision-free path. These waypoints are used by a model-based planner to generate a smooth and dynamically feasible trajectory that is executed on the physical system using feedback control. Our experiments in simulated real-world cluttered environments and on an actual ground vehicle demonstrate that the proposed approach can reach goal locations more reliably and efficiently in novel environments as compared to purely geometric mapping-based or end-to-end learning-based alternatives. Our approach does not rely on detailed explicit 3D maps of the environment, works well with low frame rates, and generalizes well from simulation to the real world. Videos describing our approach and experiments are available on the project website.
1 Introduction
LB-WayPtNav combines learning-based perception with model-based optimal control for navigation in novel, cluttered environments. It uses RGB observations to predict waypoints, then generates smooth trajectories and aims to outperform mapping-based and end-to-end alternatives.
- Unknown-environment navigation is difficult because it spans dynamics modeling, perception, localization, mapping, trajectory generation, and optimal control.
- Globally consistent mapping is computationally expensive and fragile in challenging visual conditions, while end-to-end learning is sample inefficient and system-specific.
- LB-WayPtNav predicts intermediate waypoints from sequential RGB images and uses model-based optimal control to produce smooth, dynamically feasible trajectories.
- The framework combines learning and optimal control to maneuver a robot through novel cluttered environments using a single onboard RGB camera.
- Simulations and mobile-robot experiments report better goal-reaching efficiency and smoother trajectories than end-to-end learning, with greater reliability than geometric mapping-based approaches.
- The approach transfers from simulation to unseen real-world environments without real-world finetuning or data collection.
2 Related Work
Prior navigation approaches factor the problem into geometric mapping, planning, and tracking, or use end-to-end learning. LB-WayPtNav belongs to work combining learned perception with optimal control while targeting navigation in novel environments.
- Classical Robot Navigation: Classical navigation decomposes mapping, localization, path planning, and trajectory tracking, but geometric representations can miss navigational affordances.
- Classical Robot Navigation: Mapping is difficult with RGB observations and can struggle with transparent or shiny objects, textureless scenes, and strong ambient lighting.
- End-to-End Learning for Navigation: End-to-end navigation methods seek to incorporate semantics and common-sense reasoning, with some designed to generalize to unseen environments.
- End-to-End Learning for Navigation: Because many end-to-end methods abstract away dynamics, their macro-action policies can produce jerky, inefficient stop-and-go behavior on real robots.
- Combining Optimal Control and Learning: Related work also combines learning with optimal control, including learned costs for model predictive control and learned waypoints for drone racing.
3 Problem Setup
The paper studies goal-directed ground-vehicle navigation in unknown indoor environments using RGB observations and measured vehicle state. Planning repeatedly predicts waypoints and fits smooth trajectories under vehicle dynamics and control constraints.
- The setup assumes perfect odometry and static environments, deferring imperfect odometry and dynamic-environment navigation to future work.
- The vehicle is modeled as a nonlinear three-dimensional Dubins car with bounded linear and angular speeds.
- The robot receives an RGB image and a target position in its current coordinate frame, then must reach the target without colliding in an unmapped novel environment.
- Algorithm 1 replans periodically by predicting a waypoint from the image, current motion, and goal, then fitting a spline trajectory and computing feedback control.
4 Model-based Learning for Navigation
LB-WayPtNav factorizes navigation into learned waypoint prediction and model-based planning with feedback control. RGB observations guide waypoint selection, while dynamics-aware splines and LQR produce smooth, robust trajectories.
- Perception: LB-WayPtNav uses a CNN to infer the next waypoint from an RGB image, target position, and current vehicle speed.The network uses visual cues such as floors, tables, chairs, and learned shape priors without explicitly constructing an accurate map.
- Planning and control: A spline-based planner uses each waypoint, vehicle speed, and system dynamics to generate a dynamically feasible trajectory over a finite horizon.Third-order splines yield desired state and control trajectories efficiently through a set of linear equations.
- Planning and control: Third-order splines produce smooth speed and acceleration profiles, reducing concerns about jerky motion, poor tracking, sensor errors, and hardware damage.The planner is designed for computational efficiency, though other model-based planning schemes could also be used.
- Planning and control: An LQR feedback controller tracks the generated trajectory and improves robustness to disturbances and mismatches between modeled and actual dynamics.This feedback design supports deployment from simulation to a real robot when visual conditions are similar.
- Planning and control: The robot executes control commands for H seconds, then repeatedly obtains a new image, predicts a waypoint, and replans until reaching the goal.This receding-horizon loop couples perception, planning, and feedback during navigation.
- Perception: The perception module is trained with known maps but requires only RGB images and onboard sensors at test time.Known maps provide optimal waypoint and trajectory supervision during training, while test-time navigation does not assume a map.
5 Simulation Experiments
Simulation experiments evaluate LB-WayPtNav against end-to-end learning and geometric mapping-based planning in held-out indoor environments. LB-WayPtNav improves navigation reliability, speed, and smoothness over E2E learning, while mapping performance depends strongly on depth and memory.
- Experimental setup: Experiments use 185 test episodes in a held-out building derived from real-world indoor scans, covering obstacle and room-navigation scenarios.Two buildings provide training data, while a third building is reserved for testing.
- Experimental setup: Comparisons measure success rate, time to reach the goal, acceleration, and jerk, with lower acceleration and jerk indicating smoother, more efficient execution.Success requires reaching within 0.3m of the goal without collisions.
- Comparison with end-to-end learning: LB-WayPtNav achieves a 22% higher success rate, 40% less goal-reaching time, and 50% less acceleration than end-to-end learning.Both methods use first-person RGB inputs, while LB-WayPtNav computes controls through a model-based planner.
- Comparison with end-to-end learning: LB-WayPtNav produces smooth, collision-free trajectories through narrow hallways and tight turns, whereas E2E trajectories are discontinuous and jerky.LB-WayPtNav's acceleration and jerk are reported to be as low as an expert's, while smoothing E2E outputs reduced jerk but lowered success rate.
- Comparison with geometric mapping: Geometric mapping performs near expert with ideal depth and egomotion, but real-world depth limitations reduce mapping-based performance; memory-less mapping completely fails in one comparison.Simulation results are comparable between LB-WayPtNav and a memory-less depth-based planner, while perfect-depth mapping outperforms RGB-based LB-WayPtNav.
- Learned navigation affordances: LB-WayPtNav learns navigation affordances, choosing doorways for nearby room goals and hallways for farther goals.The model focuses on walls, doorways, hallways, and obstacles, with attention depending on the target.
- Failure modes: The method performs only local reasoning without memory and can get stuck near obstacles or when backtracking is required.These are identified as the prominent failure modes of the reactive navigation framework.
6 Hardware Experiments
Hardware experiments deploy LB-WayPtNav and three baselines on a TurtleBot 2 across unseen buildings and repeated navigation trials. The method is more reliable than mapping-based alternatives, faster and smoother than end-to-end learning, and can react to moving obstacles.
- Hardware setup: LB-WayPtNav was deployed without additional training in two buildings absent from the training dataset.The hardware evaluation used a TurtleBot 2 and on-board odometry for state measurement.
- Hardware setup: Across 20 trials, the evaluation reported success rate, goal-reaching time, acceleration, and jerk for LB-WayPtNav and three baselines.Each method was tested for five trials per task.
- Results: LB-WayPtNav produced more reliable, faster, and smoother trajectories than end-to-end learning.The comparison is based on hardware experiments with repeated trials.
- Results: LB-WayPtNav solved 95% of trials, compared with 40% for the geometric mapping scheme, while the memoryless baseline completely failed.The mapping-based system was affected by missing depth measurements for shiny, dark, and thin objects.
- Results: The method navigated narrow hallways smoothly without depth sensors or explicit maps in the short-horizon tasks studied.Its learning-based scheme leveraged prior experience with similar objects and maintained smooth trajectories around obstacles.
- Dynamic environments: When a chair repeatedly blocked the path, the robot reacted to the moving obstacle and reached the target without collision.The control horizon was reduced to 0.5s to provide faster visual feedback.
7 Discussion
LB-WayPtNav combines learning with model-based control for navigation in novel indoor environments. The discussion reports stronger goal-reaching reliability than end-to-end learning or geometric mapping, while identifying domain-shift, state-estimation, reactive-policy, and dynamic-environment boundaries.
- Contribution: LB-WayPtNav combines learning and model-based control for goal-driven navigation in novel indoor environments.The framework uses model-based feedback control to generalize from simulation to physical robots.
- Conclusion: The method is better and more reliable at reaching unseen goals than end-to-end learning or geometric mapping-based approaches.This is the paper's stated comparison across the evaluated alternatives.
- Limitations: Large appearance differences between simulation and reality can cause LB-WayPtNav to predict poor waypoints.The authors suggest fine-tuning with real-world data as a possible response.
- Limitations: The framework assumes perfect state estimation and uses a purely reactive policy, which may be unsuitable for long-range tasks requiring spatial memory.The discussion identifies geometric or learned maps as potentially critical for such tasks.
- Future work: A more detailed study of dynamic environments remains future work.The discussion separately identifies dynamic-environment evaluation as an open direction.
8 Supplementary Material
The supplementary material describes MPC-based expert supervision, waypoint optimization, trajectory generation, and experiments showing how perception robustness affects navigation performance.
- 8.2 Expert supervision: MPC generates dynamically feasible waypoints and spline trajectories from mapped training environments, while privileged map information is withheld at test time.The resulting supervision supports learning navigation decisions without explicit human labeling.
- 8.2 Expert supervision: The expert cost trades off goal distance and obstacle clearance, with a stronger obstacle penalty near obstacles to discourage unsafe trajectories.Obstacle cost is applied within λ1 of an obstacle and uses a cubic penalty versus a quadratic goal-distance penalty.
- 8.2 Expert supervision: The MPC procedure repeatedly optimizes a reachable waypoint and applies the corresponding control horizon while respecting vehicle dynamics, control bounds, and boundary conditions.The waypoint and spline trajectories must be reachable from the current state and speed.
- 8.2 Expert supervision: Waypoint orientation is a crucial design variable because it changes spline shape; suitable angles avoid collisions while line-of-sight trajectories can collide with obstacles.The waypoint angle provides an extra degree of freedom for generating agile, collision-free trajectories in cluttered environments.
- 8.3 Image distortions and domain randomization: Training-time image distortions increase LB-WayPtNav success from 47.94% to 80.65% and improve generalization to real-world camera tilt.Perspective distortions are particularly useful because the camera tilt changes as the robot moves.
- 8.7 Imperfections in depth estimation: Depth estimation fails on sunlight, thin wires, power strips, shiny chair legs, bike components, and transparent monitors, limiting depth-dependent mapping approaches.These failures produce inaccurate or missing depth regions in hardware experiments.