PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Visual MPC-Policy Learning for Navigation

Noriaki Hirose, Fei Xia, Roberto Martin-Martin, Amir Sadeghian, Silvio Savarese

arXiv:1903.02749v2cs.RO

TL;DR

Robot navigation often depends on mapping and localization, while visual servoing has limited convergence and lacks collision-safety guarantees. The paper presents PoliNet, which follows image trajectories while avoiding obstacles using MPC-based training. It reports high success rates across obstacle conditions and robust visual-trajectory following in simulation and the real world, with occasional difficulty avoiding large obstacles.

  • Problem

    Visual navigation needs to follow image-defined paths safely, but visual servoing has limited convergence and no collision-safety guarantees, while traditional navigation relies on mapping and localization.

    Source: §I · p. 1

  • Method

    PoliNet takes a visual trajectory and current camera image and learns velocity commands through an MPC objective using visual-dynamics and traversability models.

    Source: §I · p. 1, §III · p. 3

  • Results

    0.997 success rate without obstacles and 0.850 with obstacles on average; PoliNet outperforms all baseline methods.

    Source: B. Evaluation of PoliNet · p. 7

  • Takeaways & Limitations

    PoliNet follows visual trajectories robustly in simulation and the real world, including trajectories generated in simulation and executed in the real world without retraining.

    Source: §VI · p. 9

    The method occasionally fails to avoid large obstacles because traversability is a soft constraint, the prediction horizon limits long detours, and large obstacles can impede image convergence.

    Source: §VI · p. 9

Abstract

from arXiv · show

Humans can routinely follow a trajectory defined by a list of images/landmarks. However, traditional robot navigation methods require accurate mapping of the environment, localization, and planning. Moreover, these methods are sensitive to subtle changes in the environment. In this paper, we propose a Deep Visual MPC-policy learning method that can perform visual navigation while avoiding collisions with unseen objects on the navigation path. Our model PoliNet takes in as input a visual trajectory and the image of the robot's current view and outputs velocity commands for a planning horizon of $N$ steps that optimally balance between trajectory following and obstacle avoidance. PoliNet is trained using a strong image predictive model and traversability estimation model in a MPC setup, with minimal human supervision. Different from prior work, PoliNet can be applied to new scenes without retraining. We show experimentally that the robot can follow a visual trajectory when varying start position and in the presence of previously unseen obstacles. We validated our algorithm with tests both in a realistic simulation environment and in the real world. We also show that we can generate visual trajectories in simulation and execute the corresponding path in the real environment. Our approach outperforms classical approaches as well as previous learning-based baselines in success rate of goal reaching, sub-goal coverage rate, and computational load.

I. INTRODUCTION

The paper introduces a visual navigation policy that follows image-defined paths while avoiding collisions, addressing limitations of visual servoing and learned navigation. It compares this approach with prior methods and evaluates navigation in simulation and the real world.

I. INTRODUCTION

  • Earlier visual servoing has a small convergence region and no collision-safety guarantees, while learning-based methods may require test-environment experience and lack explicit penalties for untraversable areas.The paper motivates visual navigation with RGB cameras as an alternative to costly, failure-prone Lidar or depth sensing.Source: §I · p. 1
  • PoliNet maps 360° camera views and a visual path to velocity commands, trained through MPC objectives using differentiable visual-dynamics and traversability models.The method is reported not to require predictions at test time; its evaluation includes 10,000 simulation tests and 110 real-world tests, including trajectories generated in simulation and followed in the real world without retraining.Source: §I · p. 1
  • VUNet-360 modifies prior mobile-robot view-synthesis method VUNet, propagating information between differently directed cameras (the 360° camera’s front and back views); the paper releases a visual-trajectory dataset and project code.Source: §I · pp. 1–2

A. Visual Servoing

  • Visual servoing can get stuck in local minima, requires costly Image Jacobian computation, and may break when the environment differs from the goal image.The paper contrasts this with MPC, a learned action-image relationship, and demonstrated robustness to simulation-to-real differences.Source: A. Visual Servoing · p. 2

B. Visual Model Predictive Control

  • Unlike Finn and Levine’s sampling-based Visual MPC for manipulation, this method uses a neural network to generate velocity commands and reduce repeated predictive-model rollouts.The paper also distinguishes its training from imitation: PoliNet is optimized through differentiable predictive models rather than trained to imitate an MPC controller.Source: B. Visual Model Predictive Control · p. 2

C. Deep Visual Based Navigation

The method follows visual paths by comparing camera images with successive image subgoals, without geometric localization or map-based path planning. It explicitly penalizes movement toward poorly traversable areas and reports higher success rates than visual path-following baselines.

C. Deep Visual Based Navigation

  • The method achieves a higher success rate than both visual path-following baselines.Its stated distinction is explicit penalization of velocities that lead toward low-traversability areas or obstacles, with evaluation in simulation and the real world.Source: C. Deep Visual Based Navigation · pp. 2–3

III. METHOD

  • The input is a visual trajectory of constant-time image subgoals and the robot’s current 360° camera image.The 360° view is represented as front and back 180° fisheye images; a trajectory can be recorded by teleoperating the robot or moving a virtual camera in simulation.Source: §III · p. 3
  • PoliNet moves toward the next subgoal by minimizing the difference between its predicted camera view and the subgoal image while avoiding obstacles, including previously unseen ones.The process switches to the next subgoal when the absolute pixel difference falls below an experimentally adapted threshold.Source: §III · p. 3
  • The robot transitions between image subgoals without geometric localization or path planning on a map.This summarizes the navigation process described for successive subgoals.Source: §III · p. 3

A. Control Policy

The image-domain MPC policy combines visual trajectory following with traversability and realistic-velocity objectives. PoliNet is learned to avoid costly direct optimization and, at inference, uses only images defining the trajectory.

A. Control Policy

  • Directly solving the image-domain MPC objective is computationally prohibitive because its cost minimization is non-convex.The policy is therefore learned to produce the MPC-policy commands.Source: A. Control Policy · p. 3
  • The policy combines image, traversability, and reference losses to balance trajectory following, safe areas, and realistic velocities.The reference loss addresses velocities that can otherwise be non-smooth or unrealistic, while the other objectives follow the visual trajectory and penalize risk.Source: A. Control Policy · pp. 3–4
  • Image loss measures mean absolute pixel difference between the subgoal and predicted images, with step weights allowing deviations to avoid collisions.The final step receives greater weight to encourage convergence to the subgoal.Source: A. Control Policy · p. 3
  • Traversability loss uses GONet probabilities from predicted front images, with kernelization emphasizing areas below the traversability threshold.The kernelized probability encourages commands to avoid areas below 1/κtrav without penalizing higher values.Source: A. Control Policy · p. 3
  • Reference loss minimizes differences between generated and teleoperator velocities to regularize commands toward smoother, more realistic motion.It is used in MPC and training, not as an inference-time requirement for geometric information or velocities.Source: A. Control Policy · p. 4

B. Predictive Model, VUNet-360

VUNet-360 provides velocity-conditioned future images for the MPC training objective. Its parallel prediction and front/back blending adapt VUNet for 360-degree inputs and multi-step prediction.

B. Predictive Model, VUNet-360

  • VUNet-360 takes a 360-degree image represented by front and back views plus N virtual velocities, then generates N future images in parallel.This addresses the original VUNet's front-view-only input and sequential multi-step predictions, which were computationally heavy and lower quality.Source: B. Predictive Model, VUNet-360 · p. 4
  • Its encoder-decoder concatenates robot velocities with the latent vector and uses a blending module to fuse front and back image information.The module generates virtual front and back images; bilinear sampling maps input pixels to predicted images.Source: B. Predictive Model, VUNet-360 · p. 4
  • The blending module uses sampled front and back images with visibility masks to form predicted views.The architecture and front/back blending are depicted in Fig. 3.Source: B. Predictive Model, VUNet-360 · p. 4
  • During PoliNet training, generated images and traversability values contribute to the loss; at test time, only the blue-shaded portion of the calculation is used.The figure depicts the forward calculation during training and distinguishes the test-time path.Source: Figure 4 · p. 4

C. Neural Model Control Policy, PoliNet

PoliNet replaces online solution of the computationally prohibitive MPC optimization with a neural policy. It learns velocity sequences from current and subgoal images using predictive and traversability models during training.

C. Neural Model Control Policy, PoliNet

  • PoliNet replaces online non-convex MPC optimization with an eight-layer convolutional network designed for fast onboard computation.A tanh output layer constrains linear and angular velocities to their respective maximum magnitudes.Source: C. Neural Model Control Policy, PoliNet · p. 4
  • Given current and subgoal images, PoliNet outputs N velocities that move toward the subgoal in image space while avoiding non-traversable areas.The policy is trained using VUNet-360 and GONet to calculate the MPC cost and its gradient; those models are not updated during PoliNet training.Source: C. Neural Model Control Policy, PoliNet · p. 4
  • Training uses teleoperator velocities for the reference loss and randomly selects a future dataset image as the target.The randomly selected target is drawn within the range Nr.Source: C. Neural Model Control Policy, PoliNet · p. 4

IV. EXPERIMENTAL SETUP

The evaluation setup uses a Turtlebot 2 with a Ricoh THETA S 360-degree camera, collecting teleoperation data in real and simulated buildings. Training balances both data sources and uses an eight-step, real-time prediction horizon.

IV. EXPERIMENTAL SETUP

  • Experiments use a Turtlebot 2 equipped with a Ricoh THETA S 360-degree camera in simulation and the real world.The platform is used as the navigation system for the learned visual MPC approach.Source: §IV · p. 5
  • Real-world teleoperation data comprise 10.30 hours of 360-degree images and velocity commands from twelve Stanford campus buildings, split into eight training, two validation, and two test buildings.The building-level split separates the training, validation, and test data.Source: §IV · p. 5
  • Simulation data use 36 buildings and 3.59 hours of teleoperation, with 26 buildings for training and five each for validation and testing.The simulator uses Stanford 2D-3D-S and Matterport3D models; the former have corresponding real-world environments, while the latter mainly contain residential buildings.Source: §IV · p. 5
  • Training balances simulator and real data equally and trains the networks iteratively with Adam at a learning rate of 0.0001.Images are resized to 128×128 before network input.Source: §IV · p. 5
  • The MPC horizon is eight steps, or 2.667 seconds, with inference at 3 Hz to support real-time navigation.The selected horizon balances longer MPC prediction with the reliability of shorter predictions from VUNet-360.Source: §IV · p. 5
  • Maximum linear and angular velocities are 0.5 m/s and 1.0 rad/s; the traversability-loss weight is 0.5 and the reference-loss weight is 0.1.The reference loss is intended to regularize and smooth velocities rather than imitate the teleoperator exactly; the subgoal image is randomly chosen within Nr = 12.Source: §IV · p. 5

V. EXPERIMENTS

The experiments compare VUNet-360 and PoliNet with baselines in simulation and real-world navigation, including changing conditions and sim-to-real transfer. They report improved image prediction, strong navigation results, and a runtime trade-off in which PoliNet is faster than online optimization baselines that meet the target cost only with slower settings.

A. Evaluation of VUNet-360

  • VUNet-360 improves SSIM and pixel difference over VUNet at all prediction stages, with 901MB memory and 19.28 Hz versus 1287MB and 5.03 Hz.It predicts more accurately in turning and forward-motion examples, using information from the back camera to handle occlusions.Source: A. Evaluation of VUNet-360 · p. 5, Figure 5 · p. 6

B. Evaluation of PoliNet

  • PoliNet achieves small cost at 325 MB and 72.25 Hz, while higher-iteration optimization baselines achieve lower costs but cannot compute at the 3 Hz real-time target.Open-loop control and imitation learning are also fast but are less robust and reactive in other experiments.Source: B. Evaluation of PoliNet · p. 6
  • In simulation, PoliNet deviates from the visual trajectory to avoid obstacles and reaches the goal without collision across varied starting poses.The evaluation uses randomized starts, obstacles on and off the trajectory, and velocity noise to test policy robustness.Source: Figure 6 · p. 6, B. Evaluation of PoliNet · p. 7
  • In real-world navigation, PoliNet achieves high success and subgoal coverage across changed conditions, outperforming eight-step imitation learning by a large margin.The qualitative examples include changes in furniture, lighting, and pedestrians, while predicted images remain close to the subgoal images.Source: B. Evaluation of PoliNet · p. 7, Table III · p. 8
  • Despite the simulation-to-real domain gap, PoliNet reaches the destination without collision in most trials using trajectories generated in simulation.The evaluation used ten trials at different days and times; qualitative examples show large image differences between simulated subgoals and real views.Source: B. Evaluation of PoliNet · pp. 7–8, Table IV · p. 8
  • 0.997 success rate without obstacles and 0.850 with obstacles on average; PoliNet outperforms all baselines in simulation.Across seven additional environments, success/coverage/SPL were (0.979, 0.985, 0.979) without obstacles and (0.865, 0.914, 0.865) with obstacles, exceeding IL and ZVI.Source: B. Evaluation of PoliNet · p. 7, Table II · p. 8

C. Ablation Study of Traversability Loss Generation

The ablation varies the weight on the traversability loss and evaluates its navigation effect. Increasing that weight trades lower traversability cost for poorer image matching and, at the largest tested weight, near-zero success.

C. Ablation Study of Traversability Loss Generation

  • At qg = 1.0, success is almost zero even without obstacles; the policy avoids narrow paths, so the authors use qg = 0.5 for evaluations.Larger qg lowers Jtrav but increases Jimg, reflecting a conservative policy that can fail to reach the target image.Source: C. Ablation Study of Traversability… · p. 8, Table V · p. 8

VI. CONCLUSION AND FUTURE WORK

The conclusion presents PoliNet as a learned MPC policy for visual navigation with a 360° RGB camera, and reports robust trajectory following in simulation and the real world. It also identifies occasional failures around large obstacles and outlines possible changes to the constraint and prediction horizon.

VI. CONCLUSION AND FUTURE WORK

  • PoliNet learns MPC objectives to generate velocities that reduce image differences to subgoals while avoiding collisions and using less computation than standard visual MPC.The reported navigation robustness spans simulation and real-world visual trajectories.Source: §VI · p. 9
  • Large obstacles are occasionally not avoided because traversability is a soft constraint, the horizon limits long detours, and obstacles can dominate a subgoal image.The authors propose testing a hard traversability constraint, increasing the prediction horizon, and improving transitions between subgoal images.Source: §VI · p. 9

APPENDIX I NETWORK STRUCTURE

The appendix describes how VUNet-360 predicts future front and back images and how PoliNet generates bounded velocity sequences. Additional qualitative examples illustrate the predicted views and navigation transitions.

A. VUNet-360

  • VUNet-360 encodes concatenated front and horizontally flipped back images with future virtual velocities, then predicts N future images in one encoder-decoder network.Its blending modules synthesize predicted views using internally predicted images and selection masks for front and back views.Source: A. VUNet-360 · p. 11

B. PoliNet

  • PoliNet generates N linear and angular velocities from image features, using tanh-scaled outputs to keep each velocity within its specified limit.The network is intended to represent relative pose and future traversability while remaining computationally light for online use.Source: B. PoliNet · p. 11

A. Additional Qualitative Result

  • In additional examples, VUNet-360 predicts views close to ground truth through the eighth step, including environments beyond the front camera view.The comparison contrasts its predictions with VUNet, which does not correctly predict the environment behind the front camera.Source: A. Additional Qualitative Result · p. 11
  • Six additional PoliNet examples show predicted image sequences transitioning from the current view to the subgoal without untraversable situations or discontinuous motion.The authors interpret these sequences as velocities that navigate to the subgoal location without collision.Source: A. Additional Qualitative Result · p. 11

B. Additional Quantitative Result

Additional simulation runs show PoliNet works well across the tested environments, while the predictive-model evaluation favors parallel VUNet-360 over sequential alternatives.

B. Additional Quantitative Result

APPENDIX III ADDITIONAL EVALUATION FOR VUNET-360

  • Sequential VUNet-360 predicts distant frames by repeatedly applying a next-frame model; training evaluates a one-step objective and an N=8-step objective.The N=8 objective matches the parallel method's objective.Source: APPENDIX III ADDITIONAL EVALUATION FOR… · pp. 12–13
  • The predictive model for parallel prediction outperforms all methods using sequential processes on L1 and SSIM, and the parallel process predicts higher-quality images, as shown in Fig. 12.Source: APPENDIX III ADDITIONAL EVALUATION FOR… · p. 13
  • The one-step objective does not teach the model to use the opposite-side camera image, despite having the same blending module as parallel VUNet-360.The trained model extends image edges instead of blending the front and back views.Source: APPENDIX III ADDITIONAL EVALUATION FOR… · p. 13
  • The N=8 sequential model appears to use the opposite-side image, but its predictions are lower quality than the parallel model's.The authors hypothesize that sequential prediction blurs images for several architectural and training-related reasons.Source: APPENDIX III ADDITIONAL EVALUATION FOR… · p. 13
  • Parallel prediction can sample current raw-image pixels for each forecast, whereas sequential prediction uses prior predictions, accumulating errors and making training slower and harder.The authors also cite the sequential model's need to encode both raw and predicted images and its deeper backpropagation path.Source: APPENDIX III ADDITIONAL EVALUATION FOR… · p. 13

APPENDIX IV ANALYSIS FOR RESULTS IN TABLE II

The appendix attributes baseline weaknesses to computational or control constraints and to mismatches with obstacle-avoiding navigation, describing distinct failure modes for imitation learning, ZVI, and visual memory.

APPENDIX IV ANALYSIS FOR RESULTS IN TABLE II

Loading 1903.02749v2…