Source-linked AI summary
Deep Visual Foresight for Planning Robot Motion
Chelsea Finn, Sergey Levine
TL;DR
Robotic manipulation needs scalable learning with less human supervision, while hand-engineered models struggle with unknown objects and environments. The paper combines a deep action-conditioned video prediction model with model-predictive control using unlabeled data. A real robot performs nonprehensile pushing, including on novel objects, without camera calibration or detailed physical modeling.
Problem
Robotic manipulation pipelines are difficult to apply to unseen objects because errors in modeling perception and physical properties can compound, motivating a general learned model connecting perception with physical prediction.
Method
The method combines a deep predictive model of images trained on 50,000 autonomous pushing attempts with model-predictive control that continuously replans actions from new observations.
Results
The real robot performs self-supervised nonprehensile pushing and generalizes to previously unseen objects without camera calibration, 3D models, depth observations, or a physics simulator.
Takeaways & Limitations
Learned predictive video models can support robotic manipulation with minimal prior engineering and flexible pixel-level goals using unlabeled real-world experience.
Takeaways & Limitations
The approach remains limited to relatively simple short-horizon tasks, while planning horizon and replanning rate are constrained by computational power.
Abstract
from arXiv · showhide
A key challenge in scaling up robot learning to many skills and environments is removing the need for human supervision, so that robots can collect their own data and improve their own performance without being limited by the cost of requesting human feedback. Model-based reinforcement learning holds the promise of enabling an agent to learn to predict the effects of its actions, which could provide flexible predictive models for a wide range of tasks and environments, without detailed human supervision. We develop a method for combining deep action-conditioned video prediction models with model-predictive control that uses entirely unlabeled training data. Our approach does not require a calibrated camera, an instrumented training set-up, nor precise sensing and actuation. Our results show that our method enables a real robot to perform nonprehensile manipulation -- pushing objects -- and can handle novel objects not seen during training.
I. INTRODUCTION
The paper addresses failures of hand-engineered robotic manipulation pipelines by asking whether a single learned model can connect perception with physical prediction. It combines deep predictive video models with model-predictive control to perform self-supervised pushing of previously unseen objects without extensive prior engineering.
- Motivation: Modular manipulation pipelines can fail on unseen objects because modeling errors compound across perception, physical prediction, and action planning.Even small errors in properties such as liquid content or friction can produce ineffective real-world actions.
- Motivation: The paper asks whether a single general-purpose learned model can replace hand-engineered pipelines by connecting low-level perception with physical prediction.
- Approach: The approach combines a learned predictive model of raw sensory observations with model-predictive control and continuously replans actions from new observations.It optimizes action sequences to move user-specified pixels or pixel groups toward desired goal positions while correcting for mispredictions.
- Contribution: The primary contribution is using deep predictive video models in a real robot to manipulate previously unseen objects.The MPC algorithm uses probabilistic inference through a learned predictive image model to plan actions that move objects to user-defined locations.
- Evaluation: Experiments focus on nonprehensile pushing and use unlabeled training data without camera calibration, 3D models, depth observations, or a physics simulator.The method learns fast visual feedback control in an entirely self-supervised setting.
- Context: The work targets a flexible learned model rather than task-specific policies, addressing the difficulty of modeling unknown physical environments for robotic manipulation.Related approaches include physics-based manipulation, model-free policies, learned image embeddings, and visual servoing, but differ in flexibility or calibration requirements.
III. BACKGROUND
The method learns an action-conditioned probabilistic video model from robotic pushing experience and uses emergent stochastic pixel flows to predict future images. Training uses raw images, actions, and states rather than explicit flow supervision.
- The dataset contains 50,000 pushing attempts collected with 10 7-DoF arms and hundreds of objects.Robots recorded gripper poses, commanded poses, and workspace RGB images at 10 Hz while commands were selected randomly.
- The model’s flow and mask representations emerge from supervision on video, actions, and states alone, without explicit flow labels.The predicted image distribution uses a constant diagonal covariance and recursive mean prediction.
- The predictive model receives recent images, end-effector poses, and future commands, then predicts a distribution over future image frames.It is trained by maximum likelihood, yielding a mean squared error objective, with independent Gaussian pixel distributions as a simplifying factorization.
- A convolutional LSTM predicts stochastic flow maps that represent pixel transitions and combine object-specific motion through normalized kernels and learned masks.The flow operator is applied recursively to generate predicted image means, while multiple channels capture separate moving objects.
- The architecture adds layer normalization after each layer and uses 5 masks instead of 10 for slightly faster computation.
IV. VISUAL MPC WITH LEARNED VIDEO PREDICTION MODELS
At test time, the system combines a learned deep video prediction model with model-predictive control to realize a user-specified visual goal. Because the model does not explicitly represent objects, goals must be expressed through image-level quantities.
- The system uses MPC with the learned video model to choose controls that realize a user-commanded goal.The goal is represented through visual rather than standard object-pose notions because the video model does not explicitly model objects.
A. Specifying Goals with Pixel Motion
Users specify manipulation objectives by selecting source pixels and corresponding goal positions in the initial image. The robot then plans to move the objects associated with those pixels.
- The goal consists of one or more selected source pixels and a corresponding goal position for each pixel.
- Moving selected pixels lets the robot command the motion of the objects to which those pixels belong.
- This representation can specify arbitrary object rearrangements, such as clearing a table, through direct image selection or higher-level planning.
B. Evaluating Actions with Implicit Pixel Advection
The controller evaluates action sequences by propagating probability distributions over designated pixel positions through the model’s stochastic flow operators. It selects actions that maximize the probability of reaching the user-specified goal.
- The predictive model’s stochastic flow operators define distributions over pixel motion conditioned on candidate action sequences.The operators can transform pixel distributions recursively across the prediction horizon.
- The controller assumes the designated pixel’s current position is known and propagates its position distribution forward through the MPC horizon.
- The action sequence is chosen to maximize the probability that the designated pixel reaches its goal location.
- For multiple pixels, success probabilities are computed separately and combined by summing their log-probabilities.
- Action evaluation uses implicit flow predictions rather than predicted images, while image prediction enables training without explicit flow supervision.
C. Sampling-Based Model-Predictive Control with Deep Models
The controller evaluates candidate action sequences with a learned predictive image model and repeatedly refines them using cross-entropy optimization. It executes the best action, then updates designated pixel positions with optical flow for real-time replanning.
- Receding-horizon control: At each timestep, the controller executes the action sequence with the highest predicted success probability, observes a new image, and replans.The procedure is summarized in Algorithm 1.
- Action evaluation: The controller samples action sequences and uses the predictive model to estimate each sequence’s probability of moving designated pixels to their goals.The probability is conditioned on recent images, states, actions, and designated pixels.
- Action optimization: Cross-entropy optimization repeatedly fits a Gaussian distribution to the highest-probability action sequences and resamples from it.The K selected sequences define the distribution used to improve subsequent samples.
- Implementation: The initial planner uses Jt = 4 iterations, M = 40 samples, and K = 10 selected samples; realtime replanning uses one iteration with M = 20.The sampling batches can be parallelized efficiently through batched forward passes.
- Online state updates: Optical flow updates designated pixel locations between observations, while video prediction runs in parallel on a GPU for speed.Optical flow is used for replanning initialization rather than for the predictive model’s own predictions.
V. EXPERIMENTS
The experiments test whether action-conditioned video prediction can manipulate novel objects and infer physical behavior from raw image pixels. They combine qualitative and quantitative evaluations to address both questions.
- Experimental questions: The evaluation asks whether the method can manipulate objects absent from training and whether raw-pixel video models infer nontrivial physical behavior.Both questions are examined through qualitative and quantitative experiments.
- Evaluation scope: The experiments prioritize flexibility of a data-driven approach over highest accuracy for precise nonprehensile manipulation.The intended demonstration uses minimal prior knowledge about objects, physics, and contacts.
- Evaluation scope: The study examines whether predictive models trained entirely on raw video can infer physical-world characteristics useful for robotic manipulation.The stated setting excludes prior knowledge about objects, physics, and contacts.
A. Experimental Setup
The setup evaluates pushing with a 7-DoF arm and an RGB camera, using short pixel-to-goal episodes on mostly novel objects. Short-horizon planning enables fast reactive replanning rather than long-term control.
- Robot and task: A 7-DoF robot arm with an over-the-shoulder RGB camera performs episodes that move designated pixels to corresponding goal locations.The experiments use pixel positions as task specifications.
- Test objects: Unless otherwise specified, experimental objects were not seen previously in the training set.The setup therefore evaluates generalization to novel objects.
- Planning configuration: Images have resolution 64×64 pixels, and the planning horizon H = 3 corresponds to about 800 ms.The short horizon supports fast feedback control.
- Data collection: Training data was collected on 10 robots with varying camera angles and positions, encouraging a calibration-invariant representation of object interactions.The figure contrasts camera configurations and robot-base positions.
- Realtime control: Controls are recomputed about every 200 ms, with one commanded action held constant across the H timesteps.The short horizon focuses the controller on fast, reactive pushing tasks rather than long-term planning.
B. Quantitative Comparisons
The quantitative comparison tests whether the learned predictive model captures useful object and interaction behavior without explicit object or physics knowledge. The method improves over the baselines, though the authors note substantial room for improvement.
- Evaluation design: The baselines and proposed method evaluate whether video prediction learns meaningful object and physical-interaction representations from data.Object identity, inertia, and contact dynamics are not explicitly provided to the model.
- Baselines: The baselines comprise random actions, calibrated goal-pixel servoing, and optical-flow-based servoing toward the goal.The latter two baselines use only the first designated pixel when multiple pixels are specified.
- Measurement and replanning: Optical flow supports replanning and final pixel-to-goal distance evaluation, but it is not used by the predictive model to make predictions.It provides the initial pixel-state distribution during replanning.
- Study scope: The comparison intentionally avoids baselines with prior object or physics knowledge because the study explores learned video prediction from scratch.The goal is not to establish superior nonprehensile manipulation performance.
- Results: The method improves over the baselines, suggesting that its predictive model makes meaningful inferences about object motion in response to the robot arm.The authors state that the results still leave significant room for improvement.
C. Qualitative Results
Qualitative experiments show that visual MPC can coordinate multi-pixel motions to rotate objects, while failures arise from self-occlusion and inaccurate physical predictions.
- Object rotation: When the arm starts away from an object, the method plans contact on one side to realize the commanded rotation.The figure shows this behavior across objects, including objects absent from the original dataset.
- Object rotation: The model plans coordinated motions for opposing pixel goals that produce the desired object rotation.This demonstrates prediction and control of multiple pixels moving in different directions.
- Failure cases: Self-occlusion can shift the designated pixel from the object to the arm, causing the controller to move the arm instead of the object.The resulting plan can make the arm bump into the object.
- Failure cases: Incorrect model predictions can push a bottle near its center of mass, causing translation rather than rotation.The failure is attributed to inaccurate prediction of the bottle’s physical behavior.
VI. DISCUSSION & FUTURE WORK
The paper presents a self-supervised predictive-model approach for basic nonprehensile manipulation and reports generalization to novel objects with minimal prior engineering. Its scope remains limited by task simplicity, prediction accuracy, and runtime computational constraints.
- Discussion: The method uses a deep predictive neural network to plan pushing motions with minimal prior engineering.A convolutional LSTM trained on autonomously collected unlabeled data predicts future camera images and image-space pixel flow from motor commands.
- Discussion: The model continuously plans pushing tasks with novel objects not seen during training through self-supervised prediction of future images and pixel flow.The training procedure supports ongoing data collection for continuous self-improvement.
- Discussion: The experiments demonstrate that the method outperforms simple geometric-heuristic and known hand-to-camera-calibration baselines.
- Future work: The approach generalizes to completely novel objects but remains limited to relatively simple short-horizon tasks.The authors expect improved video-prediction accuracy to expand its capabilities.
- Future work: Runtime planning horizon and replanning rate are limited by available computational power, particularly fast and highly parallel GPUs.The authors expect faster, more parallel platforms to improve capability and accuracy.
- Future work: Deep video prediction remains accurate only a few frames into the future, motivating hierarchical models operating at varying time scales.Such models may support planning more elaborate skills with learned predictive models.