Source-linked AI summary
Self-Supervised Visual Planning with Temporal Skip Connections
Frederik Ebert, Chelsea Finn, Alex X. Lee, Sergey Levine
TL;DR
Robotic learning from autonomous data needs predictive models without hand-designed representations, but visual control must handle changing scenes and occlusions. The paper uses direct action-conditioned video prediction with temporal skip connections, improved pixel-distance planning, and mixed discrete-continuous actions. The resulting system performs diverse self-supervised pushing tasks, including with unseen and multiple objects, while tracking remains limited over long or repeated occlusions.
Problem
Autonomous robotic learning needs predictive models that work across changing open-world scenes and preserve object information through occlusions without human supervision.
Method
The paper combines direct video prediction with temporal skip connections, a pixel-distance planning objective, and continuous-discrete action representations.
Results
The system performs a wide range of self-supervised multi-objective pushing tasks, including manipulation of unseen objects and pushing around obstructions.
Takeaways & Limitations
Video prediction trained on random pushing motions can support model-predictive control for complex manipulation despite occlusions.
Takeaways & Limitations
Experiments use relatively short behaviors, and repeated occlusions can eventually cause the model to lose track of target pixels.
Abstract
from arXiv · showhide
In order to autonomously learn wide repertoires of complex skills, robots must be able to learn from their own autonomously collected data, without human supervision. One learning signal that is always available for autonomously collected data is prediction: if a robot can learn to predict the future, it can use this predictive model to take actions to produce desired outcomes, such as moving an object to a particular location. However, in complex open-world scenarios, designing a representation for prediction is difficult. In this work, we instead aim to enable self-supervised robotic learning through direct video prediction: instead of attempting to design a good representation, we directly predict what the robot will see next, and then use this model to achieve desired goals. A key challenge in video prediction for robotic manipulation is handling complex spatial arrangements such as occlusions. To that end, we introduce a video prediction model that can keep track of objects through occlusion by incorporating temporal skip-connections. Together with a novel planning criterion and action space formulation, we demonstrate that this model substantially outperforms prior work on video prediction-based control. Our results show manipulation of objects not seen during training, handling multiple objects, and pushing objects around obstructions. These results represent a significant advance in the range and complexity of skills that can be performed entirely with self-supervised robotic learning.
1 Introduction
The paper targets autonomous robotic skill learning without human-specified rewards, demonstrations, or environment resets. It directly predicts sensory observations and introduces occlusion-aware video prediction for planning increasingly complex manipulation behaviors.
- Autonomous learning of broad robotic skills is bottlenecked by human involvement in rewards, demonstrations, and environment preparation.
- Fixed environment representations are difficult because object numbers, types, and relevant physical information can vary across open-world scenes.
- Directly predicting sensory observations avoids engineering representations for different object types and scenes while providing predictions usable for action planning.
- Occlusions require the robot to preserve information about objects that disappear from view during manipulation.
- Temporal skip connections improve object permanence, while the paper also contributes a planning objective and mixed discrete-continuous action formulation.
- The combined system performs self-supervised pushing involving unseen objects, multiple objects, obstructions, and arm motion over obstacles.
2 Related Work
Prior work studies autonomous data collection, action-conditioned video prediction, and model-based control, but faces limitations in generality, computation, sensing, and action representations. This paper positions its approach as a predictive model for varied real-world manipulation without rewards or depth-based correspondences.
- Earlier autonomous data-collection methods targeted individual skills such as grasping or obstacle avoidance, whereas this approach supports varied manipulation without success measures or reward functions.
- Action-conditioned video prediction has been explored in video games and robotic manipulation, while action-free prediction has been studied for unstructured videos and driving.
- Autoregressive future-image models can produce sharp predictions but are computationally demanding and had not been applied to real-world robotic control.
- Depth-based 3D motion prediction requires point-to-point correspondences, making it difficult to apply to previously unseen objects.
- The proposed model handles real-world object manipulation without 3D depth sensing or point-to-point frame correspondences, and extends sampling-based planning to mixed action types.
3 Preliminaries
The preliminaries formulate visual model-predictive control around moving designated image pixels to user-specified goals. A learned image-transformation model predicts pixel motion recursively, enabling action-sequence planning but not, in its prior form, tracking through occlusions.
- 3.1 Visual Model Predictive Control: Visual MPC lets a user select source pixels and destination locations, then plans an action sequence over a finite horizon to move those pixels.
- 3.1 Visual Model Predictive Control: The learned model represents each designated pixel with a spatial probability distribution initialized at its observed location.
- 3.1 Visual Model Predictive Control: At each real-world step, MPC plans multiple future actions, executes the first, and replans using the preceding one-step prediction.
- 3.1 Visual Model Predictive Control: Planning uses the cross-entropy method to resample action sequences and refit Gaussian distributions without gradients.
- 3.2 Video Prediction via Pixel Transformations: The image-transformation model uses convolutional LSTMs, object-specific transformations, and masks to composite predicted images recursively.
- 3.2 Video Prediction via Pixel Transformations: The prior single-image prediction model applies the same transformations to pixel distributions but cannot track selected pixels through occlusions.
4 Skip Connection Neural Advection Model
The SNA model extends DNA with temporal skip connections, allowing predicted frames to reuse earlier observations and preserve objects through occlusions. A simplified version uses a first-image skip connection and performs comparably in the robotic manipulation setting.
- Model design: SNA extends dynamic neural advection with temporal skip connections to track objects through occlusions.It transforms images from the observation history with separate transformations and masks, allowing occluded pixels to reappear later.
- Model design: The practical SNA variant assumes occluded objects remain static throughout the prediction horizon.This removes intermediate transformations and retains only a skip connection from the first image.
- Model design: The simplified model uses N + 1 masks and achieved similar prediction performance to a version using a transformed initial image.The simplified formulation was used in all experiments.
- Model design: Figure 2 depicts the first image I0 concatenated with transformed images ˜I(i) in the simplified SNA model.The red arrow marks the concatenation site.
- Occlusion behavior: Figure 4 compares the predicted probability of a designated pixel remaining at its original location for DNA and SNA during occlusion.Figure 5 visualizes the corresponding predicted arm motion and probability distributions as the arm passes in front of the object.
5 Visual MPC with Pixel Distance Costs
The proposed visual MPC objective replaces a terminal goal probability with an expected pixel-distance cost accumulated across future steps. This smoother objective supports directional progress during long-horizon planning and enables obstacle-avoiding paths when combined with vertical actions.
- Pixel-distance objective: The expected pixel-distance cost uses predicted position distributions to measure distance from designated pixels to their goals at each future step.The cost is computed by summing the probability-weighted distances across positions in each predicted image.
- Pixel-distance objective: Unlike a terminal goal probability, the proposed objective remains informative when long trajectories make goal probabilities close to zero.It encourages movement toward the goal at every execution step, even when the goal cannot be reached within the prediction horizon.
- Obstacle avoidance: Integrating vertical actions enables the end-effector to move over obstacles, producing more natural and shorter paths.Figure 6 shows executed trajectories alongside predictions and designated-pixel probabilities at time step 3.
- Pixel-distance objective: For multiple designated pixels, the planning costs are summed with equal weighting.
6 Sampling-Based MPC with Continuous and Discrete Actions
The controller uses a hybrid action representation combining continuous horizontal end-effector motion with discrete lifting commands, integrated into sampling-based visual MPC.
- The action space combines horizontal end-effector motion with a discrete lift action that keeps the gripper above the table for multiple time steps.The lift action takes one of N values specifying its duration.
- The hybrid representation gives the planner maneuvering freedom while constraining the search space for tabletop manipulation.
- Table 1 reports pushing-benchmark results on 20 object/goal configurations, with distances measured in pixels in 64x64 images.
- Real-valued parameters for discrete actions are sampled during CEM optimization and rounded to the nearest valid integer.
7 Experiments
Experiments evaluate visual MPC on pushing tasks, including novel objects, multi-objective arrangements, and obstacle-induced occlusions. The proposed planning cost substantially outperforms the prior cost, while SNA succeeds more often than DNA when occlusion must be handled.
- Experiments: The evaluation compares SNA and DNA on long pushes and multi-objective tabletop tasks using a Sawyer robot and varied objects.
- Pushing benchmark: 20 trajectories with 3 novel objects and 1 training object were collected for the pushing benchmark, with final goal-pixel distance annotated by humans.
- Pushing benchmark: The proposed planning cost substantially outperforms the prior planning cost, while SNA and DNA perform comparably with the new cost on tasks without occlusions.
- Multi-objective pushing: The occlusion task requires moving one object around a stationary obstacle while the gripper may occlude that obstacle.
- Multi-objective pushing: Table 2 reports improvement from the starting distance on 8 object/goal configurations, with higher pixel values indicating better performance.
- Multi-objective pushing: Across four arrangements containing two seen and two unseen objects, SNA usually finds valid trajectories whereas DNA is mostly unable to find a solution.The findings are presented as evidence for the importance of temporal skip connections.
8 Discussion and Future Work
The demonstrated system solves multi-objective pushing despite occlusions and supports more complex behaviors through mixed discrete and continuous actions. The authors identify short behaviors, tracking degradation, and limited random-data coverage as important boundaries.
- Discussion: Visual predictive models trained entirely on random pushing videos support model-predictive control for multi-objective pushing despite occlusions.
- Discussion: Combining discrete and continuous actions enables behaviors such as lifting the gripper to move over objects.
- Limitations: The experiments use relatively short behaviors, and repeated occlusions can eventually cause the model to lose track of the target pixel.
- Future work: More complex behaviors such as picking and placing may be difficult to learn from only randomly collected data.The authors expect more goal-directed data collection to improve complex-task performance.
Hyperparameters
The video prediction network was trained on randomly windowed 15-step sequences drawn from 44,000 trajectories, using Adam optimization and minibatches of 32. Visual MPC used three CEM iterations with 200 sampled action sequences and 10 selected samples per iteration.
- The network trained on randomly shifted 15-step windows from 44,000 trajectories of length 30, providing data augmentation.
- Training ran for 66,000 iterations with minibatch size 32, learning rate 0.001, and the Adam optimizer.
- Visual MPC used 3 CEM iterations, sampling 200 action sequences and selecting the best 10 at each iteration.
Prototyping in Simulation
Simulation enabled rapid, reset-free evaluation of visual MPC, including data-scaling, obstacle, viewpoint, and failure-case analyses. The model handled marked obstacles and small viewpoint changes but remained limited by distribution drift, difficult long-horizon sampling, and uncertainty from multiple collisions.
- Simulation Setup: Simulation enabled orders-of-magnitude-faster data collection and a benchmark without manual resets or object-position labeling.The benchmark used a simple block-pushing task simulated in MuJoCo.
- Data Scaling: 44,000 trajectories showed that prediction quality and visual MPC performance improved with more training data.With 5% of the data, the model failed to predict object movement; 30% produced more smeared distributions than 100%.
- Failure Cases: Visual MPC failed for substantially different object appearances, designated-pixel drift, and action sequences that did not move the pixel toward distant goals.More CEM samples could sometimes address distant goals, but this slows planning; trackers and macro-actions were proposed as potential remedies.
- Obstacle Handling: When obstacles were explicitly marked with a second designated pixel, the planner usually found a path around them.Clutter did not negatively affect performance when the object had a free path, and an additional cost function enabled obstacle avoidance.
- Failure Cases: The current model could not infer that an object should be pushed around an unmarked obstacle because multiple collisions created large uncertainty.This limitation arose when the arm pushed one object and that object collided with another.
- Viewpoint Robustness: The model was robust to viewpoint changes of several centimetres in translation and several degrees in orientation.The authors suggest training from multiple viewpoints could improve robustness but might require more data.