Source-linked AI summary
Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
Wilson Yan, Ashwin Vangipuram, Pieter Abbeel, Lerrel Pinto
TL;DR
Deformable-object manipulation requires plannable visual representations despite missing canonical states and complex dynamics. The paper jointly learns latent representations and dynamics with contrastive estimation, then uses simple MPC for manipulation. It reports substantial improvements over standard model-based methods and transfers simulation-trained policies to a real PR2 through domain randomization.
Problem
Deformable manipulation is challenging because robots lack direct state representations and must handle complex, nonlinear dynamics.
Method
The method jointly learns visual latent representations and dynamics from offline random interactions using contrastive predictive modeling, then applies simple MPC.
Results
The method substantially improves multi-task rope and cloth manipulation over standard model-learning approaches and transfers policies from simulation to a real PR2 through domain randomization.
Takeaways & Limitations
Contrastive learning produces stronger, more plannable latent representations while requiring only random environmental data for transfer to real robots.
Abstract
from arXiv · showhide
Using visual model-based learning for deformable object manipulation is challenging due to difficulties in learning plannable visual representations along with complex dynamic models. In this work, we propose a new learning framework that jointly optimizes both the visual representation model and the dynamics model using contrastive estimation. Using simulation data collected by randomly perturbing deformable objects on a table, we learn latent dynamics models for these objects in an offline fashion. Then, using the learned models, we use simple model-based planning to solve challenging deformable object manipulation tasks such as spreading ropes and cloths. Experimentally, we show substantial improvements in performance over standard model-based learning techniques across our rope and cloth manipulation suite. Finally, we transfer our visual manipulation policies trained on data purely collected in simulation to a real PR2 robot through domain randomization.
I. INTRODUCTION
Deformable manipulation is difficult because robots lack canonical object states and must model complex, nonlinear dynamics. The paper addresses these challenges with contrastive visual model-based learning and demonstrates improved manipulation and sim-to-real transfer.
- Deformable objects lack direct, canonical state representations, making shape tracking difficult for manipulation.
- Complex, nonlinear dynamics make traditional modeling, task planning, and motion planning difficult for deformable objects.
- Model-free approaches avoid explicit state and dynamics modeling but are sample-inefficient without expert demonstrations, often requiring millions of samples.
- The proposed framework jointly learns visual latent representations and deformable-object dynamics using contrastive optimization.
- Offline random interactions support latent-model learning, after which simple MPC manipulates ropes and cloths toward visual goal configurations.
- The method improves multi-goal rope and cloth manipulation over standard model-learning approaches and transfers policies to real robots without additional real-world training data.
II. RELATED WORK
Prior work uses simulation, simplified controllers, and representation-learning methods to address deformable manipulation, but large state spaces and complex dynamics remain difficult for efficient planning.
- Simulation-based planning has addressed deformable manipulation, but large state spaces make correct and computationally efficient planning difficult.
- Model-based servoing simplifies dynamics by controlling the end-effector toward a goal, yet complex dynamics can produce local minima.
- Jacobian controllers avoid requiring explicit models, providing another approximation-based strategy for deformable-object control.
- Contrastive predictive methods have learned semantic and high-level representations across words, images, video, and speech using positive and negative examples.
III. CONTRASTIVE FORWARD MODELING (CFM)
Contrastive Forward Modeling learns a latent predictive model from image-action transitions rather than planning directly in pixel space. Joint encoder-forward-model training structures the latent space for prediction and planning.
- The framework learns an encoder and latent forward model for deformable-object observations and actions.
- Training uses image, next-image, and action tuples while jointly learning the encoder and forward model.
- Latent-space prediction avoids relying on pixel-value distances, which may not correspond to true visual or task-relevant distances.
- The learned latent dynamics predict the next encoded observation, ẑt+1, from the current latent state and action.
B. Contrastive Models
The contrastive model aligns predicted and observed next-state embeddings while separating incorrect alternatives. MPC then selects actions whose predicted embeddings are closest to the goal embedding.
- The framework jointly learns an encoder and forward model with the InfoNCE contrastive loss.
- Positive predicted-next-state pairs are aligned, while negative samples representing incorrect next states are pushed apart.
- The similarity function reflects the forward model’s objective of minimizing ℓ2 distance between predicted and observed latent next states.
- MPC samples actions, predicts their next latent states, and chooses the action whose prediction is closest to the goal embedding.
IV. EXPERIMENTAL EVALUATIONS
The experiments evaluate contrastive model-based learning for deformable-object manipulation across simulated rope and cloth tasks, with questions spanning representation quality, design choices, and real-robot transfer.
- The evaluation asks whether contrastive methods improve latent spaces and forward models for planning, which design choices matter, and whether manipulation transfers to a real robot.
- The simulated environments use 64 × 64 × 3 RGB observations rendered from an overhead camera.
- Simulation Tasks: The rope is modeled with 25 geoms and a four-dimensional action space, while its initial state is randomized with 120 actions.
- Simulation Tasks: The cloth is modeled as a 9×9 grid of geoms with a five-dimensional action space, and its initial state is randomized with 50 actions.
- Evaluation: Performance is evaluated by planning toward a goal image and summing pairwise geom distances between achieved and true goal states, averaged over 1000 trials.
B. Data Collection
The study collects randomized simulation trajectories for offline learning, then compares the proposed method with several model-based and model-free baselines under one-step MPC planning.
- The dataset contains 4000 rope trajectories and 8000 cloth trajectories, each 50 steps long, totaling 200k and 400k samples respectively.
- Trajectory Comparison: The simulator trajectories in Figure 3 share the same start state and horizontal-line goal, with each trajectory run for 20 actions.
- Baselines: The baseline suite includes a random policy, pixel-space visual forward model, autoencoder with latent dynamics, PlaNet, and a joint dynamics model.
- Planning: All methods use binary RGB-threshold segmentation to constrain pick points to the rope or cloth and use MPC with one-step prediction.
- Planning: During closed-loop planning, each method samples 100 possible one-step trajectories.
D. Training Details
All models share the same convolutional encoder architecture, while the proposed method uses a latent forward model whose linear transformation parameters are predicted by an MLP.
- All models use six 2D convolutional layers with specified kernels, strides, and filter sizes, followed by flattening and a fully connected layer that produces latent z.
- The forward model is an MLP with two hidden layers of size 32 that outputs parameters for a linear transformation applied to z_t.
- For CFM, other batch elements provide 127 negative samples per positive pair in contrastive learning.
E. Does Using Contrastive Models Improve Performance?
Across rope and cloth environments, the contrastive method performs better on randomly sampled goals with and without domain randomization, while baseline behavior varies by latent-space regularization and planning space.
- The proposed method performs better on all randomly sampled goals with and without domain randomization, indicating stronger generalization in latent spaces for planning.
- Latent-Space Baselines: The autoencoder performs well in some scenarios but poorly under domain randomization, while PlaNet is reasonably competitive yet also fails with domain randomization.
- Latent-Space Baselines: The joint dynamics model performs best among baselines on domain-randomized data, but the proposed method outperforms it on every task.
- Pixel-Space Baseline: The visual forward model generally performs poorly for low-area rope goals, does better on cloth flattening, and degrades sharply under domain randomization.
- Ablation: The ablation study examines forward-model architecture and contrastive similarity function using the same lower-is-better evaluation metric as Table I.
F. Ablations on Contrastive Models
The ablations examine how contrastive similarity functions and forward-model architectures affect predictive modeling performance, with results reported using final-goal geometric distance.
- Forward Model Architectures: The forward-model ablation compares linear, small-MLP, and MLP-parameterized linear-transformation architectures.
- Evaluation Setup: The rope and cloth trajectories use 40 and 100 actions, respectively, between start and final states.
- Evaluation: The ablation measures performance as the sum of pairwise geometric distances between final and goal images.
- Contrastive Similarity Functions: The study compares the proposed similarity function with InfoNCE and log-bilinear similarity functions.
- Contrastive Similarity Functions: The proposed similarity function gives the largest performance boost because it better matches the objective of learning correct forward dynamics.
2) Forward models architectures:
The architecture study finds that more expressive latent forward models are important, while the real-robot setup uses domain randomization and MPC-based action selection.
- Forward models architectures:: Performance drops most with a linear dynamics model and slightly with an MLP used for both rope and cloth tasks.
- Forward models architectures:: These results demonstrate the need for more complex models for latent forward-dynamics learning.
- Real Robot Setup: Figure 5 compares baseline and proposed-method rope trajectories on a real robot from one start state toward a 135° goal orientation.
- Real Robot Setup: Domain randomization varies lighting, texture, friction, damping, inertia, and mass during simulator training to support real-world transfer.
- Real Robot Setup: The policy replans at every time step using MPC, segmentation-based pick locations, and 100 randomly sampled action deltas.
2) Evaluation Metrics:
Evaluation uses segmented-image intersection over repeated action sequences to compare real-world rope and cloth manipulation, where the proposed method performs best across tasks.
- 2) Evaluation Metrics:: Real-world evaluation compares random actions, an autoencoder, a joint dynamics model, and the contrastive method.
- 2) Evaluation Metrics:: The metric is the pixel intersection between each segmented final image and the segmented goal image, rather than intersection over union.
- 2) Evaluation Metrics:: The models run for 40 rope actions or 100 cloth actions, and the highest-intersection observation is selected.
- 2) Evaluation Metrics:: The proposed model performs best on all rope and cloth tasks, while the joint dynamics model is second best.
- 2) Evaluation Metrics:: The paper reports stronger and more plannable latent representations and transfer from simulation without additional real-world training.