Source-linked AI summary
R3M: A Universal Visual Representation for Robot Manipulation
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, Abhinav Gupta
TL;DR
Robotic manipulation policies trained from images are often data intensive, while robot datasets remain costly and limited in diversity. The paper pre-trains R3M on diverse human video and language using temporal, alignment, and sparsity objectives, then evaluates it as a frozen perception module. R3M outperforms training from scratch and prior visual representations across simulated tasks and supports real-world manipulation learning with limited demonstrations.
Problem
Image-based robot manipulation commonly relies on task-specific end-to-end training, which is data intensive, while available robot-interaction datasets have limited diversity.
Method
R3M pre-trains a reusable visual representation on diverse human video and language using time-contrastive learning, video-language alignment, and sparsity-oriented regularization, then freezes it for downstream policy learning.
Results
Over 10%: R3M outperforms learning from scratch and prior visual representations across 12 tasks, 9 viewpoints, and 3 simulation environments.
Takeaways & Limitations
R3M enables data-efficient imitation learning across simulated and real robot manipulation tasks, including challenging Panda-arm tasks in a cluttered apartment.
Takeaways & Limitations
The evaluation is limited to behavior cloning with a small number of task demonstrations, leaving performance in reinforcement learning settings unresolved.
Abstract
from arXiv · showhide
We study how visual representations pre-trained on diverse human video data can enable data-efficient learning of downstream robotic manipulation tasks. Concretely, we pre-train a visual representation using the Ego4D human video dataset using a combination of time-contrastive learning, video-language alignment, and an L1 penalty to encourage sparse and compact representations. The resulting representation, R3M, can be used as a frozen perception module for downstream policy learning. Across a suite of 12 simulated robot manipulation tasks, we find that R3M improves task success by over 20% compared to training from scratch and by over 10% compared to state-of-the-art visual representations like CLIP and MoCo. Furthermore, R3M enables a Franka Emika Panda arm to learn a range of manipulation tasks in a real, cluttered apartment given just 20 demonstrations. Code and pre-trained models are available at https://tinyurl.com/robotr3m.
1 Introduction
The paper asks whether diverse human-video pre-training can provide reusable visual representations for efficient robot manipulation. R3M combines temporal, semantic, and sparsity-oriented objectives and is evaluated as a frozen perception module across simulated and real tasks.
- End-to-end training from scratch is often data intensive and can limit generalization in image-based robot manipulation.
- Robot representation learning is constrained by the cost and limited diversity of datasets containing robots interacting with the physical world.
- R3M combines time-contrastive learning, video-language alignment, and an L1 sparsity penalty to pre-train a reusable visual representation from diverse human videos.
- R3M is used as a frozen perception module for downstream policy learning across simulated benchmarks and real robot manipulation tasks.
- Over 10%: R3M features outperform CLIP, supervised ImageNet, MoCo, and learning from scratch across 12 tasks, 9 viewpoints, and 3 simulation environments.On a Franka Emika Panda, R3M also reaches over 50% average success on challenging tasks with less than 10 minutes of human demonstrations.
2 Related Work
Prior robotics representations often rely on narrow in-domain data, static images, or task-specific signals. This work instead studies diverse human video and language data as reusable visual representations for data-efficient robotic manipulation.
- In-domain representation methods use target-environment and task data, which limits their scope to those domains and tasks.
- Prior general motor-control representations draw on web images, class labels, or robot data, whereas R3M uses diverse human video and language data.
- Earlier human-video approaches typically use small datasets resembling the robot environment, while this work leverages diverse datasets such as Ego4D.
- Language has commonly served as task specification or reward learning in manipulation, while R3M uses language annotations to learn reusable visual representations for control.
- Robot-data scaling efforts often include only a handful of environments, making generalization across unseen scenes difficult.
- The paper’s contribution is evaluating whether representations trained on diverse human interaction video and language improve robotic manipulation learning, rather than proposing a novel representation-learning algorithm.
3 R3M: Reusable Representations for Robotic Manipulation
R3M is a reusable visual representation pre-trained on diverse human video and language data for efficient downstream robotic manipulation. Its training combines temporal, semantic, and compactness objectives so frozen image embeddings can support policy learning across tasks and environments.
- Approach: R3M pre-trains a single reusable image encoder on diverse human video and language data for downstream robotic manipulation.The encoder maps images to continuous embeddings that can be reused as state representations for downstream policies.
- Design criteria: The representation is designed to capture temporal dynamics, semantically relevant features, and compact state information.These criteria correspond to sequential interaction, task-relevant scene content, and reduced irrelevant dimensionality.
- Time Contrastive Learning: Time contrastive learning makes embeddings of temporally closer frames more similar than embeddings of farther or cross-video frames.Training samples ordered frames from video sequences and uses different-video examples as negatives.
- Video-Language Alignment: Video-language alignment trains embeddings to capture scene features predictive of language describing task-relevant transitions.The auxiliary module scores whether an image transition completes the paired language instruction, with mismatched-video examples serving as negatives.
- Regularization: L1 and L2 regularization is used to encourage sparse, compact representations that may mitigate state-distribution shift in low-data imitation learning.The final R3M objective is a weighted sum of the contrastive losses and regularization terms.
4 Experiments
Experiments evaluate R3M as a frozen visual perception module for behavior-cloning policies across simulated environments, component ablations, data-versus-algorithm comparisons, and real-world tasks.
- Evaluation framework: The evaluation reuses frozen visual representations for downstream behavior cloning across MetaWorld, Franka Kitchen, and Adroit manipulation domains.The simulated suite contains 12 tasks, while representations are compared with standardized behavior-cloning training and multiple seeds, viewpoints, and demonstration sizes.
- Efficient imitation learning: Across 12 simulated tasks, R3M achieves approximately 62% success, exceeds training from scratch by more than 20%, and outperforms prior representations by more than 10% on average.R3M is best in all three environments and on 11 of 12 tasks; CLIP is the strongest comparison on MetaWorld, while MoCo (345) (PVR) is stronger on Franka Kitchen and Adroit.
- Component ablations: Removing crop augmentation or L1 regularization reduces average performance by approximately 2%, while removing video-language alignment has the largest negative impact across environments.The language-alignment ablation is especially damaging in Adroit, whereas the L1 ablation particularly affects Franka Kitchen and MetaWorld.
- Data versus algorithm: MoCo-Ego4D achieves an average success rate approximately 10% lower than R3M in both Franka Kitchen and Adroit, while MVP performs approximately 20% worse.These comparisons use the same or more data and compute, separating the contribution of the dataset from the training objective.
- Real-world robot learning: With just 20 demonstrations per task, R3M supports learning five household manipulation tasks with a Franka Emika Panda in a real graduate-student apartment.Compared with CLIP, R3M performs similarly on drawer closing and better on the other four tasks, yielding nearly double the average success rate.
5 Limitations and Future Work
The evaluation is limited to behavior-cloning imitation learning with few demonstrations, while future work considers reinforcement learning and richer uses of video representations.
- Limitations: The current evaluation is limited to imitation learning, specifically behavior cloning, with a small number of task demonstrations.The authors identify reinforcement learning as an important untested setting.
- Future Work: R3M currently provides only a single-frame state representation rather than representations for reward learning or task specification.The authors propose studying whether R3M embeddings or language grounding can provide a useful reward signal.
- Future Work: The authors plan to study how R3M performs in reinforcement-learning settings and what changes may improve its performance there.
A R3M Training Details
R3M trains on short Ego4D video clips paired with natural-language annotations, sampling resized and cropped frames from each clip for faster training.
- R3M Training Details: R3M uses shorter Ego4D video clips paired with natural-language annotations for training.Ego4D contains several-hour videos within scenes, subdivided into annotated sub-clips.
- R3M Training Details: For faster training, R3M parses each video clip into 224x224 resized and cropped frames and samples frames individually.
A.2 Training Architecture and Hyper-Parameters
The training architecture supports standard ResNet encoders and combines frame representations with language features through a multilayer prediction head, using fixed sampling and optimization settings.
- Training Architecture: R3M can use ResNet18, ResNet34, or ResNet50 as the visual encoding architecture.
- Training Architecture: The language prediction head is a five-layer MLP that combines visual and DistilBERT sentence-encoder outputs into a scalar score.The DistilBERT sentence encoder has output dimension 768.
- Hyper-Parameters: Each training batch contains 16 video clips with five sampled frames per clip, including initial, final, and three intermediate frames.Initial and final frames come from the first and last 20% of each clip.
- Hyper-Parameters: R3M models train for one million steps in the experiments and 1.5 million steps in released models, using learning rate 0.0001.
- Hyper-Parameters: The training objective uses λ1 = 1, λ2 = 1, λ3 = 0.00001, and λ4 = 0.00001.
A.3 Additional Implementation Details
R3M training uses multiple positive and negative video pairs to stabilize optimization, and the released code provides a simple loading interface.
- Additional Implementation Details: R3M uses three negative examples sampled from different videos in each batch for the contrastive training equations.
- Additional Implementation Details: Equation 2 includes multiple positive frame pairs within one video and corresponding negative pairs, using temporal relationships among frames.
- Additional Implementation Details: Using more positive examples from one video and multiple negatives from different videos stabilizes training.
- Example Usage: R3M can be loaded from the codebase after cloning the repository and installing it with pip install -e .
- Example Usage: The example usage imports load_r3m from the r3m package.
B.1 Simulation Environments
The evaluation spans three simulated manipulation environments and real apartment tasks, using randomized settings, visual observations, proprioception, and imitation-learning demonstrations. Comparisons include frozen or pretrained visual representations and a learned policy architecture.
- Simulation Environments: The simulated evaluation covers Franka Kitchen, MetaWorld, and Adroit environments.Franka Kitchen uses five tasks, MetaWorld uses five standard tasks, and Adroit uses Pen and Relocate.
- Simulation Environments: Randomized object, scene, or goal positions make the simulated tasks vary across episodes.Franka Kitchen randomizes kitchen position; MetaWorld randomizes target objects; Adroit randomizes visually specified goals.
- Data and Evaluation: Imitation experiments use 5, 10, or 25 demonstrations in Franka Kitchen and MetaWorld, and 25, 50, or 100 in Adroit.The real-world figure reports learning the apartment tasks from 20 demonstrations.
- Real Robot Experiments: The real-world evaluation uses a Franka Emika Panda in an apartment for five randomized manipulation tasks.Tasks include putting lettuce in a pan, pushing a mug, closing a drawer, putting a mask in a drawer, and folding a towel.
- Real Robot Experiments: Real-robot observations combine task-specific USB-webcam RGB images with end-effector position during imitation learning.The camera viewpoint differs by task, and the end-effector position is concatenated with the image embedding.
- Policy and Data: The downstream policy is a two-layer MLP preceded by BatchNorm, receiving concatenated visual embeddings and proprioceptive data.Models use a ResNet50 base architecture, while expert data comes from state-based agents, heuristics, or human teleoperation depending on the setting.
C.1 How does performance vary across viewpoint and demo dataset size?
R3M’s performance improvement remains consistent across camera viewpoints and demonstration dataset sizes. It is the highest-performing representation in all evaluated viewpoint and dataset-size conditions, whereas prior-method rankings can change with viewpoint.
- Viewpoints and Dataset Size: R3M is the highest-performing representation across all evaluated viewpoints and dataset sizes.The reported comparison uses average success rates across viewpoints and demonstration dataset sizes.
- Viewpoints: R3M’s performance improvement remains consistent across viewpoints, unlike the changing MoCo (345) and CLIP ranking.The prior-method ranking changes according to the selected viewpoint.
- Dataset Size: R3M’s performance benefit remains consistent as demonstration dataset size increases, even while absolute performance improves.This pattern is reported for the Franka Kitchen and Adroit environments.
C.2 Performance Breakdown By Task
R3M achieves the highest success rate on 11 of 12 evaluated manipulation tasks. Each task-level success rate averages results across views, demonstration sizes, and random seeds.
- Per-task Results: 11/12 tasks have R3M as the highest-performing method.The task-level success rates are averaged over three views, three demonstration sizes, and three seeds.
- Evaluation Protocol: Each reported task success rate averages performance across 3 views, 3 demo sizes, and 3 seeds.This aggregation provides the basis for the per-task comparison.