Source-linked AI summary
VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors
Yifeng Zhu, Abhishek Joshi, Peter Stone, Yuke Zhu
TL;DR
Deep imitation-learning policies can lack robustness to covariate shifts and environmental perturbations. VIOLA addresses this gap with object-proposal-based representations and a transformer policy for closed-loop manipulation. It reports superior performance to state-of-the-art baselines and successful deployment on challenging long-horizon physical-robot tasks.
Problem
Deep imitation-learning visuomotor policies can associate actions with task-irrelevant visual factors, limiting robustness to covariate shifts and environmental perturbations.
Method
VIOLA uses general object proposals from a pre-trained RPN to build object-centric representations, then applies a transformer-based policy for action generation.
Results
VIOLA outperforms state-of-the-art imitation-learning baselines in simulation and the real world and succeeds on challenging long-horizon physical-robot tasks.
Takeaways & Limitations
Object-based structural priors improve the robustness of deep imitation learning against object variations and environmental perturbations.
Takeaways & Limitations
The behavioral-cloning formulation assumes access to a set of teleoperated demonstrations.
Abstract
from arXiv · showhide
We introduce VIOLA, an object-centric imitation learning approach to learning closed-loop visuomotor policies for robot manipulation. Our approach constructs object-centric representations based on general object proposals from a pre-trained vision model. VIOLA uses a transformer-based policy to reason over these representations and attend to the task-relevant visual factors for action prediction. Such object-based structural priors improve deep imitation learning algorithm's robustness against object variations and environmental perturbations. We quantitatively evaluate VIOLA in simulation and on real robots. VIOLA outperforms the state-of-the-art imitation learning methods by $45.8\%$ in success rate. It has also been deployed successfully on a physical robot to solve challenging long-horizon tasks, such as dining table arrangement and coffee making. More videos and model details can be found in supplementary material and the project website: https://ut-austin-rpl.github.io/VIOLA .
1 Introduction
VIOLA addresses the limited robustness of deep imitation-learning visuomotor policies by using object proposals as priors for object-centric representations and transformer-based action prediction.
- Deep imitation-learning policies can associate actions with task-irrelevant visual factors, causing poor generalization under covariate shifts and environmental perturbations.
- Vision-based manipulation enables autonomous robots to interact with everyday environments through visual perception and motor behavior.
- VIOLA uses a pre-trained Region Proposal Network to obtain general object proposals and extract proposal-region features for object-centric scene representations.
- The transformer policy processes discrete tokens from object-centric representations and uses multi-head self-attention to focus on task-relevant regions for decision-making.
2 Related Work
Prior manipulation imitation-learning methods include trajectory-focused non-parametric approaches and neural parametric policies, while visual representations trade off detector dependence against generalization and robustness.
- Non-parametric methods such as DMP and PrMP learn manipulation behaviors from few demonstrations but typically generate open-loop trajectories and handle high-dimensional observations poorly.
- Parametric neural-network approaches show promise for vision-based manipulation but remain part of the broader imitation-learning design space.
- Object bounding boxes provide intermediate visual representations but require fine-tuned or category-specific detectors and generalize poorly to previously unknown objects.
- End-to-end policies trained on raw observations avoid manual intermediate representations but are prone to covariate shift and causal confusion.
3 Approach
VIOLA builds a temporally composed object-centric representation from proposal regions and context features, then uses a transformer policy to generate closed-loop manipulation actions.
- Problem Formulation: VIOLA formulates manipulation as learning a closed-loop policy π mapping raw sensory states to motor commands in a discrete-time Markov Decision Process.
- Object-Centric Representation: The method uses top K proposals from a pre-trained RPN as approximate objects, extracting visual and positional region features without ground-truth object states.
- Object-Centric Representation: VIOLA composes per-step features from the last H + 1 observations with temporal positional encodings to form z_t.
- Object-Centric Representation: Global workspace, eye-in-hand, and proprioceptive context features complement local region features for task-stage, occlusion, and robot-state information.
- Object-Centric Representation: Ablations indicate that temporal positional encoding is important for retaining feature ordering, especially with the transformer-based policy.
- Transformer-based Policy: A transformer with multi-head self-attention reasons over object-centric tokens and outputs an action-token latent vector that an MLP converts into actions.
4 Experiments
The experiments compare VIOLA with imitation-learning baselines in simulation and real-world manipulation, while testing model designs and transformer attention. Results include quantitative success-rate comparisons and qualitative evidence of task-relevant attention.
- Experimental Setup: The evaluation covers three simulation tasks and three real-world tasks spanning diverse manipulation behaviors.Simulation uses Sorting, Stacking, and BUDS-Kitchen; real-world tasks include Dining-PlateFork, Dining-Bowl, and Make-Coffee.
- Ablations: Table 2 evaluates how VIOLA model designs affect success rates in the Sorting task, marking changes larger than 2% with arrows.The supplied caption identifies the task, metric, and threshold but does not provide individual ablation values.
- Baselines: VIOLA is compared against BC, OREO, BC-RNN, and a regular-grid patch variant called VIOLA-Patch.The comparison uses 100 initial configurations, three random seeds, and identical pre-generated configurations across policies.
- Simulation Results: 50.8% success rate improvement over BC-RNN is reported in Canonical, and 44.1% across three testing variants.These results are reported in Table 1 as evidence favoring object-centric representations for visuomotor imitation.
- Attention Visualization: Transformer attention concentrates on task-relevant regions while incorporating robot and scene context for spatial reasoning.For grasping, attention focuses on the k-cup while considering robot fingers and the coffee machine; for closing, it spans k-cups, gripper, and machine.
- Real-World Results: 46.7% average success-rate improvement over BC-RNN is reported in real robot tasks.The Make-Coffee observations describe robust k-cup grasping and coffee-machine opening by VIOLA, while BC-RNN misses grasps or releases the gripper mistakenly.
5 Conclusion
VIOLA learns robust closed-loop visuomotor policies from object-centric representations built from general proposals, using a transformer to select relevant regions for action generation. The paper reports strong simulation and real-world performance but identifies offline-data limitations and dependence on an unadapted pre-trained RPN.
- Conclusion: VIOLA builds object-centric representations from general object proposals, combining proposal, context, and robot-state information for transformer-based action generation.The representation includes visual and positional proposal features together with global scene and robot-state context.
- Conclusion: VIOLA outperforms state-of-the-art baselines in simulation and the real world, with ablations validating the impact of its model components.The conclusion also states that the approach supports robust visuomotor policy learning.
- Limitations: Offline demonstration datasets constrain VIOLA, motivating future online use of its object-centric representation for improvement from rollout experience.The paper also notes that its pre-trained RPN is used without adaptation and may not generalize under aggressive distribution shifts from natural scenes.
A Additional Implementation Details
The additional implementation-details section states that it provides model implementation information beyond the main text.
- Additional Implementation Details: The section provides implementation details not included in the main text.
A.1 Model Details
The model uses shared visual and proprioceptive inputs, a ResNet-18 and ROI Align feature pipeline, transformer processing, positional encodings, augmentation, and likelihood-based training.
- Visual Features: ResNet-18 produces a 16 × 16 spatial feature map, and ROI Align pools proposal regions into 6 × 6 features before linear projection.
- Sensor Modalities: All models use workspace and eye-in-hand RGB images, joint configuration, and binary gripper state.The eye-in-hand view follows prior manipulation-imitation setups and has been empirically shown to contribute significantly to policy performance.
- Network Architecture: The policy uses four transformer encoder layers, six attention heads, 1024 hidden units per fully connected layer, and a five-mode GMM output head.
- Positional Encoding: Region positional features are computed from bounding-box coordinates, with dimensionality D matching the input tokens.
- Positional Encoding: Temporal positional encoding applies sinusoidal functions by dimension, using sine for even indices and cosine for odd indices.
- Positional Encoding: The positional-encoding frequency is 10 because the input sequence is shorter than typical natural-language sequences.The smaller value is chosen to keep input-token positional features sufficiently distinguishable.
- Data Augmentation: Color jittering and pixel shifting are applied to VIOLA and baselines, while random erasing is used for VIOLA variants to reduce overfitting to region proposals.Random erasing is omitted for baseline behavioral-cloning models because it lowers their performance relative to no erasing.
- Training Details: VIOLA is trained for 50 epochs with batch size 16, learning rate 10^-4, and negative log likelihood for GMM action supervision.The checkpoint is selected by the lowest loss over all demonstration data because validation loss does not correlate with policy performance.
B Additional Experiments
This section introduces two additional experiments supporting the authors’ design choices.
- Two additional experiments are compiled to support some of the design choices.
B.1 Choice of K in simulation
The authors assess pre-trained RPN object-proposal coverage in simulation to select the proposal count K. They choose K = 20 for simulation experiments.
- Pre-trained RPN proposals are evaluated across simulation environments using recall rates to measure coverage of scene objects, including robots.Coverage is computed at IoU = 0.5.
B.2 Visual Feature Design
VIOLA learns spatial feature maps from scratch for actionable continuous-control features. An ablation finds this outperforms directly using or fine-tuning RPN’s pre-trained FPN features, while using fewer trainable parameters.
- Visual Feature Design: VIOLA learns spatial feature maps from scratch to extract actionable visual features informative for continuous control.
- Visual Feature Design: Pre-trained FPN features without fine-tuning perform worse overall than features learned from scratch.
- Visual Feature Design: Fine-tuning FPN nearly matches, but does not outperform, the original model trained from scratch.
- Visual Feature Design: The scratch-trained convolutional encoder uses 14 MB of trainable parameters, compared with about 200 MB for FPN.
C Environment Details
The experiments use specified robot hardware, manipulation tasks, demonstrations, success criteria, and evaluation horizons. Simulation and real-world settings differ in task coverage, data volume, and success assessment.
- Overall setup: All tasks use a 7-DoF Franka Emika Panda arm with operational-space end-effector control and a binary parallel-gripper command at 20 Hz.Kinect Azure provides workspace views and Intel Realsense D435i provides eye-in-hand views.
- Task Details: The tasks include Sorting, Stacking, BUDS-Kitchen, and Dining-PlateFork, each defined by distinct object-placement or manipulation goals.
- Data Collection: The dataset contains 100 human-teleoperated demonstrations per simulation task and 50 per real-world task.Color augmentation increases visual diversity under surrounding-lighting variations.
- Task Success Determination: Simulation success requires the programmed goal function to return true for the evaluated rollout.For Sorting, both boxes must be in the bin.
- Evaluation Horizons: Canonical evaluation horizons are 1000 steps for Sorting, 800 for Stacking, and 1500 for BUDS-Kitchen.Testing variants use 200 additional steps per task.