Source-linked AI summary
Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations
Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser
TL;DR
The paper tackles the lack of effective, diverse training data for grasping systems that combine 6DoF control with closed-loop feedback. It introduces a low-cost handheld demonstration interface and trains a reinforcement-learning model using action-view rendering; the resulting system achieves reliable grasping in static and dynamic scenes and transfers to real robots.
Problem
Most grasping algorithms are limited to top-down movements or open-loop execution because effective training data for higher-dimensional, diverse settings is difficult and expensive to collect.
Method
A $600 handheld gripper with a wrist-mounted RGB-D camera collects diverse human demonstrations for reinforcement-learning visual value functions using action-view rendering during closed-loop 6DoF control.
Results
The system achieves 92% grasping success in static scenes and 88% in dynamic scenes with moving objects, while transferring to real robot platforms.
Takeaways & Limitations
The demonstrations improve grasping performance and learning efficiency, while 6DoF movement and adaptive closed-loop control handle varied environments and moving objects.
Takeaways & Limitations
View-based rendering does not model contact physics, which may matter during in-contact manipulation.
Abstract
from arXiv · showhide
Intelligent manipulation benefits from the capacity to flexibly control an end-effector with high degrees of freedom (DoF) and dynamically react to the environment. However, due to the challenges of collecting effective training data and learning efficiently, most grasping algorithms today are limited to top-down movements and open-loop execution. In this work, we propose a new low-cost hardware interface for collecting grasping demonstrations by people in diverse environments. Leveraging this data, we show that it is possible to train a robust end-to-end 6DoF closed-loop grasping model with reinforcement learning that transfers to real robots. A key aspect of our grasping model is that it uses "action-view" based rendering to simulate future states with respect to different possible actions. By evaluating these states using a learned value function (Q-function), our method is able to better select corresponding actions that maximize total rewards (i.e., grasping success). Our final grasping system is able to achieve reliable 6DoF closed-loop grasping of novel objects across various scene configurations, as well as dynamic scenes with moving objects.
I. INTRODUCTION
The paper addresses the difficulty of combining 6DoF control with closed-loop feedback by collecting diverse human demonstrations with a low-cost handheld interface. It uses reinforcement learning and action-view rendering to train grasping policies that transfer to real robots and achieve strong static and dynamic-scene performance.
- Motivation: Most data-driven grasping methods do not combine flexible 6DoF control with closed-loop feedback for unstructured and dynamic settings.6DoF allows changes in x, y, z, roll, pitch, and yaw, while closed-loop algorithms react to visual changes during execution.
- Motivation: Training effective 6DoF closed-loop policies is difficult because robot trial-and-error is expensive and the exploration space grows exponentially with action and state diversity.Random search therefore becomes increasingly unlikely to discover useful grasping trajectories.
- Approach: A $600 handheld grabber with a wrist-mounted RGB-D camera collects 6DoF human grasping trajectories during everyday tasks in diverse environments.Visual tracking recovers trajectories from gripper-centric RGB-D videos, providing higher-diversity, lower-cost data than prior work.
- Approach: The model uses reinforcement learning to estimate visual-state values and action-view rendering to simulate candidate future observations before selecting actions closed-loop.The learned value function evaluates simulated states to predict the next gripper movement that maximizes expected rewards.
- Results: 92% grasping success in static scenes and 88% in dynamic scenes with moving objects demonstrate transfer of the learned 6DoF closed-loop system to real robots.The system also grasps novel objects across configurations including sideways wall grasps and inclined bins.
II. RELATED WORK
Prior grasping research ranges from geometry-based and open-loop visual methods to closed-loop and demonstration-based learning. Existing approaches commonly trade generality, dynamic-scene handling, data diversity, or scalability for reduced training demands.
- Vision-based grasping: Classic vision-based grasping explicitly models contact forces using object geometry, pose, and dynamics, but such prior knowledge is difficult to obtain for novel objects.
- Vision-based grasping: Object-agnostic visual policies generalize to novel objects without 3D scans or pose estimates, but open-loop execution remains sensitive to calibration errors and dynamic environments.
- Vision-based grasping: Closed-loop methods use visual observations and predicted next actions, but often constrain state-action spaces to reduce training-data requirements.QT-Opt, for example, learns top-down policies from a fixed static camera.
- Grasping data acquisition: Most self-supervised grasping systems collect data in simulation or structured laboratories, while physically moving robots expands access but remains limited to simple scenarios.
- Learning from demonstration: Human annotation and teleoperation provide positive examples efficiently but require robot expertise, whereas direct hand-interaction videos face a human-hand-to-robot-gripper kinematic gap.
III. APPROACH
The approach combines a portable wrist-camera handheld gripper, diverse in-the-wild human demonstrations, and a visual 6DoF closed-loop model trained with reinforcement learning and view-based rendering.
- Data collection: The system collects human grasping demonstrations with a handheld gripper equipped with a wrist-mounted RGB-D camera.
- Grasping model: The grasping model trains visual value functions with view-based rendering to pursue reliable 6DoF closed-loop grasping of novel objects and moving scenes.
- Data collection: Participants use the tool for everyday pick-and-place tasks across shelves, bins, refrigerators, dishwashers, and floors.
- Design motivations: The interface improves data diversity by allowing untrained people to collect manipulation data in environments that robots may find difficult to access.
- Design motivations: The tool supports challenging tasks with negligible failure rates and resembles the robot end effector to narrow the demonstration-to-robot domain gap.
A. Hardware Setup
The hardware setup is a portable handheld analogue of the robot’s end effector, combining binary gripper control, RGB-D sensing, onboard computation, and portable power.
- Hardware components: The handheld device uses a plastic grabber, Dynamixel servo, 3D-printed grip, push button, RGB-D camera, compute stick, battery, and optional monitor.
- Hardware components: The complete unit costs around $600 and runs for 5 hours on a portable 12V battery.
- Sensing and computation: A wrist-mounted camera sits 25cm from the fingertips and streams 640×480 RGB-D images to the onboard Linux computer.
- Robot analogy: The handheld gripper mirrors the real setup’s 6DoF UR5 arm, binary RG2 gripper, wrist-mounted RGB-D camera, and binary open-close behavior.
B. Data Collection and Processing
The dataset distributes collection across participants and everyday environments, using RGB-D demonstrations to recover grasping trajectories and structured picking information. This process yields 7,797 valid picking attempts spanning diverse tasks and scenes.
- Collection: Eight participants collected demonstrations across varied pick-and-place tasks and environments, encouraging diverse grasping strategies.Tasks included picking from shelves and bins, rearranging objects, and picking up trash in apartments, kitchens, offices, and warehouses.
- Trajectory recovery: The handheld device’s RGB-D videos support recovery of 6DoF camera and gripper trajectories through frame-to-frame visual tracking.The calibrated rigid transform between the camera and gripper enables camera tracking to recover gripper pose and trajectory.
- Collection: Demonstrations vary for the same object according to object pose, environment, and device user, producing diversity across objects, tasks, and environments.Figure 3 illustrates RGB-D frames, recovered motion trajectories, and reconstructed 3D scene representations.
- Processing: Each picking clip is segmented around the gripper-closing signal into pre-grasp and post-grasp trajectories, with target-object masks recovered by background subtraction.Frames before button press record approach motion, while frames between button press and release record post-grasp motion.
- Dataset output: 7,797 valid picking attempts contain pre-grasp trajectories, final grasp poses, target-object masks, post-grasp placing trajectories, and picking order.These extracted elements provide structured supervision and are illustrated by example demonstrations in Figure 3.
V. 6DOF CLOSED-LOOP VISION-BASED GRASPING
The grasping task is formulated as reinforcement learning over visual states and 6DoF actions. An off-policy Q-learning policy selects end-effector motions while robot control executes the resulting poses and closes the gripper from depth-based proximity.
- Task formulation: Closed-loop grasping requires time-varying actions that move the gripper toward an object and approach from an angle supporting stable grasping.The passage states that reinforcement learning is more effective than direct supervision for this loosely rewarded pre-grasp process.
- Reinforcement learning: The problem is modeled as a Markov decision process in which actions transition visual states and produce rewards, with Q-values representing expected total rewards.The optimal policy selects actions that maximize the expected reward return.
- Learning objective: Off-policy Q-learning iteratively minimizes the temporal-difference error between the parameterized Q-function Qθ(st,at) and a target value yt.The parameter θ may denote neural-network weights.
- Action space: The action set At contains all available actions at time t.
- State and action representation: States are wrist-camera image observations, while actions are relative 6DoF rigid transforms between current and target end-effector poses.Inverse kinematics and proportional-derivative control execute the planned motion, and depth proximity triggers gripper closure.
A. View-based Rendering as Predictive Models
The method renders candidate future wrist-camera views from a fused 3D scene, then evaluates those action-view pairs with a Q-network. This representation directly links each candidate motion to its predicted visual consequence.
- Predictive rendering: View-based rendering forward-simulates possible future states conditioned on the current state and each candidate action.It serves as a predictive model f(st,at) = ŝt+1 for the grasping system.
- Pipeline: The overview proceeds from current camera observation to TSDF-fused scene representation, rendered action-view pairs, and dense Q-value prediction.The action-view formulation removes the need to learn how actions correspond to changes in state space.
- Predictive rendering: Rendered future states approximate wrist-camera observations after relative 6DoF end-effector motions and are produced at 45×80 resolution for faster rendering and inference.
- Learning efficiency: Action-view representation improves learning efficiency by directly pairing each action, such as gripper movement, with its corresponding future state.This avoids explicitly learning or memorizing the mapping from abstract action representations to state changes.
- System components: The grasping algorithm combines 3D scene reconstruction, arbitrary-view rendering, and a deep neural network that models the value function Qθ.TSDF fusion accumulates camera observations into a voxel-grid scene representation with color for downstream ray casting.
- Action selection: At each step, the system renders candidate views, evaluates them with the Q-function, and executes the action associated with the highest predicted Q-value.The action candidates encode relative rotation and translation between current and target end-effector poses.
B. Learning from Human Demonstrations
Human demonstrations bootstrap the value function, while robot trial-and-error fine-tuning addresses the domain gap between human-collected data and real-robot experience. Demonstration labels use sparse success-based targets for pretraining.
- Demonstration bootstrapping: Human demonstrations bootstrap Qθ learning, but their data distribution requires additional treatment before effectively training robot grasping algorithms.
- Pretraining targets: Positive demonstration views receive targets yt(st,apos_t) = λ^(m−t), where t is the step and m is the episode length.
- Pretraining targets: Negative actions receive target value 0 during demonstration-based pretraining.
- Pretraining targets: The labeling scheme is used only for supervised bootstrapping, reflecting sparse rewards delivered at the end of trajectories according to final grasp success.The approach simplifies n-step Q-learning from demonstrations while maintaining the Bellman equation.
- Robot fine-tuning: Real-robot fine-tuning uses standard off-policy Q-learning with ε-greedy exploration to address the domain gap from human demonstrations.Target values are predicted from the next state, and loss is backpropagated only for actions taken by the robot.
VI. EXPERIMENTS
The experiments evaluate grasping across static scene configurations, dynamic clutter, and training strategies using grasping success rate. The algorithm operates in diverse settings, benefits from demonstration pretraining, and is compared with alternative methods.
- Evaluation setup: Grasping success rate is the evaluation metric for experiments on novel objects across different scene configurations.The metric is defined as successful grasps divided by grasping episodes.
- Runtime: Each action step takes 0.18s on average, while rendering all action views takes 0.057s and runs asynchronously with reconstruction.Reconstruction runs at 30 FPS and does not block action execution.
- Static settings: Static tests cover tabletop, bin, wall, and randomly positioned bin scenes, with bin grasping requiring collision avoidance.The wall setting uses objects hung on a flat wall 1m in front of the robot.
- Static settings: The algorithm predicts relative 6DoF positions, works from any initial starting position, and improves performance by around 18% on average after setting-specific fine-tuning.The pretrained-only model performs reasonably across scene configurations because the demonstrations are diverse.
- Dynamic settings: Dynamic tests randomly shift a pile of 10 objects between grasp attempts, including translations above 0.1m and rotations above 25°, until completion or repeated failures.Ten test runs are averaged, and the results are compared with alternative approaches in the same dynamic setting.
- Training strategy: Demonstration pretraining improves both early learning speed and post-fine-tuning performance relative to on-robot trial-and-error training.The authors attribute this advantage to the greater diversity of human demonstration examples.
VII. CONCLUSIONS AND FUTURE WORK
The paper concludes that a low-cost demonstration interface and action-view-based 6DoF closed-loop grasping algorithm support varied environments. It also identifies contact physics as an important limitation of the current predictive model.
- Contributions: The work introduces a low-cost hardware interface for collecting grasping demonstrations in diverse environments.The conclusion identifies this interface as one of the paper’s main contributions.
- Contributions: The visual grasping algorithm combines 6DoF closed-loop control with action-view-based rendering.The conclusion presents this algorithm as the second main contribution.
- Findings: Training on demonstration data improves grasping performance and learning efficiency, while 6DoF adaptive closed-loop control handles varied environments.The conclusion connects these findings to the system’s ability to operate across environment types.
- Limitations and future work: The current view-based predictive model does not represent contact physics, which may matter during in-contact manipulation.The authors propose extending it with a learnable model incorporating object and contact physics.
APPENDIX A DATA COLLECTION DEVICE: HARDWARE DETAILS
The appendix lists the handheld data-collection device’s hardware and its associated CAD models, then specifies the neural-network components used in the system.
- Hardware: The handheld data-collection device is documented through a hardware component list with associated costs.The parts are listed in Table IV.
- Hardware: CAD models are provided for the device’s 3D-printed parts.The models can be downloaded from the project webpage.
- Network architecture: The current-state encoder receives a 640 × 360 RGB-D image and its corresponding surface-normal map.Its listed architecture begins with a 7-channel convolutional input followed by normalization, activation, residual blocks, and pooling.
- Network architecture: The future-state encoder uses a convolutional layer, batch normalization, ReLU, and three residual blocks.The architecture description lists 7 input channels and 128-filter residual blocks.
- Network architecture: The action-selection network stacks residual blocks and ends with a one-channel 1 × 1 convolution.The listed residual blocks reduce the final filter count from 128 to 64 before the output layer.