Source-linked AI summary
DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation
Yuzhe Qin, Binghao Huang, Zhao-Heng Yin, Hao Su, Xiaolong Wang
TL;DR
Dexterous manipulation policies face challenges generalizing across objects and transferring to real multi-finger hands. This work trains point-cloud RL policies with imagined hand points and contact-based rewards; the authors report direct sim-to-real transfer with category-level generalizability.
Problem
Dexterous manipulation remains challenging because high-dimensional hand actions complicate optimization, while RL policies still face difficulties generalizing to diverse objects and deploying on real multi-finger hands.
Source: §1 · p. 2Method
The framework trains point-cloud RL policies for dexterous hands, augmenting observations with imagined hand points and using contact-pair information in rewards rather than observations.
Source: §1 · p. 2Results
The authors report direct sim-to-real transfer with category-level generalizability for dexterous manipulation using point-cloud representation.
Source: §1 · p. 2Takeaways & Limitations
Contact-based rewards improve sample efficiency and learning stability while avoiding dependence on contact sensors unavailable for many real robot models.
Source: §1 · p. 2The method was trained and tested on only two tasks, limiting the scope of the proposed method.
Source: §5 · p. 8
Abstract
from arXiv · showhide
We propose a sim-to-real framework for dexterous manipulation which can generalize to new objects of the same category in the real world. The key of our framework is to train the manipulation policy with point cloud inputs and dexterous hands. We propose two new techniques to enable joint learning on multiple objects and sim-to-real generalization: (i) using imagined hand point clouds as augmented inputs; and (ii) designing novel contact-based rewards. We empirically evaluate our method using an Allegro Hand to grasp novel objects in both simulation and real world. To the best of our knowledge, this is the first policy learning-based framework that achieves such generalization results with dexterous hands. Our project page is available at https://yzqin.github.io/dexpoint
1 Introduction
Dexterous manipulation policies face optimization and generalization challenges, especially when transferring to real-world multi-finger hands. This work trains point-cloud reinforcement learning policies for novel-object grasping and novel-lever door opening, using imagined hand points and contact-based rewards.
1 Introduction
- Prior RL results on dexterous manipulation included Rubik’s Cube transfer, but the policy was trained on one object and did not generalize to diverse objects.Other point-cloud approaches faced limits from assuming complete object point clouds and ground-truth states; real-time object-state access is challenging under hand-object occlusion.Source: §1 · p. 2
- The framework targets grasping unseen objects and opening doors with previously unseen lever shapes using an Allegro Hand.For door opening, the robot rotates the lever to unlock the latch, then pulls it in a circular motion.Source: §1 · p. 2
- The authors report that point-cloud representation enables direct sim-to-real transfer with category-level generalizability for dexterous manipulation.Source: §1 · p. 2
- Imagined finger point clouds augment noisy, occluded sensor observations and can improve policy robustness and sample efficiency.The points are generated according to the robot kinematic model.Source: §1 · p. 2
- A novel reward uses contact-pair information without adding contact to the observation, improving sample efficiency and learning stability while avoiding dependence on contact sensors.Source: §1 · p. 2
2 Related Work
Prior dexterous manipulation methods often target a single object or assume access to object states, while point-cloud RL offers flexible interaction but faces real-world observation challenges. This work combines multi-object training and point-cloud inputs with imagined points and contact-based rewards.
2 Related Work
- Most prior dexterous manipulation methods learn for one object or assume object states from a perfect detector, while this work trains on multiple objects and uses point-cloud inputs.Source: §2 · pp. 2–3
- Point-cloud RL lets the hand adjust its pose during object interaction, unlike grasp-proposal approaches that require motion-planning feedback to assess feasibility.Noisy, occluded real-world point clouds and training with noisy inputs remain challenges.Source: §2 · p. 3
- The proposed imagined point clouds, contact information, and multi-object training address observation and transfer challenges faced by point-cloud RL.Source: §2 · p. 3
- The reward uses each robot-hand contact link to encourage reasonable grasping behavior without requiring a real tactile sensor.Source: §2 · p. 3
3 Approach
The approach trains a point-cloud policy for dexterous manipulation with the goal of sim-to-real transfer without real-world training. It combines contact-based rewards with imagined hand point clouds to address reward design and incomplete observations.
3 Approach
- The system uses an Allegro Hand and XArm6 for grasping diverse objects and opening doors, with test-time input limited to a single-view point cloud and robot proprioception.The grasping and door-opening tasks include unseen test objects or doors; the real setup uses a RealSense D435 camera.Source: §3 · pp. 3–4
3.1 Reward Design with Oracle Contact
- The reward combines reaching, multi-finger contact, lifting, and a control penalty to guide the hand from approaching an object toward grasping and moving it to a target.The contact reward is intended to encourage meaningful contact, while lifting is separately rewarded.Source: §3.1 · p. 4
3.2 Imagined Hand Point Cloud
- Imagined hand points address finger occlusion and sparse point clouds by synthesizing finger-link points from joint positions and the robot kinematic model.The imagined points are generated from finger-link meshes using forward kinematics, in both simulation and the real world.Source: §3.2 · p. 5
3.3 Training
- PPO is trained in simulation without real-world fine-tuning, using observed and imagined points, proprioception, and the goal pose in a shared feature extractor for actor and critic.The point-cloud input marks each point as observed or imagined.Source: §3 · p. 4, §3.3 · p. 5, Figure 3 · p. 5
4 Experiments
The experiments compare single- and multi-object training, ablate imagined hand point clouds and contact-based rewards, and evaluate simulator-trained policies in the real world. They assess grasping and door opening on known and unseen objects.
4.1 Experimental Setup
- Training used shared point-cloud preprocessing, including uniform downsampling to 512 points; grasp policies trained on one object or multiple category objects, while door training used fixed lever geometry.Multi-object training randomly selected one object per episode.Source: §4.1 · pp. 5–6
- Success used task-specific criteria, including grasp placement and lift thresholds or opening a door to about 45°; the EigenGrasp baseline required a complete object model and ground-truth pose.The baseline searched a grasp database and planned an open-loop trajectory.Source: §4.1 · p. 6
4.2 Comparison of Single-object and Multi-object Training
- Policies learned grasping within 600 iterations of 20K environment steps each; single-object training slightly led on known objects, while multi-object training did better on novel objects.The study trained on single YCB objects or 10 objects from a bottle or can category, selecting one object per episode for multi-object training.Source: §4.2 · p. 6, §4.1 · p. 6, Table 1 · p. 6
4.3 Ablation Results in Simulation
- Removing contact rewards left bottle and can policies with nearly zero success; encouraging finger-object contact helped avoid local minima and learn meaningful manipulation behavior.The ablation compared four settings: with or without imagined point clouds and with or without contact-based rewards.Source: §4.3 · p. 7, Table 2 · p. 7
- Imagined hand point clouds improved training and test performance for both categories, especially bottles, by revealing fingers occluded by objects.They also supported faster early learning and smaller training variance.Source: §4.3 · p. 7
- On door opening, the policy with both design techniques outperformed ablations, though training showed larger variation, possibly because lever occlusion affected PointNet feature consistency.The proposed explanation is tentative: the authors identify heavier occlusion during lever grasping as a possible reason for the variation.Source: §4.3 · p. 7
4.4 Real-World Evaluation
- Simulator-trained point-cloud policies transferred to real-world grasping and door opening without fine-tuning, handled unseen objects, and multi-object grasp training performed better than single-object training.The grasping evaluation covered 26 objects across bottles, cans, and mixed categories; the door evaluation included two novel doors.Source: Table 3 · p. 8, §4.4 · p. 8, Table 4
5 Conclusion and Limitation
The approach is presented as the first point-cloud-input dexterous manipulation RL policy to transfer to the real world. Its evaluation covers two tasks, and the authors identify this limited task scope and the lack of temporal information as limitations.
5 Conclusion and Limitation
- Experiments cover only two tasks, limiting the scope; the authors suggest recurrent networks and temporal information as future work for long-horizon tasks.The authors identify recurrent networks and temporal information as a potential improvement.Source: §5 · p. 8
- The authors describe their approach as the first dexterous manipulation RL policy trained with point cloud inputs that transfers to the real world.They state that direct sim-to-real transfer is possible for two manipulation tasks.Source: §5 · p. 8
Appendices
The supplementary material provides implementation details and directs readers to the project page for additional visualization.
Appendices
- The supplementary material provides implementation details and points to the project page for additional visualization.The project page is https://yzqin.github.io/dexpoint.Source: Appendices · p. 13
A Environment Details
The environment details describe the object sets and preprocessing used in simulation, as well as the components and design of the task reward.
A Environment Details
- Single-object experiments use one YCB mesh, while multi-object experiments train on 10 ShapeNet objects and test on another 40 in simulation.The real-world test objects are described in the main paper.Source: A Environment Details · p. 13
- Object preprocessing scales ShapeNet instances to category-specific bounding-box diagonals and uses V-HACD convex decomposition, excluding meshes with non-manifold geometry or more than 40 parts.YCB mesh scale aligns with real counterparts; scaling is needed because ShapeNet provides no scale information.Source: A Environment Details · p. 13
- The reward combines reach, contact, lift, and action penalty; lift reward is zero without contact, discouraging knocking or unstable lifts.Weights are reach 1, contact 0.5, lift 10, and penalty 0.01; lift reward is the height difference, and action penalty is the negative squared action norm.Source: A Environment Details · p. 13
B Learning Details
The learning setup uses PPO for on-policy RL, with a PointNet visual backbone and proprioceptive features combined for policy and value prediction.
B Learning Details
- PPO is used for on-policy RL training, except in the distilling experiments.Its hyperparameters are listed in Table 5.Source: B Learning Details · p. 13, Table 5 · p. 13
- The PointNet visual features are concatenated with proprioceptive features from an MLP, and the combined representation is shared by the policy and value networks.The network architecture is detailed in Table 6.Source: B Learning Details · p. 13, Table 6 · p. 13