Source-linked AI summary
UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning
Weikang Wan, Haoran Geng, Yun Liu, Zikang Shan, Yaodong Yang, Li Yi, He Wang
TL;DR
Universal dexterous grasping from realistic point clouds and proprioception is difficult across thousands of object instances. UniDexGrasp++ combines geometry-aware curriculum learning with iterative geometry-aware generalist-specialist learning, reaching 85.4% and 78.2% success on train and test sets and outperforming UniDexGrasp by 11.7% and 11.3%.
Problem
Vision-based universal dexterous grasping must handle high-dimensional actions, noisy reinforcement-learning gradients, and large variations in object geometry and pose.
Method
UniDexGrasp++ uses GeoCurriculum and geometry-aware iterative generalist-specialist learning, with specialist assignment based on geometric clustering and iterative distillation.
Results
85.4% and 78.2% success rates on the train and test sets, respectively, outperform UniDexGrasp by 11.7% and 11.3%.
Takeaways & Limitations
The full pipeline provides universal dexterous grasping across 3000+ object instances with improved generalization over UniDexGrasp.
Takeaways & Limitations
The work evaluates dexterous grasping only in simulation and leaves real-robot extension for future work.
Abstract
from arXiv · showhide
We propose a novel, object-agnostic method for learning a universal policy for dexterous object grasping from realistic point cloud observations and proprioceptive information under a table-top setting, namely UniDexGrasp++. To address the challenge of learning the vision-based policy across thousands of object instances, we propose Geometry-aware Curriculum Learning (GeoCurriculum) and Geometry-aware iterative Generalist-Specialist Learning (GiGSL) which leverage the geometry feature of the task and significantly improve the generalizability. With our proposed techniques, our final policy shows universal dexterous grasping on thousands of object instances with 85.4% and 78.2% success rate on the train set and test set which outperforms the state-of-the-art baseline UniDexGrasp by 11.7% and 11.3%, respectively.
1. Introduction
Dexterous grasping offers versatile manipulation but is difficult because dexterous hands have high-dimensional actions and vision-based universal policies must handle noisy gradients and large task variation. UniDexGrasp++ addresses these challenges with geometry-aware curriculum and iterative generalist-specialist learning, achieving higher grasping success than UniDexGrasp.
- Motivation: Dexterous hands provide diverse grasping strategies and advanced manipulation, but their 24–30 degrees of freedom make successful execution difficult.High-dimensional action spaces increase versatility while also causing difficulty in grasp execution.
- Motivation: Vision-based universal dexterous grasping is challenging because reinforcement-learning gradients are noisy and tasks vary substantially in geometry and pose.The setting is inherently a difficult multi-task reinforcement-learning problem.
- Approach: UniDexGrasp++ uses geometry-aware task curriculum learning and geometry-aware iterative generalist-specialist learning to improve policy performance and generalizability.GeoCurriculum organizes task learning by geometric similarity, while GiGSL iteratively fine-tunes specialists and distills them into a generalist.
- Results: 85.4% and 78.2% success rates on the train and test sets, respectively, outperform UniDexGrasp by 11.7% and 11.3%.The final vision-based policy performs universal dexterous grasping on 3000+ object instances.
2. Related Work
Related work spans analytical, reinforcement-learning, imitation-learning, and policy-distillation approaches for dexterous grasping and visual policy learning. UniDexGrasp++ builds on state-to-vision distillation and generalist-specialist learning while targeting realistic observations and broad task variation.
- Dexterous Grasping: Analytical dexterous-grasping methods often simplify hand and object geometries to keep planning feasible.These simplifications limit how directly such methods represent complex grasping settings.
- Dexterous Grasping: Many reinforcement-learning and imitation-learning methods assume oracle object states, which are difficult to capture accurately in the real world.More realistic approaches instead use robot proprioception with RGB images or 3D scene point clouds.
- Visual Policy Learning: Visual policy learning has used frozen pre-trained vision backbones, multi-stage training, and state-based policy distillation to ease optimization.UniDexGrasp++ is related to methods that first train state-based policies and then distill them into vision-based policies.
- Positioning: UniDexGrasp++ extends this literature with geometry-aware task organization for universal dexterous grasping from realistic point clouds and proprioceptive information.Its overview combines state-based learning, vision-based learning, GeoCurriculum, GeoClustering, and GiGSL.
- Generalist-Specialist Learning: Generalist-specialist learning trains specialists on task subsets and distills them into a generalist covering the whole task space.This framework is used in imitation learning and policy distillation for generalization across environment variations.
3. Problem Formulation
The paper formulates universal dexterous grasping as multi-task policy learning over object instances and initial rotations. Success requires reaching a target closely across random poses, thousands of objects, and unseen instances using realistic observations.
- Observation Setting: The paper targets realistic table-top grasping from robot proprioception and scene point-cloud observations.This setting is chosen to use information that can be captured more readily in the real world.
- Task Definition: A grasping task is defined as τ = (o, R), where o is an object instance and R is its initial 3D rotation.The environment samples an object, drops it to determine an initial pose, and moves its center to the table center.
- Task Definition: The hand starts from a fixed pose above the table center, and grasping succeeds when the object reaches the target within a threshold distance.The success criterion is based on the position difference between the object and target.
- Generalization: The objective is a universal policy that generalizes across random initial poses and thousands of objects, including unseen instances.The task distribution creates a multi-task policy-learning setting with substantial object and pose diversity.
4. Method
UniDexGrasp++ uses geometry-aware curriculum, iterative generalist-specialist learning, and DAgger-based distillation to train universal dexterous grasping policies across diverse tasks. Geometry features guide curriculum design, task partitioning, specialist assignment, and policy learning.
- Method Overview: The pipeline separates state-based policy learning from vision-based policy learning, using an oracle-informed state policy to ease visual policy training.The state-based policy accesses robot state, object state, and initial scene geometry; the vision policy is restricted to real-world observations.
- GiGSL: GiGSL iteratively fine-tunes specialists on task subsets and distills their knowledge into successive generalists until performance saturates.The first state-based generalist is obtained after GeoCurriculum, then specialist-generalist cycles further improve it.
- Policy Distillation: DAgger-based distillation jointly learns the student policy and critic, preserving actor-critic initialization for subsequent reinforcement-learning fine-tuning.The policy loss matches teacher and student actions, while the critic loss matches predicted values to returns estimated with GAE.
- Geometry-aware Clustering: Geometry-aware clustering assigns continuously varying tasks to specialists by nearest feature-space center, enabling finite specialist coverage of the task space.Sampled tasks are represented by state-based or vision-based geometry features and assigned online during specialist fine-tuning.
- GeoCurriculum: GeoCurriculum measures task similarity from scene geometry rather than object identity or category labels to organize multi-task reinforcement learning.It addresses object-pose variation and within-category geometric variation through hierarchical task-space partitioning.
5. Experiment
Experiments evaluate UniDexGrasp++ on the UniDexGrasp benchmark and compare its state-based and vision-based policies with prior methods and ablations. Results support geometry-aware curriculum, clustering, iterative fine-tuning, and end-to-end distillation as contributors to performance.
- Main Results: 88% and 84% success rates on the train and test sets improve over UniDexGrasp by 9% and 11% in state-based learning.These results evaluate the proposed techniques during state-based policy learning.
- Main Results: 85% and 78% success rates on the train and test sets outperform UniDexGrasp by 12% and 11% in vision-based learning.The comparison uses the full proposed vision-based pipeline against the state-of-the-art baseline.
- Training Analysis: Geometry-aware clustering groups tasks with similar features, shapes, and, for vision-based policies, grasping poses.State-based clustering uses point-cloud encoder features, whereas vision-based clustering uses task-aware vision-backbone features.
- Training Analysis: Iterative fine-tuning improves success rates on both training and test sets over a single training cycle in both state-based and vision-based learning.The reported comparisons refer to the corresponding ablations and Figure 4.
- Training Analysis: Directly distilling final state-based specialists to a vision-based generalist performs better than first distilling through a state-based generalist.This comparison evaluates end-to-end distillation against a multi-stage alternative.
- Ablation Study: 7% and 5% improvements on the training and test sets result from vision-based GeoClustering compared with random specialist division.The comparison is between the corresponding ablation rows in the vision-based setting.
6. Conclusions and Discussions
UniDexGrasp++ improves UniDexGrasp’s performance and generalization, while remaining limited to simulated dexterous grasping.
- The method improves UniDexGrasp’s performance and generalization.
- Its current scope is limited to dexterous grasping in simulation, with real-robot extension left for future work.
A. Method and Implementation Details
The complete UniDexGrasp++ pipeline is formalized in Algorithm 3.
- Algorithm 3 formalizes the complete UniDexGrasp++ pipeline.
A.1. Details about Our Method
UniDexGrasp++ combines geometry-aware curricula, iterative generalist-specialist learning, point-cloud autoencoding, and PPO-based reinforcement learning with DAgger-style distillation.
- The pipeline starts with GeoCurriculum and trains an initial state-based generalist before iterative specialist-generalist learning.
- GeoClustering assigns nearby tasks to state-based and vision-based specialists, whose policies are repeatedly distilled into generalists.
- GiGSL stops state-based or vision-based training when the current success rate improves over the previous policy by less than 0.5%.
- The point-cloud autoencoder encodes each initialized object point cloud into a latent feature and decodes it for reconstruction training with Chamfer Distance.
- PPO is used as the reinforcement-learning baseline, while DAPG, ILAD, and GSL provide related comparison methods.
- UniDexGrasp uses object curriculum learning followed by DAgger distillation from a state-based teacher to a vision-based policy.
B. Experiment Details
The implementation uses PPO for initial generalist learning, freezes the vision backbone during specialist training, and defines robot state in Table 4.
- PPO trains the first generalist during GeoCurriculum, while specialist learning trains the state-based and vision-based specialists.
- Table 4 defines the robot state used in the experiments.
- The vision backbone is frozen during vision-based specialist learning.
B.1. Environment Setup
The environment defines state- and vision-based policies, a 24-actuator dexterous-hand action space, multi-camera point-cloud observations, and a reward combining reaching, lifting, and movement objectives.
- State and observation setup: The state-based policy includes robot state and object oracle information, whereas the vision-based policy uses robot state and scene point clouds.The object oracle state includes pose, linear velocity, and angular velocity; the point cloud is sampled to 1024 points.
- Action space: The action space contains 24 normalized motor commands: six control global hand position and orientation, and 18 control the fingers.
- Camera setup: Five RGBD cameras surround and overlook the table, and their partial point clouds are fused into one scene point cloud.The camera setup places four cameras around the table and one above it.
- Reward design: The reward function combines reaching, lifting, and movement components to guide finger-object contact, object lifting, and target-directed motion.The reaching reward depends on finger-object distance, while the lifting reward activates when a reaching condition is met.
B.2. Training Details
Training uses multilayer perceptrons for state and vision policies, a PointNet-plus-Transformer vision backbone, and hyperparameters summarized for the grasping policy.
- Policy networks: The state-based and vision-based policies use four-hidden-layer MLPs with widths 1024, 1024, 512, and 512, using ELU activations.
- Training configuration: The grasping-policy hyperparameters are reported in Table 5.
- Vision architecture: The vision backbone processes robot, object, and full-scene point subsets with separate PointNets, then integrates features using a Transformer.An additional MLP encodes robot state before the features are combined.
C. Additional Results and Analysis
Additional experiments analyze GeoCurriculum, iGSL policy distillation, GeoClustering choices, and performance on Meta-World, while qualitative figures visualize task clusters and grasping trajectories.
- GeoCurriculum: Curriculum learning is essential for the challenging grasping task, and GeoCurriculum outperforms object curriculum learning by using geometric similarity of objects and poses.The ablation also compares three-, four-, and five-stage curriculum schedules.
- iGSL distillation: DAgger-based policy distillation significantly outperforms Behavior Cloning, GAIL, and DAPG in the iGSL ablation.All compared methods use GeoCurriculum, and performance is evaluated for the next specialist-generalist stage.
- iGSL distillation: The method uses the teacher checkpoint during distillation, whereas the other compared methods use only demonstrations from the teacher.
- GeoClustering analysis: Increasing the GeoClustering cluster count beyond a certain point does not improve performance and may decrease it.GeoClustering is also compared with random and category-label-based clustering.
- Meta-World results: iGSL performs well on the diverse MT-10 and MT-50 Meta-World benchmarks and outperforms baseline methods.MT-10 contains 10 tasks, while MT-50 extends the benchmark with 50 additional tasks.
- Qualitative results: Qualitative results visualize GeoClustering-based task groups, grasping poses, and trajectories for objects with different initial poses.The clustering features are extracted by the vision backbone during vision-based policy learning.