Source-linked AI summary
DexterityGen: Foundation Controller for Unprecedented Dexterity
Zhao-Heng Yin, Changhao Wang, Luis Pineda, Francois Hogan, Krishna Bodduluri, Akash Sharma, Patrick Lancaster, Ishita Prasad, Mrinal Kalakrishnan, Jitendra Malik, Mike Lambeta, Tingfan Wu, Pieter Abbeel, Mustafa Mukadam
TL;DR
DexGen addresses the difficulty of teaching dexterous robots when teleoperation lacks safe fine-grained control and sim-to-real RL faces domain gaps and task-specific rewards. It pretrains a guided generative controller on large-scale RL-generated manipulation data, then uses human teleoperation as coarse motion conditioning. Across simulation and real-world experiments, DexGen improves stability by 10-100x and enables diverse dexterous manipulation, including tool use.
Problem
Teaching dexterous manipulation is limited by unsafe human teleoperation and sim-to-real RL’s domain gap and task-specific reward engineering.
Method
DexGen pretrains a generative behavior model on diverse RL-generated grasp transitions and uses guided diffusion to convert external motion commands into robot actions.
Results
DexGen extends stable operation duration by 10-100 times in simulation and enables real-world reorientation and tool-use tasks with human guidance.
Takeaways & Limitations
DexGen provides a foundational low-level dexterity controller that complements high-level external policies for complex manipulation.
Takeaways & Limitations
The authors identify limitations for future work but do not specify a concrete limitation in the supplied passage.
Abstract
from arXiv · showhide
Teaching robots dexterous manipulation skills, such as tool use, presents a significant challenge. Current approaches can be broadly categorized into two strategies: human teleoperation (for imitation learning) and sim-to-real reinforcement learning. The first approach is difficult as it is hard for humans to produce safe and dexterous motions on a different embodiment without touch feedback. The second RL-based approach struggles with the domain gap and involves highly task-specific reward engineering on complex tasks. Our key insight is that RL is effective at learning low-level motion primitives, while humans excel at providing coarse motion commands for complex, long-horizon tasks. Therefore, the optimal solution might be a combination of both approaches. In this paper, we introduce DexterityGen (DexGen), which uses RL to pretrain large-scale dexterous motion primitives, such as in-hand rotation or translation. We then leverage this learned dataset to train a dexterous foundational controller. In the real world, we use human teleoperation as a prompt to the controller to produce highly dexterous behavior. We evaluate the effectiveness of DexGen in both simulation and real world, demonstrating that it is a general-purpose controller that can realize input dexterous manipulation commands and significantly improves stability by 10-100x measured as duration of holding objects across diverse tasks. Notably, with DexGen we demonstrate unprecedented dexterous skills including diverse object reorientation and dexterous tool use such as pen, syringe, and screwdriver for the first time.
I. INTRODUCTION
DexterityGen addresses the difficulty of teaching dexterous manipulation by combining RL-trained low-level skills with coarse human or external motion guidance. It decouples semantic motion generation from low-level control and improves dexterous manipulation in simulation and real-world tasks.
- Dexterous in-hand manipulation remains difficult despite robotic hands’ potential for complex object handling.
- Human teleoperation is limited by unsafe contact-rich motions, embodiment differences, incomplete tactile information, complex high-dimensional motion, and weak force-control interfaces.
- Sim-to-real RL faces a simulation-to-reality gap and requires highly task-specific rewards for complex manipulation.
- RL can generate transferable low-level manipulation primitives, while humans can provide coarse motion-level guidance for complex tasks.
- DexGen pretrains a generative behavior model on multitask RL data to translate coarse motion commands into safe low-level robot actions.
- DexGen improves simulated stability by 10-100 times and enables real-world dexterous tasks including object reorientation and syringe and screwdriver use.
B. Sim-to-real Reinforcement Learning
DexGen targets the scalability limits of dexterous sim-to-real RL by learning low-level action distributions from diverse simulated manipulation trajectories. Its guided diffusion controller uses external motion commands to produce executable robot actions.
- Challenges: Generalized sim-to-real dexterous manipulation is constrained by sensor and physics gaps between simulation and the real world.
- Challenges: Long-horizon contact-rich RL requires task-specific reward engineering, limiting scalability.
- Opportunities: Sim-to-real RL can provide transferable low-level manipulation primitives that complement high-level action plans.
- DexGen approach: DexGen pretrains a generative behavior model pθ(a|o) on simulation data to model prior actions conditioned on robot state.
- DexGen approach: The controller takes robot state and motion conditioning, generates intermediate motion with diffusion, and converts it into executable actions through inverse dynamics.
A. Preliminaries
DexGen combines diffusion-based guided sampling with a large RL-generated dataset of diverse grasp transitions. The resulting controller is trained across varied objects and configurations to support robust dexterous behavior.
- A. Preliminaries: Diffusion models learn complex data distributions through noising and reverse denoising, while guided sampling steers outputs toward high-energy regions.
- A. Preliminaries: The robot hand uses PD control to convert commanded joint positions into torques based on position and velocity errors.
- B. Large-Scale Behavior Dataset Generation: DexGen collects simulated manipulation trajectories with RL to cover the diverse states encountered under external control.
- B. Large-Scale Behavior Dataset Generation: Anygrasp-to-Anygrasp trains transitions between randomly initialized and nearby reachable grasps, then rolls out sequences covering hand-object interaction modes.
- B. Large-Scale Behavior Dataset Generation: The dataset uses varied objects, wrist poses, and domain randomization so manipulation is not restricted to one object or configuration.
- B. Large-Scale Behavior Dataset Generation: 1 × 10^10 transitions were collected, equivalent to 31.7 years of real-world experience, using 300 GPU hours of rollout generation.
C. DexGen Model Architecture
DexGen combines a diffusion model for safe, observation-conditioned finger-keypoint motions with an inverse dynamics model that converts those motions into executable robot actions. Guided diffusion incorporates external motion commands while preserving commanded fingertip movement.
- Diffusion motion model: DexGen uses diffusion to model future 3D finger-keypoint motions from robot observations and mode conditioning.The representation spans a future horizon T and K finger keypoints in the robot-hand frame.
- Action conversion: A residual inverse dynamics model converts generated keypoint motions into target joint-position actions for the robot hand.It predicts actions conditioned on the current robot state and motion command.
- Motion conditioning: DexGen samples motions that are both likely under the learned safe distribution and close to the input reference motion.The reference is an input commanded fingertip offset, while the distance term penalizes deviation from it.
- Data diversity: The training dataset uses diverse objects and wrist poses to support robust manipulation across varied real-world states.The testing objects include different sizes, masses, and aspect ratios and are not present in the pretraining dataset.
- Motion conditioning: Gradient guidance adjusts each diffusion sample toward the commanded fingertip motion before inverse dynamics produces robot actions.The system uses DDIM sampling for 10Hz control, with approximately 27ms total sampling time on an RTX 4090 workstation.
IV. EXPERIMENTS
DexGen is evaluated by assisting suboptimal policies in simulation and human teleoperators in real-world dexterous manipulation. It improves robustness to noisy commands and supports diverse hand-object interactions, including rejection of dangerous behavior while retaining object holding.
- Real-world evaluation: In real-world experiments, human teleoperation serves as an external high-level policy for diverse dexterous manipulation tasks.The setup uses an Allegro Hand mounted on a Franka Panda arm and retargets human hand gestures to the robot.
- Simulation evaluation: 10-100x longer stable operation is achieved when DexGen assists highly perturbed policies in simulation.DexGen can also enable success for an extremely perturbed policy when the baseline fails.
- Simulation evaluation: DexGen assists noisy and slowed expert policies on the Anygrasp-to-Anygrasp task by reducing critical failures and increasing goal achievements.The evaluation records dropped objects and achieved goals over a fixed time window.
- Simulation evaluation: A common guidance region works across different suboptimal policies, while too little guidance preserves objects without achieving goals and too much can reduce duration.The trade-off arises because DexGen lacks the goal information at low guidance, whereas external commands can dominate at high guidance.
- Real-world evaluation: DexGen can reject a dangerous palm-opening behavior while continuing to hold the object.This demonstrates action correction that preserves object holding despite the input behavior.
C. Real World Experiments:
Real-world experiments evaluate DexGen as assistance for teleoperated dexterous manipulation, including atomic skills, object reorientation, regrasping, and long-horizon tool tasks. DexGen improves stability and task success while preserving user intention, although chaining multiple manipulation stages remains difficult.
- Real-world task suite: The evaluation measures teleoperation success across object reorientation, functional grasping, in-hand regrasping, screwdriver use, and syringe use.The protocol includes two users, each familiarized with the systems for 30 minutes before evaluation.
- Performance: The raw teleoperation baseline fails completely, while DexGen helps achieve stability and success across diverse real-world setups.Table I reports success rate and normalized time-to-fall or holding time.
- Observed properties: DexGen can override dangerous finger motions to maintain object contact, producing a protective “magnetic effect” during manipulation.The model can preserve contact when a supporting-finger movement would otherwise drop the object.
- Observed properties: DexGen usually follows the user’s commanded direction while overriding dangerous actions, preserving user agency during complex tasks.This behavior agrees with the simulated finding that DexGen realizes intentions in noisy, suboptimal actions.
- Long-horizon tasks: Long-horizon tool-use tasks remain challenging because chaining several manipulation stages is difficult despite reasonable stage-wise success rates.The breakdown analysis covers syringe and screwdriver teleoperation.
V. RELATED WORKS
The paper positions DexGen as a low-level robotic foundation controller pretrained on diverse simulated trajectories rather than costly teleoperation data. It differs from prior work through high-DOF dexterous manipulation, continuous fine-grained prompting, and explicit discussion of sensing and real-world adaptation limitations.
- Foundation models and pretraining: DexGen pretrains a low-level controller on pure simulation datasets, reducing reliance on extensive real-world teleoperation data collection.The paper contrasts this with foundation-model approaches that use real-world datasets requiring substantial human effort.
- Foundation models and pretraining: The work studies generative pretraining for high-DOF robotic hands, whereas existing foundation-model studies typically consider parallel-jaw grippers.This is presented as a first demonstration of the advantage of generative pretraining in this dexterous setting.
- Foundation models and pretraining: DexGen is a low-level foundation controller that accepts continuous fine-grained guidance and can potentially integrate with high-level planning policies.Most existing robotic foundation models are described as using discrete language prompts or task embeddings.
- Shared autonomy: Unlike related shared-autonomy methods, DexGen does not use human actions during training and instead uses external guidance to prompt the controller at deployment.Related methods commonly feed external actions into an RL policy as part of its observations.
- Conclusion: The paper presents DexGen as an initial foundational low-level controller that translates coarse motion prompts into effective low-level actions for dexterous manipulation.The conclusion characterizes generative pretraining on diverse multitask simulated trajectories as producing a powerful generative controller.
- Limitations: The controller relies on proprioceptive implicit touch sensing, lacks vision, and may require real-world fine-tuning because of the sim-to-real gap.The paper identifies nonrobust fine-grained contact inference, necessary vision for accurate tool motions, and possible deployment-time adaptation as limitations.
APPENDIX
DexGen is trained by collecting diverse RL manipulation trajectories in simulation, filtering failures, and distilling them into a generative controller. The appendix describes grasp generation, stability analysis, reward design, and goal selection used to support this dataset construction.
- Training Procedure: DexGen training collects trajectories from multiple converged RL policies, filters failure transitions, and trains a controller on the resulting dataset.The stated procedure uses two stages: RL rollout collection followed by experience distillation.
- Grasp Generation: Grasp sets begin with heuristic samples and are expanded through RRT search to cover more finger configurations and object poses.The procedure samples surface contacts, checks grasp stability and collision, then expands the valid set with RRT.
- Grasp Analysis: A grasp is considered stable when the minimized net-force objective falls below a threshold after testing nontrivial contact-force combinations.The method applies forces along contact normals and evaluates the minimum solution against F_thresh.
- Reward Design: The Anygrasp-to-Anygrasp reward combines goal, style, and regularization terms, with style penalties encouraging temporal diversity in manipulation behavior.The style term penalizes fingertip velocity and can elicit fast or slow movement styles.
- Goal Dynamics: Moderate-distance goals prevent early RL plateaus, with nearest-neighbor search selecting the next cached grasp after goal achievement.Random down-sampling reduces the cost of nearest-neighbor search over large grasp sets.
C. Boosting Dataset Diversity with Diverse Rewards
DexGen increases dataset diversity by training policies with different reward settings and combining their trajectories.
- C. Boosting Dataset Diversity with Diverse Rewards: Different reward coefficients produce both fast and slow object-manipulation policies for data collection.The collected behaviors cover states generated by both good and suboptimal policies, helping the controller handle varied real-world states.
D. RL Training Setups
The RL training setup uses IsaacGym, PPO, asymmetric actor-critic observations, and MLP policy and critic networks.
- D. RL Training Setups: Training and data collection use IsaacGym with Proximal Policy Optimization and asymmetric actor-critic inputs.The actor observes proprioception and the desired goal, while the critic observes additional state information such as object position and velocity.
E. Domain Randomization
The implementation uses domain randomization and a conditioned diffusion model to generate actions, while balancing action fidelity against inference latency.
- E. Domain Randomization: Domain randomization is applied extensively during both RL training and data collection.The randomized components are listed in Table III.
- E. Domain Randomization: The diffusion controller encodes state and mode inputs, then uses a FiLM-conditioned U-Net to predict noise for action sampling.The architecture uses three encoder and decoder blocks with hidden dimension 768.
- E. Domain Randomization: Inference exposes a tradeoff between sample fidelity, interpreted as action accuracy, and latency.These factors affect user experience in different ways.
- E. Domain Randomization: The controller predicts 0.2 seconds of future motion using T = 2 and represents the hand with K = 8 finger keypoints.A K = 4 fingertip-only representation performed suboptimally and incurred a large inverse-dynamics training loss.