Source-linked AI summary
DexMimic: A Generalist Dexterous Hand-Object Tracking Policy via Value-Guided Reset Allocation
Zhexi Luo, Zhixuan Xu, Rui Yang, Zeyu Xiao, Yiwen Hou, Lin Shao
TL;DR
DexMimic addresses the limits of trajectory-specific policies by jointly training a generalist hand–object tracking policy across diverse HOI trajectories. Its Value-Guided Reset Allocation adapts episode initialization to favor difficult phases and starting states that support tracking. Experiments report stronger tracking and training efficiency, faster fine-tuning, and zero-shot transfer potential as training scales.
Problem
Physics-based HOI tracking methods typically train specialist policies for individual trajectories or small subsets, limiting reuse of learned control priors and repeating training costs as datasets grow.
Source: §I · p. 1Method
DexMimic jointly trains a shared tracking policy and uses critic-guided reset allocation to prioritize under-learned phases and starting states with higher expected tracking returns.
Source: §I · p. 2Results
DexMimic improves tracking success and training efficiency, fine-tuning speed, and zero-shot transfer; on 50 bimanual tasks it achieves 88.40% success versus 56.64% for ManipTrans.
Source: Training Efficiency over… · p. 5, C. Does DexMimic Learn a Transferable… · p. 7, §V · p. 8Takeaways & Limitations
Scaling co-training strengthens transfer to unseen trajectories and objects, and the learned behaviors also transfer to physical robots.
Source: C. Does DexMimic Learn a Transferable… · p. 7, §V · p. 8
Abstract
from arXiv · showhide
Achieving human-like dexterous manipulation remains a central challenge in robotics. Learning from human hand-object interaction (HOI) datasets offers a promising pathway to diverse dexterous skills, yet existing physics-based approaches train specialist policies for individual trajectories, thereby limiting skill reuse at scale. We introduce DexMimic, a generalist dexterous hand-object tracking policy jointly trained across diverse HOI trajectories. Efficient joint training requires allocating each rollout to both an undertrained phase and a starting state that supports continued tracking. DexMimic realizes this insight through Value-Guided Reset Allocation, which adapts the initialization distribution online to prioritize difficult phases and select reset states with higher expected tracking returns. We further introduce a benchmark spanning training, fine-tuning, and generalization. Experiments show improved tracking success and training efficiency, faster fine-tuning, and the potential for zero-shot generalization to unseen trajectories and objects; larger training sets further improve efficiency and generalization. Finally, we validate DexMimic on a real-world dexterous manipulation platform.
I. INTRODUCTION
DexMimic targets scalable learning of dexterous hand–object tracking by jointly training a shared policy across diverse trajectories and adapting where episodes begin. Its reset allocation prioritizes under-learned phases and promising starting states as policy learning progresses.
I. INTRODUCTION
- Human demonstrations encode fine-grained finger coordination, contact transitions, and regrasping that are difficult to specify through task-specific rewards.A reusable low-level control interface conditioned on demonstrations could support diverse manipulation skills despite the embodiment gap between human and robot hands.Source: §I · p. 1
- Trajectory-specific policies limit reuse of learned control priors and incur repeated training costs as the demonstration set grows.The paper investigates joint learning across diverse HOI trajectories and transfer to unseen references.Source: §I · p. 2
- Reference State Initialization can waste rollout budget through physically inconsistent retargeted contacts and uniform timestamp sampling that oversamples simple phases.These inefficiencies reduce useful interaction experience per trajectory as the training set grows.Source: §I · p. 2
- On 50 challenging bimanual tasks, DexMimic achieves 88.40% success versus 56.64% for ManipTrans and reduces average training time per trajectory by a factor of 15.5.Source: §I · p. 2
III. METHOD
- DexMimic adapts its initialization distribution alongside policy learning, using critic estimates to prioritize under-learned phases and starting states that support continued tracking.It jointly trains a shared tracking policy across diverse HOI trajectories in physics simulation.Source: §III · p. 2, B. Reset distributions for… · p. 2
A. Problem Formulation
- The reference-conditioned MDP state includes trajectory index, reference frame, and hand–object simulator state; episodes jointly sample a trajectory, starting frame, and initial simulator state.The shared policy maximizes expected discounted tracking return.Source: A. Problem Formulation · pp. 2–3
B. Reference-Conditioned Tracking Policy
The policy tracks reference hand–object interactions using recent state history and future targets, while producing residual control actions. Value-Guided Reset Allocation updates the initialization distribution from rollout experience as the policy and critic learn.
B. Reference-Conditioned Tracking Policy
- The policy observation combines recent hand–object states, geometry and mass properties, contact forces, future reference targets, a kinematic base action, and remaining reference steps.Its residual outputs control wrist translation, wrist rotation, and finger joint positions, combining with the base action to produce forces, torques, and finger targets.Source: B. Reference-Conditioned Tracking Policy · p. 3
- A shared task-agnostic tracking reward covers hand and object pose, velocity matching, reference-guided contact, and mechanical-power regularization; episodes end at reference completion or excessive tracking error.The actor and critic share an MLP backbone and are jointly trained with PPO [51].Source: B. Reference-Conditioned Tracking Policy · p. 3
C. Value-Guided Reset Allocation
- Value-Guided Reset Allocation adapts which reference phase and physical starting state are sampled, using critic estimates to favor under-learned phases and promising states.Rollouts update the policy and critic, refine reset candidates and phase priorities, and shape the initialization distribution for later rollouts.Source: C. Value-Guided Reset Allocation · pp. 3–4
- Each training iteration samples a phase and state, collects physics rollouts, updates the policy and critic with PPO, and then updates reset buffers and phase priorities.The resulting initialization distribution is formed for the next iteration.Source: C. Value-Guided Reset Allocation · p. 4
1) Reset state allocation:
Reset state allocation reuses rollout states that support continued tracking, while reset phase allocation shifts sampling toward under-learned interactions and their preparation phases.
1) Reset state allocation:
- Phase-conditioned reset buffers store K candidate states and critic scores, initialized with the phase-aligned reference state, to reuse promising starts found in physical rollouts.Candidates are sampled according to critic-weighted values, with probability α reserved for the fixed reference anchor to limit drift from demonstrations.Source: 1) Reset state allocation · p. 4
- A rollout state is eligible for buffer replacement only if its survival fraction exceeds cmin and its critic score beats the lowest-scored stored slot.Stored scores decay over time, allowing candidates from later policies to enter while reference anchors remain unchanged.Source: 1) Reset state allocation · p. 4
2) Reset phase allocation:
- Bin difficulty is estimated from normalized critic values, with larger gaps from the reward-defined maximum indicating phases that remain under-learned.The critic value is normalized by remaining discount weights to account for different remaining reference lengths.Source: 2) Reset phase allocation · p. 4
- Difficulty propagates backward over a window of bins with geometrically decaying weights, raising reset priorities before difficult interactions so the policy can learn their preparation.Terms crossing trajectory boundaries are omitted, and priorities are smoothed across training iterations with an exponential moving average.Source: 2) Reset phase allocation · p. 4
- The smoothed bin scores are normalized into phase-sampling probabilities; a bin is sampled first, followed by a uniform reference frame within that bin.Phase priorities adapt as new rollout values are collected.Source: 2) Reset phase allocation · pp. 4–5
3) Online training loop:
DexMimic couples policy learning with online allocation of reset states and reference phases. On 50 challenging bimanual tasks, it outperformed trajectory-specific baselines in success and per-trajectory training time.
3) Online training loop:
- Each iteration uses shared physics rollouts to update the policy and critic with PPO, refine reset candidates, and adjust phase priorities.The updated allocation favors under-learned interactions from starting states that support continued tracking.Source: 3) Online training loop · p. 5
A. Experimental Settings
- The benchmark evaluates 1,944 tasks across 140 objects, using 65,536 parallel environments for training and 50 complete rollouts per task at evaluation.Evaluation initializes from the original reference’s first frame; reset allocation is applied only during training.Source: A. Experimental Settings · p. 5
Training Efficiency over Trajectory-Specific Methods?
- 88.40% success rate—31.76 and 48.72 percentage points above ManipTrans and SPIDER—with 0.12 h per trajectory, 15.5× and 6.5× lower time costs, respectively.DexMimic also achieved the lowest object rotation and translation errors, at 0.07 rad and 3.76 mm.Source: Table I · p. 5, Training Efficiency over… · p. 5
- The baseline comparison covers 50 challenging bimanual tasks and uses identical initial states, rollout counts, and success criteria for evaluation.DexMimic jointly trains one policy across all 50 trajectories; each baseline trains or optimizes separately for each trajectory.Source: Training Efficiency over… · p. 5
C. Does DexMimic Learn a Transferable Interaction Prior?
The benchmark finds that larger co-training sets improve zero-shot success across three generalization levels, while pretraining speeds adaptation to unseen trajectories. Ablations associate reset-state allocation with fewer fast-fail resets and phase allocation with better worst-trajectory performance and greater training-time savings at larger scales.
C. Does DexMimic Learn a Transferable Interaction Prior?
- The benchmark tests zero-shot transfer to similar trajectories with seen objects, unseen trajectories with seen objects, and trajectories with unseen objects, plus fine-tuning on unseen references.Zero-shot policies are trained on nested subsets of 20, 50, 200, and 500 trajectories and frozen at test time.Source: C. Does DexMimic Learn a Transferable… · pp. 5–6
- At 500 trajectories, zero-shot SR reached 68.50% (L1), 55.89% (L2), and 51.65% (L3), versus 8.00%, 10.63%, and 2.00% at 20.Success rates improved across all three generalization levels as the training set grew from 20 to 500 trajectories.Source: C. Does DexMimic Learn a Transferable… · pp. 6–7, Figure 6 · p. 7
- Fine-tuning the pretrained policy reached 80% mean training SR across 20 unseen trajectories 5.3× faster than training from scratch.The benchmark jointly adapts one policy on 20 unseen trajectories.Source: C. Does DexMimic Learn a Transferable… · pp. 6–7, Figure 6 · p. 7
D. How Does Value-Guided Reset Allocation Improve Generalist Training?
- On 500 shared reference trajectories, DexMimic achieved the highest SR and lowest object and hand tracking errors, while cutting training time versus Uniform from 18.60 min to 5.76 min.The reported training-time reduction was 69.0%.Source: D. How Does Value-Guided Reset… · p. 7, Table II · p. 7
- Across 20–500 trajectories, DexMimic beat both ablated variants in success rate and average training time per trajectory; at 500, reset-state allocation exceeded RSI by 18.22 percentage points.Compared with Uniform, phase allocation's training-time reduction grew from 15.0% at 20 trajectories to 69.0% at 500.Source: Table III · p. 7, D. How Does Value-Guided Reset… · pp. 7–8
- With reset-state allocation, SR rose from 66.51% to 84.73% and FastFail fell from 96.34% to 41.70% versus RSI.FastFail is the share of training failures occurring within 60 frames after reset; Fig. 7 illustrates corrections to physically infeasible initial states.Source: D. How Does Value-Guided Reset… · p. 7, Figure 7 · p. 8
- With reset-phase allocation, WorstTraj SR rose from 0.00% to 76.00% and average SR from 82.59% to 84.73% versus Uniform.WorstTraj SR exposes learning imbalance across trajectories that average SR can obscure.Source: D. How Does Value-Guided Reset… · p. 7, Table II · p. 7
E. Real-World Deployment
DexMimic is evaluated through physical-robot replay and across four dexterous hand models, while the conclusion reports improved tracking, efficiency, adaptation, and zero-shot generalization. The authors note that available HOI dataset scale remains a constraint.
E. Real-World Deployment
- A 7-DoF xArm with a 20-DoF Wuji hand replays successful simulation rollouts open-loop, without strict temporal alignment to the human demonstration.The setup estimates the initial object pose, aligns the wrist using inverse kinematics, and follows the wrist trajectory during replay.Source: E. Real-World Deployment · p. 8
F. Performance across Dexterous Hands
- Fig. 8 compares reset phase allocation with uniform phase sampling across four trajectories; color encodes normalized iterations to success, and light red marks regions still unsuccessful at the training-budget end.Source: Figure 8 · p. 8
- One policy per hand is trained on the same 20 tasks, and Table IV reports each hand’s success rate over 1,000 evaluation rollouts.Source: F. Performance across Dexterous Hands · p. 8
V. CONCLUSION
- Experiments demonstrate improved tracking success and training efficiency, faster adaptation, zero-shot generalization potential, and transfer of learned behaviors to physical robots.Source: §V · p. 8
- Available HOI dataset scale constrains DexMimic; future work targets tracking robust to reconstruction noise to learn from more abundant in-the-wild videos.Source: §V · p. 8
A. POLICY AND ENVIRONMENT DETAILS
The policy is conditioned on a reference trajectory and simulator state, with observations combining hand–object histories and future reference targets. The observation design includes temporal offsets, contact-related features, and shared actor–critic inputs.
A. POLICY AND ENVIRONMENT DETAILS
- The task state includes reference trajectory index, reference frame, and hand–object simulator state; the notation also defines simulator state, observation, and residual action.Source: A. POLICY AND ENVIRONMENT DETAILS · p. 10
A.1. Observation Space
- Observations combine recent hand–object state histories and interaction features with reference targets, including future hand–object states, tracking errors, and desired contact patterns.Actor and critic receive the same observation, which also includes the kinematic base action and remaining reference steps.Source: §A.1 · p. 10
- The observation uses history offsets H = {0, −1, −2, −4, −8} and future offsets F = {0, 1, 2, 4, 8}; wrist position is retained, and object position is relative to the wrist.Missing history frames repeat the reset state for both feature blocks.Source: §A.1 · p. 10
- Additional observation features include fingertip contact forces, reference hand-point position and velocity information, fingertip–object distances, object-to-hand distances, and a 128-dimensional object geometry encoding.Source: §A.1 · p. 10
A.2. Action Space
The action combines retargeted reference-based control with learned residuals for wrist motion and finger targets. Wrist and finger commands are smoothed before application.
A.2. Action Space
- The action comprises normalized wrist translation, wrist rotation, and finger-target residuals for the actuated joints.Source: §A.2 · p. 11
- The kinematic base action comes from the retargeted hand reference; wrist position and rotation errors are clipped, while the finger base is the normalized retargeted target.The wrist position error is clipped to 0.10 m and the axis–angle error to 1 rad.Source: §A.2 · p. 11
- Wrist residuals augment base force and torque, while finger residuals are added in normalized joint coordinates and mapped to joint limits.Source: §A.2 · p. 11
- Wrist forces, torques, and finger targets are temporally smoothed with α = 0.4 before application, and bimanual actions concatenate both hands’ residuals.Source: §A.2 · p. 11
A.3. Reward Function
A task-agnostic reward shared across references combines tracking, contact, and mechanical-power terms. Its components cover hand and object pose, velocity, reference-guided contact, and power.
A.3. Reward Function
- The shared task-agnostic reward combines hand pose, object pose, velocity, reference-guided contact, and mechanical-power groups, with λ_g = 1 for all five groups.Source: §A.3 · p. 11
- Reward components measure current-reference pose errors for the hand and object, hand-link errors, and average absolute velocity differences.Source: §A.3 · p. 11
- Reference fingertip–object distance gates simulated fingertip force in the contact term, while separate terms account for mechanical power.Source: §A.3 · p. 11
- For bimanual rewards, per-hand terms are summed, including an object term for each hand; component rewards and weights are dimensionless, with exponential scales assuming SI units.Source: §A.3 · p. 11
B. TRAINING AND RESET ALLOCATION DETAILS
The training configuration combines a shared PPO policy-and-critic network with a phase-specific reset buffer. The buffer retains scored rollout states that support continued tracking, alongside a fixed reference anchor.
B.1. PPO and Network Configuration
- The policy and critic share an ELU trunk, with separate Gaussian-policy and scalar-value heads updated using clipped PPO and generalized advantage estimation.Observation blocks are normalized and passed through a feature MLP before the shared trunk; policy samples are bounded to [−1, 1] before residual action synthesis.Source: §B.1 · pp. 11–12
B.2. Reset Buffer Update
- Each reference phase has a buffer of K scored states initialized from its aligned reference state, while a separate branch samples the fixed reference anchor.Stored states include hand and object configurations and velocities, finger joint positions and velocities, and controller targets.Source: §B.2 · p. 12
- Candidates are processed by score and replace the lowest-scored buffer entry only if their survival fraction exceeds cmin and their score is higher.After a nonempty episode-finalization batch, stored candidate scores decay; the fixed reference anchor is used if candidate sampling weights vanish.Source: §B.2 · p. 12
B.3. Reset Phase Allocation
Reset-phase allocation uses critic estimates to prioritize under-learned trajectory phases, while the evaluation passages report per-trajectory success patterns and failure outcomes. The supplied hyperparameter passage identifies a table of principal training and reset parameters without giving its values.
B.3. Reset Phase Allocation
- Trajectories are divided into 15-frame bins, and critic-based difficulty scores propagate to preceding bins to update phase priorities for reset sampling.Sampling first selects a bin according to normalized priorities, then selects uniformly among that bin’s valid frames.Source: §B.3 · p. 12
B.4. Hyperparameters
- Table S2 summarizes the principal training and reset parameters, though the supplied passage does not specify their values.Source: §B.4 · p. 12
C.2. Failure Case Analysis
- 81.219% of 1,017 evaluation rollouts succeeded (826 cases), and estimated failures were dominated by object-tracking errors.Fig. S3 reports estimated failure-category proportions; Fig. S4 shows representative examples for the five categories.Source: Figure S3 · p. 13, §C.2 · p. 13, Figure S4 · p. 13
C.1. Per-Trajectory Success Rates
- The evaluation covers 1,302 OakInk and HOT3D trajectories, and Fig. S2 plots per-trajectory success rate against reference length by hand side.The figure distinguishes right-hand and left-hand trajectories.Source: §C.1 · p. 13