PaperlayerView exact revision on arXiv

Source-linked AI summary

HiFun: a Hierarchical Framework for Efficient Functional Dexterous Manipulation Learning

Linyi Huang, Guowei Huai, Weibin Liu, Shulong Jiang, Ping Tan, Weixuan Zhang, Hui Zhang, Jie Song

arXiv:2610.19518v1cs.RO

TL;DR

Functional dexterous manipulation demands precise contact, coordinated arm-hand motion, and recovery, while simulation and full arm-hand demonstrations have limitations. HiFun learns contact-aware hand skills separately from skill-conditioned coordination and uses DMP rollouts and human corrections. Across six real-world tasks on two hands, it achieves 98.3% average success within one hour of online training and shows robustness to disturbances.

  • Problem

    Functional dexterous manipulation requires precise contact, synchronized arm-hand control, and recovery from execution errors, while contact dynamics are difficult to model in simulation and full arm-hand demonstrations are costly and noisy.

    Source: §1 · p. 2

  • Method

    HiFun learns contact-aware hand skills separately from a coordination policy for arm motion and skill activation, using DMP-augmented rollouts and human corrections during online learning.

    Source: §1 · p. 2

  • Results

    98.3% average success within one hour of online training across six real-world tasks and two dexterous hands, with robustness to disturbances.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    HiFun provides a real-world learning approach for functional dexterous tasks that factors contact execution from arm-hand coordination and recovery.

    Source: §1 · p. 2, §5 · p. 9

    Tasks requiring contact behaviors beyond the learned hand-skill set may fail and require additional priors, online adaptation, or reusable skill libraries; the method targets low-level execution rather than open-ended planning or language-conditioned generalization.

    Source: §6 · p. 9

Abstract

from arXiv · show

While multi-fingered dexterous robot hands have the potential to provide human-level dexterity, using them for functional manipulation remains difficult because task success depends on precise contact, coordinated arm-hand motion, and reactive recovery from execution errors. Precise contact depends on complex contact dynamics that are difficult to model accurately in simulation, motivating real-world learning. However, real-world demonstrations in the full arm-hand control space are costly and noisy because each trajectory must coordinate arm motion with high-DoF multi-finger contact. Small execution deviations can further cause slipped contact or premature force application, making recovery from off-nominal contact states important for robust execution. To deal with these challenges, we propose HiFun, a hierarchical real-world RL framework for functional dexterous manipulation tasks. Our key insight is to decouple the control complexity into a functional hand skill for precise contact execution and an arm-hand coordination policy for arm control and hand-skill activation. This design enables effective real-time human intervention for reactive recovery learning and efficient data collection with Dynamic Movement Primitive (DMP) augmentation. Across six real-world tasks on two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances. The project page is available at https://hly-123.github.io/HiFun/.

1 Introduction

Functional dexterous manipulation requires precise contact, synchronized arm-hand control, and recovery from execution errors. HiFun addresses these challenges by learning contact-aware hand skills separately from skill-conditioned coordination, achieving high success across real-world tasks.

Challenge

  • Functional dexterity requires precise fingertip placement and force regulation, synchronized arm-hand motion, and recovery from slipped contact, missed insertion, partial progress, or premature force.Small contact errors can cause slipping, trapping, or collisions during tasks such as extracting a foam-embedded handle through a narrow opening.Source: §1 · p. 2

Motivation

  • Simulation, demonstrations, and direct real-world RL each leave gaps: contact dynamics are difficult to simulate, demonstrations are costly and noisy, and full arm-hand exploration and correction are inefficient.Demonstrations also provide limited evidence for recovery after failures.Source: §1 · p. 2

Contribution

  • 98.3% average success within one hour of online training; HiFun outperforms IL and flat human-in-the-loop RL baselines and remains robust to disturbances.The framework factorizes hand-skill learning and skill-conditioned coordination, with DMP-augmented rollouts and intervention-weighted recovery learning.Source: §1 · p. 2
  • HiFun separates residual-RL hand-skill learning from arm-motion and skill-activation coordination, using DMP rollouts and human corrections to support efficient learning and reactive recovery.The hand skill adapts kinesthetic priors with real contact feedback; coordination uses a multi-head critic and intervention advantage weighting.Source: §1 · p. 2

2 Related Work

Prior approaches address dexterous manipulation through modeling, simulation, demonstrations, real-world reinforcement learning, and hierarchical skills, but functional contact and recovery remain challenging. HiFun uses kinesthetic priors and contact-aware hand skills to train downstream coordination.

Dexterous Manipulation

  • Functional contact dynamics involving deformation, friction variation, and fine-grained force feedback remain difficult to simulate, while demonstrations for dexterity are costly, noisy, and provide limited recovery evidence.Simulation and imitation learning have enabled dexterous behaviors, but contact transfer and coverage of fine-grained contacts and dynamic arm-hand coordination remain difficult.Source: §2 · p. 3
  • HiFun obtains hand-motion priors through kinesthetic teaching, adapts them with residual RL, and uses the resulting skills to collect coordination data without directly teleoperating every hand degree of freedom.The related-work discussion contrasts this with specialized teleoperation and human-to-robot retargeting, which can introduce contact noise under morphology mismatch [34] [36].Source: §2 · p. 3

Real-World Reinforcement Learning

  • Real-world RL captures physical dynamics, but high-dimensional arm-hand control increases exploration burden and makes consistent real-time correction difficult.Prior hierarchical methods reuse lower-level behaviors; HiFun shifts online learning toward skill-conditioned coordination and recovery.Source: §2 · p. 3

3 Method

HiFun separates contact-aware hand-skill learning from arm-hand coordination, which chooses arm motion and when to activate a frozen skill. DMP rollouts initialize coordination, while online corrections and value-based components support contact recovery and activation timing.

3 Method

  • HiFun learns a residual-RL hand skill and a coordination policy for arm motion and skill activation, then executes arm movement with the frozen skill for approach and fine-grained contact.Coordination uses DMP rollouts for warm start, a multi-head critic, and intervention advantage weighting (IAW).Source: Figure 2 · p. 3, §3 · pp. 3–4

3.1 Problem Formulation

  • The coordination action combines an arm end-effector increment with a scalar that modulates the hand skill from a neutral configuration; the skill determines finger contact while coordination determines where and when it is used.The policy observes wrist and external images, arm-hand proprioception, and fingertip contact forces.Source: §3.1 · p. 4

3.2 Contact-Aware Hand Skill Learning

  • The hand skill adapts kinesthetic reference poses with residual corrections and contact feedback, and can rewind to a recent contact-consistent reference when measured contact deviates substantially.Its reward combines trajectory tracking, contact incentive, and sparse skill success; after convergence, the skill is frozen as both a coordination primitive and a human intervention interface.Source: §3.2 · p. 4

3.3 Skill-Conditioned Arm-Hand Coordination

  • A modulation reward penalizes premature skill activation and rewards successful activation, while separate critic heads distinguish task-reward navigation from activation-timing feedback.The actor uses stopped gradients so the navigation head shapes arm alignment and the modulation head shapes activation timing.Source: §3.3 · pp. 4–5
  • DMP-generated arm trajectories and aligned skill-activation labels provide initial coordination coverage, while online real-robot rollouts and skill-level corrections refine alignment, timing, and recovery.IAW weights intervention imitation by estimated advantage, and separate navigation and modulation value heads help shape arm alignment and activation timing.Source: §3.3 · p. 5

4 Experiments

HiFun achieves 98.3% average success across six tasks and two hands, with 18.8 minutes of RL data collection and 46.2 minutes of online training. The experiments compare its hierarchical approach with learning baselines and test robustness, data sources, coordination components, and hand-control alternatives.

4.1 Experimental Setup

  • Evaluation spans six real-world tasks, compares hierarchical HiFun with flat HIL-SERL and offline imitation baselines, and trains RL methods online for up to one hour.The comparisons include controls for the hand-skill intervention interface and for sharing a frozen hand skill; BC and DP use 200 successful demonstrations per task.Source: §4.1 · pp. 5–6

4.2 Main Results

  • 98.3% average success across six tasks and two hands, with 18.8 minutes of RL data collection and 46.2 minutes of online training, is the strongest reported performance.Table 1 reports success over 50 trials; averages use final-task success and count “–” as zero.Source: §4.2 · p. 6, Table 1 · p. 6
  • Against shared-backbone HIL-SERL, the flat policy using HiFun’s intervention interface reaches 13.7% average success, while 200-demonstration BC and DP remain weak on contact-rich tasks.BC reaches 15% on Power-Drill Actuation and 12% on Pipette Dispensing; DP reaches 36% on Bit Removal and 26% on Pipette Dispensing.Source: §4.2 · p. 7
  • HiFun succeeds in 20/20 perturbation trials on four tasks and 18/20 on Power-Drill Actuation and Pipette Dispensing, with local hand-skill recovery also tested under orientation and force disturbances.With the arm fixed near the execution region, Angle-Spreader Retrieval achieves 19/20 under specified orientation changes and 18/20 under external force; Bit Removal achieves 18/20 with drill yaw up to 30°. Figure 3 shows thumb-contact restoration after collision without arm re-alignment.Source: §4.2 · pp. 7–8, Figure 3 · p. 7

4.3 Ablation and Analysis

  • DMP warm-start rollouts achieve the same or higher final success than human demonstrations, reaching 100% with about 4× less collection time.The comparison concerns warm-start data for coordination-policy training.Source: §4.3 · p. 8, Table 2 · p. 8
  • Removing the multi-head critic drops success to 85% on Angle-Spreader and 75% on Bit Removal; the critic also keeps 95% of activations within 2.00 cm of the pre-contact region.The single-head variant increases the 95th-percentile alignment error to 4.58 cm; the authors attribute mistimed activation to conflated value signals for terminal success and modulation penalties.Source: §4.3 · p. 8, Table 2 · p. 8
  • Uniform intervention cloning drops success to 60%, and removing intervention imitation drops it to 40%, while HiFun’s cycle time is 6.69 s versus 9.21 s and 10.03 s.IAW weights intervention-imitation losses by critic-estimated advantage; its reported mean weights separate weak and strong corrections at 0.175 and 0.884.Source: §4.3 · pp. 8–9, Table 2 · p. 8
  • The learned residual hand skill succeeds in all 20 trials on every task, whereas kinesthetic replay and retargeted teleoperation fail on several tasks.Replay succeeds only partially on Bit Removal (5/20); retargeted teleoperation reaches 20/20 on Pipette but drops on constrained retrieval and Bit Removal.Source: §4.3 · p. 9, Table 3 · p. 9

5 Conclusion

HiFun separates contact-aware hand skills from coordination of arm motion, skill activation, and recovery. Across six real-world tasks on two dexterous hands, it achieves high success within one hour of online training and remains robust to dynamic disturbances.

5 Conclusion

  • 98.3% average success across six real-world tasks on two dexterous hands within one hour of online training.HiFun also remains robust to dynamic disturbances.Source: §5 · p. 9
  • HiFun freezes contact-aware hand skills for a coordination policy that optimizes arm motion, skill activation, and recovery without exploring or correcting all arm-hand DoFs directly.The framework uses DMP-augmented rollouts, a multi-head critic, and prioritized human interventions for recovery learning.Source: §5 · p. 9
  • The results support structuring dexterous learning around functional hand skills as a practical path toward efficient, reliable real-world manipulation with multi-fingered hands.Source: §5 · p. 9

6 Limitations

HiFun assumes task-relevant hand skills learned from kinesthetic priors and real contact feedback. Its scope is reliable low-level execution, not open-ended planning or language-conditioned generalization.

6 Limitations

  • HiFun assumes task-relevant hand skills learned from kinesthetic priors and real contact feedback.Tasks requiring contact behaviors beyond the learned skill set may fail and require additional priors, online adaptation, or reusable hand-skill libraries.Source: §6 · p. 9
  • The method targets reliable low-level execution rather than open-ended planning or language-conditioned generalization.Source: §6 · p. 9
  • HiFun is complementary to VLA policies and world models, and combining pretrained reasoning with HiFun controllers remains future work.The passage proposes HiFun for contact-rich low-level control and consistent autonomous rollouts, with pretrained models providing task selection and semantic grounding.Source: §6 · p. 9

A Method Details

HiFun augments a single demonstrated arm trajectory with Cartesian DMP rollouts to warm-start coordination, then collects physical-robot transitions for offline pre-training and online replay. The setup uses Franka Research 3 arms with Sharpa or XHand dexterous hands.

A Method Details

  • DMP augmentation randomizes initial arm poses from one reference trajectory while preserving its goal and phase-aligned hand activation, broadening approach coverage beyond repeated demonstrations.The generated trajectories are executed on the physical robot to collect state-action transitions.Source: A Method Details · p. 14, Figure 4 · p. 14
  • The arm action label is the clipped displacement from the current robot state to the next DMP waypoint, and transitions seed offline pre-training and the online replay buffer.When activation exceeds a threshold, the frozen hand-skill policy generates finger commands; otherwise the hand remains at its reset pose.Source: A Method Details · p. 14
  • DMP rollouts systematically cover different approach states from the same task goal, providing coordination a varied warm start without repeated human arm-hand demonstrations.Human demonstrations provide realistic executions but are time-consuming and vary with operator timing.Source: A Method Details · p. 14, Figure 4 · p. 14
  • SAC is used for offline pre-training because DMP data comes from successful reference motions and supplies coarse coverage for later online fine-tuning.The authors report that IQL behaves similarly to behavior cloning and CQL adds little benefit when successful state regions are already covered.Source: A Method Details · pp. 14–15
  • On a representative disassembly task, the total target Q-value remains stable during offline pre-training, while separated critic heads reveal a scale gap between arm-related and hand-activation values.Using SAC for offline pre-training and online fine-tuning also avoids a value-calibration gap between pessimistic offline objectives and real online returns.Source: A Method Details · p. 15, Figure 5 · p. 15

B.1 Hardware Setup

  • All tasks use Franka Research 3 arms with either a Sharpa hand or an XHand, with wrist RGB observations and side-view cameras when needed.Figure 6 summarizes the six task setups, training and evaluation workspace regions, and camera views.Source: §B.1 · p. 15, Figure 6 · p. 15

B.2 Task Descriptions

The six tasks span precise tool use, constrained-space retrieval, and contact-rich rotation, with XHand tasks testing portability. Main evaluation uses randomized object poses within the trained workspace, while perturbation trials test recovery from off-nominal contact states.

B.2 Task Descriptions

  • Power-Drill Actuation tests stable drill grasping, bit alignment, and trigger use; premature activation or excessive contact force can shift the hand and misalign the bit.Source: §B.2 · p. 16
  • Angle-Spreader, Thin-Handle, and Thin-Shaft Tool Retrieval test constrained-space extraction through fingertip access, levering, or contact sequences adapted across hand embodiments.Thin-Handle retrieval requires a precision grasp while avoiding tray collisions; Thin-Shaft retrieval tests cross-hand deployment with XHand.Source: §B.2 · p. 16
  • Power-Drill Bit Removal tests sustained rotation under changing contact forces, while Pipette Dispensing tests thumb force, grasp stability, and tip alignment during liquid dispensing.Source: §B.2 · p. 16

B.3 Evaluation Protocol

  • Main evaluation tests each task over 50 trials with randomized object poses inside the trained workspace, measuring repeatable functional execution rather than zero-shot workspace generalization.Success credit is 1 for full success and 0 for failure, with rare horizon-limited baseline near-completions receiving 0.5.Source: §B.3 · p. 16
  • Perturbation evaluation uses 20 trials per task, including object-pose changes, approach displacements, and drill-chuck re-tightening that require re-alignment or delayed skill activation.Source: §B.3 · p. 16

C.1 Policy Training Details

The appendix describes policy-training details, baseline protocols, and supplementary experimental materials, including videos of all six tasks.

C.1 Policy Training Details

  • Across both hands, policies share a high-level action interface and RL pipeline, while active camera streams and online fine-tuning interactions vary by task.Thin-Shaft Tool Retrieval uses the wrist camera; Pipette Dispensing uses wrist and side cameras.Source: §C.1 · pp. 16–17, Table 4 · p. 17
  • Kinesthetic priors typically take under five minutes per task to record; slow teaching supports consistent force signals, while residual RL starts near zero before increasing its action scale.The small initial residual keeps early training close to the prior while collecting contact data.Source: §C.1 · p. 17
  • Coordination data collection takes 4–13 minutes per task; classifier quality depends on camera visibility of completion states versus near-success failures.Across six task datasets, collection includes 10–35 positive frames and roughly 1.9k–7.9k negative frames.Source: §C.1 · p. 17, Table 5 · p. 17

C.2 Baseline Details

  • Both HIL-SERL baselines use HiFun’s RLPD backbone and offline DMP buffer but retain flat full arm-hand actions, separating intervention-interface effects from hierarchical-architecture effects.One baseline uses glove-based hand teleoperation and Gello arm control; the other shares HiFun’s hand-skill intervention interface.Source: §C.2 · pp. 17–18
  • BC and Diffusion Policy use the same 200 human demonstrations, while additional same-source variants train on successful HiFun trajectories, including DMP rollouts and intervention segments.The project website shows all six tasks, contact recovery, tool-size transfer, and a long-horizon multi-skill sequence.Source: §C.2 · p. 18, D Additional Experimental Results · p. 18

D.1 Online Training Curves and Autonomy

Across six tasks, HiFun’s intervention rate falls toward zero within one hour, and its converged cycle time is lower than HIL-SERL (Ours Interv.).

D.1 Online Training Curves and Autonomy

  • Intervention rate tracks how often human guidance remains necessary, whereas cycle time tracks execution efficiency; successful corrections mean success rate alone cannot show autonomy.Figure 7 plots running averages over 20 episodes for both measures.Source: §D.1 · p. 18, Figure 7 · p. 19
  • Within one hour—and in as little as ∼15 minutes on the fastest tasks—HiFun’s intervention rate falls toward zero on every task, while its converged cycle time is consistently lower than HIL-SERL (Ours Interv.).The baseline retains a high intervention rate and does not become autonomous within the same one-hour budget.Source: §D.1 · p. 18, Figure 7 · p. 19
  • Lower intervention rate indicates more autonomous execution, while lower cycle time indicates more efficient task completion.Source: Figure 7 · p. 19

D.2 Additional IL Baseline Results

Offline imitation on HiFun trajectories does not materially improve BC or Diffusion Policy success, while Skill-HIL remains below HiFun across four matched Sharpa tasks.

D.2 Additional IL Baseline Results

  • Offline BC and DP action prediction provides no value-based selection of recovery-like actions, so recovery-rich data alone does not yield reliable recovery behavior without value-guided online RL.Policies on HiFun data often approach objects faster and hesitate less, but still fail to reliably complete precise alignment and contact recovery.Source: §D.2 · pp. 19–20

D.3 Skill-HIL Comparison

  • Sparse outcomes complicate credit assignment between arm alignment and skill activation, while human corrections vary in recovery efficiency and consistency.HiFun addresses these coordination challenges with separate value learning and intervention advantage weighting.Source: §D.3 · p. 20
  • Across four Sharpa tasks under matched settings and training budgets, Skill-HIL achieves 46% average success versus 100% for HiFun.Skill-HIL combines HiFun’s frozen hand skill with standard HIL-SERL coordination; its failures often involve contact regions or activation timing.Source: §D.3 · p. 20
  • On Angle-Spreader Retrieval, doubling Skill-HIL’s training data yields 35/50 successful trials versus 50/50 for HiFun, without closing the gap.The comparison assesses HiFun’s coordination design as a whole; the multi-head critic and intervention advantage weighting are ablated separately.Source: §D.3 · p. 20

D.4 Coordination Component Analysis

The analysis links coordination components to activation localization and correction weighting, then contrasts residual hand skills with replay and retargeting on contact-rich tasks.

D.4 Coordination Component Analysis

  • The multi-head critic reduces p95 activation error from 4.58 cm to 2.00 cm and activations outside the 2 cm pre-contact band from 6.0% to 3.7%.The analysis attributes closer-to-target activation localization to separating arm-alignment and skill-activation value learning.Source: Figure 8 · p. 20, §D.4 · p. 20
  • IAW assigns mean imitation weights of 0.175 versus 0.884 to weak and strong corrections, and high-weight corrections occur at lower estimated policy-value states.This supports focusing imitation on useful corrections rather than cloning every intervention uniformly.Source: Figure 8 · p. 20, §D.4 · pp. 20–21

D.5 Hand Motion Control Failure Cases

  • Kinesthetic replay loses contact-force information, while DexPilot∗ remains sensitive to morphology differences; the residual skill uses real contact feedback for closed-loop force modulation.The comparisons cover Thin-Shaft Tool Retrieval, Angle-Spreader Retrieval, and Thin-Handle Retrieval.Source: Figure 9 · p. 21, §D.5 · p. 21

Related papers

Loading 2610.19518v1…