Source-linked AI summary
In-Hand Object Rotation via Rapid Motor Adaptation
Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, Jitendra Malik
TL;DR
Generalizable fingertip-only in-hand manipulation remains difficult because robots must rotate varied objects while maintaining force closure. The paper trains a reinforcement-learning controller on cylindrical objects and adapts it online from proprioception history, achieving real-world z-axis rotation across diverse objects without fine-tuning.
Problem
Generalizable in-hand manipulation across objects with different shapes, sizes, masses, and materials remains an unsolved robotics challenge.
Method
The method learns a base policy with compact object-property extrinsics and trains an adaptation module to estimate them online from proprioception and action history.
Results
The controller rotates dozens of real-world objects over the z-axis after simulation training only on cylindrical objects, without fine-tuning.
Takeaways & Limitations
Stable finger gaits emerge from reinforcement-learning control, while online extrinsics estimation supports generalization across object shapes and physical properties.
Takeaways & Limitations
The method relies only on proprioception and is unaware of precise contact positions, with many real-world failures caused by incorrect contacts and unstable force closure.
Abstract
from arXiv · showhide
Generalized in-hand manipulation has long been an unsolved challenge of robotics. As a small step towards this grand goal, we demonstrate how to design and learn a simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then - without any fine-tuning - can be directly deployed to a real robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the controller to the object properties using only proprioception history. Furthermore, natural and stable finger gaits automatically emerge from training the control policy via reinforcement learning. Code and more videos are available at https://haozhi.io/hora
1 Introduction
Generalizable in-hand manipulation remains an unsolved robotics challenge. This paper demonstrates fingertip-only z-axis object rotation with an adaptive controller that handles diverse objects while maintaining force closure.
- Generalizable in-hand manipulation must accommodate objects differing in shape, size, mass, and materials.
- The demonstrated task rotates diverse objects over the z-axis using only the fingertips of a multi-fingered robot hand.The task simplifies general SO(3) reorientation but still requires dynamic or static force closure because the palm cannot support the object.
- The approach is inspired by reinforcement-learning methods that learn compact representations of environmental properties for rapid online controller adaptation.Here, object properties perceived by the fingertips are likewise represented compactly and inferred from proprioceptive history.
2 Related Work
Prior in-hand manipulation work includes model-based control, simulation trajectory optimization, and reinforcement-learning approaches. The paper’s approach is situated among methods that seek real-world robustness but notes that existing systems often manipulate limited object sets.
- Classic control methods typically require analytical models of object and robot geometry for planning finger movements.Related approaches use geometric assumptions, finger tracking, or optimization-generated trajectories for object manipulation.
- Reinforcement-learning methods use simulation variation or large-scale training to improve robustness and transfer manipulation skills to real robots.
- Existing reinforcement-learning examples described here remain focused on manipulating a limited number of objects.
3 Rapid Motor Adaptation for In-Hand Object Rotation
The method combines a privileged-information base policy with a learned extrinsics embedding and an adaptation module that estimates object properties online from proprioception and action history. Reinforcement learning shapes rotation, stability, energy, and pose behavior without prescribing finger gaits.
- The deployment policy infers a low-dimensional embedding of object properties from proprioception and action history, then uses it to rotate the object.
- Base Policy Training: Privileged information comprises simulated object position, size, mass, friction coefficient, and center of mass, represented as a 9-dimensional vector.The embedding µ maps this vector to an 8-dimensional extrinsics vector used by the policy.
- Base Policy Training: The base policy receives joint positions, the previous predicted action, and extrinsics, while additional history supplies velocity and acceleration information.
- Base Policy Training: PPO jointly optimizes the policy and embedding using rotation reward together with penalties for pose deviation, linear velocity, work, and torque.The reward combines these terms through weighted summation.
- Base Policy Training: Stable finger-gaiting behavior emerges from energy and pose-deviation constraints rather than an imposed heuristic contact pattern.Unlike a cited baseline, the method does not explicitly require at least three fingertips to remain in contact.
- Adaptation Module Training: Because simulated extrinsics are unavailable on the real robot, an adaptation module estimates them from discrepancies between proprioception and commanded-action histories.
- Adaptation Module Training: The adaptation module is trained with Adam by minimizing the ℓ2 distance between ground-truth and estimated extrinsics.
4 Experimental Setup and Implementation Details
Experiments use an Allegro Hand and IsaacGym simulation with large-scale parallel training, compare adaptive control against several baselines, and evaluate within- and out-of-distribution settings using defined stability, rotation, velocity, and torque metrics.
- Hardware and Simulation Setup: The Allegro Hand has four fingers with four degrees of freedom each, and its 16 joints are position-controlled at 20 Hz.Target positions are converted to torque through a PD controller operating at 300 Hz.
- Hardware and Simulation Setup: IsaacGym training uses 16384 parallel environments containing simulated Allegro Hands and randomized cylindrical objects.Simulation runs at 120 Hz, control at 20 Hz, and each episode lasts 400 control steps or 20 seconds.
- Baselines: The baselines include robust domain randomization, online explicit system identification, and no online adaptation, alongside an expert using privileged information.
- Evaluation Metrics: TTF measures normalized episode duration before falling, while RotR, ObjVel, and Torque measure rotation reward, object linear velocity, and commanded-torque magnitude.
5 Results and Analysis
The method uses online adaptation to generalize in-hand z-axis rotation across heavy, irregular, and unseen objects. Its learned extrinsics reflect object properties, while stable finger gaits emerge without explicit gait heuristics.
- Generalization via Adaptation: Online adaptation achieves the best performance across rotated angle, Time To Fall, and torque-based energy efficiency on heavy objects.DR is slower, SysID is unstable, and NoAdapt fails the task.
- Generalization via Adaptation: Across 500K simulated episodes, online adaptation gives the best performance and improves generalization to out-of-distribution object parameters.The evaluation averages results over five models trained with different seeds.
- Generalization via Adaptation: Adaptation outperforms all baselines on irregular objects, including holes, soft objects, and deformable objects absent from training.DR rotates conservatively and slowly, while SysID is faster but less stable.
- Understanding and Analysis: Estimated extrinsics respond to object diameter and mass, with different object sizes and weights occupying separate regions in the embedding space.Irregular shapes produce within-object variation because perceived scale changes during rotation.
- Understanding and Analysis: Training on cylindrical objects produces stable, high-clearance finger gaits, whereas spherical-object training yields gaits that fail to generalize to complex objects.The policy does not enforce heuristic finger gaiting behavior.
- Real World Qualitative Results: The real-world evaluation covers 30 objects spanning diverse shapes and physical properties, including porous and non-rigid objects.Training uses only cylindrical objects of different sizes and aspect ratios, without real-world fine-tuning.
6 Discussion and Limitations
The paper demonstrates proprioceptive z-axis rotation but identifies contact uncertainty, very small objects, and extreme shapes as important boundaries.
- Discussion and Limitations: The task simplifies general SO(3) reorientation to z-axis rotation, although three axis-specific policies could reach arbitrary target poses.The authors present multi-axis rotation as an important future extension.
- Discussion and Limitations: Incorrect contact points cause many real-world failures because proprioceptive sensing does not reveal precise fingertip-object contact positions.Incorrect contacts can produce unstable force closure.
- Discussion and Limitations: Objects smaller than 4.0 cm are a failure case because finger collisions prevent the robot from maintaining grasp balance.More extreme and sophisticated shapes are also harder to manipulate.
A.1 Per-Object Result Result Analysis.
Per-object analysis shows strong stability across many object properties, while challenging geometries and small objects reduce rotation duration.
- Per-Object Result Result Analysis: 22 of 33 objects achieve almost perfect stable and continuous in-hand rotation.These objects vary in scale, mass, friction coefficient, and shape.
- Per-Object Result Result Analysis: The policy remains stable and dynamic for objects with very high centers of mass, including a plastic bottle and paper towel.Cubes, high-aspect-ratio heavy objects, and small objects typically support about 10 to 20 seconds of rotation.
- Per-Object Result Result Analysis: Average commanded torque correlates almost linearly with object mass, with lighter objects receiving smaller torque commands for energy efficiency.The analysis connects policy behavior to object weight.
A.2 Multi-Axis In-Hand Object Rotation
The approach is extended to multi-axis in-hand object rotation, where the policy produces smooth transitions between rotation about different axes. This task is harder than single-axis training and requires about 1.5× the training time.
- Multi-axis in-hand object rotation is qualitatively demonstrated as an extension of the approach.
- The multi-axis task requires about 1.5× the training time of single-axis training.
- The policy achieves smooth gait transitions between rotation over different axes.
B Ablation Experiments
Ablations examine physical randomization, proprioceptive history, and temporal-input baselines. Performance improves with longer history until saturating at T=30, while removing randomization or extending baseline inputs degrades performance, especially out of distribution.
- Removing physical randomization causes much worse performance, especially in the out-of-distribution case.
- Performance increases with proprioceptive history length but saturates at about T=30.The final experiments use T=30 for inference efficiency.
- Both MLP and LSTM domain-randomization baselines suffer optimization difficulty with longer observation sequences, reducing performance.
C Implementation Details
The implementation trains with randomized cylindrical-object physics, grasp and reward constraints, and a proprioception-based adaptive policy. The reward combines rotation with pose, motion, energy, and torque considerations to encourage stable, efficient gaits.
- Training uses cylindrical objects with randomized physical parameters, disturbances, and joint-position noise, plus sphere and cube objects for out-of-distribution evaluation.
- The grasp setup requires fingertip proximity, contact from at least two fingers, and an object height above the palm center.
- The reward clips rotation, penalizes hand-pose deviation, object linear velocity, energy consumption, and torque.
- The reward design aims to prevent excessively fast rotation and encourages stable, smooth gaits that improve sim-to-real performance.
- The policy commands smaller torque for lighter objects, supporting energy efficiency.
- Optimization jointly trains the base policy and object-property encoder using PPO, while the adaptation encoder uses Adam with a 3e−4 learning rate.
- Spherical and light objects are easier to rotate than irregular objects, while the paper towel achieves the fewest rotations because of finger collisions.
- More than half of the objects remain stable for over 30 seconds, and every object rotates for more than 15 seconds.