PaperlayerView exact revision on arXiv

Source-linked AI summary

In-Hand Object Rotation via Rapid Motor Adaptation

Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, Jitendra Malik

arXiv:2210.04887v1cs.ROcs.AIcs.CVcs.LG

TL;DR

Generalizing in-hand manipulation beyond a few objects remains an unsolved challenge, even though fingertip-only z-axis rotation is a difficult simplification. This paper trains an adaptive controller on simulated cylinders and deploys it without fine-tuning, reporting real-world rotation of dozens of diverse objects.

  • Problem

    In-hand rotation with real multi-fingered hands has been shown for only a few objects, while generalizable manipulation remains an unsolved robotics challenge.

    Source: §1 · p. 1

  • Method

    The controller infers object properties from proprioception and action history during deployment, then uses that embedding to guide a base rotation policy.

    Source: §3 · p. 3

  • Results

    Across 30 objects with varied shapes and physical properties, the method performs reasonably well and outperforms baselines on challenging objects beyond those seen in training.

    Source: §5.1 · p. 7, §5.3 · p. 8

  • Takeaways & Limitations

    The work demonstrates the feasibility of purely proprioceptive in-hand rotation for diverse objects over the z-axis.

    Source: §6 · p. 8

    Real-world failures often stem from incorrect contact points that destabilize force closure, which proprioceptive sensing cannot precisely identify.

    Source: §6 · p. 8

Abstract

from arXiv · show

Generalized in-hand manipulation has long been an unsolved challenge of robotics. As a small step towards this grand goal, we demonstrate how to design and learn a simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then - without any fine-tuning - can be directly deployed to a real robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the controller to the object properties using only proprioception history. Furthermore, natural and stable finger gaits automatically emerge from training the control policy via reinforcement learning. Code and more videos are available at https://haozhi.io/hora

1 Introduction

The paper presents a fingertip controller for rotating diverse objects about the z-axis, adapting online from proprioception history without requiring vision or tactile sensing. On a real Allegro Hand, it rotates over 30 objects and develops a smooth finger gait.

Problem

  • Fingertip rotation about the z-axis remains challenging because the fingers must maintain force closure without support from the palm or another surface.The task simplifies general in-hand reorientation, while prior real-world demonstrations had handled only one or a few objects [2] [3] [4].Source: §1 · p. 1

Approach

  • The approach compresses fingertip-perceived shape, mass, and size into learned extrinsics, inspired by locomotion methods that estimate terrain properties online [5] [6].The controller uses this compact representation to adaptively manipulate objects with different physical properties.Source: §1 · p. 2
  • The method trains an adaptive policy with object extrinsics, then estimates them online from proprioception history to adapt during real-world deployment.The adaptation module is trained solely in simulation using supervised learning on discrepancies between observed proprioception history and commanded actions.Source: §1 · p. 2

Results

  • On an Allegro Hand, the method rotated over 30 real-world objects spanning 4.5–7.5 cm and 5–200 g, including deformable or soft objects.A smooth adaptive finger gait emerged, and the controller used proprioception without vision or tactile sensing; estimated extrinsics correlated with mass and scale and had a low-dimensional structure.Source: §1 · p. 2

2 Related Work

Prior approaches to in-hand manipulation rely on analytical models, face sample-efficiency and sim-to-real challenges, or handle limited object diversity. This work uses model-free reinforcement learning and adaptation to target transfer across diverse objects.

Classic Control for In-Hand Manipulation

  • Classic controllers plan using analytical object and robot models or optimized trajectories, while real-world object diversity remains limited [4].In contrast, this work uses model-free reinforcement learning and adaptation to generalize.Source: §2 · p. 2

Reinforcement Learning for In-Hand Manipulation

  • Real-world reinforcement learning has produced simple or narrow skills due to sample inefficiency [23] [24] [25], while diverse simulated skills remain difficult to transfer [26] [27] [28], [29] [30].This work instead trains the policy entirely in simulation for direct real-world transfer.Source: §2 · p. 2

Sim-to-Real Transfer via Domain Randomization

  • Domain-randomized methods and [2] manipulate limited object sets, while [3] transfers cube finger-gaiting with the hand facing down but not fingertip-only manipulation.This work targets diverse objects and can be trained within a few hours.Source: §2 · p. 3

Sim-to-Real via Adaptation

  • Sim-to-real adaptation estimates system parameters or encodes environment conditions in embeddings [5] [6], but applying these methods to in-hand manipulation is not straightforward.This work designs the reward and training environment to produce a natural, stable controller that transfers to the real world; learning exact physical values may be suboptimal given simulation inaccuracies.Source: §2 · p. 3

3 Rapid Motor Adaptation for In-Hand Object Rotation

The method trains a base policy with simulator-provided object properties, then adapts it at deployment by inferring those properties from proprioception and action history.

3.1 Base Policy Training

  • The base policy maps joint positions, recent actions, and an 8D embedding of the simulator’s 9D object-property vector to PD-controller targets.The property vector includes position, size, mass, friction, and center of mass.Source: §3.1 · pp. 3–4
  • Energy and initial-pose penalties yield stable finger gaits without prescribing gait behavior or requiring three fingertips to remain in contact [28].The reward combines rotation with penalties for pose deviation, object linear velocity, energy use, and torque.Source: §3.1 · p. 4
  • Training randomizes cylinder dimensions, mass, center of mass, friction, and grasp initialization to promote real-world generalization.Objects and robot joints are sampled around a canonical grasp until a stable precision grasp is achieved.Source: §3.1 · p. 4

3.2 Adaptation Module Training

  • The adaptation module infers extrinsics from proprioception and action history, then is trained iteratively to minimize L2 error against ground-truth extrinsics [34].Training trajectories are collected using predicted extrinsics, and the setup uses the same object and dynamics randomization as base-policy training.Source: §3.2 · p. 4

4 Experimental Setup and Implementation Details

The experiments evaluate an Allegro-hand fingertip rotation controller in IsaacGym against adaptive and non-adaptive baselines, using measures of survival, rotation, stability, and torque.

Experimental Setup and Implementation Details

  • The Allegro Hand's 16 joints use position control at 20 Hz and PD torque conversion at 300 Hz; simulation uses 16,384 environments at 120 Hz with 20 Hz control.Each episode lasts 400 control steps (20 s); the hand has four fingers with four degrees of freedom each.Source: §4 · p. 4
  • 222.27±21.20 RotR and 0.82±0.02 TTF within-distribution; 160.60±10.22 RotR and 0.68±0.07 TTF out-of-distribution, with ours best among baselines.The corresponding ObjVel/Torque pairs are 0.29±0.05/1.20±0.19 within-distribution and 0.47±0.07/1.20±0.17 out-of-distribution; the table says performance closely emulates the privileged-information Expert.Source: §4 · p. 5, Table 1 · p. 5
  • Comparators include a domain-randomized robust policy [3] [2] [36], explicit SysID, frozen-at-start NoAdapt, periodic expert-action replay, and a privileged-information Expert upper bound.The robust policy uses the same reward without privileged information; SysID predicts system parameters, while NoAdapt estimates extrinsics once and holds them fixed.Source: §4 · p. 5
  • Evaluation tracks episode survival (TTF), rotation reward (RotR), real-world net z-axis radians, simulated object linear speed (ObjVel), and commanded-torque cost (Torque).TTF is normalized by maximum episode duration; ObjVel is scaled by 100 and measured only in simulation, while real-world rotations measure net rotation relative to the world's z-axis.Source: §4 · p. 5

5 Results and Analysis

The results show that online adaptation supports in-hand rotation and generalization from cylindrical-object training to diverse real-world objects. Estimated extrinsics track object properties, and training with cylinders produces gaits that generalize better than sphere-trained gaits to complex objects.

5.1 Generalization via Adaptation

  • 23.96 average Rotations (0.8 rad s−1 equivalent rotation velocity) accompanied consistent rotation of heavy objects, and our method outperformed baselines on heavy and irregular objects.In simulation, online adaptation achieved the best performance among the baselines and generalized better to out-of-distribution object parameters.Source: Figure 3 · p. 6, §5.1 · pp. 6–7, Figure 4 · p. 7

5.2 Understanding and Analysis

  • Estimated extrinsics tracked object diameter and mass during a continuous six-object run, while t-SNE separated objects by size and weight.The diameter-related extrinsic also varied while rotating irregular objects, consistent with changes in their perceived diameter.Source: §5.2 · p. 7, Figure 5 · p. 7, Figure 6 · p. 8
  • Training with cylindrical objects produced stable, high-clearance finger gaits, whereas sphere-only training produced a dynamic gait that failed to generalize to complex objects.The sphere-trained gait worked well on balls.Source: §5.2 · p. 8

5.3 Real World Qualitative Results

  • The real-world evaluation covered 30 objects spanning 4.5–7.5 cm in diameter and 5–200 g in mass, although training used only cylinders of different sizes and aspect ratios.The proposed approach estimates a low-dimensional representation of fingertip-perceived object shape from proprioceptive history to support generalization.Source: §5.3 · p. 8

6 Discussion and Limitations

The paper demonstrates proprioceptive in-hand rotation about the z-axis for diverse objects, while identifying contact, object-size, and shape limits. The authors propose three axis-specific policies as a path toward arbitrary target poses and real-world data as future work.

6 Discussion and Limitations

  • The z-axis rotation task simplifies general SO(3) reorientation, and the authors propose three policies for rotation along three principal axes to reach any target pose.They describe this broader capability as an important future extension.Source: §6 · p. 8
  • The experiments show the feasibility of proprioception-only in-hand object rotation across a diverse set of objects.Source: §6 · p. 8
  • Most real-world failures stem from incorrect contact points and unstable force closure, which proprioceptive-only sensing cannot detect precisely.Source: §6 · p. 8
  • Objects under 4.0 cm in diameter often cause finger collisions that prevent the robot from maintaining grasp balance.Source: §6 · p. 8
  • More extreme or sophisticated shapes are harder to manipulate and may require tactile or visual feedback; using real-world data, such as through meta-learning, is proposed as future work.The policy was not improved using real-world experience because the study focused on generalization via adaptation.Source: §6 · p. 8

A Additional Results and Analysis · A.1 Per-Object Result Result Analysis.

The per-object analysis reports stable fingertip rotation across objects with varied properties, while also identifying shorter-duration performance on difficult objects and drops from incorrect contact positions.

A.1 Per-Object Result Result Analysis.

  • 22 of 33 objects achieved almost perfect, stable, continuous in-hand rotation across varied scales, masses, friction coefficients, and shapes.The successful set included objects with different scales, masses, coefficients of friction, and shapes.Source: §A.1 · p. 12
  • Objects with very high centers of mass, including a plastic bottle and paper towel, still underwent stable, dynamic in-hand rotation.Source: §A.1 · p. 12
  • Cubes, heavy high-aspect-ratio objects such as Pear and Kiwi Fruit, and small objects achieved about 10 to 20 seconds of rotation.Source: §A.1 · p. 12
  • Most failure cases involved objects falling from the fingertips because of incorrect contact positions.Source: §A.1 · p. 12
  • Figure 7 shows an almost linear correlation between object mass and the average torque commanded by the policy.Source: §A.1 · p. 12
  • The policy commands smaller torque for lighter objects, which the paper describes as supporting energy efficiency.Source: §A.1 · p. 12

A.2 Multi-Axis In-Hand Object Rotation

Preliminary qualitative results demonstrate multi-axis in-hand rotation, a harder task than single-axis rotation that requires more training but produces smooth transitions between rotation axes.

A.2 Multi-Axis In-Hand Object Rotation

  • Multi-axis rotation is harder than single-axis training, needs about 1.5× the training time, and yields smooth gait transitions between axes.The results are preliminary and demonstrated qualitatively on the project website.Source: §A.2 · p. 12

B Ablation Experiments

The ablations show that physical randomization is important, especially for out-of-distribution performance, while longer proprioceptive history helps until performance saturates around T=30. Extending temporal inputs in the robust domain-randomization baseline reduces performance and creates optimization difficulties.

B Ablation Experiments

  • Without physical randomization during training, performance is much worse, especially out of distribution.Table 2 reports the No Rand entry against the method; the supplied passage does not include numeric values.Source: Table 2 · p. 12
  • Performance increases with proprioceptive history length but saturates around T=30, which is used in final experiments for inference efficiency.The approach is described as not sensitive to this hyperparameter once the history is long enough.Source: B Ablation Experiments · p. 12, Table 3 · p. 12
  • With longer temporal inputs, MLP performance decreases and is hard to optimize; LSTM performs better than MLP at longer lengths but also has optimization difficulty, while temporal convolution fails to produce usable policies.The comparison extends robot observations from 3 to 30 timesteps; the temporal convolution network produces random policies.Source: B Ablation Experiments · p. 13, Table 4 · p. 13

C Implementation Details

The implementation combines randomized simulation training and filtered precision-grasp initialization with a reward that balances z-axis rotation against stability and effort. An MLP policy uses an adaptation module that infers object properties from recent joint-position and action history.

Physical Randomization Parameter

  • Training randomizes object physics and scale on cylinders, then tests broader ranges with stronger disturbances, joint noise, and sphere and cube objects.Training disturbances have scale 2m and decay by 0.9 every 80 ms; out-of-distribution testing raises the scale to 4m. The added joint-position noise is sampled from U(0, 0.005).Source: C Implementation Details · p. 13, Table 5 · p. 14

Stable Precision Grasp Generation

  • Grasp poses are retained only when fingertips are within 10 cm, at least two fingers contact the object, and its height exceeds 14.5 cm above the palm.The procedure perturbs a canonical grasp by U(−0.25, 0.25) rad, simulates 0.5 s, and pre-samples 50,000 poses per discretized object scale.Source: C Implementation Details · pp. 13–14

Reward Definition

  • The reward clips z-axis rotation between −0.5 and 0.5 and penalizes hand-pose deviation, work, object linear velocity, and torque.The pose penalty discourages unnatural gaits and supports breaking and establishing new contacts; work and linear-velocity penalties encourage smoother, more stable behavior. Training terminates falls at about 14.5 cm above the palm, versus about 10.0 cm during evaluation.Source: C Implementation Details · p. 14

Network Architecture

  • The 96-dimensional-state policy maps state and an 8-dimensional extrinsics vector to 16 actions, while an encoder estimates extrinsics from nine inputs.The policy uses four MLP layers [512, 256, 128, 16]; the adaptation module processes 30 timesteps (1.5 seconds) of joint positions and actions using per-step MLP embeddings and three temporal 1-D convolutional layers.Source: C Implementation Details · p. 14

Optimization Details

  • PPO jointly trains the policy and extrinsics encoder using 16,384 environments, for 100,000 gradient updates.Each iteration collects eight steps per environment and performs five epochs; training uses about 500 million agent steps, equivalent to roughly 7,000 hours of real-world time.Source: C Implementation Details · p. 15

Related papers

Loading 2210.04887v1…