Source-linked AI summary
A System for General In-Hand Object Re-Orientation
Tao Chen, Jie Xu, Pulkit Agrawal
TL;DR
In-hand object reorientation is difficult because it combines high-dimensional actuation with frequent contact changes. This paper develops a model-free framework that handles upward- and downward-facing hands and distills policies toward real-world-accessible observations, demonstrating reorientation across more than 2000 diverse objects. The results also show shape-agnostic manipulation of many previously unseen objects, although failures concentrate on long, small, or thin objects.
Problem
In-hand reorientation remains difficult because dexterous hands require high-dimensional control while finger-object contact states change frequently.
Method
The system trains model-free teacher policies and distills them toward reduced-state or RGBD point-cloud observations, using gravity curriculum and stable object initialization for downward manipulation.
Results
The system reorients more than 2000 geometrically diverse objects, while policies without shape information manipulate many previously unseen objects across the evaluated settings.
Takeaways & Limitations
General in-hand reorientation can be learned with shape-agnostic policies, including upward- and downward-facing hand settings and manipulation using external support.
Takeaways & Limitations
Experiments are limited to simulation, and failures primarily involve long, small, or thin objects.
Abstract
from arXiv · showhide
In-hand object reorientation has been a challenging problem in robotics due to high dimensional actuation space and the frequent change in contact state between the fingers and the objects. We present a simple model-free framework that can learn to reorient objects with both the hand facing upwards and downwards. We demonstrate the capability of reorienting over 2000 geometrically different objects in both cases. The learned policies show strong zero-shot transfer performance on new objects. We provide evidence that these policies are amenable to real-world operation by distilling them to use observations easily available in the real world. The videos of the learned policies are available at: https://taochenshh.github.io/projects/in-hand-reorientation.
1 Introduction
The paper targets general in-hand reorientation across hand orientations, support conditions, object shapes, and real-world-accessible observations. It proposes a model-free system combining teacher-student learning, gravity curriculum, and stable initialization, while finding that shape information is unnecessary for manipulating many unseen objects.
- 1 Introduction: In-hand reorientation is difficult because dexterous hands have high-dimensional control and frequent finger-object contact-state transitions.The task includes upward-facing manipulation, downward-facing manipulation with table support, and unsupported downward manipulation.
- 1 Introduction: Prior approaches often relied on object models, simple geometries, simulation, specialized sensing, or the upward-facing setting, limiting demonstrated generality.The cited prior work did not establish all four desired capabilities together.
- 1 Introduction: The paper seeks one system that handles upward- and downward-facing hands, external support, novel object shapes, arbitrary orientations, and readily available sensory data.The target observations include RGBD images and hand joint positions.
- 1 Introduction: The framework uses model-free reinforcement learning with teacher-student learning, a gravity curriculum, and stable object initialization, without object or manipulator models or contact dynamics.The experiments are conducted in simulation, while the paper provides evidence for future real-world transfer.
- 1 Introduction: Policies without shape information manipulate many previously unseen objects across the three evaluated settings.The authors describe this as evidence that shape-agnostic control can achieve general in-hand reorientation.
2 Method
The method learns a privileged teacher policy with model-free reinforcement learning, then distills it into policies using reduced state or direct RGBD point-cloud observations. Goal point clouds, recurrent visual processing, stable initialization, gravity curriculum, and Chamfer-distance evaluation support general reorientation.
- 2 Method: Model-free reinforcement learning avoids explicit modeling of nonlinear, frequently changing hand-object contacts and supports operation from raw point-cloud observations.Privileged state information is used to train better policies faster before distillation.
- 2 Method: The teacher is trained on more than two thousand objects, then distilled with DAgger into policies using reduced state information or RGBD point clouds and hand joint positions.The vision student takes a voxelized scene point cloud, joint positions, and the previous action as inputs.
- 2 Method: Unsupported downward manipulation uses stable object initialization and gradually introduced gravity because gravity makes exploration substantially harder.A table provides support in the supported downward setting by preventing objects from falling during exploration.
- 2 Method: The vision policy represents the desired orientation as a goal object point cloud, showing the policy how the object should appear at the end.This avoids manually defining a per-object coordinate frame for a goal quaternion.
- 2 Method: The visual architecture processes voxelized point clouds with sparse 3D convolutions, combines them with joint positions and previous actions, and uses recurrence to output actions.The architecture extends IMPALA-style processing with Minkowski Engine sparse convolutions.
- 2 Method: The vision policy evaluates success with either orientation error or Chamfer distance, addressing symmetric objects whose distinct orientations can produce identical point clouds.The success criterion is (∆θ ≤¯θ) ∨(dC ≤¯dC).
3 Experimental Setup
Experiments use a simulated 24-DoF Shadow Hand and diverse EGAD and YCB object datasets, with upward and downward initial configurations and vision observations from RGBD cameras. Success is measured over randomized start and goal orientations.
- 3 Experimental Setup: The experiments use a simulated anthropomorphic Shadow Hand with 24 degrees of freedom and a fixed base.Twenty joints are tendon-actuated and four are under-actuated.
- 3 Experimental Setup: The object benchmark combines EGAD’s 2282 geometrically diverse textureless meshes with 78 scanned YCB object models representing daily-life objects.YCB objects include varied shapes and textures and are proportionally scaled for in-hand manipulation.
- 3 Experimental Setup: Vision experiments use two RGBD cameras above the upward-facing hand, combining their data into a voxelized colored point-cloud observation.The point cloud voxel resolution is 0.003 m.
- 3 Experimental Setup: Figure 3 distinguishes upward-facing initialization from three downward-facing cases: random poses, table support, and lifted poses.The downward configurations represent different support and initialization conditions.
- 3 Experimental Setup: Initial and goal orientations are randomly sampled from SO(3), and performance is the percentage of successful episodes over 100 trials per training object.Reported means and standard deviations come from three seeds; non-vision success uses an orientation-error threshold.
4 Results
The learned policies reorient diverse objects across upward- and downward-facing hand settings, including with and without external support. They also show transfer across object datasets, robustness to perturbations, and identifiable failure modes.
- General evaluation: More than 2,000 diverse objects were used to evaluate upward- and downward-facing hand reorientation and generalization to unseen shapes.The evaluation explicitly analyzes policies with the hand facing upward and downward.
- Upward-facing hand: Over 90% success on EGAD and over 80% on YCB were achieved by full-state RNN policies without explicit object-shape knowledge.These results are reported for policies trained separately on the two datasets.
- Failure modes: 60% of YCB failures involved long, small, or thin objects, while 19% were near-goal reorientations that missed the success threshold.Aspect-ratio diversity and scaling constraints made some objects too small, large, or thin; sphere-like objects were easiest and knives and forks hardest.
- Robustness: Domain randomization made policies more robust to dynamics variations and observation/action noise.The paper also reports good success rates after adding randomization and noise.
- Robustness: Around 75% success was retained when testing a YCB policy on objects whose masses were changed to [0.3, 0.5]kg.This is a zero-shot test on modified object dynamics.
- Table-supported reorientation: 95.31% ± 0.9% on EGAD and 81.59% ± 0.7% on YCB were obtained by full-state MLP policies using external support.The experiments support model-free RL as an alternative for learning manipulation strategies that use support surfaces.
- Downward-facing hand: 97.80% success on EGAD and 90.11% on YCB were achieved for downward-facing reorientation after pose initialization from successful lifting rollouts.A separate RNN policy was trained for each dataset, with objects first grasped and lifted from the table.
- Zero-shot transfer: Policies trained with gravity curriculum still performed well on the untrained dataset in cross-dataset tests.The downward-facing transfer used RNN policies trained with gravity curriculum.
5 Related Work
Prior dexterous-manipulation methods commonly rely on analytical models, trajectory optimization, or restricted task settings. This work addresses broader reorientation tasks and reports high success without object-shape information.
- Model-based approaches: Analytical and planning methods often simplify hand-object geometry because many active contacts make optimization or planning difficult.Prior work includes trajectory optimization, kinodynamic planning, and assumptions preventing contact changes.
- Scope of this work: Recent shape-conditioned work addressed z-axis reorientation with an upward-facing hand, whereas this work considers diverse objects, arbitrary SO(3) goals, and both hand orientations.The comparison also emphasizes shape-agnostic policies.
6 Discussion and Conclusion
The paper shows that simple model-free RL can reorient geometrically diverse objects without shape information across three settings. It also identifies object-shape-dependent performance and limits the experiments to simulation while presenting transfer evidence.
- Model-free RL with simple deep architectures reorients a large set of geometrically diverse objects without explicit shape information.The same broad finding supports using shape-agnostic policies as a baseline for future work incorporating object shape.
- Downward-facing manipulation benefits from lifting-based pose initialization and a gravity curriculum, while the policy can use a table as external support.
- Experiments are limited to simulation, although domain randomization and torque analysis provide evidence supporting possible real-world transfer.
- Success rates vary substantially with object shape, motivating future shape-based training curricula.
Appendix A Evidence indicating transfer to real-world
The appendix evaluates real-world transfer readiness through simulation evidence, accessible observations, randomized conditions, and torque feasibility. These results support potential transfer but do not constitute real-robot experiments.
- Experiments are limited to simulation, with transfer evidence provided rather than direct real-robot evaluation.
- Student policies use observations intended to be readily available in practice, including joint positions with object pose or scene point clouds.
- Domain randomization varies physical parameters and adds observation and action noise to approximate real-world variability.
- Randomization and noise preserve good success rates, indicating potential transfer despite imperfect simulation replication.
- Peak torque values remain in a reasonable range for the Shadow hand, suggesting lower likelihood of motor overload.
B.2 Dataset
The experiments use diverse EGAD and YCB object datasets, augmenting each mesh through random scaling and approximate convex decomposition. The setup also specifies camera placement for scene observations.
- The study uses EGAD and YCB datasets and creates five randomly scaled variants of each object mesh.Bounding-box longest-side ranges are 0.05–0.08 m for EGAD and 0.05–0.12 m for YCB; masses are sampled from 0.05–0.15 kg.
- EGAD and YCB objects have geometrically different, complex shapes that remain diverse after V-HACD convex decomposition.
- Two RGBD cameras are positioned above the hand to capture the scene.
- Figure B.2 presents EGAD examples in the first row and YCB examples in the second row.
- Figure B.3 contrasts visual meshes with corresponding collision meshes after V-HACD decomposition.
Appendix C Experiment Setup
The experiment setup trains teacher policies with MLP or RNN architectures using PPO, then supports vision policies based on point clouds and recurrent processing. Training includes task-specific rewards, large-scale parallel environments, and dynamics randomization.
- Non-vision policies use either a three-layer MLP or an RNN with a 256-dimensional GRU and ELU activations.
- Vision policies process scene point clouds with Sparse3D-IMPALA-Net and combine visual, joint-position, and previous-action embeddings recurrently.
- PPO trains the policies with up to 40K parallel environments, while vision experiments use 360 environments and 15,000 scene points plus 5,000 goal points.
- The reorientation reward encourages target orientation and penalizes large action commands, while the lifting reward uses object-to-hand height difference.
- Dynamics randomization and observation/action noise improve robustness, although some distilled MLP policies collapse under randomized dynamics in C3 and H3.
- A gravity curriculum improves learning for downward-facing YCB manipulation.
D.1 Hand faces upward
The upward-facing-hand policies learn successfully on EGAD and YCB, including without explicit object-shape information, while full-state inputs accelerate learning. Failures mainly involve unsuitable object size or initialization and near-miss target orientations.
- Learning curves: Both MLP and RNN policies learn well on the EGAD and YCB datasets.YCB requires substantially more environment interactions than EGAD.
- Learning curves: Full-state information speeds policy learning and improves final performance.
- Testing performance - Teacher: Success rates exceed 90% on EGAD and 70% on YCB without explicit object-shape knowledge.
- Testing performance - Student: Distillation can reduce EGAD success by 36% and YCB success by 47% when the student policy lacks temporal information.The reduced state space makes temporal information necessary for the student policy.
- Testing performance - Student: Domain randomization makes the distilled policy more robust to dynamics variations and observation/action noise.
- Failure cases: Failures occur with objects that are too small, thin, or large, border-near initialization, and reorientations that do not meet the target threshold.The target criterion includes Δθ ≤ θ̄.
D.4 Torque analysis
The torque analysis records finger-joint torques across many YCB reorientation episodes, while accompanying examples illustrate downward-hand manipulation of cups and sponges.
- Torque analysis: Torque analysis records every finger-joint torque at each time step across 20,000 episodes.The episodes come from 100 randomly sampled YCB objects, each reoriented 200 times.
- Examples: The appendix provides episode visualizations for downward-hand reorientation of a cup and a sponge.
D.5 Vision experiments with noise
Vision policies are trained with point-cloud perturbations to evaluate noisy observations. Training augmentation improves data efficiency, although final performance can sometimes be slightly lower.
- Noise transformations: Four point-cloud transformations are used: translation, point jitter, point dropout, and color jitter.
- Noise transformations: Each transformation is applied independently with 40% probability to every point cloud at every time step.
- Results: Training with data augmentation improves vision-policy data efficiency, even though final performance may be slightly lower.
- Results: For the mug, augmentation significantly improves final testing performance by helping escape a handle-misaligned local optimum.