PaperlayerView exact revision on arXiv

Source-linked AI summary

Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?

Tyler Ga Wei Lum, Kushal Kedia, C. Karen Liu, Jeannette Bohg

arXiv:2606.26428v2cs.ROcs.AI

TL;DR

Precise dexterous assembly remains difficult because contact-rich interactions and sparse rewards limit existing robot-learning approaches, and it is unclear which play-pretraining factors matter downstream. Play2Perfect learns a general manipulation prior through RL play, then finetunes it for assembly, achieving 33× greater sample efficiency than training from scratch and zero-shot real-world transfer.

  • Problem

    It remains unclear which aspects of play pretraining matter for downstream finetuning to precise, sparse-reward assembly with dexterous hands.

  • Method

    Play2Perfect pretrains a goal-conditioned RL policy on diverse objects and poses, then finetunes the resulting manipulation prior for sparse-reward precise assembly.

  • Results

    33× greater sample efficiency than RL from scratch, with zero-shot real-world transfer achieving 60% success on 0.5 mm-clearance insertions.

  • Takeaways & Limitations

    Object diversity, training objectives, trajectory diversity, and goal precision all affect downstream assembly performance, while the learned prior transfers zero-shot to real-world tasks.

  • Takeaways & Limitations

    The system learns short-horizon skills rather than a complete autonomous assembly pipeline, with sequencing, part selection, and goal poses specified externally.

Abstract

from arXiv · show

Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. These tasks are contact-rich, making data collection for imitation learning difficult, and sparse-reward, making direct exploration with reinforcement learning (RL) intractable. Consequently, prior work has made progress by structuring the problem with specialized grippers, tool attachments, and environment fixtures. In this work, we argue that before a robot can perfect precise assembly, it must first learn to play. We further ask the question: what factors in the process of learning to play matter for precise assembly? We propose Play2Perfect, an RL framework for task-agnostic pretraining through play on diverse objects and goals, which is then perfected on precise assembly. The goal of play is to acquire reusable manipulation priors, such as grasping, in-hand reorientation and pose reaching. Finetuning then adapts this general prior to assembly, focusing exploration on the final contact-rich, high-precision interactions needed for success. We systematically study key design choices in play pretraining, including object diversity, training objective, trajectory diversity, and goal precision. We show that our prior is 33x more sample-efficient than RL training from scratch, even when provided with dense, multi-stage rewards. We demonstrate zero-shot sim-to-real transfer, achieving 60% success on tight insertions with only 0.5 mm contact clearance, and over 50% success on long-horizon multi-part assembly and screwing.

1 Introduction

Precise assembly remains difficult for dexterous robot learning because contact-rich control and sparse final-goal rewards provide little training signal. Play2Perfect addresses this by pretraining a task-agnostic manipulation prior on diverse objects and goals, then finetuning it for precise assembly while studying which play-design choices transfer.

  • Motivation: Multi-fingered robots remain limited on precise assembly because controlling many contact-mediated degrees of freedom is challenging.The contact-rich nature of assembly also makes dexterous teleoperation difficult, leading imitation learning toward lower-precision pick-and-place tasks.
  • Motivation: Sparse final-goal rewards force random-policy RL to discover grasping, in-hand reorientation, alignment, and insertion before receiving feedback.This limited training signal motivates learning reusable manipulation skills before attempting precise assembly.
  • Play2Perfect: Play2Perfect pretrains a goal-conditioned RL policy on diverse primitive objects and random target poses, then finetunes it in sparse-reward assembly environments.The pretraining induces a task-agnostic manipulation prior that is adapted to assembly benchmarks and their final-part-configuration rewards.
  • Play2Perfect: The study evaluates how object diversity, trajectory diversity, training objectives, and goal precision affect transfer from play pretraining to precise assembly.The framework learns a play prior on general objects and goals before perfecting it on new CAD-defined assembly tasks.

2 Related Work

Prior work studies dexterous manipulation through imitation learning, reinforcement learning, and task-agnostic play, while precise assembly often depends on task-specific hardware or environmental structure. Broad dexterous pretraining is emerging, but existing datasets are largely centered on parallel-jaw grippers.

  • Manipulation with Multi-Fingered Robots: Imitation learning depends on high-quality demonstrations, yet contact-rich tasks make demonstrations difficult to collect because of the human–robot embodiment gap.Demonstrations have been obtained through motion-capture gloves, VR devices, cameras, and exoskeleton systems.
  • Manipulation with Multi-Fingered Robots: Dexterous reinforcement learning commonly targets free-space grasping and in-hand reorientation, while contact-rich extensions rely on dense rewards, human motion references, or teleoperation warm starts.Task-agnostic play controllers trained across diverse objects are the closest related approach identified.
  • Precise and Contact-Rich Assembly: Precise assembly has mainly advanced through specialized gripper attachments, tools, and fixtures that simplify manipulation or reduce grasping, alignment, and insertion uncertainty.These approaches are effective but require task-specific setup for each assembly problem.
  • Pretraining for Dexterous Manipulation: Broad dexterous priors are increasingly pretrained from diverse-task datasets and human videos, but existing large datasets are largely concentrated on parallel-jaw gripper robots.Vision Language Action models are an example of pretraining on large datasets containing diverse tasks.

3 Play2Perfect

Play2Perfect first trains a task-agnostic, goal-conditioned dexterous manipulation policy on diverse primitive objects, then finetunes it with sparse rewards derived from assembly CAD designs. The framework targets reusable skills such as grasping, lifting, in-hand reorientation, and precise contact-rich assembly.

  • Play pretraining: Play pretraining learns a goal-conditioned policy that manipulates procedurally generated primitive objects to random 6D poses in free space.The policy receives robot proprioception, current and target object poses, and object geometry, while controlling both the arm and hand.
  • Assembly finetuning: CAD-derived sparse contact goals include final assembled poses, aligned pre-insertion poses for insertion, and fixed 90° rotational offsets along threads for screwing.These goals are generated by reversing the assembly motion.
  • Play pretraining: The play objective sequences grasping and lifting with subsequent in-hand translation and rotation control without dropping objects.Translation teaches workspace motion, while rotation encourages in-hand reorientation.
  • Play pretraining: Play episodes randomize goal sequences, sampling broad initial goals and nearby later goals with significant rotations to promote in-hand manipulation.This avoids learning simple arm movements with fixed grasps.
  • Assembly finetuning: Assembly finetuning adapts the pretrained prior to contact-rich, high-precision interactions using sparse success rewards and environments constructed from assembly CAD designs.Assembly steps are generated by reversing feasible part removals, with each step instantiated as an RL environment containing randomized part and fixture poses.

4 Experiments

Experiments show that Play2Perfect substantially accelerates assembly learning, while ablations identify diverse-object, orientation-aware play as important for transfer. Assembly-specific RL finetuning is necessary for tight insertion, and the resulting policies transfer zero-shot to real-world tasks.

  • Learning Efficiency: Play2Perfect solves all tasks within roughly 2–5 hours, while both scratch baselines produce no successful rollouts after 24 hours.On simplified Tight-Insertion (Fixtured), Scratch (dense reward) requires over 100 hours to reach near-perfect success, whereas Play2Perfect reaches the same success rate in 4 hours, a 33× speed-up.
  • Pretraining Ablations: Final success rates consistently increase from 10 to 1000 pretrained objects, while orientation-aware 6D control provides the in-hand reorientation prior needed for assembly.The ablation varies object diversity, training objective, trajectory diversity, and goal precision; all four choices affect downstream finetuning.
  • Assembly Finetuning: Play2Perfect reaches 95% success at 4 mm, 92% at 1 mm, and 80% at 0.2 mm clearance, whereas Play-only drops from 75% at 40 mm to nearly 0% by 4 mm.Play-only supplies grasping and reorientation behaviors but does not achieve precise assembly without task-specific RL finetuning.
  • Assembly Finetuning: Play2Perfect learns local search, contact-corrective motions, and insertion commitment after alignment, unlike Play-only’s direct goal-directed motion that treats contact as a disturbance.These behaviors explain why assembly finetuning converts general play priors into precise contact-rich insertion strategies.
  • Sim-to-Real Transfer: Real-world Tight-Insertion success is 10/10 at 10 mm, 9/10 at 2 mm, and 6/10 at 0.5 mm clearance without real-world finetuning.The policy also achieves 8/10 on Assemble-Beam Step 1 and 7/10 on Step 2, with each step completed in under 7 s on average.

5 Discussion and Limitations · Appendix

Play2Perfect finetunes a dexterous manipulation prior learned through play for precise, contact-rich assembly and transfers zero-shot to challenging real-world tasks. Its limitations include externally specified sequencing and goals, per-task finetuning, short-horizon skills, and dependence on object-pose estimates.

  • 5 Discussion and Limitations: Play2Perfect finetunes a dexterous prior learned through play to adapt quickly to precise contact interactions in assembly tasks.The prior is learned through play across diverse objects.
  • 5 Discussion and Limitations: Object diversity, training objectives, trajectory diversity, and goal precision all affect downstream assembly performance.These factors were identified through ablation studies.
  • 5 Discussion and Limitations: Play2Perfect transfers zero-shot to challenging real-world dexterous assembly tasks.The passage presents this as a final result of the framework.
  • Appendix: The appendix reiterates Play2Perfect’s framework, ablation findings, zero-shot transfer, and deployment limitations described in the discussion.No separate appendix passage was supplied beyond the provided discussion material.
  • 5 Discussion and Limitations: The system learns short-horizon assembly skills rather than a complete autonomous assembly pipeline.The limitation concerns the scope of learned capabilities.
  • 5 Discussion and Limitations: Task sequencing, active-part selection, and goal poses are specified externally, while policies are finetuned per task or benchmark family.These requirements limit autonomous, broadly shared deployment.
  • 5 Discussion and Limitations: Future work could combine the learned skills with sequencing, scene memory, recovery, and broader multi-task finetuning.These directions target a more complete assembly system.
  • 5 Discussion and Limitations: Real-world deployment depends on object-pose estimates, which can fail under occlusion or fast motion.Pose-estimation failures create a practical deployment limitation.

A Additional Ablation Results · B Simulation and Computational Resources

Per-task ablations broadly reproduce the averaged trends: diverse objects and precise 6D in-hand control produce stronger assembly-transfer priors. Training uses large-scale Isaac Sim parallelization, with fewer environments during finetuning because contact-rich assembly requires more GPU memory.

  • A Additional Ablation Results: Across four assembly tasks, the same four play-pretraining ablations are repeated and each policy is finetuned with Play2Perfect’s sparse assembly reward.The tasks are Tight-Insertion, Assemble-Beam Step 1, Assemble-Beam Step 2, and Screw-Leg.
  • A Additional Ablation Results: Across tasks, pretraining transfers best when it teaches precise 6D in-hand object control across diverse objects and goal trajectories.The ablation comparisons include 6D Pose, Rot-Only, Trans-Only, and goal precisions of 1 cm, 5 cm, and 10 cm.
  • A Additional Ablation Results: Increasing object diversity generally improves finetuning stability and final performance across individual downstream tasks.This indicates that exposure to diverse object geometries and inertial properties benefits the play prior before assembly adaptation.
  • A Additional Ablation Results: Orientation control is essential: translation-only pretraining performs poorly because grasping and transport do not require in-hand reorientation.Rotation-only pretraining is described as substantially different, but the supplied passage truncates before stating the complete result.
  • B Simulation and Computational Resources: 120 Hz physics simulation and 60 Hz policy actions run on a single NVIDIA RTX A6000 GPU.All policies are trained in Isaac Sim.
  • B Simulation and Computational Resources: 24,576 parallel environments support 7 days of play pretraining, while 12,228 environments support 1 day of downstream assembly finetuning.Fewer environments are used during finetuning because contact-rich assembly interactions require more GPU memory than free-space play.

C Policy Architecture and RL Algorithm

The play policy uses SAPG, a population-based PPO variant designed to improve exploration in massively parallel environments. Its architecture combines an LSTM-based actor, an MLP action head, and an asymmetric actor–critic design.

  • RL algorithm: SAPG trains the play policy as a population-based variant of PPO for improved exploration in massively parallel environments.SAPG stands for Split and Aggregate Policy Gradients.
  • Policy architecture: The actor uses an LSTM to integrate interaction history and infer unobserved object properties.
  • Policy architecture: A multilayer perceptron converts the actor’s integrated representation into arm and hand actions.

D Play Pretraining Details

Play pretraining uses procedurally generated two-part objects, randomized initialization, and sequential 6D goals to train reusable manipulation behavior. A keypoint-based pose representation, structured observations, shaped rewards, and domain randomization define the training setup.

  • Pretraining Environment and Procedural Objects: Each episode places a procedurally generated two-primitive object at a random table pose, requiring grasping, lifting, and manipulation through randomly sampled 6D goal poses.Objects combine two cuboid or capsule primitives, with the primary component defining the graspable region and dimensions sampled from [5] [30] cm.
  • Episode Initialization and Goal Sampling: Initialization randomizes robot joints by ±0.1 rad, object position within ±10 cm, object orientation, and reachable first goals spanning x ∈[−0.35, 0.35] m, y ∈[−0.1, 0.2] m, and z ∈[0.15, 0.52] m.Subsequent goals are sampled after the robot reaches a goal.
  • Keypoint-Based Pose Representation: Four local-frame keypoints represent each 6D object pose, while reward computation uses fixed dimensions srew = [0.14, 0.03, 0.03] m to balance translation and rotation consistently across objects.Policy observations instead define keypoints from the object’s primary-component dimensions.
  • Policy Observations and Action Space: The policy receives a 140-dimensional observation containing proprioception, current and goal object poses, and an object geometry descriptor.Proprioception includes 29 joint positions and velocities, previous joint-position targets, palm pose, and five fingertip positions relative to the palm.
  • Play Reward and Success Criterion: The play reward combines smoothness, grasping, lifting, and progress toward sequential 6D goals, with success defined by reaching a goal within ϵ = 1 cm.The object is considered grasped after being lifted by 10 cm, then the policy receives goal-progress rewards and a sparse success bonus.
  • Domain Randomization: Domain randomization perturbs observations, action execution, object geometry, and environment dynamics to improve sim-to-real transfer.Randomization includes pose-estimation latency, joint-velocity noise, geometry-descriptor noise, table-height variation, and random object forces and torques.

E Pretraining Ablation Implementations

The ablation isolates four play-pretraining factors—object diversity, training objective, trajectory diversity, and goal precision—while holding the environment, architecture, rewards, randomization, budget, and finetuning procedure fixed. Variants change only the factor under study relative to default Play2Perfect.

  • Ablation design: Four components are ablated: object diversity, training objective, trajectory diversity, and goal precision.All variants retain the same environment, policy architecture, reward coefficients, domain randomization, training budget, and downstream assembly-finetuning procedure.
  • Object Diversity: Object diversity varies procedurally generated pretraining objects among 10, 100, and 1000, with 1000 as default.All sets use the same procedural distribution, while goal sampling and optimization remain unchanged.
  • Training Objective: The training objective compares default joint 6D pose reaching with Translation-only and Rotation-only variants.Translation-only evaluates position distance, whereas Rotation-only requires grasping, lifting, moving to a pose, and repeated in-hand reorientation.
  • Trajectory Diversity: Trajectory diversity compares online goal-trajectory sampling with fixed banks of 10 or 100 trajectories.The fixed trajectories preserve the default sampling distribution, workspace bounds, and maximum consecutive-goal translation and rotation.
  • Goal Precision: Goal precision varies play-goal success tolerance from the default ϵ = 1 cm to ϵ = 5 cm or ϵ = 10 cm.Sampled goals, pose representation, reward coefficients, and all other training settings remain identical.

F Assembly Finetuning Details

Assembly finetuning uses CAD-derived geometry and goal sequences, sparse rewards tied only to reaching those goals, and domain randomization to improve robustness. Contact-critical interfaces receive higher-fidelity SDF representations, while intermediate goals structure insertion and screwing behaviors.

  • Environment Construction from CAD: Contact-critical holes and insertion components use 256-resolution signed distance fields, while most other geometry uses convex decomposition for efficient simulation.The SDF treatment avoids distortions to narrow holes and mating interfaces that could alter clearance and contact dynamics.
  • Episode Initialization and Reset Distribution: Initial part positions are sampled independently within x, y ∈[−0.1, 0.1] m, with uniformly random orientation and randomized stable poses produced by dropping.The fixture is also placed flat at a random table location.
  • CAD-Derived Assembly Goal Sequences: CAD-derived goal sequences are invariant to randomized fixture placement and include reversed-disassembly intermediate goals for contact-rich assembly.Insertion uses an aligned pre-insertion pose followed by the inserted pose, while screwing uses successive thread goals with 90◦ rotational offsets.
  • Assembly Reward, Goal Progression, and Success: Finetuning removes play’s grasping, lifting, and dense pose-progress rewards, retaining action smoothness and sparse bonuses only for CAD-goal achievement and final retraction.Intermediate goals advance within ϵ = 1 cm; the final retraction requires the palm to move more than 0.2 m away, with Bsucc = Bretract = 1000.
  • Domain Randomization During Finetuning: Domain randomization perturbs observed part and goal poses and randomizes fixture yaw, object-pose latency, action, and proprioception latency to improve finetuning robustness.These perturbations target perception errors, control latency, and contact-dynamics mismatch.

G Inference Time Pipeline

At deployment, Play2Perfect reuses CAD models and online pose estimation to drive a 60 Hz closed-loop policy through CAD-derived sparse assembly goals. The learned controller alone produces contact-rich behaviors, advancing goals when pose error falls below a threshold without scripted insertion, screwing, or recovery controllers.

  • Deployment inputs: CAD models support online part-pose estimation, fixture-pose estimation, CAD-derived goal computation, and grasp-bounding-box definition.FoundationPose tracks the manipulated part, while the fixed fixture is estimated once per rollout to compute desired part poses from assembly transforms.
  • Closed-loop control: At 60 Hz, the finetuned policy receives proprioception, part pose, goal pose, and grasp bounding box, then outputs joint-position targets for the 7-DoF arm and 22-DoF hand.The part-pose tracker runs at 30 Hz, and the controller reuses the latest estimate between tracking updates.
  • Learned behaviors: The learned policy produces local search, corrective motions, regrasping, and in-hand spinning without additional scripted insertion, screwing, or recovery controllers.These contact-rich behaviors are generated by the policy and executed through low-level joint-position controllers.
  • Goal sequencing: The controller advances through sparse CAD-derived goals when dpose(ot, gm) < ϵgoal, considering assembly complete after reaching the final goal gM.The active goal starts at g1 and progresses sequentially as pose distance falls below the goal threshold.

H Real-World Experiment Additional Analysis

Real-world analysis shows that Play2Perfect uses closed-loop local search, corrective manipulation, and recovery after failures rather than relying on a single open-loop motion. Remaining failures mainly stem from perception errors and contact dynamics that differ between simulation and deployment.

  • Qualitative behavior: Across real-world tasks, the policy recovers from misalignment through additional in-hand reorientations and immediately regrasps dropped parts before continuing.These behaviors are difficult to obtain from a single open-loop assembly motion.
  • Qualitative behavior: During tight insertion, the policy approaches with a tilted strategy, contacts near the fixture, searches locally with small corrective motions, and commits after alignment.This closed-loop behavior becomes especially important at tighter clearances, where small pose errors can block direct insertion.
  • Qualitative behavior: After failed grasps or drops, the policy continues from the new state, regrasps the part, retries the assembly motion, and completes tasks without scripted recovery control.This recovery behavior is shown in representative real-world Assemble-Beam rollouts.
  • Failure modes: Perception failures arise from fast motion, hand-object occlusion, visually similar objects, and approximate 90◦ rotational symmetries that confuse pose estimation during screwing.Screwing is especially challenging because the policy must track object rotation while manipulating a rectangular leg.
  • Failure modes: Control failures typically occur during final contact-rich insertion because real fixtures can move or comply, unlike rigid immovable simulation fixtures.These unmodeled fixture behaviors can cause corrective motions learned in simulation to struggle in deployment.
Loading 2606.26428v2…