PaperlayerView exact revision on arXiv

Source-linked AI summary

IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality

Bingjie Tang, Michael A. Lin, Iretiayo Akinola, Ankur Handa, Gaurav S. Sukhatme, Fabio Ramos, Dieter Fox, Yashraj Narang

arXiv:2305.17110v1cs.RO

TL;DR

Robotic assembly remains difficult to address through simulation because contact modeling is costly and RL tasks are long-horizon. IndustReal combines simulation-oriented RL methods with a deployment-time action integrator, and reports successful simulated tasks and real-world transfer.

  • Problem

    Simulation has had limited impact on assembly because accurate contact simulation is costly and long-horizon RL has high sample complexity; real-world-only learning can also require prolonged training and manual resets.

    Source: §I · p. 1, 2) Learning-Based Approaches · p. 14

  • Method

    IndustReal combines SAPU, SDF-based dense rewards, and sampling-based curricula for simulation learning with a policy-level action integrator for deployment and a real-world assembly system.

    Source: §I · pp. 1–2

  • Results

    Real-world system achieved success rates of 83-99% over 600 trials, and simulated benchmark tasks achieved success rates of 82-99%.

    Source: §I · p. 2

  • Takeaways & Limitations

    The work demonstrates, to the authors’ knowledge, a first RL-based sim-to-real system for end-to-end assembly without policy adaptation and provides tools for replication.

    Source: §IX · p. 10

    The work primarily investigates tasks inspired by the NIST benchmark.

    Source: §VIII · p. 9

Abstract

from arXiv · show

Robotic assembly is a longstanding challenge, requiring contact-rich interaction and high precision and accuracy. Many applications also require adaptivity to diverse parts, poses, and environments, as well as low cycle times. In other areas of robotics, simulation is a powerful tool to develop algorithms, generate datasets, and train agents. However, simulation has had a more limited impact on assembly. We present IndustReal, a set of algorithms, systems, and tools that solve assembly tasks in simulation with reinforcement learning (RL) and successfully achieve policy transfer to the real world. Specifically, we propose 1) simulation-aware policy updates, 2) signed-distance-field rewards, and 3) sampling-based curricula for robotic RL agents. We use these algorithms to enable robots to solve contact-rich pick, place, and insertion tasks in simulation. We then propose 4) a policy-level action integrator to minimize error at policy deployment time. We build and demonstrate a real-world robotic assembly system that uses the trained policies and action integrator to achieve repeatable performance in the real world. Finally, we present hardware and software tools that allow other researchers to reproduce our system and results. For videos and additional details, please see http://sites.google.com/nvidia.com/industreal .

I. INTRODUCTION

IndustReal addresses contact-rich assembly with simulation-based RL, a deployment-time action integrator, and a reproducible real-world system. It reports successful simulated tasks and transfer to real-world assembly.

I. INTRODUCTION

  • Accurate simulation of complex geometry requires 1k-10k contacts per rigid-body pair, while long-horizon RL requires faster-than-realtime simulation.The paper notes that meeting these accuracy and performance requirements has only recently become possible.Source: §I · p. 1
  • SAPU rewards reliable simulation predictions, SDF-based dense rewards measure geometric alignment, SBC prevents curriculum-stage overfitting, and PLAI reduces steady-state error under unmodeled dynamics.The first three methods support simulation learning; PLAI is proposed for sim-to-real deployment.Source: §I · pp. 1–2
  • 83-99% success rates over 600 trials were achieved by the real-world system, while simulated assembly tasks achieved success rates of 82-99%.The real-world system transfers simulation-trained policies; the simulated tasks cover pick, place, and insertion for pegs-and-holes and gears.Source: §I · p. 2
  • IndustRealKit provides 3D-printable CAD models and a purchasing list, while IndustRealLib supports Isaac Gym policy deployment on a real Franka Panda and includes training assistance.The tools are intended to help researchers reproduce the hardware and robot behaviors.Source: §I · p. 2

A. Sim-to-Real Transfer for Assembly

  • Prior sim-to-real assembly studies often focused on insertion with pre-grasped objects, whereas IndustReal addresses detection, grasping, alignment, and insertion without real-world policy adaptation.The paper also avoids task-specific grippers and force/torque sensors and uses part clearances of ≤0.5-0.6 mm.Source: A. Sim-to-Real Transfer for Assembly · p. 2

A. Problem Setup

  • The setup uses movable plugs with randomized 3-DOF poses and bolted sockets, and targets peg-hole, gear-gearshaft, and connector-receptacle assemblies.The parts are inspired by NIST Task Board 1.Source: A. Problem Setup · p. 2

B. Problem Decomposition

  • Each assembly is decomposed into Pick, Place, and Insert phases: grasp a plug, move it near its socket, then contact and insert it with alignment where needed.The plug/socket terms cover pegs, gears, and connectors and their corresponding holes, gearshafts, or receptacles.Source: B. Problem Decomposition · pp. 2–3

IV. POLICY LEARNING IN SIMULATION

The paper develops simulation-trained policies for contact-rich assembly, using staged task training and a PPO formulation intended for transfer to the real world. It specifies task-independent incremental pose actions and task-dependent rewards.

A. Training Environments

  • Peg-and-hole Pick and Place policies are fine-tuned from Reach, while Insert policies are trained from scratch; gears use a separately trained Insert policy.For gears, the corresponding Peg and Hole policies are later used for Pick and Place; connector tasks receive no phase-specific training in simulation.Source: A. Training Environments · p. 3

B. Formulation

  • The policies are learned with PPO in an MDP, with an actor mapping observations to actions and a critic estimating the on-policy value function.The stated deployment aim is to transfer simulation-trained policies without policy adaptation in the specific real environment.Source: B. Formulation · p. 3

C. Observations, Actions, and Rewards

  • Actors use task-dependent subsets of joint angles and object, gripper, or target poses, while noisy joint velocities and torques are reserved for critic training.The formulation uses asymmetric actor-critic training.Source: C. Observations, Actions, and Rewards · p. 3
  • Task-independent actions are incremental position and quaternion targets for a task-space impedance controller; rewards combine weighted dense terms and terminal success bonuses.Incremental rather than absolute targets avoid encoding task-specific biases and needing selection from a large spatial range; the authors choose TSI because Franka provides a high-performance implementation, while OSC relies on an accurate dynamics model.Source: C. Observations, Actions, and Rewards · pp. 3–4

D. Randomization and Noise

The section describes randomized training conditions and SAPU, which uses interpenetration information to discourage policies that exploit inaccurate collision dynamics. Filter and Weight performs best among the tested SAPU strategies.

D. Randomization and Noise

  • Training randomizes initial end-effector and object poses and adds observation noise for Insert policies, but does not randomize dynamics because the authors have strong priors on them.The passage states that pose and observation perturbations are critical for robustness to initial conditions and real-world sensor noise.Source: D. Randomization and Noise · p. 4

E. Simulation-Aware Policy Update (SAPU)

  • SAPU encourages policies to avoid spurious interpenetrations, which can let agents exploit inaccurate collision dynamics and learn policies unlikely to transfer.The interpenetration module samples 1,000 points on or inside the plug mesh and uses maximum depth to weight cumulative reward during policy updates.Source: E. Simulation-Aware Policy Update (SAPU) · p. 4
  • 84.9–87.6% success rates were achieved by Filter and Weight across interpenetration thresholds, while the Baseline, Filter Only, and Weight Only strategies stayed below 40%.Success is also evaluated only for episodes below specified maximum interpenetration distances; the figure reports medians and IQRs.Source: E. Simulation-Aware Policy Update (SAPU) · p. 4, Figure 3 · p. 4

F. SDF-Based Dense Reward

The section introduces an SDF-based dense reward for geometric alignment and a sampling-based curriculum for insertion. Their evaluations report improved insertion outcomes over the compared reward formulations and curriculum strategies.

F. SDF-Based Dense Reward

  • The SDF reward maps a 3D point to its signed distance from a surface, providing a dense alignment metric for geometrically complex objects.The paper motivates it against keypoint rewards, which can underspecify non-axisymmetric parts or overspecify symmetric parts.Source: F. SDF-Based Dense Reward · p. 4
  • SDF Query Distance achieved 88.6% success and 96.6% engagement, with 3.80 mm position error and 0.086 rad rotation error.The other reward formulations had 1.8–54.2% success, with position errors of 11.7–31.0 mm and rotation errors of 0.13–0.99 rad.Source: F. SDF-Based Dense Reward · p. 5

G. Sampling-Based Curriculum (SBC)

  • SBC exposes the agent to the full range of initial-state distributions from the start, then raises the lower bound as curriculum stages advance.The authors developed it after naive curricula that increased initial distance from the goal were ineffective for insertion.Source: G. Sampling-Based Curriculum (SBC) · p. 5
  • Sampling-Based curriculum achieved 88.6% success and 3.80 mm position error, while the other strategies reached at most 66.8% success and 10.7 mm position error.All strategies had 0.086–0.096 rad rotation errors; Baseline and Sampling-Based had 89.2–96.6% engagement rates.Source: G. Sampling-Based Curriculum (SBC) · p. 5

H. Joint Evaluation

The joint insertion evaluation reports high success under moderate state randomization and observation noise. The deployment section describes the robot communication and perception pipelines, dynamics adjustments, and PLAI’s error-reduction results.

H. Joint Evaluation

  • Pegs-and-Holes insertion achieved 88.6% success and 96.6% engagement, while Gears-and-Gearshafts achieved 82.0% and 85.2% under moderate randomization and observation noise.Across evaluations, performance ranged from 67.88% when gearshaft position randomization was doubled to 92.4% with no randomization or noise.Source: H. Joint Evaluation · p. 5, Figure 4 · p. 6

A. Communications Framework

  • IndustRealLib converts Isaac Gym policy checkpoints into targets for a Franka robot, with commands passed through frankapy, ROS, franka-interface, and libfranka.The real robot uses 1 kHz low-level control and 60–100 Hz policy control; training used 120 Hz physics and a 60 Hz control rate.Source: A. Communications Framework · p. 6

B. Perception Pipeline

  • The perception pipeline estimates part poses in the robot frame through camera calibration, workspace mapping, and object detection from a single RGB image.Bounding-box centroids and a specified height are used to construct targets.Source: B. Perception Pipeline · p. 6

C. Dynamics Strategy

  • Removing arbitrary energy dissipation from the simulator and asset descriptions partially resolved deployment steady-state error.Heuristic friction and damping values were often chosen without physical consideration.Source: C. Dynamics Strategy · pp. 6–7

D. Policy-Level Action Integrator

  • PLAI integrates policy actions by applying each action to the last desired state rather than the current state, while continuing to evaluate the policy on current observations.The method is related to a standard policy with PID control and can experience windup in contact; Leaky PLAI clamps accumulated control effort.Source: D. Policy-Level Action Integrator · p. 7
  • In simulation, PLAI maintained approximately 2 mm error across ideal, friction, and gravity conditions, outperforming Nominal and consistently lowering error versus PID.In the real world, PLAI also achieved approximately 2 mm error and lower error and variance than Nominal; Franka’s RL-free TSI implementation had 4.45 mm error.Source: D. Policy-Level Action Integrator · p. 7

VI. REAL-WORLD EXPERIMENTS

The real-world system was exercised on Pick, Place, Sort, Insert, and Pick-Place-Insert tasks; the reported results here show high pick success, measured placement accuracy, and repeatable sorting.

A. Pick Experiment

  • The pick policy achieved 98.8% success across six randomly placed pegs.The two failures were one missed peg detection and one grasp that included both a peg and its tray.Source: A. Pick Experiment · p. 8

B. Place Experiment

  • The place policy reached a mean distance-to-goal of 4.23 ± 1.96 mm.The robot guided a laser mounted to its gripper toward targets centered on randomly placed trays.Source: Experimental Setup · p. 8

C. Sort Demonstration

  • The sorting demonstration showed highly repeatable performance while sorting pegs and gears into separate bins.The setup used six pegs and three gears, with ±45° angular perturbations applied to the trays.Source: C. Sort Demonstration · p. 8

D. Insert Experiment

IndustReal’s real-world insertion tests demonstrated transfer across pegs, gears, and connectors, including an end-to-end pick-place-insert demonstration. The paper also provides reusable hardware and software while identifying limits of its current sim-to-real scope.

D. Insert Experiment

  • Insert policies achieved engagement rates of 86.7% for pegs, 95.0% for gears, and 100% for connectors, with full-insertion success rates of 76.7%, 92.5%, and 85%.The Pegs and Holes Insert policy also generalized to NEMA connectors.Source: D. Insert Experiment · p. 8
  • Engagement failures were almost exclusively due to gripper-object slip, while full-insertion failures were almost exclusively due to wedging; gear insertion also showed recovery after perturbation.The paper reports connector-insertion search behavior informally.Source: D. Insert Experiment · p. 8, Figure 7 · p. 9

E. Pick, Place, and Insert (PPI) Demonstration

  • 80%/88.3% peg, 97.5%/100% gear, and 100%/100% connector success/engagement rates accompanied an end-to-end RL assembly demonstration without real-world policy adaptation.The authors describe this as the first such system to their knowledge.Source: E. Pick, Place, and Insert (PPI)… · p. 8

VII. INDUSTREALKIT AND INDUSTREALLIB

  • IndustRealKit provides printable part models and a purchasing list, while IndustRealLib supplies policy deployment and training code, including trained Peg and Hole policies.The library supports deployment from Isaac Gym to Franka robots and includes implementations of SAPU, SDF-based reward, and SBC.Source: §VII · p. 9

VIII. LIMITATIONS & FUTURE WORK

  • The work primarily studies NIST-inspired tasks; its current framework may not train and deploy smoothly when a task’s simulator is fundamentally inaccurate or inefficient.The authors cite elastic-cable assembly onto pulleys as an example and propose a real-to-sim feedback loop as a possible strategy.Source: §VIII · p. 9

IX. CONCLUSIONS

  • The authors report careful simulation and real-world evaluations of their algorithms and provide hardware and software for reproducing the results.They characterize the end-to-end RL-based sim-to-real assembly system without policy adaptation as the first of its kind to their knowledge.Source: §IX · p. 10

APPENDIX

The appendix situates IndustReal among classical and learning-based assembly methods and describes its simulation platform and real-robot communications and calibration procedures.

1) Classical Approaches:

  • Classical assembly methods use geometry, dynamics, mechanical design, and sensing, but can be sensitive to modeling, sensing, calibration, and unseen-asset errors.The cited approaches address issues including compliance, jamming, wedging, and initial misalignment.Source: 1) Classical Approaches · p. 14

2) Learning-Based Approaches:

  • Learning-based assembly work spans model-based, off-policy, on-policy, demonstration-based, and non-RL approaches, each with reported trade-offs or engineering demands.The appendix notes contact dynamics challenge model-based methods, while on-policy methods can be sample-inefficient and demonstrations can require substantial infrastructure and collection time.Source: 2) Learning-Based Approaches · p. 14
  • Prior learning-based assembly methods have shown success, repeatability, perturbation robustness, or part generalization, but most rely on demonstrations, human initialization, or on-policy corrections.Purely real-world methods can be difficult to parallelize and may require long training and manual resets.Source: 2) Learning-Based Approaches · p. 14

3) Simulation:

  • IndustReal uses Factory as its simulation platform, citing its integration of accurate contact-dynamics capabilities within a widely used robotics simulator.The cited passage identifies Factory among the state-of-the-art contact-rich simulators considered by the authors.Source: 3) Simulation · p. 14

1) Communications:

  • The communications pipeline takes an Isaac Gym policy checkpoint through IndustRealLib and outputs torque commands for the robot.Figure S8 depicts the communications framework; Figure S9 evaluates the Place policy in simulation and reality, with and without training-time damping.Source: 1) Communications · p. 14, Figure S8 · p. 14, Figure S9 · p. 15

2) Camera Calibration:

  • Camera calibration uses manufacturer-provided RGB intrinsics and estimates the camera’s robot-frame pose from AprilTag observations and robot poses.The extrinsic estimate is validated against a corresponding transformation in a CAD assembly.Source: 2) Camera Calibration · p. 15

3) Object Detection:

The perception pipeline detects parts, maps their image locations into the robot frame, and converts them into end-effector targets for the real-world system.

3) Object Detection:

  • Mask R-CNN models were fine-tuned on augmented asset images combining random translations, rotations, scaling, and color-jittered composites; after 50 epochs with 4,000 training images, precision and recall were typically above 85%.Source: 3) Object Detection · p. 15

4) Workspace Mapping:

  • Workspace mapping converts detected part centroids and yaw from image space through the camera frame into the robot frame, with a one-time offset calibration to correct systematic bias.The mapping’s distance relation is most accurate in the plane containing the AprilTag and loses accuracy farther from that plane.Source: 4) Workspace Mapping · pp. 15–16

5) Constructing Targets:

  • The pipeline sets end-effector height from a human-specified nominal value and uses detected position and yaw as the remaining target coordinates.For industrial-style parts, the authors report that specifying height is trivial and requires little to no iteration.Source: 5) Constructing Targets · p. 16

6) Experimental Setup:

7) Summary:

The appendix summarizes the perception pipeline’s calibration and per-trial steps, then explains how PLAI integrates policy actions into a setpoint tracked by a low-level controller.

7) Summary:

  • Camera calibration is performed once, while detection and workspace mapping are run at the start of each trial requiring detections.These per-trial steps take approximately 10 seconds on the authors’ non-realtime system.Source: 7) Summary · p. 16

1) Standard PLAI:

  • PLAI accumulates policy actions into a control setpoint tracked by a P controller, rather than converting the accumulated term directly into control effort as PI does.The authors compare the policies in the context of mismatches between training and deployment dynamics.Source: 1) Standard PLAI · pp. 16–17
  • For PLAI, policy actions are applied to the previous desired state and interpreted as the change to the current desired state.The cited prose defines the P-controller error as desired state minus current state.Source: 1) Standard PLAI · p. 17
  • After a disturbance, PLAI’s setpoint does not change instantaneously, so the impedance controller continues pulling toward that setpoint; PI’s accumulated error may instead act on an unintended state.The passage describes this difference for a robot pushed away from its current state.Source: 1) Standard PLAI · p. 17

2) PLAI for 6-DOF Pose:

PLAI extends from Cartesian position control to orientation control using rotation-matrix operations. The leaky variant adds a threshold transformation to the desired-state update, and simulation evaluation reports lower steady-state error for PLAI than PID under randomized gravity.

2) PLAI for 6-DOF Pose:

  • For 6-DOF pose control, PLAI replaces addition and subtraction with rotation-matrix operations for orientation updates.The text defines B as a frame’s rotation relative to O and R as the end-effector’s incremental rotation between time steps.Source: 2) PLAI for 6-DOF Pose · p. 17

3) Leaky PLAI:

  • Leaky PLAI applies a threshold transformation ϵ when updating the desired state after the PLAI update.Source: 3) Leaky PLAI · p. 17
  • The simulation evaluation compares PLAI with Nominal and PID across three environmental conditions, including an ideal condition without perturbations.Source: Figure S13 · p. 17
  • Under randomized gravity, PLAI’s steady-state error was 1.6 mm, versus 3.5 mm for PID, while Nominal frequently failed to converge.The comparison covered 20 trials.Source: Figure S14 · p. 18

D. Additional Simulation Parameters & Results

This section specifies the simulation RL formulation and supplemental evaluation and implementation parameters. It describes partial observability, mesh-interpenetration visualizations, policy evaluation conditions, and tables documenting training and real-world settings.

1) MDP formulation & Parameters in Simulation:

  • The simulation RL formulation specifies an MDP objective and assumes observations do not fully reveal the state in simulation or reality.Real-world observations include noise in robot signals and object-pose estimates; object states inside the gripper are unobserved without tactile sensing or a 6-DOF tracker.Source: 1) MDP formulation & Parameters in… · p. 18

2) Simulation-Aware Policy Update:

4) Additional Real Parameters & Results:

3) Joint Evaluation in Simulation:

  • The joint simulation evaluation tests Insert policies for pegs/holes and gears/gearshafts across randomized conditions, using five seeds and 1,000 trials per test.The table describes in- and out-of-distribution testing, with moderate plug/socket randomization and observation noise.Source: 3) Joint Evaluation in Simulation · p. 18, Table X · p. 20

Related papers

Loading 2305.17110v1…