PaperlayerView exact revision on arXiv

Source-linked AI summary

Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching

Kejia Ren, Gaotian Wang, Andrew S. Morgan, Kaiyu Hang

arXiv:2605.09789v1cs.RO

TL;DR

The paper tackles the sim-to-real gap in dexterous manipulation, where uncertain contact dynamics and perception noise make transfer difficult. It proposes Domain-Randomized Instance Set (DRIS), which simultaneously represents and propagates multiple randomized instances, and demonstrates robust zero-shot transfer on reactive catching with a flat plate.

  • Problem

    Uncertain contact dynamics and perception noise make dexterous manipulation difficult to transfer from simulation to the real world.

  • Method

    DRIS simultaneously propagates multiple randomized manipulation instances and maps their variable-size state set into a fixed-dimensional latent representation for policy learning.

  • Results

    68% success rate was achieved by the DRIS-trained policy in zero-shot deployment, with all tested balls successfully caught.

  • Takeaways & Limitations

    DRIS-based propagation provides superior robustness with modest computational overhead compared with massive environment parallelization under 2σ observation noise.

  • Takeaways & Limitations

    The conclusion bounds the evidence to a DRIS-based strategy demonstrated for dexterous manipulation and reactive catching, with experiments used to show improved robustness and sim-to-real performance over conventional baselines.

Abstract

from arXiv · show

Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, making sim-to-real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we propose Domain-Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shot sim-to-real transfer.

I. INTRODUCTION

Dexterous manipulation is highly sensitive to uncertain contact dynamics, creating a substantial sim-to-real gap. The paper addresses this with DRIS, which propagates multiple randomized instances simultaneously and evaluates the approach on reactive catching with a flat plate.

  • Dexterous manipulation requires coordinated contact, timing, and motion, while small discrepancies in geometry, friction, inertia, or compliance can cause task failure.
  • Conventional domain randomization samples one randomized physical instance per rollout, limiting structured reasoning about how uncertainty affects manipulation outcomes.
  • DRIS propagates multiple instances with different physical properties under a shared action, allowing their state evolutions to jointly inform policy updates.
  • The reactive catching setup uses a flat, low-friction plate instead of mechanically stabilizing cups, nets, or articulated hands.
  • The paper proposes DRIS as an uncertainty representation and learning paradigm, with theoretical analysis and simulation and real-world validation for zero-shot transfer.

B. Domain-Randomized Instance Set (DRIS)

DRIS replaces single-instance domain randomization with simultaneous propagation of multiple randomized manipulation instances, allowing policy updates to reflect uncertainty across possible dynamics. A size-agnostic encoder supports training on these sets and deployment on a single real-world instance.

  • Conventional domain randomization samples one parameter instance per episode and predicts individual next states without explicitly representing their uncertainty.
  • DRIS constructs a discrete set of N manipulation instances and propagates their possible states simultaneously under a shared robot action.
  • A size-agnostic encoder maps the DRIS state to a fixed-dimensional latent vector, allowing the policy input dimension to remain independent of N.
  • Theorem III.1 characterizes DRIS as an exact particle approximation for belief propagation and states that it improves optimization stability, policy robustness, and sim-to-real generalization.
  • The instances evolve independently with different physical parameters, while their state transitions under the same action jointly inform policy updates.
  • During real-world inference, the trained policy encodes the observed single state as a DRIS of size N = 1 and generates actions without fine-tuning.

IV. REACTIVE CATCHING CASE STUDY

The case study evaluates DRIS on reactive ball catching with a flat plate, requiring the robot to manage contact, impact energy, and balance using a compact motion-frame representation. The state tracks ball displacement and velocity, while actions command plate translation and tilt.

  • The task requires an M-DoF robot with a rigidly attached flat plate to catch, decelerate, stabilize, and balance an incoming ball.
  • The motion frame is fixed relative to the ball at manipulation start, with origin at the plate center, vertical Z-axis, and X-Z plane through the ball.
  • The state is a six-dimensional pair of the ball’s displacement from the plate center and linear velocity, both expressed in the motion frame.
  • The five-dimensional action combines a three-dimensional plate-center displacement command with two target tilting parameters.
  • The reward combines a velocity term favoring lower ball speed with a constant penalty when the ball moves beneath or outside the plate.

B. Training with DRIS

Training represents randomized ball properties and states as a DRIS, compresses that set with a pretrained autoencoder, and conditions an RL policy on the resulting latent feature and plate orientation.

  • Each episode samples N balls with independently randomized radius, static friction, dynamic friction, and restitution coefficient to form the DRIS.
  • The DRIS state concatenates the relative displacements and velocities of its ball instances and is encoded into a fixed-dimensional feature independent of N.
  • A 6D point-cloud autoencoder is pretrained to reconstruct DRIS states, and its encoder is then used for downstream policy learning.
  • The policy applies FiLM modulation to the encoded feature using the observed plate orientation before an MLP generates the action.
  • The FiLM scaling and bias networks and the action-generation MLP are jointly optimized through gradient-based updates.
  • Plate orientation conditions the policy because it changes tangential gravity and contact direction, thereby affecting ball acceleration during catching.

D. Control Implementation on High-DoF Manipulator

The study trains DRIS policies in simulation and evaluates their robustness to observation noise, execution errors, and unseen physical parameters in a reactive catching task.

  • Simulation setup: 128 parallel simulation environments trained policies on multiple balls per environment for up to 20 steps, or one second of simulated time.The balls were randomly positioned 1.0–2.0 m from the plate and assigned velocities aimed at a predefined catching region.
  • Policy comparison: Policies were trained with DRIS sizes N = 1, 10, 50, and 200, while evaluation used a single ball to match the real task.An E2E policy trained directly from the observed single-ball state served as the baseline.
  • Observation noise: Under observation noise, E2E and N = 1 policies degraded significantly, whereas DRIS policies with even 10 balls showed substantially improved robustness.Observation noise scaled base standard deviations of 1 cm for position and 5 cm/s for velocity by factors from 1 to 4.
  • Execution error: Under execution errors, DRIS policies were more robust than E2E, and even 10 balls provided an obvious performance boost.Execution perturbations were uniformly sampled from [−0.05, 0.05] rad in desired joint positions.
  • Out-of-distribution physics: With unseen restitution coefficients in [0.7, 0.8], larger DRIS policies generalized better than both E2E and the single-ball DRIS policy.Training used restitution values only within [0.4, 0.7].

C. Scalability and Computational Efficiency

DRIS achieves higher robustness than massively parallelized single-instance training while using far less simulation memory. Real-world tests further show zero-shot transfer across diverse balls, ramps, and object shapes, with limitations in high-dimensional or strongly divergent settings.

  • C. Scalability and Computational Efficiency: 0.89 success rate: DRIS under 2σ observation noise outperformed the massively parallelized E2E baseline at 0.73.Both configurations matched total ball interactions, with 128 × 50 single-ball environments for E2E and 128 environments with DRIS size 50.
  • C. Scalability and Computational Efficiency: 0.71 GB VRAM: DRIS required only a modest increase from the 0.65 GB baseline, versus 2 GB for massively parallelized E2E.The comparison used equal total numbers of ball interactions.
  • C. Scalability and Computational Efficiency: The real-world evaluation varied ball properties and release speeds using four balls and three interchangeable ramp sections.The ramp radii were R = 0.13, 0.20, and 0.32 m.
  • C. Scalability and Computational Efficiency: 68% success rate: the DRIS-trained policy transferred zero-shot to the real robot and caught all four tested ball types.The policy used N = 200 and was evaluated across every combination of ball type and ramp.
  • C. Scalability and Computational Efficiency: Qualitative tests showed successful human-throw catching, direct balancing-task generalization, and occasional catching of irregular objects.The wooden cube remained difficult because of unpredictable impact dynamics.
  • C. Scalability and Computational Efficiency: DRIS can become costly for high-dimensional inputs or complex multi-body contacts, and excessive instance divergence can destabilize shared-action learning.These constraints limit scaling and are especially relevant when simultaneous collision manifolds are expensive to resolve.

APPENDIX A. IMPLEMENTATION DETAILS

The implementation samples physical instances from a prior, propagates them under a shared policy action, and optimizes average return across their trajectories. The appendix also relates DRIS to conventional domain randomization and summarizes the computational and real-time deployment setup.

  • APPENDIX A. IMPLEMENTATION DETAILS: The training pipeline ran on one NVIDIA GeForce RTX 3060 GPU with 12 GB memory, while real-time inference used a 3.4 GHz AMD Ryzen 9 5950X CPU.Actions were generated at 20 Hz, and real-world ball trajectories were estimated from two cameras operating at 80 FPS.
  • APPENDIX A. IMPLEMENTATION DETAILS: DRIS samples N independent physical instances from the prior and represents their states as an ensemble.Each instance has its own sampled physical parameters and state trajectory.
  • APPENDIX A. IMPLEMENTATION DETAILS: A single consensus action generated from the ensemble state is applied simultaneously to all instances.The shared action couples policy decisions across the parallel instances.
  • APPENDIX A. IMPLEMENTATION DETAILS: The policy objective maximizes expected average return across the trajectories of all instances.The trajectories are conditioned on their respective physical-parameter realizations.
  • APPENDIX A. IMPLEMENTATION DETAILS: N = 1 reduces DRIS to conventional domain randomization with one sampled physics parameter per episode.The ensemble becomes a singleton, recovering the standard Monte Carlo objective.

B. Interpretation: Exact Particle Propagation

DRIS is interpreted as exact propagation of a particle approximation to the system belief over states and physical parameters. This interpretation connects simultaneous instance simulation to belief dynamics governed by the Liouville equation.

  • B. Interpretation: Exact Particle Propagation: DRIS simulation steps exactly propagate an empirical particle approximation of the system belief.The particles represent joint uncertainty over state and physical parameters.
  • B. Interpretation: Exact Particle Propagation: The belief evolves under the Liouville equation for probability mass, with static physics parameters enforced during propagation.The Dirac-delta term δ(c′ − c) keeps each physical-parameter realization fixed.
  • B. Interpretation: Exact Particle Propagation: Updated particle states correspond precisely to the discrete set of updated simulation states.The result follows by substituting the empirical measure into the Liouville equation and applying the Dirac-delta sifting property.
  • B. Interpretation: Exact Particle Propagation: The resulting distribution is mathematically equivalent to propagating the belief state through the exact system dynamics.This provides the theoretical interpretation of DRIS beyond independent single-instance sampling.

C. Gradient Variance Reduction

Under a permutation-invariant policy, DRIS produces exchangeable instance gradients whose averaged estimator can have lower variance than conventional domain randomization. The reduction is strict when cross-instance covariance is sufficiently smaller than marginal covariance, and increasing instance count lowers variance toward a nonzero floor.

  • Permutation-invariant policies make DRIS instance gradients exchangeable, with common mean, covariance, and cross-covariance.
  • DRIS achieves strict gradient-variance reduction over conventional DR when ρ < 1, meaning aggregate cross-covariance is below marginal covariance.
  • Independent physics perturbations typically yield ρ < 1, so DRIS with N > 1 is expected to reduce variance relative to N = 1.
  • As N →∞, total variance decreases from the conventional DR baseline σ2 toward the nonzero floor ρσ2.Averaging suppresses instance-specific, uncorrelated gradient variability.

D. Robustness Analysis

DRIS changes the empirical optimization landscape by suppressing single-instance noise and exposing a robustness signal associated with physics-induced variance. As the instance set grows, optimization increasingly favors state-action regions invariant to physical uncertainty.

  • DRIS and conventional DR share the same expected objective but have fundamentally different empirical optimization landscapes.
  • As N increases, linear noise is suppressed while the robustness signal concentrates in the empirical local cost.
  • The limiting nonnegative variance penalty improves the global optimization Signal-to-Noise Ratio.
  • For N = 1, linear noise can dominate the objective and obscure the variance penalty.The optimizer may chase random first-order fluctuations.
  • DRIS updates favor state-action regions that are invariant, and therefore robust, to physical uncertainty.

E. Sim-to-Real Generalization

The sim-to-real analysis separates finite-sample estimation error from physics-distribution mismatch. Increasing DRIS size reduces the former, but mismatch between simulated and real physics remains irreducible when the simulator does not cover the target world.

  • The analysis explicitly models sim-to-real transfer as a shift between simulation and real-world physics distributions.
  • The DRIS objective averages trajectory costs over N physics instances sampled from the simulator distribution pS(c).
  • The transfer bound decomposes error into a reducible finite-sampling generalization gap and an irreducible physics mismatch term dLΘ(pS, pT).
  • The physics-mismatch discrepancy dLΘ(pS, pT) does not vanish as N →∞.
  • If simulated and real physics distributions are disjoint, the discrepancy can remain large, while larger DRIS still reduces the modeled-uncertainty generalization gap.

F. Discussion on Theory and Practical Implementation

The theory assumes access to the full ensemble state, whereas the implementation uses a learned compressed set representation. The encoder is optimized to preserve task-relevant information, but formal equivalence remains unproved.

  • The theoretical analysis assumes access to the full ensemble state, unlike the practical policy’s learned compressed representation.
  • The encoder is optimized to preserve task-relevant information from the underlying ball states.
  • Formal equivalence between the theoretical analysis and learned representations is not proved and is deferred to future work.
Loading 2605.09789v1…