X-Reset: Scaling Object-Centric Reinforcement Learning via Cross-Embodiment Resets

1 Introduction

• High-DoF manipulation makes exploration from scratch difficult, while prior approaches rely on high-quality robot demonstrations, per-task states, or restricted behaviors.Teleoperation remains especially expensive for dexterous multi-fingered hands.Source: §1 · p. 3
• Policies depend only on object state and goal; demonstrations enter training through resets rather than reference tracking.This differs from motion-tracking approaches and accommodates randomizations such as novel goals and external forces.Source: §1 · p. 3
• X-RESET trains generalist RL policies on 20 objects with task-agnostic rewards, framing human hand-object states as resets to address exploration.The approach kinematically retargets hand-object trajectories, filters states unstable in simulation, and samples retained states as resets.Source: §1 · p. 3
• 11.0% to 66.6% average reorientation success with object-only rewards, and 42.1% to 59.6% with hand-object rewards, across three embodiments.The three embodiments are a five-fingered hand on two arms and a parallel-jaw gripper.Source: §1 · p. 3
• The approach scales with training objects, generalizes zero-shot to unseen objects, tolerates hand-pose estimation errors, and transfers behaviors zero-shot to the real world.The stated sim-to-real result includes a 22-DoF hand on a 6-DoF arm and seen and unseen objects.Source: §1 · pp. 3–4

Dexterous Manipulation with RL

• X-RESET addresses exploration through resets from diverse human data while retaining generic object-centric rewards and learning behaviors across diverse objects.The passage contrasts this with tool-handle representations and top-down power grasps.Source: §2 · p. 4

Reference State Initialization

• Unlike reference-tracking methods, X-RESET injects noisy retargeted resets and trains policies on current object state and goal.Reference tracking faces diversity challenges under novel goal poses and external forces.Source: §2 · p. 4

Learning from Human Hand-Object Data

• The pipeline retargets human hand-object states to noisy robot states, retains stable simulation states, and samples diverse resets during RL training.The policy consumes object keypoints at inference time.Source: Figure 2 · p. 5
• Human-motion imitation can fail under embodiment mismatch and robot-object contact dynamics, while human-guided RL often produces single-task policies.X-RESET uses human data neither as an imitation target nor as a per-task reference.Source: §2 · pp. 4–5

3 Problem Formulation

• The task is to approach, grasp, and reorient objects to goal poses through object-centric RL in simulation, with resets from human demonstrations.The object is denoted η, and each trajectory contains T tracked timesteps.Source: §3 · p. 5
• Each human hand-object trajectory pairs 21 hand keypoints with the object's position and rotation at each timestep, and its final object pose defines the goal.Object poses are represented by 64 sampled surface keypoints.Source: §3 · p. 5
• The dataset for an object η is denoted D_η and consists of N human hand-object trajectories.Each trajectory is written as ξ_η in the formulation.Source: §3 · p. 5

4.1 Generating Reset States from Human Hand-Object Demonstrations

• Kinematic retargeting maps human hand poses to robot joints, while offline filtering discards states that violate joint limits or are unstable when spawned.The generated robot states include joint configurations, object pose, and goal.Source: §4.1 · p. 6

4 Approach

• X-RESET generates robot reset states from human hand-object poses, then trains object-centric RL with occasional resets to those states.These demonstration resets expose the policy to states where later-stage success signals can help avoid local optima from scratch.Source: §4 · p. 5, §4.2 · p. 6

4.2 Supplementing Object-Centric Reinforcement Learning with Simulation Resets

• The shared object-centric reward combines hand proximity, goal reorientation and success, smooth actions, and safety penalties.Reorientation uses mean distance between corresponding object keypoints, jointly capturing position and orientation differences.Source: §4.2 · p. 6
• Training samples from the default reset distribution with probability 1 − p and from retained demonstration frames with probability p; the policy observes object keypoints, goals, and robot proprioception.The policy also uses interaction history and an asymmetric critic with privileged information.Source: §4.2 · pp. 6–7

5.1 Cross-Embodiment Resets

• Across three embodiments, success rose from 11.0% to 66.6% for Obj-Only Reward and from 42.1% to 59.6% for Hand-Obj Reward with X-RESET.From-scratch Obj-Only Reward reached 11.0% overall, while direct kinematic retargeting averaged 4.9%.Source: §5.1 · p. 8

5.2 Scaling and Generalization

• On EBench, success rose from 64.8% to 89.6% for Easy objects and from 16.8% to 43.5% for Hard objects as training diversity increased.Zero-shot transfer surpassed full-budget EBench scratch training with as few as 10 DexYCB training objects.Source: Figure 4 · p. 9, §5.2 · p. 9
• 85.9% average reorientation success followed 5k epochs of DexYCB pretraining and 5k EBench fine-tuning, versus 39.1% for 10k EBench scratch training.Both policies trained on the same EBench objects, without demonstration resets during EBench training.Source: Figure 4 · p. 9, §5.2 · p. 9

5.3 Reset Distribution

• With demonstration-reset probability varied, returns converged between 55.45 and 61.09, above the 8.28 return at 0%; higher probabilities accelerated early learning.The benefit held across a broad range of probabilities rather than requiring p = 0.9.Source: §5.3 · p. 9, Figure 5 · p. 10

5.4 Real-World Capabilities

• Noisy hand poses yielded returns of 51.46–56.07 versus 8.28 without demonstration resets, and the policy transferred to seen and unseen real objects.Object-pose estimation error was the primary observed real-world failure mode.Source: §5.4 · pp. 9–11, Figure 7 · p. 11

• X-RESET uses retargeted human hand-object states as a reset distribution for generalist dexterous object-centric RL.Source: §6 · p. 11
• Resets make task-agnostic rewards tractable where from-scratch exploration fails; policies also scale with training objects, tolerate noisy hand tracking, and transfer zero-shot to hardware.Source: §6 · p. 11
• Because policies consume object pose, they inherit tracking noise; visual or visuo-tactile distillation is proposed as a natural remedy.Source: §6 · p. 11
• Scaling to large egocentric datasets remains future work and requires extracting hand-object states from video for rigid, articulated, and deformable objects.Source: §6 · p. 11

A.1 Data and Representation

• A common table-aligned transform maps demonstration hands and objects into simulation, while robot-base transforms map wrist targets into arm coordinates.Source: §A.1 · p. 15
• The representation samples 64 ordered object-local surface points, using collision surfaces for DexYCB and visual surfaces for EBench.Source: §A.1 · p. 15
• Point indices define current-goal correspondence, and the actor receives paired features with temporal context from its LSTM.Source: §A.1 · p. 15

A.2 Retargeting, Filtering, and Resets

• PyRoki jointly optimizes Sharpa hand joints and root poses, Robotiq uses an analytical hand-to-gripper mapping, and cuRobo solves collision-aware arm IK with the hand fixed.Source: §A.2 · p. 15
• Filtering holds IK-valid configurations at their initial joint targets with gravity disabled, rejecting terminations, non-finite measurements, or threshold violations.Source: §A.2 · p. 15
• Each environment samples retained demonstration frames with probability p and uses the demonstration’s terminal object goal; otherwise it resets to the home robot configuration and demonstrated object start.Human-derived training uses p=0.9 unless ablated; synthetic EBench training uses p=0.Source: §A.2 · p. 15

A.3 Rewards

• The proximity reward penalizes capped sums of fingertip distances to nearest object-surface keypoints and the analogous palm distance.Source: §A.3 · p. 15
• The object-only reward includes a success term with a distance threshold of 0.02.Source: §A.3 · p. 16
• A geometric proximity gate multiplies the two goal terms.Source: §A.3 · p. 16
• Out-of-bounds objects and terminal robot-environment contact each incur a -500 safety penalty; no dense contact penalty is used.Source: §A.3 · p. 16
• The hand-object variant replaces proximity with object-local fingertip targets that specify a grasp region, not a time-indexed trajectory.Source: §A.3 · p. 16

A.4 Policy and Training

• The actor observes joint positions, controller targets, fingertip-to-object geometry, and paired point features; the critic additionally receives clean geometry and state, contact, reward, and perturbation information.UR7e+Sharpa has 648 pre-encoder observations and 28 actions; dimensions vary by embodiment. Actions increment persistent joint-position targets, with embodiment-specific controller scales and output heads.Source: §A.4 · p. 16, Table A2 · p. 16
• Training uses 10k-epoch checkpoints on one GPU per seed, with matched within-seed budgets; proximity-gated disturbances include bounded object forces, arm pushes, and spin impulses.Fine-tuning uses 5k DexYCB plus 5k EBench epochs; scratch uses 10k EBench epochs. Event probabilities are log-uniform in [0.001, 0.1]; Table A3 covers training randomization.Source: §A.4 · pp. 16–17, Table A3 · p. 17

A.5 Evaluation

• Evaluation covers Easy, Medium, and Hard EBench cohorts, uses perturbation seeds distinct from training, and aggregates simulation plots as mean ± SE over three training seeds.Object averages are computed within each seed before aggregation; episode returns use default resets. Table A4 covers evaluation protocols and cohorts.Source: §A.5 · p. 17, Table A4 · p. 17

A.6 Offline Hand-Pose Noise

• Offline noise is added to wrist position and palm-frame finger keypoints before retargeting and filtering, while object poses and untracked sentinel frames remain unchanged.Finger-keypoint noise has eight-frame correlation; vector norms are clipped, coordinates rotated to world space, and bone lengths restored. Table A5 reports hand-pose noise in millimeters.Source: §A.6 · p. 17, Table A5 · p. 18
• The High noise tier rejects 2.45× as many resets as Clean, and corruption changes both reset quality and retained-bank composition.All tiers use the same retargeting/filter, object-only reward, p=0.9, and 10k-epoch budget; Clean reuses the clean-policy runs.Source: §A.6 · p. 18