PaperlayerView exact revision on arXiv

Source-linked AI summary

Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts

Antonia Bronars, Younghyo Park, Kushal Kedia, Pulkit Agrawal

arXiv:2610.23849v1cs.RO

TL;DR

The paper asks whether proprioception alone can support precise assembly through light contact and millimeter-scale clearances. It trains blind policies in randomized simulation and finds that they generalize better than matched visual policies in simulation; a peg policy also transfers zero-shot to real-world assembly, succeeding in 73% of trials.

  • Problem

    It is unclear whether joint sensing, which supports legged and humanoid motion with large, forceful contacts, is sufficient for precise assembly through light contact.

    Source: §1 · p. 1

  • Method

    The authors train reinforcement-learning policies in simulation using joint histories and actions alone, with randomized parts, grasps, fixtures, and contacts.

    Source: §1 · p. 2

  • Results

    Blind policies generalize better to unseen part shapes and sizes than matched visual policies in simulation, while the zero-shot peg-insertion policy succeeds in 173 of 237 real-world trials (73%).

    Source: §1 · p. 2, §3.4 · p. 9

  • Takeaways & Limitations

    Proprioception supports peg insertion, gear meshing, and nut threading, and the authors provide mjfit to generate tight-clearance training assemblies.

    Source: §1 · pp. 1–2

    The peg policy assumes it starts in contact with the fixture and searches locally around its starting point.

    Source: §5 · p. 10

Abstract

from arXiv · show

Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit them to walk, run, and track whole-body motions without vision, but there the contacts are large and forceful. Is joint sensing also enough for precise, contact-rich assembly, where the robot must find a millimeter-scale opening, mesh gear teeth, or catch a thread through light contact? We study this question by training blind assembly policies that observe only a short history of joint positions, velocities, and their own actions, without visual, force, torque, or tactile sensing. The poses of both the fixture and the part are unknown to the policy, and the grasp is randomized substantially. The policy must therefore assemble the parts through contact, sensed exclusively through its own proprioception. Trained with reinforcement learning entirely in simulation over randomized parts, grasps, and contacts, such policies learn peg insertion, gear meshing, and nut threading. The peg insertion policy transfers zero-shot to the real world and inserts pegs of different shapes and sizes in 73% of trials. Controlled comparisons with matched visual policies in simulation show that blind policies generalize far better to unseen part shapes and sizes, though they can be slower. Training such policies needs thousands of different assemblies, each simulated with its clearance intact, so we also release an open-source generator that builds them procedurally from exact convex pieces for MuJoCo; it is more accurate and faster than convex decomposition or MuJoCo's signed-distance collision. Videos and interactive simulations are at https://younghyopark.me/blind-proprioceptive-assembly/.

1 INTRODUCTION

The paper tests whether joint sensing alone can support precise, light-contact assembly, where contact effects on the joints are easy to miss. It trains proprioception-only policies in simulation and reports real-world peg insertion, stronger generalization than matched visual policies, and a procedural simulator for tight-clearance parts.

1 INTRODUCTION

  • The work asks whether joint sensing can support assembly through light contact despite millimeter-scale clearances and subtle joint effects.Source: §1 · p. 1
  • 73% of 237 real-world trials succeeded with a zero-shot peg-insertion policy, including pegs outside its training distribution.The policy received no hole location, grasp pose, part size, or clearance.Source: §1 · p. 1, Figure 1 · p. 2
  • Policies trained entirely in simulation use joint positions and velocities, gripper pose and twist, and previous actions, with no images or force signals.Training randomizes part geometry, contact properties, fixture poses, and grasps.Source: §1 · pp. 1–2
  • Across peg insertion, gear meshing, and nut threading, blind policies search reactively from contact and generalize better to unseen part shapes and sizes than matched visual policies.Their learned proprioceptive representations contain task-relevant information used to adjust behavior.Source: §1 · p. 2
  • The authors introduce mjfit, which procedurally generates diverse MuJoCo assemblies from exact convex pieces while preserving millimeter clearances.The generator supports pegs and holes, threads, and gears.Source: §1 · p. 2

2 LEARNING BLIND ASSEMBLY

Blind policies learn three contact-rich assembly tasks from proprioception, while mjfit generates accurate collision geometry for training at scale. The section describes the tasks, simulator assets, policy architecture, and training setup.

2.1 TASKS

  • The 7-DOF arm must search for alignment and assemble randomized parts using joint feedback, across peg insertion, gear meshing, and nut threading.Tasks use 1.0–1.5 mm peg clearance per side, 0.2–0.6 mm gear backlash, and 0.05–0.2 mm nut-thread clearance.Source: §2.1 · p. 3

2.2 SIMULATING TIGHT INSERTION AT SCALE

  • mjfit generates arbitrary-outline holes and mating pegs as exact-clearance convex geometry, with companion generators and sampling for gears, nuts, and thousands of assemblies.Its curved walls may add corner play but do not intrude into the true hole; the user specifies clearance directly.Source: §2.2 · p. 4
  • mjfit has the best speed–accuracy trade-off on GPU and is an order of magnitude faster than mesh SDF on CPU at equal or better accuracy.At tight clearances, approximate decomposition leaves material inside holes and mesh SDF misses contacts; a centered peg jams on baseline plates but falls through mjfit’s plate.Source: §2.2 · p. 4

2.3 POLICY AND TRAINING

  • The policy maps a 0.6 s history of proprioception and the current observation to joint-position increments, without images, fixture pose, part geometry, or wrench estimates.PPO training uses sparse rewards and an asymmetric critic; a contact decoder adds training-only supervision to the encoder, but its predictions never reach the actor.Source: §2.3 · pp. 5–6
  • Blind policies reach the same near-perfect training success as wrist-camera visual policies across all three tasks.Visual policies use the same task setup, resets, randomization, reward, actions, and success check, but receive image histories instead of proprioception.Source: Figure 8 · p. 6, §2.3 · p. 6

3 RESULTS

The results show that proprioception alone supports precise assembly and reactive contact search, while blind policies generalize better than matched visual policies to unseen parts. The peg policy also transfers to the real world without fine-tuning.

3.1 PROPRIOCEPTION IS SUFFICIENT

  • All three tasks achieved near-perfect success on in-distribution geometries, showing joint histories carried enough contact information to find openings, mesh teeth, and engage threads.The tasks were peg insertion, gear meshing, and nut threading.Source: §3.1 · p. 6
  • Blind peg insertion followed a less direct path and seated later than visual insertion; the gap was smaller for gear meshing and nut threading.For those tasks, the grasped part can occlude more of the mating part from the camera.Source: §3.1 · p. 6

3.2 VISUALLY BLIND BUT INTELLIGENT

  • History latents separated action-relevant contact modes on all three tasks more accurately than current observations alone (61–73% vs. 41–64%; chance 14%).The probe was fitted in simulation, and the peg probe decoded the same mode sequence on the real robot.Source: §3.2 · p. 7, Figure 10 · p. 8
  • Cross-mode latent swaps changed actions 1.3–3.0× more than within-mode swaps, indicating that inferred contact states informed action choice.The modes included states such as lifted, engaged, and stuck.Source: §3.2 · p. 7
  • The reactive blind policy traveled a median 37 mm before seating, versus 95 mm for an optimistic spiral baseline given the peg orientation and best starting direction.The comparison used the same starts.Source: §3.2 · p. 7

3.3 GENERALIZATION TO UNSEEN PARTS

  • Blind policies were more reliable than matched visual policies on unseen shapes and extreme sizes across all three tasks, despite similar in-distribution reliability.Visual policies consistently underperformed on parts with different shapes or extreme sizes, while blind policies learned a more transferable search strategy.Source: §3.3 · pp. 7–8, Figure 11 · p. 8

3.4 ZERO-SHOT TRANSFER TO THE REAL WORLD

  • 173 of 237 trials (73%) succeeded with the frozen peg policy in the real world, including 66%, 66%, and 72% on tighter, larger, and taller-handled pegs.Trials began with the peg touching the fixture; the policy used encoder-derived state without real-world data or fine-tuning.Source: Figure 12 · p. 9, §3.4 · p. 9

4 RELATED WORK

Prior insertion methods often rely on known grasps, localized search patterns, or additional sensing; related proprioceptive work has focused on different contact regimes. The paper also situates mjfit among approaches to simulating tight contact.

Robotic insertion

  • Classical peg-in-hole strategies use compliance with spiral, raster, or tilting searches, but assume a known grasp and a hole within the pattern and are tuned for each part.Some such strategies omit force feedback.Source: §4 · p. 9
  • Learned insertion policies commonly add force/torque sensing, wrist vision, or both [Inoue et al., 2017; Beltran-Hernandez et al., 2020; Schoettler et al., 2020; Bogunowicz et al., 2020; Spector & Di Castro, 2021; Haugaard et al., 2021; Seo et al., 2025; Lee et al., 2019; Luo et al., 2021; Watson et al., 2020].The paper contrasts these with a policy using proprioception alone and no hole pose, part model, or force signal.Source: §4 · p. 9

Proprioceptive policies

  • Prior proprioceptive manipulation includes object reorientation and Blind Dexterity, but the cited tasks involve forceful contact or centimeter tolerances.Legged-robot policies also use histories of onboard state [Hwangbo et al., 2019; Lee et al., 2020; Siekmann et al., 2021b; Kumar et al., 2021; Radosavovic et al., 2024].Source: §4 · p. 9

Simulating tight contact

  • Convex decomposition may exceed assembly clearances, while SDF accuracy depends on field resolution and contact search; mjfit automates exact polygon partitions for bulk assembly generation.The paper compares collision geometries while keeping MuJoCo’s solver fixed.Source: §4 · p. 10

5 LIMITATIONS AND CONCLUSION

The peg policy is a last-mile skill with a local operating range: it begins in contact with the fixture and corrects nearby misalignments.

5 LIMITATIONS AND CONCLUSION

  • The policy requires a separate approach to bring the peg into contact near the hole and cannot handle events needing more than local correction, such as a peg slipping from the grasp and landing far away.Every training episode starts with the peg touching the fixture a few centimeters from the hole.Source: §5 · p. 10

A TRAINING DETAILS

The study uses narrower training ranges to test generalization systematically, while the deployed peg policy is trained on the broadest ranges feasible under physical and geometric constraints.

A TRAINING DETAILS

  • For generalization tests, training ranges were narrower than the deployed peg policy’s feasible ranges, allowing each task property to be widened beyond training.The deployed policy uses the full ranges of geometry, physics parameters, and object pose feasible under physics stability and constraints such as gripper width.Source: A TRAINING DETAILS · p. 14

A.1 STUDY POLICIES

The study policies use task-specific episode conditions and contact labels to train peg insertion, gear meshing, and nut threading. Their actors use recent proprioceptive observations and prior actions, while critics also receive privileged task state.

Tasks

  • Peg, gear, and nut episodes start in distinct misaligned states and succeed at seating, meshing, or threading, with task-specific horizons and rewards.Peg episodes last up to 5 s, gear episodes 10 s, and nut episodes 15 s; rewards include first capture and final success, with additional task-specific milestones.Source: §A.1 · p. 14
  • Contact labels classify 64 sampled points on task-specific part surfaces into states such as free, contacting, inside, or engaged.Peg labels cover its bottom edge; gear labels cover the shaft rim and tooth tips; nut labels cover its bore rim, including thread capture.Source: §A.1 · p. 14

Architecture

  • A temporal convolution encodes 30 observations spanning 0.6 s into a 32-dimensional latent, which an MLP maps with the current observation to joint-position increments.The actor observes joint positions and velocities, gripper pose and twist, and its previous action; the nut actor uses a 36-value orientation representation.Source: §A.1 · p. 14
  • Critics receive privileged part-to-fixture pose and contact labels, plus thread geometry for nut threading, unlike the proprioceptive actor.The visual policies instead receive wrist RGB images over the same 30-step history.Source: §A.1 · p. 14

PPO

  • PPO trains across 12,288 worlds, using 32-step rollouts and a contact decoder with class-weighted labels.The decoder uses two 128-unit hidden layers; the stated class weights are 0.25 for free, 3.0 for contact, and 2.0 for engaged classes.Source: §A.1 · p. 14

A.2 DEPLOYED AND FULL-RANGE POLICIES

The deployed peg policy and full-range gear and nut policies expand the randomized training conditions beyond the study policies. The full-range gear and nut policies achieve high success on unseen examples drawn from their training distributions.

Deployed peg policy

  • The deployed peg policy trains on random convex cross-sections under a handle, with wider contact and pose ranges and starts a median 2 cm from the hole center.It uses the gear and nut controller and a staged safety penalty after learning without safety limits.Source: §A.2 · p. 14
  • The deployed peg training distribution includes polygonal, smooth, and mixed outlines within a stadium handle, subject to fitting the printed fixture insert.The outlines are sampled in equal shares; scale, aspect ratio, and clearance remain those of the study policy.Source: Table 1 · p. 15, Table 2 · p. 16

Full-range gear and nut policies

  • The full-range gear policy succeeds in 98% of 8,192 episodes on 4,096 unseen gear sets, and the nut policy succeeds in 93% on 512 unseen pairs.Both evaluations use deterministic actions and unseen geometries drawn from the respective training distributions.Source: §A.2 · p. 17
  • The full-range gear policy covers module 2–3.5 mm, 12–28 teeth, and 0.5–1 mm bore clearance, while the nut policy covers 870 ISO metric pairs from M24–M42.The policies also use wider pose and contact ranges; the nut starts a median 19 mm off-axis and 60–170° short of thread capture.Source: §A.2 · p. 17

B COLLISION BENCHMARK DETAILS

The collision benchmark measures geometric error against simulation time for concave-hole methods at clearances relevant to assembly. Its setup compares automatic collision representations against exact geometry under specified numerical and hardware conditions.

Simulation conditions

  • Scenes use a 1 ms time step and zero margin, with exact watertight plate meshes for baselines and an analytic circle for ground truth.The peg is always an exact MuJoCo primitive, so differences are attributed to the hole representation.Source: B COLLISION BENCHMARK DETAILS · p. 17

Measurement boundary

  • The stated S1 probe cannot resolve geometric errors below about 20 µm because frictionless drops let pegs up to 20 µm oversize slip in for every method.The passage notes that play probes can resolve errors below this level.Source: B COLLISION BENCHMARK DETAILS · p. 17

Setup

  • The benchmark tests 16 mm round and square holes with radial clearance from 20 to 200 µm, plus M10×1.5 bolt-and-nut cases at 20 and 100 µm flank clearance.The threaded cases are probed for axial, radial, and tilt play and for screwing under load.Source: B COLLISION BENCHMARK DETAILS · p. 17

Timing setup

  • CPU timings use one environment on one thread, while GPU timings use 512 MuJoCo Warp worlds, except figure 6b, which uses 256.Figure 6b also measures error on Warp.Source: B COLLISION BENCHMARK DETAILS · p. 18

C ADDITIONAL RESULTS

Additional results probe generalization, learning speed, and the policies’ use of proprioceptive history. Blind policies outperform matched visual policies on unseen convex pegs, while task-specific speed comparisons and probes reveal trade-offs and scope limits.

Behaviour

  • 36.2% of unseen random convex-peg episodes succeed for the blind policy, versus 3.9% for the visual policy, despite the pegs’ median area being 53% of the matching stadium’s.The blind policy also travels farther than the visual policy from 99% of matched starts; evaluation pegs have the same clearance as matching stadiums and one vertex on each side of their bounding box.Source: C ADDITIONAL RESULTS · p. 18, Figure 13 · p. 18

Speed on gear meshing and nut threading

  • The blind gear policy’s median success time is 0.58 s versus 0.64 s for vision, while the blind nut policy’s is 0.76 s versus 0.64 s.Among mutually successful starts, the blind policy takes longer on 43% of gear starts and 98% of nut starts, by a median 0.10 s for nut threading.Source: C ADDITIONAL RESULTS · p. 18

Contact supervision

  • Without contact loss, peg, gear, and nut policies require 2.7, 3.8, and 5.1 times as many environment steps to reach 90% training success.The unsupervised peg policy reaches 97.8% after 486M steps, while supervised policies pass 99% by 242M.Source: C ADDITIONAL RESULTS · p. 18, Figure 14 · p. 19

Probe details

  • The peg history latent achieves 70% balanced accuracy over seven modes, compared with 56% from current observation alone and 33% from a randomly initialized encoder.Latent swaps between searching and inside states change actions 1.4–1.9× as much as swaps within a mode, while near/far swaps change them 0.99–1.11× as much.Source: C ADDITIONAL RESULTS · p. 18, Figure 15 · p. 19
  • On the robot, the peg-state probe correctly labels 62% of inside states, versus 100% in simulation from the same start states, and labels 17 of 24 successful-trial final states inside.Source: C ADDITIONAL RESULTS · p. 19
  • Gear and nut history-latent probes reach 61% and 73% balanced accuracy across seven modes, exceeding current-observation and random-encoder probes.For engaged episodes visiting at least five modes, decoded modes agree with true modes on 79% of gear steps at the 75th percentile and a median 69%, and on a median 74% of nut steps.Source: C ADDITIONAL RESULTS · p. 19

Unseen parts

Real-world pegs

  • Real-world peg trials use 30 planned resets per peg, counted by the first attempt that reaches a verdict; a 20 s horizon counts as failure.The pegs include training-distribution cross-sections, a tighter-clearance polygon, an enlarged outline, and a taller handle.Source: C ADDITIONAL RESULTS · pp. 20–21

Related papers

Loading 2610.23849v1…