Source-linked AI summary
AlphaNIST: Emergent Assembly Skills from Binary Task Rewards
Octi Zhang, Iretiayo Akinola, Rosario Scalise, Mateo Guaman Castro, Patrick Yin, Tobias Widmer, Miles Macklin, Chris Amevor, Dylan Turpin, Dirk Van Gelder, Jan Carius, Ankur Handa, Gavriel State, Yunrong Guo, Yashraj Narang
TL;DR
Robotic assembly remains difficult because it combines precise contact-rich control with long-horizon recovery and coordination, while prior learning-based methods had not solved all tasks on the NIST board. AlphaNIST trains a shared policy across the 20 tasks using sparse binary rewards and diverse experience. In simulation, it reaches 98.2% overall training success and also completes ten assembly pairs without a prescribed order.
Problem
Robotic assembly requires precise contact-rich control and long-horizon recovery, and learning-based methods had not solved all tasks on any NIST Assembly Task Board.
Source: §I · p. 1Method
AlphaNIST trains a single policy across all 20 tasks with sparse binary rewards, diverse cached resets, and success-guided sampling, without demonstrations or per-task tuning.
Source: §I · p. 2, B. Reset Coverage and Experience… · p. 3Results
98.2% overall training success: the shared policy solves all 20 ATB1 tasks and the framework also trains policies to complete ten pairs without a prescribed order.
Source: §I · p. 2, A. Assembly-Pair Heterogeneity Enables… · p. 7, D. Emergent Strategies over Long… · p. 8Takeaways & Limitations
The findings show that diverse experience under a shared objective can become a source of reusable dexterity and coordination in simulation.
Source: §VII · p. 8Results are in simulation; robustness to perception error, latency, and contact-model mismatch remains unestablished, and extending multi-pair assembly to 20 parts remains a research goal.
Source: §VI · p. 8
Abstract
from arXiv · showhide
The NIST Assembly Task Board #1 remains a challenging benchmark for learning-based methods in robotic manipulation. Over short horizons, it demands precise, contact-rich control of small, tight-fitting parts. Over long horizons, it requires recovery and coordination across extended action sequences. Demonstrations, task decomposition, and dense progress rewards have enabled research advances by providing guidance on how to assemble. But how much of the strategy must be specified? The progression from AlphaGo to AlphaGo Zero showed that RL at scale can discover superhuman strategies without prescribing how to win. AlphaNIST asks whether a sparse binary task reward and task diversity admitted by this task-agnostic reward can similarly let a robot discover how to assemble. In simulation, we train a single policy from scratch across all 20 assembly tasks on the NIST Assembly Task Board #1, reaching 98.2% overall training success without demonstrations, progress shaping, scripted primitives, or per-task tuning. The framework also trains policies to complete ten assembly pairs in a single episode without a prescribed order, with task success rewarded only when all ten pairs are simultaneously complete. The learned policies discover dexterous and planning-like behaviors: choosing grasps for later assembly, using surrounding geometry to reorient parts, and preparing one part, assembling another, then returning to finish the first without an explicit planner. For videos, see our website: https://alpha-nist.github.io/
I. INTRODUCTION
ATB1 combines heterogeneous, contact-rich assembly tasks that prior learning-based methods had not solved as a complete board. AlphaNIST tests whether one policy can learn these tasks from sparse binary rewards and diverse experience, without specifying assembly strategies.
I. INTRODUCTION
- ATB1 poses challenges in task ordering, long action sequences with recovery, and precise contact among parts as small as 4 mm with clearances as low as zero.Parts can become misaligned or jam during insertion or threading, requiring recovery and reexecution.Source: §I · p. 1
- The paper asks whether sparse binary rewards and joint training across diverse assembly pairs can support long-horizon robotic strategy discovery.The authors frame task diversity as a potential resource rather than a burden.Source: §I · p. 2
- A single policy reaches 98.2% overall training success across all 20 ATB1 tasks and also learns to complete ten pairs without a prescribed order.The ten-pair reward is issued only when all pairs are complete.Source: §I · p. 2
- AlphaNIST combines cached assembly-relevant reset states, success-guided sampling, a unified sparse binary reward, and multi-GPU training without demonstrations or per-task tuning.The same training configuration is shared across tasks.Source: §I · p. 2
- The policy handles all 20 tasks from a wide range of initial poses, and multi-task policies outperform individual specialists on three tasks the specialists struggle to solve.The work presents this as the first multi-task RL policy capable of solving all 20 ATB1 tasks.Source: §I · p. 2
- The learned repertoire includes pushing, sliding, in-hand pivoting, sequential regrasps, extrinsic reorientation, and nut threading, alongside returning to partially finished tasks.These behaviors were not specified in advance.Source: §I · p. 2
II. RELATED WORK
Prior work supplies scalable simulation, reset coverage, and sampling methods, while AlphaNIST applies these tools to shared training across heterogeneous assembly tasks. Its multi-task approach uses a common assembly objective and standard PPO rather than task-specific optimization machinery.
A. Scalable Learning for Contact-Rich Assembly
- GPU-parallel physics simulation supports large-scale on-policy learning, and AlphaNIST uses this throughput to collect experience across heterogeneous assembly pairs.Earlier Factory work demonstrated simulation of 1,000 simultaneous nut-and-bolt interactions.Source: A. Scalable Learning for Contact-Rich… · p. 2
- OmniReset supplies diverse manipulation-centric reset states, and SGS allocates experience near the changing capability frontier; AlphaNIST retains SGS while sharing one policy across all 20 ATB1 tasks.The shared policy learns assemblies left unsolved by single-task training.Source: A. Scalable Learning for Contact-Rich… · p. 2
B. Multi-Task Reinforcement Learning
- Prior multi-task methods have used specialist distillation, task-specific value normalization, or gradient projection to address challenges in joint training.The passage also notes evidence that shared-network baselines can match specialized multi-task optimizers.Source: B. Multi-Task Reinforcement Learning · p. 3
- AlphaNIST gives threading, gear meshing, and insertion the same binary success criterion, training one geometry-conditioned policy and value function with standard PPO.It uses no task-specific value heads, gradient projection, or specialist pretraining.Source: B. Multi-Task Reinforcement Learning · p. 3
C. Learning-Based Robot Assembly
Prior assembly learning systems often structure exploration with staged initialization, demonstrations, curricula, or prescribed subskills. AlphaNIST frames the tasks as a shared-policy finite-horizon MDP, with task variation represented by point clouds rather than an explicit task identifier.
C. Learning-Based Robot Assembly
- Earlier assembly systems shorten horizons or structure learning with staged states, demonstrations, curricula, specialist policies, or prescribed subskills.Some approaches also use dense auxiliary shaping whose relative influence can vary with task geometry and motion.Source: C. Learning-Based Robot Assembly · p. 3
- The paper asks whether binary task success can support strategy discovery in continuous, contact-rich control with one shared policy learning heterogeneous assemblies end to end.This question is motivated by AlphaGo Zero’s use of terminal win/loss objectives without intermediate reward shaping.Source: C. Learning-Based Robot Assembly · p. 3
A. Task Definition and MDP
- The MDP samples an assembly task and initial state per episode, and a shared policy acts on partial observations to maximize return.Task variation is represented through surface point clouds rather than an explicit discrete task identifier.Source: A. Task Definition and MDP · p. 3
IV. METHOD
AlphaNIST trains a single policy across all 20 assembly tasks using standard PPO, a shared binary task reward, diverse reset states, and success-guided sampling.
IV. METHOD
- The framework combines standard PPO with one policy across all 20 tasks and a binary reward for completed assembly.Diverse cached reset states and beta-distributed success-guided sampling shape the training distribution.Source: A. AlphaNIST Training Framework · p. 3
- The reset bank determines which states are available, beta sampling determines which states are drawn, and the binary reward defines success.The three components therefore address state coverage, experience allocation, and the success criterion, respectively.Source: A. AlphaNIST Training Framework · p. 3
B. Reset Coverage and Experience Allocation
AlphaNIST pairs each cached reset state with an assembly identity, then samples from this heterogeneous bank to initialize training episodes. The bank spans configurations from ungrasped parts to partial assemblies, while success-guided sampling reallocates experience without prescribing a task sequence.
B. Reset Coverage and Experience Allocation
- Each reset-bank entry pairs an assembly identity with a robot, board, and movable-part state, so sampling selects both the task and its initial configuration.The bank covers ungrasped workspace, pickup, transport, near-mating, and partially assembled configurations.Source: B. Reset Coverage and Experience… · p. 3
- Reset families define regions of the training distribution rather than a prescribed sequence, and the policy receives no stage identifier.Fig. 4 illustrates examples ranging from random object poses to in-hand grasps and partial assemblies.Source: B. Reset Coverage and Experience… · p. 3, Figure 4 · p. 4
- Beta sampling uses recent success rates to favor intermediate-performing reset entries, while εq > 0 keeps every entry sampleable; the uniform ablation samples entries equally.Each episode outcome updates the rate for its sampled entry.Source: B. Reset Coverage and Experience… · p. 4
C. Binary Full-Assembly Objective
The task objective is full assembly, expressed relative to each part’s mating geometry and rewarded as a binary success. Control costs regularize actions without supplying assembly-progress information.
C. Binary Full-Assembly Objective
- An assembly-relative coordinate Φe(α) parameterizes the part in its mating frame, with Φe(1) denoting the fully assembled pose.The distance to that pose defines the assembly error used by the objective.Source: C. Binary Full-Assembly Objective · p. 4
- For a single-pair episode, the binary task reward is issued upon reaching the assembly-distance threshold, and the episode terminates.The threshold is ϵ = 0.001 m; intermediate progress receives no task reward.Source: C. Binary Full-Assembly Objective · p. 4
- A control cost penalizes large actions, rapid joint motion, and excessive torque, biasing learning toward smoother, lower-effort behavior without measuring assembly progress.In multi-pair episodes, reward remains zero until every active pair is assembled, with no prescribed pair order or intermediate completion reward.Source: C. Binary Full-Assembly Objective · p. 4
D. Shared Policy and Robot Control
AlphaNIST uses one shared policy and training configuration across assembly tasks, with point-cloud and robot-state observations. Its sparse-reward results and distributed PPO procedure are reported alongside hardware-aligned simulation constraints.
D. Shared Policy and Robot Control
- The shared policy observes robot state and surface point clouds without a discrete task identifier, and PPO batches combine rollouts from sampled assembly–reset pairs.Policy, reward, controller, and training configuration are shared across the 20 tasks.Source: D. Shared Policy and Robot Control · pp. 4–5
- The simulated Panda uses an implicit impedance controller and hardware-matched parameters, with joint-speed and torque limits set to a hardware-realistic envelope.The stated constraints are intended to prevent exploitation of unrealistic speeds or torques.Source: D. Shared Policy and Robot Control · pp. 4–5
- Above 98% success across all 20 tasks was achieved by the shared policy, including three precision assemblies that corresponding specialists rarely solve.Fig. 5 compares per-task shared and separate policies; dense shaping does not show the same benefit.Source: Figure 5 · p. 5
- Joint sparse training converges faster than the mean of 20 separately trained policies, while sparse-reward success benefits strongly from parallel experience.Fig. 6 compares shared and separate-policy learning curves; Fig. 7 encodes reward by color and reset sampling by line style.Source: Figure 6 · p. 5, Figure 7 · p. 5
E. Distributed Training and Scale
Training scales synchronous PPO across GPUs and environments with fixed network and PPO settings. Experiments compare shared and separate policies, evaluate scale and reset sampling, and explore multi-pair assembly and emergent behaviors.
E. Distributed Training and Scale
- Synchronous PPO scales from 1–256K environments, increasing GPUs, environments, and rollout batch size together while keeping architecture and PPO settings fixed.Multi-GPU runs use 4K environments per GPU and average actor and critic gradients across GPUs.Source: E. Distributed Training and Scale · p. 5
V. EXPERIMENTS
- Fig. 8 illustrates contact strategies including regrasping, geometry-assisted reorientation, pivoting, rolling, tapping, and fingertip shoveling across assembly parts.The panels proceed from left to right.Source: Figure 8 · p. 6
- At 64K environments, the shared policy reaches 98.2% training success under matched per-run interaction budgets against separately trained task policies.The comparison otherwise uses identical settings.Source: §V · p. 6
- The scaling evaluation uses stochastic actions and a common frozen mixed-reset bank sampled uniformly within each task; success requires full assembly within 15 seconds without simultaneous safety failure.The reset-sampling ablation compares beta with uniform sampling over the same bank.Source: §V · p. 6
- The reward ablation adds end-effector proximity and target-orientation alignment terms, with coefficients c2 = c3 = 0.01; success rates cover all reset configurations.The coefficients match those used in OmniReset [10].Source: §V · p. 6
A. Assembly-Pair Heterogeneity Enables Learning
A shared policy outperforms separately trained policies in success and training cost, including on small-part tasks that separate policies rarely solve. The results also contrast sparse binary rewards with dense shaping and show coordination across multiple assembly pairs.
A. Assembly-Pair Heterogeneity Enables Learning
- Separately trained policies achieve ≤0.3% success on M4 nut threading, 4 mm peg insertion, and rod insertion; the shared policy produces successful rollouts on those tasks.The passage attributes the separate policies’ difficulty to reliably grasping the small parts from the table.Source: A. Assembly-Pair Heterogeneity Enables… · p. 6
- In the ten-pair sequence, the policy stages a USB-A connector, assembles an 8 mm rod, then returns to insert the connector.During M12 nut threading, it also makes small turns to avoid the neighboring RJ45 connector.Source: Figure 9 · p. 7
- Dense shaping may favor different partial solutions across geometries, whereas the binary task reward gives every task a unified full-assembly objective.The passage presents this as one possible explanation for the difference between joint dense training and the binary-reward policy.Source: A. Assembly-Pair Heterogeneity Enables… · p. 7
- 98.2% training success in 410 GPU-hours for the shared policy, versus 81.4% average success for separate policies at 39.3× more device-hours.Joint dense training reaches 17.8% success at 25K iterations, while separately trained dense-reward policies average 78.3% at their final checkpoints.Source: A. Assembly-Pair Heterogeneity Enables… · p. 7
B. Sparse Binary Task Rewards Benefit from Scale
Sparse binary rewards benefit from increased parallel experience, and beta sampling adds gains over uniform resets. The same policy develops dexterous assembly strategies without demonstrations or rewards that specify individual maneuvers.
B. Sparse Binary Task Rewards Benefit from Scale
- With beta sampling, evaluation success rises from 38.6% at 1K environments to 78.9% at 4K and 86.9% at 8K; dense-reward success stays at 11.8–12.0% over 1K–4K environments.The passage describes rapid improvement with increased parallel experience for the sparse binary task reward.Source: B. Sparse Binary Task Rewards Benefit… · p. 7
- At 256K environments, sparse-reward success is 97.4% with beta sampling versus 84.3% with uniform sampling; dense-reward success is 77.4% versus 34.3%.The passage reports these results as beta sampling’s additional gain over uniform reset sampling.Source: B. Sparse Binary Task Rewards Benefit… · p. 7
C. Emergent Strategies over Short Horizons
- The policy adapts grasps, reorients parts through surrounding geometry, and uses tapping and pivoting without demonstrations or rewards specifying those maneuvers.For example, it drops and regrasps a center-held peg near one end to expose enough shaft for insertion, and taps a nut to advance its thread.Source: C. Emergent Strategies over Short… · p. 7
D. Emergent Strategies over Long Horizons
In multi-pair episodes, AlphaNIST exhibits coordination without a prescribed task order or explicit planner. The paper’s results are simulation-only, and scaling the multi-pair framework to 20 parts remains future work.
D. Emergent Strategies over Long Horizons
- The policy can prepare one assembly, complete another, and return to finish the first, with reward only when every active pair is assembled and no prescribed order or explicit planner.Episodes allow 15 seconds per pair and terminate upon success; the policy’s actions for one assembly can adapt to neighboring parts.Source: D. Emergent Strategies over Long… · p. 8, §VII · p. 8
VI. LIMITATIONS
- All results are in simulation; hardware-matched velocity and torque limits do not establish robustness to perception error, latency, or contact-model mismatch.Extending the multi-pair framework to 20 parts remains a research goal.Source: §VI · p. 8
VII. CONCLUSION
- The conclusion argues that diverse experience under a shared objective can become a source of reusable dexterity and coordination, challenging the assumption that task complexity and diversity must be reduced.The paper frames this as a step toward policies that discover strategies rather than requiring them to be specified in advance.Source: §VII · p. 8