Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection

• Prior work often isolates segmentation, pose estimation, fixed targets, or simplified geometries, whereas this work targets broader physical assembly planning.The paper also identifies computational cost as a concern for reinforcement-learning approaches.Source: §I · p. 1
• The proposed planning pipeline generates trajectories with RRT-connect and optimizes contact points, while physical execution remains future work.The motion and contact stages are separated from downstream execution policies.Source: §I · pp. 1–2
• The framework plans assembly sequences, part movements, and contact points from target blueprints and assembly parts.These stages are illustrated through colored sequence numbers, motion arrows, and contact arrows.Source: Figure 1 · p. 1, §I · p. 2
• PAST autoregressively infers feasible assembly sequences from target-blueprint and part geometry information, supporting novel and diverse blueprints.The model predicts the next part and recursively generates the full sequence.Source: §I · p. 2
• D4PAS provides assembly trajectories, enumerated sequences, and viable contact points for training and future robotic-assembly studies.The dataset is constructed by enumerating feasible assembly sequences.Source: §I · p. 2

II. RELATED WORKS

• Prior methods span blueprint segmentation, pose regression, reinforcement learning, CAD precedence, and assembly-by-disassembly strategies.Their scopes include fixed or single-category targets and simplified geometric settings.Source: §II · p. 2
• Assembly-by-disassembly can simplify rigid-part planning by reversing disassembly sequences, but these methods often rely on time-consuming physical simulations.The paper uses GPU-based simulations and assembly-by-disassembly to generate D4PAS.Source: §II · p. 2
• After sequence inference, part motions and contact locations provide additional guidance for manipulation and execution.The paper builds on sampling-based motion planning and contact optimization while leaving physical execution for future work.Source: §II · p. 2

III. PROBLEM OVERVIEW

• The formulation assumes one moving part at a time while other parts remain stationary during trajectory planning.This assumption structures the assembly task into sequence, motion, and contact sub-problems.Source: §III · p. 2
• Feasible assembly sequences form a constrained subset of permutations because collisions can make arbitrary orders impossible.For example, a washer must be placed before a screw is tightened.Source: §III · p. 2
• PAST predicts the next feasible part from target and unassembled-part information, then iteratively determines the complete assembly sequence.The overall formulation sequentially solves assembly inference, motion planning, and contact selection.Source: §III · pp. 2–3

• The section introduces the multi-level approach to assembly planning.Source: §IV · p. 3
• The approach contains three planning levels whose details are developed separately.Source: §IV · p. 3
• The section transitions from the overall planning formulation to detailed treatment of each planning level.Source: §IV · p. 3

A. Part Assembly Sequence Transformer (PAST)

• PAST ingests target-blueprint and remaining-part point clouds, then iteratively selects the next part with maximum assembly probability until the sequence is complete.At each step, the selected part is removed from the remaining-part set.Source: A. Part Assembly Sequence Transformer… · pp. 3–4
• DGCNN encodes point-level target features and pooled feature vectors for each remaining part before transformer-based refinement.The target and part representations use positional and normal point-cloud features.Source: A. Part Assembly Sequence Transformer… · p. 3
• PAST refines target and part representations through self-attention followed by cross-attention between target and part features.The model applies these operations across L transformer blocks.Source: A. Part Assembly Sequence Transformer… · p. 3
• The model computes P(Mi) with an MLP and chooses mk = argmaxi P(Mi), while also regressing each part’s 6D assembled pose as an auxiliary task.The auxiliary pose objective is intended to improve geometric interrelation modeling.Source: A. Part Assembly Sequence Transformer… · pp. 3–4
• PAST is trained with supervised losses for assembly feasibility probabilities and predicted-versus-target assembled poses.The probability task uses mean squared error, while pose error covers translation and axis-angle rotation.Source: A. Part Assembly Sequence Transformer… · p. 4

B. Dataset for Part Assembly Sequences (D4PAS)

• D4PAS samples contain target-blueprint point clouds, remaining-part point clouds, and feasibility labels for parts at each assembly step.The labels indicate whether a part can be assembled at the current step and still lead to a successful final assembly.Source: B. Dataset for Part Assembly Sequences… · p. 4
• The dataset enumerates all feasible sequences, although multiple viable next-part candidates may exist at each step.Figure 3 presents one representative sequence for each target blueprint, with colors and numbers indicating order.Source: B. Dataset for Part Assembly Sequences… · p. 4, Figure 3 · p. 4
• The disassembly planner searches over part poses, simulates removing candidate parts, and reconstructs assembly sequences from successful states.Breadth-first search expands executable novel poses and considers all possible dragging forces for remaining parts.Source: B. Dataset for Part Assembly Sequences… · p. 4
• Disassembly applies forces along each part’s moment of inertia and uses corresponding torque to enable rotational movement.The force is defined from the part mass, inertia, and a one-hot axis vector.Source: B. Dataset for Part Assembly Sequences… · p. 4
• Compared with, the generator adds parallel Isaac Gym simulation, inertia-aligned forces, and robot-execution constraints.The constraints require collision-free grasp or push points, making generated plans more suitable for robotic applications.Source: B. Dataset for Part Assembly Sequences… · p. 4

C. Part Motion and Contact Planning

• RRT-connect searches for collision-free trajectories that move each part from its resting pose to its assembled pose in sequence order.The planner operates after PAST determines the assembly sequence.Source: C. Part Motion and Contact Planning · p. 4
• Two-contact grasps are enumerated and scored with Ferrari-Canny metrics, then filtered using collision checks along the assembly trajectory.FCL verifies collisions between grasp points and assembled parts.Source: C. Part Motion and Contact Planning · pp. 4–5
• When no feasible grasp exists, the method optimizes a single pushing point and force using friction-cone, grasp-map, and movement-force constraints.The grasp map G(c) maps contact force to part movement force, while Fc is constrained by the friction cone at c.Source: C. Part Motion and Contact Planning · p. 5
• The pushing optimization becomes a semidefinite program for a fixed contact point and is solved by sampling points from the part cloud and comparing optimized outcomes.The hybrid procedure updates results across sampled pushing points to identify the best solution.Source: C. Part Motion and Contact Planning · p. 5

V. EXPERIMENTS

• The experiments section is described as validating the model and dataset against related methods.No experimental findings or comparison values are supplied in the passage.Source: §V · p. 5

Dataset

• 8,670 target blueprints yield 84,326 assembly sequences spanning three-step, four-to-seven-step, and longer assemblies.The dataset also augments sequences by varying unassembled-part choices and identifying all viable next parts.Source: A. Dataset and Network Details · p. 5

Network Details

• PAST training samples target blueprints at 1,024 points and each part mesh at 512 points, using a 256-dimensional DGCNN encoding.The model uses eight transformer blocks and is implemented in PyTorch.Source: A. Dataset and Network Details · p. 5

Evaluation

• Table I evaluates assembly sequence inference with one-step accuracy, full-sequence accuracy, and computation time.These metrics are denoted 1-Acc, Seq-Acc, and CT.Source: Table I · p. 5

Training Setup

• The experiments reserve 500 multi-part assemblies for testing while preprocessing blueprints through recentering, unit-ball normalization, rotation, and jitter augmentation.Training uses AdamW with a One-Cycle learning-rate scheduler.Source: A. Dataset and Network Details · p. 5

Evaluation Protocol

• The study measures one-step accuracy, full-sequence accuracy, and computation time against adapted baselines because no prior method directly targets part assembly sequence inference.A full sequence is correct when it matches any valid dataset sequence.Source: B. Baselines and Ablations · p. 5

Baselines

• NSM adapts a transformer for two-part shape mating to accommodate multiple part inputs.It is derived from Neural Shape Mating.Source: B. Baselines and Ablations · p. 5
• DGL adapts graph-based assembly pose regression by adding a global node to represent the target assembly.This adaptation supports assembly sequence inference.Source: B. Baselines and Ablations · p. 5
• ATA solves assembly through runtime physical simulation, following the assembly process represented by Algorithm 1.It serves as a simulation-based comparison method.Source: B. Baselines and Ablations · p. 5

Ablations

• Seg-PAST replaces PAST’s final pose-regression layer with blueprint segmentation during pretraining, while NoAux-PAST removes auxiliary pose regression.Both variants retain the same core network for sequence prediction.Source: B. Baselines and Ablations · p. 5

Multi-level assembly planning

• 82.9% assembly success is achieved in 1,565.9 ms across 500 multi-part assemblies, compared with 63.9% for RL-based assembly planning.Motion planning averages 1,476.9 ms and contact optimization 36.8 ms, each with 100% success in the reported setting.Source: C. Experimental Results · p. 6

Generalization to novel assemblies

• PAST consistently outperforms NSM, DGL, Seg-PAST, and NoAux-PAST on one-step and full-sequence prediction tasks.The reported explanation attributes this to direct target-blueprint features and point-level feature aggregation.Source: C. Experimental Results · p. 6

Ablation results

• Auxiliary pose regression or part segmentation significantly improves PAST performance relative to the ablated variants.Pose regression performs better than target segmentation, possibly because assembled target points lack supervision.Source: C. Experimental Results · p. 6

Sensitivity analysis

• Increasing the number of parts reduces sequence accuracy and increases auxiliary pose-regression error.Figure 5 presents this sensitivity analysis, which the authors relate to greater assembly complexity.Source: C. Experimental Results · p. 6, Figure 5 · p. 6

Disassembly planning

• GPU-accelerated parallel disassembly reduces planning time from 24,312.6 ms to 3,592.0 ms, while part-centric forces raise success from 83.8% to 92.7%.These changes support enumeration of feasible assembly sequences within a manageable timeframe.Source: C. Experimental Results · p. 6

Contributions

• The framework combines assembly-sequence inference, part-motion planning, and contact-point optimization for blueprint-based assembly.PAST infers feasible sequences from target blueprints and part geometries before motion and contact planning.Source: §VI · p. 6
• D4PAS provides thousands of physically validated assembly sequences as a benchmark for part assembly planning.The dataset includes assembly trajectories, enumerated sequences, and viable contact points.Source: §VI · p. 6

Limitations

• The approach assumes one moving part at a time, which may limit practical robotic execution.Additional mechanisms may be needed to hold the assembly.Source: §VI · p. 6
• The study does not address robotic execution and assumes ideal part sensing and localization.Future work targets real-world robotic assembly and tool use.Source: §VI · p. 7