Generative 3D Part Assembly via Dynamic Graph Learning

1 Introduction

• Part assembly is laborious because it requires exploring a large solution space and satisfying many relations and constraints for physical stability.The introduction compares the task to assembling furniture without procedural guidance.Source: §1 · p. 1
• The method predicts each input part’s 6-DoF pose through iterative, coarse-to-fine refinement, dynamically updating part features and relations in a graph.Its key modules are dynamic part relation reasoning and dynamic part aggregation.Source: §1 · p. 2
• On synthetic PartNet data, the approach outperforms three strong baselines by significant margins despite no prior work addressing the exact setting.The setting uses fine-grained input geometry with variable part counts and no semantic knowledge.Source: §1 · p. 2
• Central parts such as chair backs and seats learn faster and sketch the shape backbone before peripheral parts adjust through graph message passing, without direct supervision.The authors present this emergent behavior as evidence of the dynamic graph framework’s effectiveness.Source: §1 · p. 2

2 Related Work

• Earlier methods construct shapes from mesh repositories, learn probabilistic graphical models of component relationships, or assemble shapes conditioned on scans, artist models, or partial shapes [7] [30].Source: §2 · p. 2
• Data-driven methods generate parts and predict transformations, often using accumulated shape priors and assuming canonical part rotations; this work instead poses given detailed parts, while [17] requires image guidance.Source: §2 · p. 2
• Structure-aware generative networks model joint structural and geometric variation, whereas this work targets structural variation conditioned on fixed detailed part geometry [15] [21] [22] [8] [35] [23].Source: §2 · p. 3

3 Assembly-Oriented Dynamic Graph Learning

• The method predicts each input part's 6-DoF pose and assembles the transformed parts into a complete 3D shape.Its framework combines iterative coarse-to-fine refinement with dynamic relation reasoning and part aggregation.Source: §3 · p. 3

3.1 Iterative Graph Neural Network Backbone

• The backbone represents each part as a node and iteratively updates edge and node features to refine poses over five coarse-to-fine message-passing iterations.Point-cloud features initialize the nodes, and the graph is fully connected.Source: §3.1 · pp. 3–4, Figure 1 · p. 4

3.2 Dynamic Relation Reasoning Module

• At each step, estimated poses explicitly inform evolving part relations, which then contribute to subsequent pose refinement.The method reasons about the current assembly, including part disconnections and overall shape geometry.Source: §3.2 · pp. 4–5

3.3 Dynamic Part Aggregation Module

• The aggregation module alternates between dense part nodes and sparse nodes that combine geometrically equivalent parts, then unpools features back to the dense set.This enables information exchange among equivalent parts while allowing their predicted poses to diverge.Source: §3.3 · p. 5

3.4 Training and Losses.

• The Min-of-N loss balances assembly quality and diversity by encouraging at least one noise-conditioned prediction to approach the ground truth.The implementation samples five noise particles and combines local part and global shape losses.Source: §3.4 · p. 6

4.1 Dataset

• The evaluation uses PartNet chairs, tables, and lamps with default splits, fine-grained segmentation, and 1,000 sampled points per part.The dataset contains 6,323 chairs, 8,218 tables, and 2,207 lamps; parts are zero-centered in canonical PCA spaces.Source: §4.1 · p. 6

4.2 Baseline Approaches

• The three baselines adapt part retrieval, sequential LSTM pose estimation, and one-shot pose regression with global shape features to the assembly setting.Baseline training uses the same losses and termination strategy as the proposed method.Source: §4.2 · pp. 6–7

4.3 Evaluation Metrics

• Minimum Matching Distance evaluates shape fidelity across multiple noise-sampled assemblies, alongside part accuracy, shape Chamfer distance, and connectivity accuracy.Part accuracy measures the percentage of parts matching ground-truth parts within a Chamfer-distance threshold of 0.01; connectivity accuracy evaluates correctly connected parts.Source: §4.3 · p. 7

4.4 Results and Comparisons

• The algorithm outperforms all compared baseline approaches by a significant margin on most columns, especially part and connectivity accuracy, and achieves the best assembly results in visual comparisons.Source: §4.4 · p. 7
• Ablations show the full model performs best and that the iterative GNN, dynamic relation reasoning, and dynamic aggregation contribute to performance.For Table part accuracy, five iterations score 49.51 versus 45.51 for three and 44.6 for seven; more input points and max pooling also improve performance, and all three loss terms contribute.Source: §4.4 · p. 8

4.5 Dynamic Graph Analysis

• Relation weights shift with graph updates: even iterations emphasize messages from central parts to peripheral parts, while odd iterations show relatively greater influence from peripheral parts.The even-step graph is updated from dense nodes; the odd-step graph aggregates geometrically equivalent parts into sparse nodes.Source: §4.5 · p. 8, Figure 3 · p. 9
• Averaged relation weights are higher for central than peripheral parts, supporting the analysis that central parts guide assembly.The reported average emitting relation weights are 1.22 for central parts and 0.94 for peripheral parts.Source: §4.5 · p. 9
• Across chair iterations, central parts are more accurate, while peripheral-part accuracy improves relatively more at even iterations, consistent with central parts guiding peripheral poses.The time-varying assemblies show central poses determined first, followed by peripheral adjustments, before predictions converge to stable assemblies.Source: §4.5 · p. 9, Figure 4 · p. 9

5 Conclusion

• The method achieves state-of-the-art performance over three baseline methods by learning an effective assembly-oriented relation graph.Source: §5 · p. 9
• An iterative GNN dynamically evolves node and edge features within the part graph, alongside dynamic relation reasoning and part aggregation modules.Source: §5 · p. 9
• Figure 5 summarizes part accuracy and its relative improvement at each time step.Source: Figure 5 · p. 9

Broader Impact

• The authors say part-assembly research could enable robots to free people from manual assembly work and benefit the economy and society.Source: Broader Impact · p. 10
• Because human trust in machines remains a problem, the authors say assembly algorithms should assist and cooperate with humans rather than fully replace them.Source: Broader Impact · p. 10

6 Appendix

• The appendix covers additional ablations, dynamic-graph analysis, training details, failure cases, structural variation, and qualitative results.Source: §6 · p. 10

A. Additional ablation study

• The tested variants replace graph learning with an MLP, add either dynamic module, exchange node-set order, use ground-truth adjacency, or reason from transformed point clouds.Source: A. Additional ablation study · p. 10
• The iterative GNN backbone, dynamic relation reasoning, and dynamic part aggregation each contribute significantly to assembly quality.Source: A. Additional ablation study · p. 10, Table 3 · p. 11
• Exchanging sparse and dense node-set order makes little difference, while replacing learned relations with binary adjacency or geometry-based reasoning degrades performance.Source: A. Additional ablation study · pp. 10–11

B. Additional analysis of the dynamic graph

• In eight-part chairs, the back, seat, and head emit larger relation weights than peripheral parts.Source: B. Additional analysis of the dynamic… · p. 11
• Peripheral chair parts, including legs, arms, and footrests, have larger received relation weights.Source: B. Additional analysis of the dynamic… · p. 11
• Similar central-emitter and peripheral-receiver patterns appear in the lamp and table categories, each shown with four parts.Source: B. Additional analysis of the dynamic… · p. 11

C. Training details

• The graph network runs for five iterations because performance approaches saturation there, with no obvious improvement from additional iterations.Pose supervision is applied at every time step to accelerate convergence.Source: C. Training details · p. 11
• Geometrically equivalent parts are filtered by bounding-box dimension differences above 0.1, then by pairwise Chamfer distances below 0.2.Source: C. Training details · p. 12

D. Failure cases and future work

• Missing hard physical connection constraints can leave parts floating or disconnected, motivating future joint-centric reasoning about relative displacement and rotation.The authors describe the current method as part-centric and its learned relations as soft.Source: D. Failure cases and future work · p. 12, Figure 7 · p. 12

• With part geometry provided, the modeled diversity is structural; some shapes admit different assemblies, while others have only one deterministic result.Source: E. Additional results of structural… · p. 12
• The top five examples show structural variation, while the bottom three predict the same assembly results because the input part set is limited.Source: E. Additional results of structural… · p. 12

• Additional qualitative comparisons cover the PartNet Chair, Table, and Lamp datasets.Source: F. Additional qualitative results · p. 12, Figure 9 · p. 14, Figure 10 · p. 15
• Additional generated examples show structural variation in some assemblies and no variation in cases with very limited parts.Source: Figure 8 · p. 13