PaperlayerView exact revision on arXiv

Source-linked AI summary

ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling

Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas

arXiv:1708.01841v2cs.GR

TL;DR

Choosing compatible components and placements for partial assemblies is difficult when repositories lack consistent part labels and segmentations. ComplementMe jointly learns part embeddings, probabilistic retrieval, and placement from unlabeled repository decompositions, achieving stronger benchmark retrieval than state-of-the-art alternatives while supporting low-input modeling. Its limitations include sensitivity to small/thin components and possible placement violations.

  • Problem

    Choosing compatible components and placements for partial assemblies is difficult when repositories lack consistent part labels and segmentations.

    Source: §1 · p. 1

  • Method

    ComplementMe jointly learns part embeddings, probabilistic retrieval, and placement from unlabeled repository decompositions.

    Source: §5 · p. 4, §5.1 · p. 5

  • Results

    The method outperforms state-of-the-art retrieval techniques and supports a modeling tool requiring minimal or no user input.

    Source: §1 · p. 2, §6.1 · p. 6, §6.2 · pp. 6–7

  • Takeaways & Limitations

    The framework enables interactive and fully automatic assembly-based modeling with diverse component proposals and predicted placements.

    Source: §1 · p. 2, §6.1 · p. 6, §7 · p. 10

    Small or thin components may be underrepresented, and placement predictions may float or overlap components, requiring user correction.

    Source: §7 · p. 10

Abstract

from arXiv · show

Assembly-based tools provide a powerful modeling paradigm for non-expert shape designers. However, choosing a component from a large shape repository and aligning it to a partial assembly can become a daunting task. In this paper we describe novel neural network architectures for suggesting complementary components and their placement for an incomplete 3D part assembly. Unlike most existing techniques, our networks are trained on unlabeled data obtained from public online repositories, and do not rely on consistent part segmentations or labels. Absence of labels poses a challenge in indexing the database of parts for the retrieval. We address it by jointly training embedding and retrieval networks, where the first indexes parts by mapping them to a low-dimensional feature space, and the second maps partial assemblies to appropriate complements. The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input. Thus, instead of predicting a single output, we train our network to predict a probability distribution over the space of part embeddings. This allows our method to deal with ambiguities and naturally enables a UI that seamlessly integrates user preferences into the design process. We demonstrate that our method can be used to design complex shapes with minimal or no user input. To evaluate our approach we develop a novel benchmark for component suggestion systems demonstrating significant improvement over state-of-the-art techniques.

1 INTRODUCTION

ComplementMe targets component suggestion for 3D assembly without explicit part annotations, using inconsistent unlabeled components from online repositories. It jointly learns complementary-part retrieval and placement, and outperforms state-of-the-art retrieval techniques while supporting minimal or no user input.

1 INTRODUCTION

  • Explicit part annotations are costly, motivating component suggestion from unlabeled decompositions of online 3D repositories.Prior part-wise assembly interfaces relied on heavily supervised, consistently segmented and semantically labeled datasets.Source: §1 · pp. 1–2
  • The method jointly learns an embedding network for indexing parts and a retrieval network that predicts multiple plausible complements as a probability distribution.A placement network then predicts locations for selected complementary components.Source: §1 · p. 2
  • The approach supports a modeling tool requiring minimal or no user input and outperforms state-of-the-art retrieval techniques without heavily curated datasets.The paper also introduces a benchmark for component suggestion methods.Source: §1 · p. 2

2 RELATED WORK

Prior assembly systems commonly depend on consistently segmented, labeled repositories, while ComplementMe learns complementary parts and placements with neural representations from less curated data. Its retrieved-part representation is designed for high-fidelity incremental assembly rather than geometry synthesis from scratch.

2 RELATED WORK

  • Earlier part-based completion methods rely on consistently segmented shapes with part labels, limiting applicability because annotation is expensive.Chaudhuri et al. [2011] and related systems use heavily curated repositories, while Yi et al. [2016] document annotation costs.Source: §2 · p. 2
  • Chaudhuri and Koltun [2010] avoid labels but use handcrafted descriptors and assumptions about coarse completeness and stable placement, whereas ComplementMe learns complementary parts and positions.The learned representation is intended to transfer across shape categories without manual parameter tuning.Source: §2 · p. 2
  • ComplementMe applies neural networks to incremental interactive assembly and retrieves repository components instead of synthesizing geometry from scratch, enabling high-fidelity meshes.This distinguishes its output representation from neural methods that synthesize volumes, skeletons, graphs, or point clouds.Source: §2 · p. 2
  • The paper introduces an embedding specialized for retrieval by placing complementary components from similar contexts nearby and mapping partial objects to their complements.This differs from prior embeddings designed for reconstruction, denoising, synthesis, or hierarchical part detection.Source: §2 · p. 2

3 OVERVIEW

The system turns partial assemblies into diverse complementary-component proposals and predicted placements using point-cloud networks. It addresses ambiguous retrievals through a learned embedding and probability distribution, then applies a placement network to each proposal.

3 OVERVIEW

  • The test-time pipeline samples complementary components from the predicted embedding distribution and then predicts their placements relative to the query assembly.The distribution is modeled as a mixture of Gaussians.Source: Figure 2 · p. 3
  • Given a partial object, the system proposes diverse compatible components and placements intended to preserve stylistic and functional alternatives for the designer.The proposals should form a valid shape with the partial assembly.Source: §3 · p. 3
  • Training data pairs connected partial assemblies with adjacent complementary components from decomposed repository shapes, without requiring consistent segmentation, correspondences, or labels.A proximity-based graph avoids training on disconnected examples.Source: §3 · p. 3
  • Because one partial assembly can have multiple valid outputs, the retrieval network models P(Y |X) as a Gaussian mixture over a learned low-dimensional part-embedding space.Contrastive loss jointly trains retrieval and embedding networks so compatible parts are nearby.Source: §3 · p. 3
  • A regression placement network takes the partial object and retrieved component as input and predicts the component's position.At test time, retrieval produces high-probability components and placement generates a gallery of assembly candidates.Source: §3 · p. 3

4 DATA PREPROCESSING

Preprocessing converts repository geometry into contact graphs whose components are large enough and locally meaningful for training. Small, overlapping, and repetitive elements are merged before graph partitioning creates partial-assembly/complement pairs.

4 DATA PREPROCESSING

  • The preprocessing pipeline builds contact graphs and partitions connected subgraphs into partial assemblies with adjacent complementary components.Later networks do not require the resulting components to be consistent or labeled.Source: §4 · p. 3
  • Components are initialized from reasonable scene-graph geometry nodes, then small elements are merged to improve geometric boundaries and embedding discrimination.Larger components also make additions more visible in the assembly.Source: §4 · p. 4
  • Merging uses proximity, size, overlap, and similarity criteria, including τproximity = 0.05, τsize = 0.2, and τHausdorff tol = 0.05.Identical or symmetric components can be grouped, and training graphs contain at most Nmax CC = 8 components.Source: §4 · p. 4
  • Components are represented for retrieval and placement by Npoints = 1000 randomly sampled points recentered at the origin.Source: §4 · p. 4
  • Figure 3 contrasts input connected components with final components after small, overlapping, and repetitive elements are grouped.Source: Figure 3 · p. 4

5 METHOD

The method retrieves multiple complementary components for a partial assembly by jointly learning an embedding space and a probabilistic retrieval model, then predicts placement coordinates for selected components.

5 METHOD

  • The system combines retrieval, embedding, and placement networks to propose components that can be added to a partial shape.Point clouds represent the input geometry, and the output consists of several database components with predicted placements.Source: §5 · p. 4

5.2 Embedding Network

  • The embedding network jointly learns to place interchangeable, stylistically compatible components nearby so proximity search can retrieve suitable complements.Joint learning couples the embedding with the retrieval distribution and clusters candidates sharing stylistic and functional attributes.Source: §5 · p. 4, §5.2 · p. 5

5.1 Retrieval Network

  • The retrieval network predicts a Gaussian-mixture distribution over a 50-dimensional embedding space because one partial assembly can have multiple valid complements.The mixture uses as many modes as the maximum number of connected components and represents each mode with a weight, mean, and standard deviation.Source: §5.1 · p. 5

5.3 Joint Training for Retrieval and Embedding

  • Training uses positive and negative components in a triplet contrastive loss to prevent the embedding from collapsing to a single point.Positive components are adjacent to a sampled connected subgraph, while negatives are non-adjacent; the margin is set to m = 10.Source: §5.3 · p. 5

5.4 Placement Network

  • The placement network predicts translation coordinates from the partial assembly and retrieved component while preserving the component’s source orientation.It concatenates features from separate PointNet networks processing the assembly and component, then predicts 3D coordinates.Source: §5.4 · p. 6

6 RESULTS

The paper demonstrates interactive and automatic assembly tools, then evaluates complement retrieval, embeddings, geometric compatibility, style compatibility, and placement using a dedicated benchmark. Results show strong functional retrieval and placement performance, while style retrieval depends on how complete the partial query is.

6.1 Assembly-Based Geometric Modeling

  • Interactive modeling samples complementary components for user selection, predicts their placement, and iteratively synthesizes the growing shape.Automatic synthesis instead samples components repeatedly from a random initial component, supporting diversity and complexity control through branching choices.Source: §6.1 · p. 6

6.2 Quantitative Evaluations and Comparisons

  • The proposed benchmark evaluates complement retrieval, part embedding, and part placement because end-to-end assembly quality is difficult to assess objectively.The evaluation compares functional, geometric, and stylistic compatibility against random suggestions and state-of-the-art alternatives.Source: §6.2 · p. 6
  • Our method outperforms CK10 on functional-label retrieval across all evaluated categories except guitars, where both methods perform well.The experiment uses 100 queries per category and measures top-5 retrieval mAP on six ShapeNet categories with part annotations.Source: §6.2 · p. 7
  • The method retrieves geometrically more similar missing parts than CK10, although it performs slightly worse on guitars.Geometric compatibility is measured by average Hausdorff distance between top-five retrieved parts and the excluded original component, with shapes normalized to unit radius.Source: §6.2 · p. 8, Table 2 · p. 8
  • Style retrieval is better with only one part given, whereas CK10 performs better when the query is nearly complete.The style metric uses top-five mAP over fine-grained ShapeNet classes and can differ from compatibility between individual retrieved parts.Source: §6.2 · p. 9, Table 3 · p. 9
  • The learned embedding performs on par with or slightly better than MVCNN features, while producing diverse plausible components and enabling placement prediction from new-component geometry.Placement errors are reasonably small across categories, and cross-category synthesis works best when categories share components, such as tables, sofas, and chairs.Source: §6.2 · p. 9

7 CONCLUSION

The paper presents a component retrieval and placement method that learns from inconsistent, unlabeled repository components and supports minimally interactive assembly modeling. It also identifies retrieval and placement failures that user interaction can correct.

7 CONCLUSION

  • The method jointly learns complementary-part retrieval and low-dimensional organization from non-curated, inconsistently segmented online-repository components.It is designed to retrieve parts with functional, geometric, and stylistic compatibility.Source: §7 · pp. 9–10
  • It predicts target part positions from normalized geometry, extending component suggestion beyond retrieval alone.Source: §7 · p. 10
  • The framework can produce automatically synthesized shapes with minimal user input, but some outputs contain conflicted or unmatched components.The paper attributes these failures partly to small or thin components being underrepresented during retrieval.Source: §7 · p. 10
  • The placement network can violate physical constraints by producing floating or overlapping components, although user interaction can fix these issues.Source: §7 · p. 10
  • Future work will synthesize or deform retrieved parts and provide user control over high-level attributes, rough shapes, and rough placements.Source: §7 · p. 10

A APPENDIX

The appendix expands Equation 1 into a log-sum-exp form and introduces a computation trick for numerical stability. The derivation operates dimension by dimension over the embedding values.

A APPENDIX

  • Equation 1 is expanded into a log-likelihood expression over mixture components and embedding dimensions.Source: A APPENDIX · p. 12
  • The resulting log-sum-exp computation uses a numerical-stability trick based on the maximum dimension value.The appendix defines x_d as the d-th dimension value of x.Source: A APPENDIX · p. 12

Related papers

Loading 1708.01841v2…