Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds

1 INTRODUCTION

• Structural representations can clarify why objects are designed or appear as they do, but are difficult to acquire automatically.Prior approaches have used manual annotations to attach structural information to object shapes.Source: §1 · p. 1
• Point clouds can support geometrically fitting but structurally ambiguous cuboid representations, causing abstraction to degenerate into surface fitting.Sparse points can be divided into arbitrary groups and fitted with different cuboids, leaving primitives without structural properties.Source: §1 · p. 2, Figure 2 · p. 2
• Joint segmentation uses dataset-level common structure to reduce ambiguity and degeneration in geometry-focused shape abstraction.The approach learns consistent point-to-cuboid assignments across shapes rather than relying directly on geometric proximity.Source: §1 · p. 2
• A VAE maps point clouds to latent codes and decodes them into parameterized cuboids, while high-level features support point-cuboid segmentation.The branches are jointly trained to keep segmentation and reconstructed cuboids consistent without structural annotations.Source: §1 · p. 2
• The framework supports abstract shape reconstruction, shape interpolation, virtual shape synthesis, and structural shape clustering.Its contributions include jointly learning cuboid abstraction and part segmentation without manual part or relationship annotations.Source: §1 · p. 2

2 RELATED WORK

• Supervised structure methods organize parts sequentially, flatly, or hierarchically, with many tree-based approaches requiring ground-truth part relationships.Other methods use recursive networks for shape generation, image-conditioned decoding, point-cloud segmentation, or controllable shape synthesis.Source: §2 · p. 2
• Unsupervised shape abstraction uses geometric primitives, but prior point-cloud methods rely on primitive-surface sampling for bidirectional Chamfer supervision.Earlier approaches include cuboid abstraction [Tulsiani et al. 2017; Sun et al. 2019] and multiple parametric primitives [Smirnov et al. 2020; Paschalidou et al. 2019]; this paper instead introduces a single-direction point-to-cuboid loss.Source: §2 · p. 3
• Unsupervised segmentation methods learn part consistency or correspondences, while the cited medial-axis approach does not model common structure across shapes.This paper jointly learns abstraction and segmentation while preserving structural consistency across instances.Source: §2 · p. 3

3 OUR APPROACH

• Each object point cloud is reconstructed as M cuboids parameterized by translation, rotation, and scale.The rotation is represented by a quaternion.Source: §3 · p. 3
• A fixed cuboid count and order across same-category objects provides part correspondence between instances.An existence indicator for each cuboid accommodates structural variation, such as chairs with or without armrests.Source: §3 · p. 3
• A VAE encodes each point cloud and decodes cuboid parameters, while an attention-based segmentation network builds point-cuboid correlations.The two branches are jointly trained with losses that enforce consistency between part segmentation and reconstructed cuboids without manual structural annotations.Source: §3 · p. 3

3.1 Feature embedding network

• Two EdgeConv layers produce 64D point features each, which are concatenated into 128D pointwise features.A fully connected layer and max pooling aggregate the point features into a 1024D global feature.Source: §3.1 · p. 3
• Two fully connected branches predict the latent Gaussian mean and standard deviation, each in R^512.The latent code is obtained by reparameterizing standard-normal noise using the predicted distribution.Source: §3.1 · p. 3

3.2 Shape abstraction network

• Each cuboid branch combines the latent code with a one-hot cuboid code to form a cuboid-specific feature that preserves implicit part correspondence.The fixed cuboid order and one-hot codes let the decoder encode both cuboid geometry and the structure of a specific part.Source: §3.2 · p. 3
• The feature embedding, shape abstraction, and cuboid-associated segmentation modules jointly produce a cuboid-based structural representation and point-cloud segmentation.Source: Figure 3 · p. 4
• A cuboid parameter module predicts each cuboid’s geometric parameters and existence probability, with its encoders and decoder sharing parameters across cuboids.Source: §3.2 · p. 4

3.3 Cuboid-associated segmentation network

• The branch treats point features and cuboid features as query and key vectors to compute an affinity between every point and cuboid.It first maps both feature types to 64D vectors.Source: §3.3 · p. 4
• A row-wise softmax turns each point’s cuboid affinities into a probability distribution over cuboid-associated parts.Source: §3.3 · p. 4
• Taking the argmax of the probabilistic point-to-cuboid allocation matrix yields the point-cloud segmentation result.Source: §3.3 · p. 4

3.4 Loss functions

• Four losses jointly supervise geometric coherence and compactness: reconstruction, cuboid compactness, cuboid existence, and latent-code KL divergence.The KL-divergence loss constrains the latent code to a standard Gaussian distribution.Source: §3.4 · p. 4

3.4.1 Reconstruction loss.

• The reconstruction loss weights point-to-cuboid distances by segmentation assignments, jointly optimizing cuboid parameters and allocation.It is a weighted single-direction distance that avoids differentiable cuboid-surface sampling and explicitly learns assignments; normal-matched surfaces and normal-direction sampling address degeneration.Source: §3.4.1 · pp. 4–5, Figure 4 · p. 5

3.4.2 Cuboid compactness loss.

• The compactness loss applies an L0.5 norm to point-allocation proportions to encourage sparse cuboid use, since L1 makes no update under their sum-to-one constraint.The loss penalizes using many cuboids, favoring concise representations.Source: §3.4.2 · p. 5, Figure 5 · p. 5

3.4.3 Cuboid existence loss.

• Cuboid existence targets are approximated from allocated-point proportions, with a threshold of 0.05, and trained using binary cross-entropy.Source: §3.4.3 · p. 5

3.4.4 Latent code KL-divergence loss.

• KL-divergence regularizes the 512D latent code toward an independent standard Gaussian, supporting high-order shape generation and manipulation.The network is trained end-to-end with the weighted sum of reconstruction, compactness, existence, and KL-divergence losses.Source: §3.4.4 · p. 5

3.4 Loss functions

• The method outperforms two state-of-the-art cuboid abstraction methods in Chamfer Distance reconstruction on four categories.Source: Table 1 · p. 6

4.1 Structured shape reconstruction

• Our method outperforms VP [Tulsiani et al. 2017] and HA on four object categories, producing more concise and precise reconstructions.The comparisons use symmetric L2 Chamfer Distance; the qualitative results highlight fine structures such as chair armrests and table-leg connection bars.Source: §4.1 · p. 6

4.2 Shape co-segmentation

• The method ranks second overall and best among cuboid-based approaches for co-segmentation, with consistent parts across instances and fine details segmented.The compared methods include VP, SQ, HA, BAE, and BSP; the best overall method, BSP-Net, uses more complicated shape primitives.Source: §4.2 · p. 7, Table 2 · p. 7, Figure 7 · p. 7

4.3 Ablation study and analysis

• P2C-Seg outperforms P2C-Dis and Chamfer-Dis on three categories; explicit segmentation allocations yield more compact abstractions than scattered hard assignments.The hard-assignment variants can get stuck with four chair legs merged into one cuboid.Source: §4.3 · p. 8, Figure 8 · p. 8, Table 3 · p. 8
• Normal-similar projection and random point sampling produce satisfactory reconstruction and better noise robustness than minimum-distance projection or no normal-direction sampling.Minimum-distance projection can degenerate into thin, stacked cuboids when normal consistency is ignored.Source: §4.3 · p. 8, Figure 9 · p. 9, Table 4 · p. 9
• Increasing the compactness-loss weight reduces used cuboids but can worsen reconstruction and lose structure; varying cuboid count also trades reconstruction fit against model size.Across weight settings, common parts remain structurally consistent among instances.Source: §4.3 · pp. 8–9, Figure 10 · p. 9, Table 5 · p. 9
• The method produces reasonable, consistent structural representations for chair point clouds sampled at 256, 1024, and 4096 points.The tested point densities yield similar abstraction and segmentation results for the same shape.Source: §4.3 · p. 9, Figure 11 · p. 9

4.4 Applications

• The learned representation supports plausible diverse shape generation, smooth interpolation between shapes, and clustering by shared cuboid-existence patterns.The clustering examples are table shapes whose geometric details vary while their structures remain common within each group.Source: §4.4 · p. 10, Figure 12 · p. 10, Figure 13 · p. 10

4.5 Failure cases, limitation and future works

• Unique structures may be reconstructed poorly, and uniformly sampled point clouds can leave fine parts with too little geometric information.The method also trains separately for each object category; learning across classes is identified as a future direction.Source: §4.5 · pp. 10–11, Figure 15 · p. 10

• The method uses shape co-segmentation to extract common structure within an object category and mitigate sparse-point-cloud ambiguity.Source: §5 · p. 11
• The authors report that their method preserves structural and semantic consistency in cuboid shape abstraction and point-cloud segmentation.Source: §5 · p. 11
• The generative network supports shape generation, point-cloud segmentation, shape interpolation, and structural classification.Source: §5 · p. 11