SDM-NET: Deep Generative Network for Structured Deformable Mesh

• Triangle meshes preserve compactness, fine surface details, controllability, and visual quality, but their irregular tessellations hinder conventional deep-learning operations.Source: §1 · p. 1
• Existing mesh generative models are limited by genus-zero topology, shared connectivity, or seams and incoherence from unstructured patches.Source: §1 · p. 1
• SDM-NET decomposes complex shapes into box-homeomorphic parts whose fine geometry is recovered through deformation.Source: §1 · p. 1
• Its two-level VAE uses PartVAE for part geometry and SP-VAE to jointly learn global part structure and local geometry.Source: §1 · p. 2
• SDM-NET generates closed deformable mesh parts arranged according to global relations such as symmetry and support, with a varying number of parts up to a maximum.Source: §1 · p. 2
• Experiments report quality meshes, flexible structure, fine details, plausible structure-changing interpolation, and support for mesh editing and related modeling tasks.Source: §1 · p. 2

Voxel grids

• Voxel representations are CNN-friendly but impose high memory costs at high resolution and limit geometric detail through voxel or octree resolution.Source: §2 · p. 2

Multi-view images

• Multi-view images are regular and efficient, but omit full 3D information and may fail to recover geometric details without reconstruction processing.Source: §2 · p. 2

Point clouds

• Point clouds flexibly represent scanner data, but their irregular structure creates challenges for deep learning.Source: §2 · p. 2

Meshes and multi-chart representations

• Template-based mesh deformation captures fine details but requires shared connectivity, while multi-chart methods remain constrained by topology or produce seams and disconnected patches.Source: §2 · pp. 2–3

Implicit representations

• Implicit representations support diverse shape tasks including autoencoding, generation, interpolation, completion, and single-view reconstruction.Source: §2 · p. 3

Shape structures

• Earlier structure-generation methods use probabilistic models or separate structure and geometry networks, whereas SDM-NET jointly encodes both and produces deformable mesh parts.Source: §2 · p. 3
• Compared with GRASS, SDM-NET handles support information, requires consistent semantic segmentation rather than symmetry hierarchies, and outputs deformable meshes instead of voxel-filled bounding boxes.Source: §2 · p. 3

3 METHODOLOGY

• The method represents each labeled shape as a structured set of deformable bounding boxes, allowing coarse boxes to capture part structure while deformation recovers geometry details.Each part is modeled from an axis-aligned bounding box and a common unit-cube template.Source: §3 · p. 3, §3.1 · p. 4, §3.2 · p. 4

3.1 Encoding of a Shape Part

• Each part code combines existence, support, symmetry, position, and PartVAE geometry information into a representation vector.The code includes relations indicating which parts support or are supported by the current part, together with symmetry and geometric deformation information.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 PartVAE for Encoding Part Geometry

• PartVAE encodes fixed-connectivity, genus-zero part meshes as deformations of template boxes and learns a separate model for each part type.Its input describes local vertex-neighborhood deformation using rotation axes, rotation angles, and scaling factors, then reconstructs the deformation representation.Source: §3.2 · pp. 4–5, Figure 4 · p. 5

3.3 Supporting Structure Analysis

• Support and symmetry relations provide the structural representation, with detected support relations converting part adjacency into a directed support graph.The support encoding accommodates multiple parts supporting one part and one part being supported by multiple parts.Source: §3.3 · p. 5, Figure 5 · p. 5

3.4 SP-VAE for Structured Deformable Mesh Encoding

• SP-VAE jointly encodes the layout, structural relations, and PartVAE geometry codes so generated part geometry remains coherent with global structure and compatible styles.The concatenated part representation is encoded in a consistent order, and the Gaussian latent space supports sampling and interpolation.Source: §3.4 · p. 6

3.5 Shape Generation and Refinement

• Refinement optimization enforces symmetry, equal-length, stable-support, and support constraints to produce more plausible and physically valid generated shapes.The optimization addresses typical structural artifacts while balancing its objective terms with α fixed to 10 in the experiments.Source: §3 · p. 4, §3.5 · p. 6

Data

• The dataset combines pre-aligned ShapeNet Core V2 and ModelNet meshes, but ModelNet lacks semantic segmentation.Some source models also lack sufficiently detailed segmentation for support-structure descriptions.Source: §4.1 · p. 7
• SDM-NET requires consistently segmented inputs, registers each part from its bounding box by non-rigid deformation, and supports substantial geometric variation within part types.Test-shape segmentation can be obtained with supervised methods such as PointNet++.Source: §4.1 · p. 7

Network

• The network contains PartVAE for per-part deformation encoding and SP-VAE for jointly encoding global structure and part geometry.PartVAE is trained separately for each part type, while SP-VAE encodes the combined part representation.Source: §4.2 · p. 7
• SP-VAE uses a concatenated all-part representation with fully connected layers of dimensions 1024, 512, and 256, followed by a 128-dimensional latent space.Leaky ReLU is used as the activation function.Source: §4.2 · p. 7

Evaluation

• The experiments use a 75% training and 25% test split for tables, and average bidirectional Chamfer reconstruction errors across 10 evaluations.The lowest reported SP-VAE error uses λ1 = 1.0 and λ2 = 0.5.Source: §4.3 · pp. 7–8

Efficiency

• Generating one shape takes about 36 milliseconds, while structure optimization takes about 100 milliseconds.Training both VAEs takes about 300 minutes for a typical category.Source: §4.4 · pp. 8–9

Shape Reconstruction

• SDM-NET reconstructs finer-detailed geometry than PSG, AtlasNet, and Adaptive O-CNN, whose outputs exhibit coarseness or patch artifacts.Adaptive O-CNN recovers planar chair regions but not curved regions such as the chair-back top.Source: §5 · p. 9
• SDM-NET achieves the best performance on nearly all reported JSD, Coverage, and MMD metrics.MMD and Coverage are evaluated with both Chamfer Distance and Earth Mover’s Distance.Source: §5 · pp. 9–10

Shape Generation

• Compared with GRASS, SDM-NET produces physically stable and well-connected results, avoiding structural noise such as disconnections and asymmetries.The paper attributes GRASS’s noise partly to inconsistent automatically generated symmetry hierarchies for new inputs.Source: §5 · p. 10
• SDM-NET generates novel monitor shapes by combining structural and part information from SP-VAE with deformation information from PartVAE.The monitor examples use training data from ModelNet.Source: §5 · p. 10

Shape Interpolation

• Latent-space interpolation produces smoother and more realistic transitions than AtlasNet, including natural gradual disappearance of chair armrests.Jointly training geometry and structure learns their implicit joint distribution.Source: §5 · p. 10, Figure 16 · p. 10
• SDM-NET interpolates naturally between shapes with different topologies and substantial geometric changes.These results are attributed to its flexible structure representation.Source: §5 · p. 10, Figure 17 · p. 10

Ablation Studies

• Support constraints yield physically more stable results, while decoupling structure and geometry produces inconsistent part styles.Joint encoding makes generated part geometry consistent with structural interpretations such as sofas.Source: §5 · pp. 11–12
• Training separate PartVAEs for each part type captures unique geometric features better than a single PartVAE trained across all categories.The single-PartVAE alternative leads to poor geometric reconstruction.Source: §5 · p. 12

Conclusions

• SDM-NET represents shapes as deformable boxes and uses a two-level VAE for local part geometry and global structure.The PartVAE encodes individual part geometry, while the structural VAE encodes the structure and geometries of all parts.Source: §6 · p. 13
• The representation achieves flexible topology and fine geometric details while outperforming state-of-the-art methods for shape generation and interpolation.Source: §6 · p. 13

Future Work

• The method could be extended to reconstruct 3D shapes from images by conditioning the SP-VAE on learned image part segmentations.Source: §6 · pp. 13–14

Limitations

• Current failures include parts with holes, complex chair legs, and grid backs because every part must have fixed genus-zero-box topology.Source: Figure 26 · p. 14
• Fixed high-resolution primitives preserve geometric detail but produce large storage requirements, motivating adaptive resolutions for parts with different geometric richness.Source: §6 · p. 14

Support Relationship Constraint

• Projected bounding boxes of parts i and j must satisfy either ˜bi ⊆ ˜bj or ˜bj ⊆ ˜bi.Source: APPENDIX: SUPPORT RELATIONSHIP… · p. 15

Optimization

• The nesting requirement is formulated as an integer programming problem and solved efficiently during optimization.Source: APPENDIX: SUPPORT RELATIONSHIP… · p. 15

Integer Formulation

• A large positive constant M makes the constraint conditional on binary variables δi,j.M is chosen larger than any possible shape coordinate.Source: APPENDIX: SUPPORT RELATIONSHIP… · p. 15
• The formulation requires at most one of δi,j^1 and δi,j^2 to equal 1, so at least one variable is 0.Source: APPENDIX: SUPPORT RELATIONSHIP… · p. 15

Containment Cases

• When δi,j^1 = 0, the equations enforce ˜bi ⊆ ˜bj; when δi,j^2 = 0, they enforce ˜bj ⊆ ˜bi.Source: APPENDIX: SUPPORT RELATIONSHIP… · p. 15