Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks

1. Introduction

• Primitive-based representations offer interpretable part arrangements useful for shape editing, physics-based applications, and graphics simulation.Source: §1 · pp. 1–2
• Simple primitive parametrizations limit geometric expressivity, so existing methods need many parts for accurate reconstructions, often at the cost of interpretability.The resulting primitives may not correspond to recognizable parts such as legs or arms.Source: Figure 1 · p. 1, §1 · p. 2
• Neural Parts defines each primitive with an INN implementing a homeomorphism between a simple genus-zero shape and the target, enabling implicit and explicit representations.The inverse mapping supports computing both a primitive’s surface relation and its tessellation.Source: §1 · p. 2
• The model learns primitive arrangements without primitive annotations by training their assembly to reconstruct the target object.The passage describes learning directly from unstructured 3D point clouds.Source: §1 · p. 2
• Across ShapeNet, D-FAUST, and FreiHAND, Neural Parts captures complex geometries with an order of magnitude fewer primitives while producing expressive, semantically meaningful abstractions.The paper reports that its primitives achieve geometrically accurate and interpretable reconstructions.Source: Figure 1 · p. 1, §1 · p. 2

2. Related Work

• The paper focuses on primitive-based methods among several 3D reconstruction output categories and leaves a thorough discussion of INNs to [3].Source: §2 · p. 2
• Voxels, point clouds, meshes, and implicit models each have stated limitations, while Neural Parts predicts a high-resolution mesh for each part without post-processing.Point clouds lack surface connectivity; mesh approaches may need a deformable template or multiple mappings; implicit models lack part interpretability and require iso-surfacing.Source: §2 · pp. 2–3
• Traditional primitives trade reconstruction quality against parsimony and semantic interpretability, whereas Neural Parts can represent complex geometries with few primitives.Structured abstraction methods may use primitive-parameter supervision or learn without part-level annotations; [33] decomposes space into implicit functions but does not permit extracting a single part.Source: §2 · p. 3
• Prior deformation work includes sphere-to-shape MLP mappings whose inverse is infeasible and invertible ellipsoid deformations that do not model part decomposition.Neural Parts uses inverse mappings to form a union of primitives, discard internal surface points, and apply volumetric and surface constraints.Source: §2 · p. 3

3.1. Primitives as Homeomorphic Mappings

• Each primitive is a homeomorphic deformation of a sphere conditioned on the input, with forward and inverse mappings between latent and primitive spaces.The inverse mapping supports determining whether a 3D point lies inside, outside, or on the primitive surface.Source: §3 · p. 3, §3.1 · pp. 3–4
• Applying the forward mapping to sphere vertices produces a primitive mesh while preserving its face arrangement, allowing arbitrary mesh resolutions without post-processing.Applying the inverse mapping to 3D points provides the primitive's implicit representation.Source: §3.1 · pp. 3–4

3.2. Network Architecture

• A ResNet-18 feature extractor combines the image representation with a learnable embedding for each primitive, and a conditional Real NVP implements the mappings.Its affine coupling layers transform one point dimension based on the other dimensions and the primitive embedding; an MLP expands point features before conditioning.Source: §3.2 · pp. 4–5, Figure 3 · p. 5

3.3. Training

• Training uses a watertight target mesh and combines surface, normal-consistency, and occupancy losses with overlap and coverage constraints on the predicted primitives.The surface and occupancy losses model complementary characteristics; overlap discourages points from lying inside too many primitives, while coverage encourages each primitive to contain target points.Source: §3.3 · pp. 5–6
• On D-FAUST, Neural Parts outperforms CvxNet, SQs, and H-SQs in IoU and Chamfer-L1 across primitive configurations, including with just 2 primitives.Its reconstructions are also competitive with OccNet, which is not primitive-based and requires post-processing to extract meshes.Source: Figure 4 · p. 6

4.1. Representation Power

• 67.3% IoU with 5 primitives versus 62% for CvxNet with 50 primitives shows Neural Parts is more accurate at a given primitive count.Neural Parts is also described as competitive with OccNet, which does not reason about parts.Source: §4.1 · p. 7
• Neural Parts captures human pose and limbs with as little as 1 or 2 primitives, while CvxNet misses the arms even with 10.Source: §4.1 · p. 7

4.2. Reconstruction Accuracy

• With 5 primitives, Neural Parts produces human-body part assemblies close to the target, while CvxNet with 50 primitives lacks part-level semantic interpretation.Source: §4.2 · p. 7
• On FreiHAND, Neural Parts captures fine hand details such as thumb position better than CvxNet and SQs; Figure 7 reports stronger IoU and Chamfer-L1 results than primitive baselines.Source: §4.2 · p. 7, Figure 7 · p. 7
• On ShapeNet, Neural Parts is more accurate than CvxNet with either 5 or 25 primitives; increasing CvxNet to 25 improves quality but loses semantic interpretation.For chairs, CvxNet with 5 primitives misses entire object parts; Figure 8 compares Neural Parts with 5 primitives against CvxNet with 5 and 25.Source: §4.2 · p. 8, Figure 8 · p. 8

4.3. Ablation Study

• Without the inverse mapping, primitives redundantly cover the whole shape, lose semantic structure, and cannot use occupancy loss to model empty space.Source: §4.3 · p. 8
• Removing occupancy loss leaves human geometry accurate but connects the legs, while removing reconstruction loss yields tiny primitives displaced from the body.Source: §4.3 · p. 8
• Using five SMPL-X vertices, Neural Parts is more semantically consistent than CvxNet with 5 or 50 primitives; Figure 10 reports closest-primitive label accuracy.Source: §4.3 · p. 8, Figure 10 · p. 8

• Neural Parts aims to bridge interpretable and high-fidelity primitive-based reconstruction, preserving each part’s semantic interpretation while learning geometrically accurate shapes.Source: §5 · pp. 8–9
• The authors identify mesh-free learning through differentiable rendering and replacing the sphere with more complex shapes as future directions.Source: §5 · p. 9

• The supplement title identifies Neural Parts as the work it accompanies.Source: Supplementary Material for Neural… · p. 13
• The title specifies that the work concerns learning expressive 3D shape abstractions with Invertible Neural Networks.Source: Supplementary Material for Neural… · p. 13
• The listed authors are Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler.Source: Supplementary Material for Neural… · p. 13

Abstract

• The supplement covers architecture and training, ablations, added results on D-FAUST, FreiHAND, and ShapeNet, and further evidence about representation consistency.Source: Abstract · p. 13
• The supplementary ablations examine how system components affect single-view 3D reconstruction.Source: Abstract · p. 13

A. Implementation Details

• Its implementation section describes the network architecture, training protocol, sampling strategy, metric computation during training and testing, and baseline details.Source: A. Implementation Details · p. 13

A.1. Network Architecture

• The architecture pairs an image feature extractor that predicts per-primitive shape embeddings with a conditional homeomorphism between the sphere and target shape.The feature encoder uses ImageNet-pretrained ResNet-18; the homeomorphism uses four Real NVP coupling layers conditioned on each embedding.Source: §A.1 · pp. 13–14

A.2. Training Protocol

• Training uses Adam without data augmentation, with batch sizes and iteration counts varying by primitive count.The protocol uses 100k iterations for two primitives, 150k for five, and 200k for eight or ten.Source: §A.2 · pp. 14–15

A.3. Sampling Strategy

• Sphere-surface samples are not uniform on the deformed primitive, although the authors report no observed negative training impact and leave uniform primitive sampling for future work.The method samples 200 points per primitive during training.Source: §A.3 · p. 15

A.4. Metrics

• Evaluation estimates volumetric IoU from 100,000 bounding-volume samples and Chamfer-L1 from 10,000 surface samples.Chamfer-L1’s two terms measure completeness and accuracy, respectively.Source: §A.4 · pp. 15–16

A.5. Baselines

• The baselines include unsupervised superquadrics, hierarchical superquadrics, convex hulls, and a non-part-based implicit model; all use the same ResNet-18 feature extractor.OccNet is included as a representative implicit method, though it is not directly comparable as a part-based model.Source: §A.5 · p. 16

B. Ablation Study

• The ablation study evaluates the INN, pθ(·), and individual loss terms on single-view D-FAUST reconstruction, generally using five primitives for 300 epochs.Each component is tested by comparing variants with the relevant mapping or loss removed.Source: B. Ablation Study · p. 17, §B.1 · p. 17

B.1. Invertibility

• Neural Parts outperforms both the no-inverse variant and AtlasNet-sphere on IoU and Chamfer-L1, while AtlasNet-sphere’s primitives have zero volume and inverted normals.Without the inverse mapping, the model cannot enforce non-interpenetration, so its primitives lack interpretability.Source: Table 1 · p. 17, §B.1 · p. 18

B.2. Effect of pθ(·)

• Removing pθ(·) makes convergence slower and worsens reconstruction quality on both IoU and Chamfer-L1.The point embedding increases the input point’s dimensionality before concatenation with the higher-dimensional shape embedding, helping the network predict point-dependent scale and translation.Source: §B.2 · pp. 18–19

Loss Functions

• The model combines reconstruction, occupancy, normal-consistency, overlap, and coverage losses to learn geometrically accurate and semantically meaningful primitives without primitive annotations.The ablation compares five variants, each omitting one loss term.Source: §B.3 · p. 19

w/o Reconstruction Loss

• Removing Lrec produces degenerate primitive arrangements that fail to capture object geometry.Lrec is a bidirectional Chamfer loss between target and predicted surface points.Source: §B.3 · p. 20

w/o Occupancy Loss

• Removing Locc leaves empty space uncaptured, even when primitives accurately represent human-body parts.Surface points covering empty space can remain close to target points, so reconstruction loss alone may not penalize them sufficiently.Source: §B.3 · pp. 20–21

w/o Normal Consistency Loss

• Without Lnorm, primitives still fit body-part geometry but meet with inconsistent normals and unnatural connections.The full model instead produces smoother, more natural transitions at joins.Source: §B.3 · p. 21, Figure 18 · p. 21, Figure 19 · p. 22

w/o Overlapping Loss

• Removing Loverlap preserves geometric fit but lets primitives penetrate one another and describe the same object parts, undermining semantic decomposition.The loss discourages multiple primitives from containing the same target points.Source: §B.3 · p. 21, Figure 20 · p. 22

w/o Coverage Loss

• Without Lcover, some primitives collapse into thin sheets with minimal volume that do not cover object parts or contribute to reconstruction.The coverage loss encourages every primitive to cover part of the target object.Source: §B.3 · p. 21, Figure 21 · p. 23, Figure 22 · p. 23

C. Experiment on D-FAUST

• D-FAUST uses 70%/20%/10% train, test, and validation splits, filters the first 20 frames of each sequence, and retains variation in human body size rather than normalizing meshes to a unit cube.The dataset contains 38,640 human meshes performing varied tasks; retaining size variation makes reconstruction harder.Source: C. Experiment on D-FAUST · p. 24
• Neural Parts outperforms all primitive-based baselines on D-FAUST with only 2 primitives, reporting volumetric IoU and Chamfer-L1 against the ground-truth mesh.The authors report that baseline reconstruction improves with more parts, whereas Neural Parts’ performance is independent of the number of predicted primitives.Source: Table 4 · p. 24, C. Experiment on D-FAUST · p. 24
• With 5 primitives, Neural Parts combines semantic interpretability and high reconstruction quality, unlike CvxNet and H-SQs, whose primitives lack semantic interpretation, or SQs, whose parsimonious abstractions are inaccurate.The comparison uses CvxNet and SQs with 50 primitives and H-SQs with 32.Source: §C.1 · pp. 24–25, Figure 24 · p. 25

D. Experiment on FreiHAND

• The FreiHAND experiment uses 5,000 hand poses rendered from a fixed orientation, with 70%/20%/10% splits and mesh post-processing to create occupancy pairs.The post-processing addresses MANO meshes not being watertight.Source: D. Experiment on FreiHAND · p. 26

• Neural Parts produces semantically meaningful parts that accurately capture hand geometry.Source: §D.1 · p. 26
• Arbitrarily bendable primitives capture flexible fingers, including the thumb, which the compared methods cannot model this way.Source: §D.1 · p. 26
• The comparison uses Neural Parts with 5 primitives, SQs and CvxNet with 5, and H-SQs with 8.Source: §D.1 · p. 26, Figure 25 · p. 26

E. Experiment on ShapeNet

• Neural Parts outperforms CvxNet with both 5 and 25 primitives on airplane IoU and Chamfer-L1.Source: Figure 27 · p. 28, E. Experiment on ShapeNet · p. 29, Table 6 · p. 29
• With 5 primitives, Neural Parts captures lamp and chair geometry that CvxNet with 5 misses; CvxNet with 25 is more accurate but less semantically meaningful.The reported missing details include lamp shades or bodies and chair legs.Source: E. Experiment on ShapeNet · pp. 29–30, Figure 28 · p. 29, Table 7 · p. 30

F. Semantically Consistent Abstractions

• The same predicted primitives consistently represent the same human parts across different humans.Source: Figure 30 · p. 31

F.1. Temporal Consistency

• The same primitive represents the same body part across motion of varying breadth, an emergent property not enforced by a loss.Source: §F.1 · pp. 31–32, Figure 31 · p. 32

F.2. Semantic Consistency

• The five evaluation vertices mark the left and right thumbs, left and right toes, and nose.Source: Figure 32 · p. 32, §F.2 · p. 33
• Semantic consistency is measured by classifying five representative mesh vertices using the label of their closest primitive.Source: §F.2 · pp. 32–33