Source-linked AI summary
BSP-Net: Generating Compact Meshes via Binary Space Partitioning
Zhiqin Chen, Andrea Tagliasacchi, Hao Zhang
TL;DR
BSP-Net addresses the limited direct generation of polygonal meshes and the expensive iso-surfacing used by implicit shape models. It learns shapes as convex decompositions from BSP-based planes, directly extracts compact watertight meshes, and achieves competitive or state-of-the-art performance with favorable fidelity-complexity trade-offs.
Problem
Polygonal meshes are underrepresented in deep shape modeling, while existing implicit approaches typically require expensive iso-surfacing and direct mesh methods face structural or visual limitations.
Method
BSP-Net is an unsupervised network that predicts planes, forms convex parts through a BSP-tree, and assembles them into polygonal meshes using learned binary connections.
Results
BSP-Net achieves state-of-the-art performance across shape auto-encoding, segmentation, part correspondence, and single-view reconstruction, while generating compact watertight meshes in about 0.5 seconds without iso-surfacing.
Takeaways & Limitations
The method provides compact polygonal meshes with sharp geometric details and a favorable fidelity-complexity trade-off compared with methods using comparable primitive counts.
Takeaways & Limitations
BSP-Net represents shapes only as unions of convexes, so concave objects may require many small convex pieces and waste representation capacity.
Abstract
from arXiv · showhide
Polygonal meshes are ubiquitous in the digital 3D domain, yet they have only played a minor role in the deep learning revolution. Leading methods for learning generative models of shapes rely on implicit functions, and generate meshes only after expensive iso-surfacing routines. To overcome these challenges, we are inspired by a classical spatial data structure from computer graphics, Binary Space Partitioning (BSP), to facilitate 3D learning. The core ingredient of BSP is an operation for recursive subdivision of space to obtain convex sets. By exploiting this property, we devise BSP-Net, a network that learns to represent a 3D shape via convex decomposition. Importantly, BSP-Net is unsupervised since no convex shape decompositions are needed for training. The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built on a set of planes. The convexes inferred by BSP-Net can be easily extracted to form a polygon mesh, without any need for iso-surfacing. The generated meshes are compact (i.e., low-poly) and well suited to represent sharp geometry; they are guaranteed to be watertight and can be easily parameterized. We also show that the reconstruction quality by BSP-Net is competitive with state-of-the-art methods while using much fewer primitives. Code is available at https://github.com/czq142857/BSP-NET-original.
1. Introduction
BSP-Net directly generates compact, watertight polygonal meshes by learning plane-based convex decompositions without ground-truth decompositions. Its BSP-tree supports reconstruction, correspondence, segmentation, and single-view reconstruction while preserving sharp features.
- Polygonal meshes are compact and coherent but remain underused in deep learning because irregular tessellations do not naturally support conventional convolution and pooling.
- Existing approaches often convert voxels, point clouds, or implicit functions into meshes during post-processing, with limitations including over-tessellation, restricted topology, seams, and difficult manipulation.
- BSP-Net predicts planes whose binary space partitions are grouped into convex parts and merged into an implicit field representing the output shape.
- At inference, classic CSG extracts polygonal surfaces from BSP-tree planes and binary connections, typically producing a compact mesh without iso-surfacing in about 0.5 seconds per mesh.
- Training is self-supervised: all shapes use the same learned BSP-tree of planes and convexes, so ground-truth convex decompositions are unnecessary.
- BSP-Net directly outputs compact, watertight meshes with arbitrary topology and structure variety, while its learned tree provides primitive-level correspondence and shape segmentation.
- The network adapts to shape auto-encoding and single-view 3D reconstruction, including structured reconstruction of segmented shapes from single unstructured object images.
- Experiments across auto-encoding, segmentation, correspondence, and single-view reconstruction report state-of-the-art performance and a favorable fidelity-complexity trade-off, including sharp-feature recovery.
2. Related work
Related work spans grid, surface, implicit, decomposition, structured, binary, and capsule representations for 3D learning. BSP-Net differs by directly learning plane-based convex assemblies without supervision and by targeting compact meshes with sharp features.
- Grid models: Grid-based models represent shapes with volumetric occupancy but are typically limited to resolutions of at most 64^3 because memory grows cubically.
- Implicit models: Implicit models treat reconstruction as indicator-function classification, but mesh generation still requires expensive iso-surfacing at inference.
- Shape decomposition: Shape-decomposition methods use primitives such as boxes, Gaussians, super-quadrics, or indicator-function parts; BSP-Net instead learns a plane-subdivision tree for bottom-up assembly.
- Shape decomposition: Compared with CvxNet, BSP-Net targets low-poly sharp-feature reconstruction, learns the appropriate primitive count automatically, and does not use a hard-coded compositional tree.
- Structured models: Structured 3D models commonly require part segmentations and labels, whereas BSP-Net is unsupervised but is not designed to infer shape semantics.
- Binary and capsule networks: BSP-Net’s binary tree optimization resembles binary and capsule networks, but it uses binary weights in only one layer and continuous relaxation during early training.
3. Method
BSP-Net provides a trainable, interpretable representation of geometry by extracting planes, grouping them into convexes, and assembling those parts into shapes. A two-stage training scheme relaxes and then discretizes the representation for more accurate reconstruction.
- Representation: BSP-Net represents geometry with a differentiable BSP-tree whose outputs are collections of convex polytopes.The representation is designed to be both trainable through implicit functions and interpretable through convex primitives.
- Network modules: An input-conditioned MLP extracts plane parameters, whose signed distances classify points relative to the planes.For point x, multiplying homogeneous coordinates by the plane-parameter matrix produces signed distances for all planes.
- Network modules: A binary grouping matrix combines hyperplanes into convex primitives, and a subsequent min-pooling layer assembles those convexes into possibly non-convex shapes.During training, summation can replace max-pooling so gradients reach all planes; shape assembly uses min-pooling.
- Training strategy: Two-stage training first optimizes a continuous approximate solution, then quantizes the grouping weights and fine-tunes with a perfect union for finer reconstruction.The discrete stage uses threshold λ = 0.01 to binarize the grouping matrix before fine-tuning.
- Training objective: The network is trained by matching its implicit field to sampled ground-truth inside/outside indicators, while relaxed adjacency and merge weights are optimized toward discrete structure.Training samples include random points in the unit box and points near the shape boundary; adjacency values are continuously relaxed to [0, 1].
4. Results and evaluation
BSP-Net is evaluated on synthetic 2D shapes, 3D shape autoencoding and segmentation, and single-view reconstruction, with emphasis on reconstruction quality, correspondence, compactness, and sharp features.
- 4.1. Auto-encoding 2D shapes: Near-perfect 2D reconstructions emerge after fine-tuning, while overlap losses reduce the number of convexes per part.The synthetic dataset contains diamonds, crosses, and hollow diamonds with varying sizes.
- 4.1. Auto-encoding 2D shapes: BSP-Net creates natural correspondence across shapes because shared convexes and hyper-planes are imposed by the static structure T.For example, hollow diamonds consistently use the same four convexes in the same relative positions.
- 4.2. Auto-encoding 3D shapes: Semantic correspondence is obtained at the convex level, with manual grouping assigning shared colors to convexes belonging to the same semantic part.A convex is labeled once and that assignment is propagated to all shapes containing the same learned convex.
- 4.2. Auto-encoding 3D shapes: In 3D autoencoding, BSP-Net achieves significantly better reconstruction quality while maintaining high segmentation accuracy.The comparison uses five ShapeNet classes and reports Chamfer Distance, Normal Consistency, Light Field Distance, and per-label IoU.
- 4.3. Single view reconstruction (SVR): BSP-Net matches state-of-the-art Chamfer Distance, outperforms most methods in Light Field Distance, and produces meshes with fewer polygons.The method is also reported as the only tested approach capable of representing sharp edges, with an average of 654 inferred polygons.
5. Conclusion, limitation, and future work
BSP-Net is presented as an unsupervised method for compact, structured polygonal meshes using convex decomposition, with superior sharp-detail quality at comparable primitive counts. Its main limitations are restricted CSG expressiveness, significant training cost, and uncertain generalization beyond curated datasets.
- Conclusion: BSP-Net learns compact, structured polygonal meshes through unsupervised convex decomposition.The network uses a shared BSP-tree built on learned planes and convexes to minimize reconstruction loss.
- Conclusion: Meshes generated by BSP-Net exhibit superior visual quality, particularly for sharp geometric details, when using comparable numbers of primitives to state-of-the-art methods.
- Limitation and future work: BSP-Net can only represent shapes as unions of convexes, so concave shapes may require many small convex pieces and waste planes and convexes.The paper identifies difference operations as a better representation for some concave shapes and proposes broader CSG operations as future work.
- Limitation and future work: Training is costly: the SVR task required 6 days for 4,096 planes and 256 convexes across all categories, although inference is fast.The fixed total plane count is needed to represent a large shape set, motivating adaptive primitive counts based on input complexity.
- Limitation and future work: Generalization beyond curated datasets and training from only RGB images remain critical directions, while RGBD applicability could leverage auto-decoder ideas.