Source-linked AI summary
ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds
Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, Radomír Měch
TL;DR
Design and graphics applications need editable, high-fidelity parametric surfaces from raw 3D point clouds. ParSeNet jointly learns decomposition and fitting for assemblies of geometric primitives and B-spline patches, with optional geometric refinement. It reports improved segmentation and reconstruction parametrization against RANSAC and SPFN, while remaining limited by small parts, sparse high-curvature regions, noisy boundaries, and scan-noise modeling.
Problem
Design and graphics applications need editable, high-fidelity parametric surfaces from raw 3D point clouds.
Method
ParSeNet jointly learns point-cloud decomposition and primitive fitting, including SplineNet for B-spline patches, with optional geometric postprocessing.
Results
14.93% and 13.13% segmentation improvements over RANSAC and SPFN accompany 50% and 47.64% relative-error reductions for patch parametrization.
Takeaways & Limitations
The resulting architecture produces editable and interpretable predictions that modelers can refine using standard CAD operations.
Takeaways & Limitations
Small parts, sparse high-curvature regions, noisy boundaries, and realistic scan-noise modeling remain limitations or future challenges.
Abstract
from arXiv · showhide
We propose a novel, end-to-end trainable, deep network called ParSeNet that decomposes a 3D point cloud into parametric surface patches, including B-spline patches as well as basic geometric primitives. ParSeNet is trained on a large-scale dataset of man-made 3D shapes and captures high-level semantic priors for shape decomposition. It handles a much richer class of primitives than prior work, and allows us to represent surfaces with higher fidelity. It also produces repeatable and robust parametrizations of a surface compared to purely geometric approaches. We present extensive experiments to validate our approach against analytical and learning-based alternatives. Our source code is publicly available at: https://hippogriff.github.io/parsenet.
1 Introduction
ParSeNet automates conversion of 3D point clouds into compact, editable assemblies of parametric surface patches, including B-splines and basic primitives. It is trained end-to-end and outperforms analytical and learning-based baselines in segmentation and patch parametrization.
- Motivation: ParSeNet converts a 3D point cloud into a piecewise parametric surface representation composed of non-overlapping, seamlessly placed patches.Patch decomposition supports diverse geometric features, surface topologies, interaction, and editing.
- Representation: The representation targets smoothness, editability, and flexibility through geometric primitives, including open or closed B-spline patches.Control points or axes provide intuitive degrees of freedom for modifying surfaces.
- Architecture: ParSeNet combines a decomposition module with primitive fitting, including SplineNet for B-spline patches, in a fully differentiable end-to-end pipeline.An optional geometric postprocessing step further refines the output.
- Evaluation: 14.93% and 13.13% improvements over RANSAC and SPFN, respectively, are reported for segmenting point clouds into patches.For parametrizing each patch for surface reconstruction, the reported relative error reductions are 50% and 47.64%, respectively.
- Contributions: The contributions include an end-to-end differentiable point-cloud representation, novel decomposition and fitting modules, and evaluation against analytical and learning-based methods.SplineNet is described as a differentiable network for fitting cubic B-spline patches.
2 Related Work
Prior work represents shapes with parametric surfaces or fits collections of simple primitives, but expressive, jointly learned decomposition into general curved patches remains limited. ParSeNet addresses this gap with differentiable decomposition and fitting for B-spline and other primitives.
- Parametric surfaces: Parametric surfaces map a typically compact subset of R2 to R3 and are commonly used for curved CAD surfaces represented as spline patches.Spline formulations include Bézier, B-spline, and NURBS patches built from weighted combinations of control parameters.
- Parametric surfaces: B-spline patches are bounded, smoothly curved parametric surfaces defined by a sparse grid of control points and basis functions over parameters (u, v).
- Primitive fitting: Analytical fitting methods commonly use Hough transforms, RANSAC, clustering, or heuristics to approximate 3D data with primitive collections or NURBS patches.
- Learning-based fitting: Learning-based shape parsers broaden primitive combinations but often rely on simple primitives such as cuboids, ellipsoids, superquadrics, spheres, cubes, or cylinders.
- Gap addressed: Existing curved-surface methods impose symmetry assumptions or lack jointly learned multi-patch decomposition and fully differentiable fitting.ParSeNet instead combines decomposition with fitting for multiple regions and includes B-spline patches.
- ParSeNet: ParSeNet’s pipeline uses decomposition, primitive fitting, SplineNet, joint end-to-end training, and optional postprocessing.
3 Method
ParSeNet reconstructs point clouds by decomposing them into typed segments, fitting parametric patches, and optionally refining their geometry. Its learned embeddings, adaptive clustering, primitive fitting, and SplineNet support automatically determined multi-patch surfaces.
- Pipeline: The pipeline decomposes an input point cloud into typed regions, predicts each patch’s shape parameters, and optionally optimizes boundaries for seamless surfaces.
- Pipeline: Inputs are 3D points with optional normals, and outputs are an automatically determined set of sphere, plane, cone, cylinder, or open/closed B-spline patches.Each patch also has a real-valued vector of geometric parameters.
- Decomposition: The decomposition module learns point embeddings in which points from the same patch form distinct clusters.EdgeConv layers extract local-neighborhood representations before normalization to 128-dimensional unit-sphere embeddings.
- Decomposition: Mean-shift clustering discovers a shape-specific number of segments without requiring the target cluster count.A von Mises–Fisher kernel with bandwidth β updates embeddings until convergence, followed by non-maximum suppression of cluster centers.
- Decomposition: Segment types are predicted from per-point representations and assigned by majority voting over each segment’s points.
- Fitting: Basic primitives use least-squares parameter fitting, while SplineNet predicts a fixed-size control-point grid for B-spline patches.SplineNet combines point-wise EdgeConv features with a global representation and outputs a 20×20 control-point grid.
- Post-processing: Postprocessing matches tessellated B-spline vertices to segment points, applies ARAP deformation, and re-estimates control points to improve coverage and alignment.The control-point grid is iteratively upsampled until a prescribed Chamfer-distance fitting tolerance is achieved.
4 Training
ParSeNet is trained on CAD shapes represented as surface patches and primitives, using losses for decomposition, classification, patch fitting, and B-spline control-point and derivative agreement. A staged training procedure separately pre-trains decomposition and SplineNet components before combining them for end-to-end learning.
- 4.1 Dataset: 32K CAD models with at least one B-spline patch form ABCPartsDataset, split into 24K training, 4K test, and 4K validation models.A separate SplineDataset contains 32K closed and open B-spline patches with the same split proportions.
- 4.1 Dataset: Each shape is sampled with 10K surface points and perturbed with uniform positional and normal noise before training.The positional noise range is [−0.01, 0.01] along normals, while normal directions receive noise in [−3, 3] degrees.
- 4.2 Loss functions: The training objective combines embedding, classification, control-point regression, Laplacian, and patch-distance losses across decomposition and fitting modules.Patch distance applies to both basic primitives and B-spline patches; B-spline-specific losses supervise standardized control-point grids and surface derivatives.
- 4.2 Loss functions: Metric-learning embeddings pull points from the same surface patch together and push points from different patches apart to form decomposition clusters.The embedding loss uses point triplets after mean-shift clustering.
- 4.2 Loss functions: B-spline control-point supervision standardizes variable-sized patches to 20 × 20 grids, using least-squares reparameterization without practical surface-quality loss.The loss accounts for control-point permutations, with 8 permutations for open and 160 for closed B-splines.
- 4.3 Training procedure: Staged training first pre-trains decomposition with embedding and classification losses, then pre-trains SplineNet on B-spline patches before joint training.The staged procedure was reported to converge faster and reach better minima than starting from scratch with all losses.
5 Experiments
The experiments evaluate segmentation, labeling, B-spline fitting, and overall reconstruction against analytical and learning-based alternatives on ABCPartsDataset and SplineDataset. ParSeNet achieves stronger segmentation and reconstruction results, while ablations examine losses, optimization, and upsampling.
- Evaluation scope: The evaluation covers segmentation and segment classification, B-spline patch fitting, and overall surface reconstruction.These experiments compare ParSeNet with nearest-neighbor, RANSAC, and SPFN baselines.
- Segmentation and labeling: 13.13% and 8.66% improvements in segmentation mIOU and label mIOU respectively are obtained over SPFN with points and normals.ParSeNet with points and normals performs better than the points-only setting, and end-to-end training provides further gains.
- Segmentation and labeling: ParSeNet produces more reasonable patch segmentations than competing methods in qualitative comparisons.Figure 4 compares SPFN, ParSeNet without post-processing optimization, full ParSeNet, and ground-truth surfaces.
- B-spline fitting: B-spline fitting is evaluated using Chamfer distance on densely sampled predicted and ground-truth patches from the SplineDataset test set.The ablation varies control-point, distance, and Laplacian losses, post-processing optimization, and upsampling for open and closed B-splines.
- Reconstruction: 47.64% and 50% reductions in relative error are achieved versus SPFN and RANSAC respectively using points and normals, joint end-to-end training, and post-processing optimization.Residual error and P-coverage follow the trend of the segmentation metrics, and the comparison includes all, geometric, and spline primitives.
6 Conclusion
ParSeNet reconstructs point clouds as editable, interpretable assemblies of geometric primitives and CAD-style surface patches. Its main limitations are errors on small parts, oversegmentation in high-curvature regions, and difficulty producing seamless boundaries from noisy, sparse data.
- 6 Conclusion: ParSeNet reconstructs point clouds by predicting geometric primitives and surface patches common in CAD design.The method combines 3D deep learning with CAD modeling practices.
- 6 Conclusion: The resulting predictions are editable and interpretable, allowing modelers to refine them using standard CAD modeling operations.
- 6 Conclusion: Small parts are often misclassified because clustering merges them with larger patches.
- 6 Conclusion: Sparse sampling in high-curvature areas can cause ParSeNet to produce more segments than the ground truth.
- 6 Conclusion: Noise and sparsity make seamless patch boundaries difficult, motivating realistic scan-noise simulation as future work.
7 Supplementary Material
The supplementary material details B-spline representations, datasets, architectures, training, differentiable clustering, and evaluations of robustness and reconstruction. It also reports results on ABCPartsDataset and TraceParts, including improvements from end-to-end training, normals, and optimization.
- 7.1 Background on B-spline patches: A B-spline patch is a smoothly curved, bounded, parametric surface defined by a sparse grid of control points.The surface is evaluated over parameters (u, v) using polynomial B-spline basis functions.
- 7.1 Background on B-spline patches: ParSeNet focuses on cubic B-splines with uniform interior knots, predicting either open or closed patches.Closed surfaces match control points on opposite grid edges.
- 7.2 Dataset: ABCPartsDataset contains 32k shapes split into 24k training, 4k validation, and 4k test examples, each containing at least one B-spline patch and at most 50 patches.
- 7.3 Architecture: The decomposition module predicts per-point embeddings and primitive types, while SplineNet predicts a control-point grid approximating each input patch.Both modules use dynamic graph convolution networks.
- 7.3 Architecture: Removing non-differentiable non-max suppression from the computational graph enables partial-gradient backpropagation through clustering as a straight-through estimator.The authors report this end-to-end training approach as effective in Table 1.
- 7.4 Robustness analysis of SplineNet: Nearest-neighbor upsampling to 1.6K points reduces SplineNet error for sparse inputs, where performance is otherwise low at 100 points per surface patch.The error is measured using Chamfer distance, and the horizontal axis is logarithmic.
- 7.6 Evaluation on TraceParts Dataset: On TraceParts, ParSeNet achieves 79.91% segmentation mIOU and 97.39% label mIOU with points, compared with 76.4% and 95.18% for SPFN.