Source-linked AI summary
Supervised Fitting of Geometric Primitives to 3D Point Clouds
Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, Leonidas Guibas
TL;DR
Precise primitive fitting from 3D point clouds is hindered by low-level representations and the parameter tuning required by RANSAC-based methods. SPFN predicts supervised per-point properties and uses a differentiable estimator to recover a varying number of primitive types and parameters. On ANSI mechanical CAD models, it significantly outperforms RANSAC-based fitting and direct parameter prediction without user control.
Problem
Point clouds and meshes lack shape-aligned structural information, while RANSAC fitting requires careful thresholds that limit scaling across noisy, diverse shapes.
Method
SPFN predicts per-point membership, normals, and primitive types, then uses a differentiable model estimator to compute primitive parameters for varying numbers of primitives.
Results
SPFN significantly outperforms RANSAC-based fitting and direct parameter prediction, including when RANSAC receives higher-resolution point clouds.
Takeaways & Limitations
Supervised per-point properties and differentiable estimation support accurate fitting of primitives at different scales without user control.
Takeaways & Limitations
The supervised training setup relies on primitive membership, normals, and bounded primitive-surface information extracted from CAD models.
Abstract
from arXiv · showhide
Fitting geometric primitives to 3D point cloud data bridges a gap between low-level digitized 3D data and high-level structural information on the underlying 3D shapes. As such, it enables many downstream applications in 3D data processing. For a long time, RANSAC-based methods have been the gold standard for such primitive fitting problems, but they require careful per-input parameter tuning and thus do not scale well for large datasets with diverse shapes. In this work, we introduce Supervised Primitive Fitting Network (SPFN), an end-to-end neural network that can robustly detect a varying number of primitives at different scales without any user control. The network is supervised using ground truth primitive surfaces and primitive membership for the input points. Instead of directly predicting the primitives, our architecture first predicts per-point properties and then uses a differential model estimation module to compute the primitive type and parameters. We evaluate our approach on a novel benchmark of ANSI 3D mechanical component models and demonstrate a significant improvement over both the state-of-the-art RANSAC-based methods and the direct neural prediction.
1. Introduction
SPFN addresses the challenge of fitting diverse geometric primitives precisely to noisy 3D point clouds without per-input user control. It predicts per-point properties and differentiably estimates primitive parameters, outperforming RANSAC-based and direct neural approaches on ANSI mechanical CAD models.
- Motivation: Low-level point clouds and meshes lack structural information aligned with shape semantics, limiting geometric manipulation and editing.Primitive types and parameters can support plausible shape editing.
- Motivation: RANSAC-based fitting is difficult to scale because unsuitable residual thresholds can over-segment noisy data or miss small primitives.The problem affects both scanned point clouds and discretized repository meshes.
- Method: SPFN predicts point-to-primitive membership, surface normals, and primitive types before differentiably estimating parameters for planes, spheres, cylinders, and cones.This avoids directly regressing primitive parameters and makes the fitting loss backpropagable.
- Results: The ANSI dataset contains 17k CAD models, and SPFN outperforms RANSAC-based fitting despite category separation between training and testing.It also achieves better fitting accuracy than RANSAC when RANSAC receives higher-resolution point clouds.
- Method: SPFN detects a varying number of primitives with different scales through an end-to-end supervised neural network.The approach uses ground-truth primitive surfaces and memberships as supervision.
- Method: The differentiable primitive model estimator solves linear least-square problems, enabling end-to-end training on a novel mechanical-component CAD dataset.The estimator is a central component of the proposed pipeline.
2. Related Work
Prior work includes RANSAC-based methods for detecting multiple primitive types and neural approaches for supervised or unsupervised fitting. Earlier neural methods generally support fewer primitive types or lower fitting accuracy than the targeted precise-fitting setting.
- Scope: Object-level primitive-fitting research includes methods designed for scanned point clouds of individual mechanical parts rather than scenes.The review focuses on methods matching this target use case.
- RANSAC-based Primitive Fitting: RANSAC and its variants are widely used for detecting multiple primitives of different types in dense point clouds.Follow-up optimization methods refine extracted primitives using relations among them.
- Network-based Primitive Fitting: Earlier neural approaches are limited by fitting accuracy or by supporting a restricted number of primitive types.Some methods predict only cuboids for rough shape abstraction, while CSGNet predicts more variety with low accuracy.
3. Supervised Primitive Fitting Network
SPFN predicts per-point properties from an input point cloud and differentiably estimates primitive types and parameters, supporting four primitive types and varying numbers of primitives. Its supervised pipeline reorders predictions to match ground truth, estimates parameters through least-squares-based modules, and weights bounded primitive surfaces equally so small primitives are not overlooked.
- Architecture: The framework supports planes, spheres, cylinders, and cones, with the number of primitives allowed to vary by shape.Ground-truth supervision includes membership, normals, and bounded primitive surfaces.
- Architecture: SPFN predicts point-to-primitive membership, unoriented normals, and primitive types before estimating primitive parameters.The network uses PointNet++ with separate fully connected layers for the three per-point predictions.
- Primitive Reordering: Hungarian matching based on Relaxed Intersection over Union aligns predicted and ground-truth primitive columns without assuming a consistent ordering.The matched ordering is injected back into the network for loss computation and gradient propagation, preserving differentiability almost everywhere.
- Primitive Model Estimation: Differentiable model estimation computes primitive parameters from the point cloud, predicted normals, and membership weights.Plane estimation uses homogeneous least squares and differentiable SVD; cylinder estimation first finds the axis, then fits a circle, while sphere and cone estimation use related weighted formulations.
- Loss Function: Equal weighting of bounded primitive surfaces in the residual loss helps detect small primitives regardless of their scale.The loss samples each surface uniformly and weights every primitive equally rather than according to surface area.
4. Experiments
Experiments evaluate SPFN on a category-held-out ANSI mechanical-component benchmark using fitting metrics, comparisons with Efficient RANSAC and DPPN, ablations, and real scans. SPFN maintains strong coverage across primitive scales and outperforms Efficient RANSAC, while ablations identify segmentation and axis losses as important components.
- Dataset and metrics: The benchmark uses ANSI CAD mechanical components, with up to 100 models sampled from each of 504 categories and category-disjoint training and test splits.Primitive boundaries are extracted from CAD models, adjacent surfaces with identical parameters are merged, tiny primitive pieces are discarded, and 512 points are sampled per primitive surface for residual evaluation.
- Dataset and metrics: Evaluation reports segmentation mean IoU, primitive type accuracy, point-normal difference, primitive-axis difference, and mean/std. residual metrics after primitive reordering.When fewer than K predicted primitives exist, type, axis, and residual statistics are averaged only over matched pairs; axis difference is measured only for correctly predicted types.
- Comparison to Efficient RANSAC: SPFN outperforms Efficient RANSAC on every metric with both identical 8k inputs and higher-resolution 64k inputs.At residual threshold ϵ = 0.01, both {Sk} and P coverage show large margins favoring SPFN, indicating more precise primitive fitting.
- Comparison to Efficient RANSAC: Neural segmentation improves Efficient RANSAC coverage, but predicted point normals and primitive types do not close the gap to SPFN.The hybrid pipeline remains below SPFN, and adding predicted normals and types does not itself improve RANSAC’s {Sk} and P coverage.
- Scale sensitivity: SPFN provides consistently high {Sk} coverage across primitive scales, whereas Efficient RANSAC remains weak for small primitives even after using neural segmentation.The comparison uses ϵ = 0.01.
- Ablation study: Removing segmentation loss causes the largest coverage drop, while removing axis loss reduces axis accuracy and coverage, especially {Sk} coverage.Using more accurate jet-fitting normals instead of predicted normals also lowers coverage, and using predicted types in the residual loss gives slightly worse results.
- Results with real scans: On real scans of 3D-printed test models, SPFN trained with synthesized noise successfully reconstructs all primitives, including small segments.The scans were acquired with a DAVID SLS-2 3D Scanner.
5. Conclusion
The conclusion presents SPFN as a fully differentiable network that predicts variable numbers of geometric primitives from potentially noisy 3D point clouds. It reports better results than RANSAC-based fitting and direct parameter prediction, alongside a new ANSI mechanical-component dataset and evaluation metrics.
- Conclusion: SPFN predicts a varying number of geometric primitives from potentially noisy 3D point clouds using a fully differentiable architecture.The network predicts per-point properties and derives primitive parameters with a differentiable model estimator.
- Conclusion: SPFN predicts per-point properties before deriving primitive parameters, rather than directly predicting primitive parameters.The approach is supported by strong supervision and targets primitives at different scales without user control.
- Conclusion: Experiments report significantly better results for SPFN than for both the RANSAC-based method and direct parameter prediction.The comparison includes comprehensive evaluation metrics, ablation studies, and the introduced ANSI mechanical-component dataset.
- Conclusion: The work introduces the ANSI mechanical component dataset and a set of comprehensive evaluation metrics for primitive fitting.These resources support the paper’s comparisons and ablation studies.
S.1. Numerical Stability Control
The differentiable model estimator solves homogeneous and unconstrained least-square problems, but both can encounter numerical instability. SPFN controls these cases with regularization and gradient-blocking strategies.
- The estimator solves homogeneous and unconstrained least-square problems, and both can experience numerical stability issues.
- SVD solves the homogeneous least-square problem, but its backpropagated gradients can diverge when input singular values are not distinct.
- An l2-regularizer is added to the unconstrained least-square formulation to handle singular or degenerate cases.
- When the condition number of diag(w)X exceeds 10^5, the least-square problem is trivialized by setting X = 0 to prevent gradient flow.The regularizer uses λ = 10^-8.
S.2. Training Details
SPFN and its ablation studies use PointNet++ default training hyperparameters, Adam optimization, and 100 training epochs. The longest experiment took 50 hours on one Titan Xp GPU.
- Training uses PointNet++ default hyperparameters with batch size 16, initial learning rate 10^-3, and staircase learning decay 0.7.
- All neural network models are trained for 100 epochs using the Adam optimizer.
- The longest SPFN and ablation-study experiment took 50 hours to train on a single Titan Xp GPU.Total-loss decay was not substantial after 50 epochs, and the authors planned to release the source code.
S.3. DPPN Architecture
DPPN outputs a fixed collection of 4Kmax primitives across four primitive types. Auxiliary membership and type assignments are then constructed from each point’s closest predicted primitive for comparison with SPFN.
- Figure S1 presents the DPPN architecture.
- DPPN outputs 4Kmax primitives: Kmax planes, Kmax spheres, Kmax cylinders, and Kmax cones.
- For comparison with SPFN, each input point is assigned to its closest predicted primitive to construct auxiliary membership and type matrices.
S.4. Primitive Correspondences
SPFN’s column ordering is loss-invariant, yet the network implicitly learns a preferred ordering that yields correspondence among similar primitives across shapes. Figure S2 visualizes these consistent column indices by color.
- Figure S2 uses consistent colors to indicate column indices in Ŵ across shapes within the same category.
- Changing the ordering of columns in Ŵ does not affect the SPFN loss.
- SPFN implicitly learns a preferred ordering in which the same columns represent similar primitives across different shapes.
- The learned correspondences provide insight into possible design variations for the same category of shapes.
S.5. Additional Experiments
Section S.5 presents additional experiments, including evaluation of high-resolution point-cloud testing despite training-time memory limitations. Table S1 reports these experiments using multiple metrics with metric-specific optimization directions.
- S.5. Additional Experiments: SPFN’s additional experiments are summarized in Table S1, whose first row matches row 13 of Table 1.The table provides the evaluation results for the experiments described in Section S.5.
- Evaluation: Table S1 evaluates the experiments with lower-is-better and higher-is-better metrics.The caption specifies that the third through fifth metrics are lower-is-better, while the remaining metrics are higher-is-better.
- High-resolution point clouds: PointNet++ limits training on high-resolution point clouds because memory consumption increases with resolution.The architecture can therefore be trained using lower-resolution point clouds when high-resolution testing is desired.
- High-resolution point clouds: PointNet++ is reported to remain robust to changes in point-cloud resolution at test time.This motivates evaluating high-resolution inputs at test time after lower-resolution training.