Source-linked AI summary
PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling
Yue Qian, Junhui Hou, Sam Kwong, Ying He
TL;DR
PUGeo-Net addresses the challenge of generating uniform dense point clouds from sparse, irregular inputs. It learns local geometric structure to sample in a 2D parametric domain, lift samples to 3D, and project them onto the surface. The paper reports improved accuracy and efficiency across 4–16× upsampling while jointly producing coordinates and normals.
Problem
Dense, detailed point clouds are costly and time-consuming to acquire, while irregular and unordered point sets make direct adaptation of image-super-resolution methods difficult.
Method
PUGeo-Net learns a local parameterization and normal direction, adaptively samples a 2D domain, lifts samples to a tangent plane, and projects them onto the curved surface.
Results
PUGeo-Net consistently outperforms state-of-the-art methods in accuracy and efficiency for 4–16× upsampling and jointly generates dense point coordinates and normals.
Takeaways & Limitations
The geometry-centric design supports upsampling for sharp-featured CAD models and detailed scanned models while providing normals useful for surface reconstruction.
Takeaways & Limitations
Global parameterization requires constraints to keep parameters continuous across cuts, so the method uses local parameterization instead.
Abstract
from arXiv · showhide
This paper addresses the problem of generating uniform dense point clouds to describe the underlying geometric structures from given sparse point clouds. Due to the irregular and unordered nature, point cloud densification as a generative task is challenging. To tackle the challenge, we propose a novel deep neural network based method, called PUGeo-Net, that learns a $3\times 3$ linear transformation matrix $\bf T$ for each input point. Matrix $\mathbf T$ approximates the augmented Jacobian matrix of a local parameterization and builds a one-to-one correspondence between the 2D parametric domain and the 3D tangent plane so that we can lift the adaptively distributed 2D samples (which are also learned from data) to 3D space. After that, we project the samples to the curved surface by computing a displacement along the normal of the tangent plane. PUGeo-Net is fundamentally different from the existing deep learning methods that are largely motivated by the image super-resolution techniques and generate new points in the abstract feature space. Thanks to its geometry-centric nature, PUGeo-Net works well for both CAD models with sharp features and scanned models with rich geometric details. Moreover, PUGeo-Net can compute the normal for the original and generated points, which is highly desired by the surface reconstruction algorithms. Computational results show that PUGeo-Net, the first neural network that can jointly generate vertex coordinates and normals, consistently outperforms the state-of-the-art in terms of accuracy and efficiency for upsampling factor $4\sim 16$.
1 Introduction
PUGeo-Net addresses the costly, difficult problem of densifying irregular sparse point clouds by performing upsampling through learned local geometry rather than abstract feature replication. It generates dense points and normals, and reportedly outperforms prior methods across 4–16× upsampling factors.
- 1 Introduction: Dense point clouds are costly and time-consuming to acquire, while irregular point structure makes image-super-resolution techniques difficult to transfer directly.The target is a uniform dense point cloud that faithfully represents the underlying surface.
- 1 Introduction: PUGeo-Net learns local parameterizations, samples adaptively in 2D, lifts samples to 3D, and projects them onto the curved surface.Its geometric pipeline uses a learned linear transformation and normal displacement based on the learned second fundamental form.
- 1 Introduction: PUGeo-Net consistently outperforms state-of-the-art methods in accuracy and efficiency for 4–16× upsampling factors.The method is evaluated on CAD models with sharp features and scanned models with rich geometric details and complex topologies.
- 1 Introduction: PUGeo-Net jointly generates dense point coordinates and normals, supporting downstream surface reconstruction algorithms.The paper describes it as the first neural network to provide both outputs for upsampled point clouds.
- 1 Introduction: The paper reports that PUGeo-Net handles both sharp-featured CAD models and scanned models with rich geometric details.Figure 1 illustrates upsampling the Retheur Statue model to fill missing cloth and facial details.
2 Related Work
Prior point-cloud upsampling methods either rely on geometric assumptions and attributes or adapt image-inspired deep learning techniques to irregular point data. Their limitations include weak multiscale preservation, annotation requirements, and insufficient treatment of spatial geometry.
- 2 Related Work: Optimization-based methods work well on smooth surfaces but struggle with multiscale structures, sharp features, or dependence on normals and parameter tuning.EAR preserves sharp features but requires given normal information and parameter tuning.
- 2 Related Work: PU-Net uses multi-branch MLPs and joint reconstruction–repulsion losses, but does not guarantee uniform samples because it ignores spatial relations.It nevertheless surpasses earlier optimization-based approaches.
- 2 Related Work: EC-Net preserves sharp edges with point-to-edge loss but requires tedious edge and surface annotations for training.This adds supervision beyond the input point cloud.
- 2 Related Work: MPU supports large upsampling factors such as 16× through cascaded 2× networks, but replicates features with a 1D code rather than modeling full spatial geometry.Its design is inspired by image super-resolution techniques.
3 Proposed Method
PUGeo-Net upsamples point clouds through learned local surface parameterizations: it samples adaptively in 2D, lifts samples to tangent planes, and projects them onto curved surfaces while refining normals.
- 3.1 Motivation & Overview: PUGeo-Net first learns local parameterizations for each input point, using geometric features to represent neighborhoods without requiring an expensive global parameterization.The local surface is characterized through first- and second-order geometric information, with local parameterization preferred for arbitrary geometry and topology.
- 3.3 Parameterization-based Point Expansion: The predicted transformation encodes tangent directions and the normal, mapping 2D coordinates to the tangent plane while providing a coarse normal for each input point.The augmented Jacobian construction uses the tangent vectors and their cross product to form the transformation and normal.
- 3.2 Hierarchical Feature Learning and Recalibration: The network extracts hierarchical point-wise features, recalibrates multi-scale representations with self-gating, and jointly optimizes coordinates and normals.Its architecture combines hierarchical feature learning, feature recalibration, parameterization-based expansion, local shape approximation, and a joint loss.
- 3.3 Parameterization-based Point Expansion: Adaptive 2D samples are learned from local surface features, then lifted to each point’s tangent plane using a predicted 3×3 linear transformation.The expansion module combines learned parametric-domain sampling with a learned linear transformation rather than duplicating abstract feature vectors.
- 3.4 Updating Samples via Local Shape Approximation: After tangent-plane expansion, PUGeo-Net regresses normal displacements to warp samples onto the curved surface and predicts normal offsets for the generated points.The displacement approximates local second-order geometry, while a separate refinement updates the coarse normals.
4 Experimental Results
Across quantitative, visual, robustness, and ablation experiments, PUGeo-Net produces accurate, efficient, geometrically detailed point-cloud upsampling with jointly predicted normals.
- Comparison with State-of-the-art Methods: PUGeo-Net achieves the best performance across all metrics and upsampling factors, while maintaining a fixed network size smaller than MPU.Table 1 averages results over 13 testing models; PU-Net’s size grows with the upsampling factor.
- Comparison with State-of-the-art Methods: PUGeo-Net reconstructs scanned surfaces with more geometric details and stronger geometric structures than competing methods after upsampling.The comparison uses the same reconstruction procedure across methods.
- Comparison with State-of-the-art Methods: On CAD models, PUGeo-Net preserves geometric and topological features after upsampling, whereas low-resolution SPSR fails to reconstruct the input geometry.The visual comparison uses 5,000 input points and an upsampling factor of 16.
- Comparison with State-of-the-art Methods: PUGeo-Net generates more uniform 2D samples than PU-Net and MPU at upsampling factor 16.Generated points are distinguished by their source in the visual comparison.
- Effectiveness of Normal Prediction: Normal prediction improves reconstructed-surface quality, while PCA neighborhood choice strongly affects results and naive joint MLP regression performs worst.Table 2 evaluates R = 8; the ablation attributes gains to geometry-aware design and additional modules.
- Geometric Analysis: Predicted transformation matrices closely match analytic Jacobians, while normal displacements reflect local curvature on spheres and sharp cube edges.Most matrix-vector angle errors are below 3 degrees; cube edges produce wider displacement ranges than cube faces.
5 Conclusion and Future Work
PUGeo-Net is a geometry-centric framework that learns local geometric structure, adaptively samples new points, and computes normals alongside coordinates. Extensive evaluation reports improved accuracy and efficiency, while future work targets incomplete data and end-to-end surface reconstruction.
- PUGeo-Net explicitly learns first- and second-order local geometry, enabling recovery of geometry up to rigid motion.
- Adaptive learned sampling helps PUGeo-Net preserve sharp features and rich geometric details during upsampling.
- The network computes normals for both input and generated points, supporting surface reconstruction as a preprocessing application.
- Extensive evaluation reports that PUGeo-Net outperforms state-of-the-art deep learning methods in accuracy and efficiency for 3D point cloud upsampling.
- Future work includes applying PUGeo-Net to incomplete datasets and developing an end-to-end network for surface reconstruction.
- The learned local geometry may support feature-preserving simplification, denoising, and compression in future 3D processing tasks.