Source-linked AI summary
PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths
Xin Wen, Peng Xiang, Zhizhong Han, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Yu-Shen Liu
TL;DR
Point cloud completion must recover missing geometry, but latent-code generation struggles to capture detailed topology and structure in unordered points. PMP-Net instead deforms each incomplete point through shortest, uniquely regularized paths, using multi-step coarse-to-fine searching. Experiments on Completion3D and PCN demonstrate advantages over state-of-the-art completion methods.
Problem
Latent-code generation of unordered point clouds makes detailed topology and structure difficult to capture during completion.
Method
PMP-Net formulates completion as multi-step deformation, moving each incomplete point toward the target with Earth Mover’s Distance-based path regularization and recurrent coarse-to-fine searching.
Results
Experiments on Completion3D and PCN demonstrate PMP-Net’s superiority over other point cloud completion methods.
Takeaways & Limitations
Point moving paths establish point-level correspondence that captures detailed topology and structure between incomplete and complete shapes.
Abstract
from arXiv · showhide
The task of point cloud completion aims to predict the missing part for an incomplete 3D shape. A widely used strategy is to generate a complete point cloud from the incomplete one. However, the unordered nature of point clouds will degrade the generation of high-quality 3D shapes, as the detailed topology and structure of discrete points are hard to be captured by the generative process only using a latent code. In this paper, we address the above problem by reconsidering the completion task from a new perspective, where we formulate the prediction as a point cloud deformation process. Specifically, we design a novel neural network, named PMP-Net, to mimic the behavior of an earth mover. It moves each point of the incomplete input to complete the point cloud, where the total distance of point moving paths (PMP) should be shortest. Therefore, PMP-Net predicts a unique point moving path for each point according to the constraint of total point moving distances. As a result, the network learns a strict and unique correspondence on point-level, which can capture the detailed topology and structure relationships between the incomplete shape and the complete target, and thus improves the quality of the predicted complete shape. We conduct comprehensive experiments on Completion3D and PCN datasets, which demonstrate our advantages over the state-of-the-art point cloud completion methods.
1. Introduction
PMP-Net reframes point cloud completion as multi-step deformation, moving incomplete-shape points toward a complete target rather than generating all points from a latent code. Earth Mover’s Distance regularization encourages unique point paths and point-level correspondences, while coarse-to-fine recurrent searching refines them.
- Point cloud completion addresses missing regions caused by scanner view angles or occlusions, which limits raw scans and affects downstream 3D vision tasks.
- PMP-Net moves points from the incomplete shape toward the complete shape instead of directly generating all point coordinates from a latent code.
- Point-level correspondence captures detailed topology and structural relationships between incomplete and complete shapes.
- Earth Mover’s Distance regularization resolves multiple possible point movements by encouraging a unique path arrangement and meaningful point-wise correspondence.
- The multi-step path search progressively reduces its radius and uses aggregated previous-path information through the Recurrent Path Aggregation module.
2. Related Work
Related work spans traditional completion based on handcrafted features or patch retrieval and deep learning methods that infer complete shapes from learned representations. For point clouds, prior methods mainly use encoder-decoder generation, whereas PMP-Net moves input points and uses multi-step deformation for missing geometry.
- Traditional methods infer missing regions using handcrafted surface or symmetry features, or retrieve similar patches from large complete-shape datasets.
- Deep learning methods extract geometric features from incomplete shapes and infer complete shapes without predefined handcrafted features.
- Point cloud completion is attractive because point clouds are direct scanner outputs requiring less storage and processing than volumetric data.
- Prior point-cloud methods such as TopNet, PCN, and SA-Net use encoder-decoder frameworks to generate complete point clouds from global features.
- PMP-Net moves all points from the initial input and combines multi-step searching with point-moving-distance regularization to infer missing-region geometry.
3. Architecture of PMP-Net
PMP-Net completes point clouds by iteratively predicting point displacements, aggregating movement history, and constraining paths toward unique, efficient correspondences.
- Multi-step framework: PMP-Net predicts displacement vectors that move each input point toward the target complete point cloud over three steps.Each step uses the previously deformed point cloud, enabling step-by-step refinement.
- Point displacement prediction: PointNet++ extracts global shape features and propagates them to produce per-point features for displacement prediction.The implementation uses three hierarchical feature-propagation levels.
- Multi-scaled searching: Coarse-to-fine searching reduces the maximum stride by powers of 10, while Chamfer and Earth Mover distances regularize the deformed shape.The shrinking search range helps preserve earlier decisions during subsequent steps.
- Recurrent path aggregation: RPA aggregates path history between steps and levels using GRU-inspired update and reset gates.Its gates selectively preserve or forget previous movement information while emphasizing current input features.
- Unique paths: The EMD-inspired correspondence constraint selects a minimum-distance bijection between source and target points, addressing multiple possible correspondences.This constraint supports meaningful point-wise correspondence and helps capture topology and structure relationships.
- Optimized searching: The point moving distance loss minimizes the sum of displacement magnitudes across all three steps and constrains each step toward shorter paths.This reduces redundant movement decisions and improves searching efficiency.
4. Experiments
Experiments evaluate PMP-Net on Completion3D and PCN, including quantitative, qualitative, generalization, and component analyses. Results support deformation-based completion, PMP loss, recurrent path aggregation, and coarse-to-fine searching.
- Evaluation setup: Completion3D evaluation uses per-point L2 Chamfer distance, with lower values indicating better completion.
- Completion3D results: PMP-Net achieves the best average Chamfer distance across Completion3D categories and outperforms GRNet in 6 of 8 categories.
- Qualitative comparison: PMP-Net produces more accurate Completion3D shapes, while competing methods show failures on severely incomplete tables and boats.
- Cross-dataset generalization: On ScanNet chairs without finetuning, PMP-Net completes shapes with less noise than GRNet and preserves residual input points.
- Dense completion: PMP-Net learned on sparse 2,048-point shapes is applied to dense 16,384-point completion on PCN, with comparable performance to the state-of-the-art method.
- Model analysis: The RPA baseline performs best, while directly adding history information performs worst; RPA better prioritizes current-step information than GRU.
- Model analysis: Removing PMP loss significantly worsens performance, supporting path regularization for capturing detailed topology and structure.
- Searching strategy: Decreasing searching radii improves performance by discouraging revisions of earlier decisions, but excessively large ratios approximate one-step deformation and can harm completion.
5. Conclusions
The conclusion presents PMP-Net as a multi-step deformation method that moves points from incomplete to complete shapes. It reports refined topology, point-level correspondence, and superiority on Completion3D and PCN.
- PMP-Net completes point clouds through multi-step shape deformation that consistently refines predicted structure and topology.
- The method establishes point-level correspondence between incomplete and complete shapes while achieving reported superiority on Completion3D and PCN.
A. Detailed Settings
Detailed settings describe PMP-Net’s PointNet++-based framework, encoder structure, and training configuration.
- PMP-Net uses the single-scale grouping version of PointNet++ with its feature propagation module as the basic framework.
- The encoder and other components are specified through detailed structures in Tables 6 and 7.
- Training uses AdamOptimizer with initial learning rate 10^-3, decay factor 0.5 every 20 epochs, batch size 24, and 150 epochs on one GTX 2080TI GPU.
B.1. Dimension of Noise Vector.
The noise vector perturbs points from their original locations, and its dimension and standard deviation control disturbance strength.
- The noise vector pushes points away from their original locations during network processing.
- As noise dimension or standard deviation approaches zero, disturbance disappears; larger values produce larger disturbance.
B.2. Standard Deviation of Noise Distribution.
The standard deviation of the noise vector affects PMP-Net completion performance, with larger disturbances helping until the influence becomes weak around 10^-1.
- Larger noise-vector standard deviations improve completion performance until their influence weakens around 10^-1.This conclusion is drawn from the completion results reported for different standard deviations.
B.3. Visual Analysis of Multi-step Searching.
Multi-step searching consistently refines the point deformation process, producing a cleaner chair-back shape than shorter searching settings.
- The 4-step searching setting shapes the chair-back empty space more cleanly than the other three settings.The comparison is visualized in Figure 10, where rectangles highlight the chair-back region.
B.4. Visualization of Completion Results on PCN dataset.
The PCN visualizations present completion examples across categories by juxtaposing incomplete inputs, PMP-Net predictions, and ground-truth shapes.
- Figures 11 and 12 show PCN completion results for each category.These figures supplement the category-level visualization of shape completion.
- For each category, the first row is the incomplete input, the second row is the predicted complete shape, and the third row is the ground truth.