Source-linked AI summary
Cascaded Refinement Network for Point Cloud Completion
Xiaogang Wang, Marcelo H Ang, Gim Hee Lee
TL;DR
Point-cloud completion must recover dense, complete shapes from sparse and incomplete inputs while handling lost details and complex point distributions. The paper proposes a cascaded refinement network with coarse-to-fine processing and a patch discriminator, and reports state-of-the-art performance across completion datasets. Its shape-prior initialization assumes a known reflection-symmetry plane, although some objects are asymmetric.
Problem
Sparse, incomplete point clouds make it difficult to recover complete dense shapes, while existing methods often lose fine object details and struggle to learn complex point distributions.
Method
A cascaded refinement generator combines partial-input details with global shape information through coarse-to-fine refinement, while a patch discriminator learns local point distributions.
Results
The method achieves state-of-the-art quantitative and qualitative performance on different 3D point-cloud completion datasets.
Takeaways & Limitations
The framework preserves details from partial inputs while synthesizing missing object parts with high quality and refining point positions locally and globally.
Takeaways & Limitations
The mirror-based shape prior assumes a known xy-plane reflection-symmetry plane, although 40 of 1200 testing objects are asymmetric.
Abstract
from arXiv · showhide
Point clouds are often sparse and incomplete. Existing shape completion methods are incapable of generating details of objects or learning the complex point distributions. To this end, we propose a cascaded refinement network together with a coarse-to-fine strategy to synthesize the detailed object shapes. Considering the local details of partial input with the global shape information together, we can preserve the existing details in the incomplete point set and generate the missing parts with high fidelity. We also design a patch discriminator that guarantees every local area has the same pattern with the ground truth to learn the complicated point distribution. Quantitative and qualitative experiments on different datasets show that our method achieves superior results compared to existing state-of-the-art approaches on the 3D point cloud completion task. Our source code is available at https://github.com/xiaogangw/cascaded-point-completion.git.
1. Introduction
The paper introduces a cascaded refinement network and coarse-to-fine strategy for completing sparse, incomplete point clouds while preserving input details and generating realistic missing parts. A patch discriminator learns local point distributions, and experiments report superior completion results.
- Motivation: Sparse, incomplete, and irregular points make it difficult to learn accurate features and generate complete, dense 3D shapes.The completion task matters for robotic navigation, scene understanding, and augmented reality.
- Existing limitations: Voxel methods are memory-inefficient at high resolution, while mesh methods are constrained by fixed vertex connectivity and topology.Point clouds allow new points to be added more easily.
- Proposed approach: The proposed network uses cascaded refinement and a coarse-to-fine pipeline to preserve partial-input details and generate missing parts with fine detail.The strategy refines point positions both locally and globally.
- Proposed approach: A patch-based discriminator forces local patches of generated point clouds to match patterns in real complete point clouds.Adversarial training penalizes inaccurate points and targets complex point distributions.
- Results: Experiments on different datasets show superior quantitative and qualitative results over existing methods for 3D point cloud completion.The paper reports this outcome as a state-of-the-art result.
2. Related work
Related work spans point-cloud generation, point-cloud upsampling, and 3D shape completion across voxel, mesh, distance-field, and point-cloud representations. These approaches differ in their generative architectures and resolution or topology constraints.
- 3D Generation: Point-cloud generation has used GANs, VAEs, fully connected generators and discriminators, and graph neural networks.PointNet helped inspire subsequent point-cloud generation methods.
- 3D Upsampling: Point-cloud upsampling methods generate dense, uniform points from sparse, non-uniform sets using point-feature extraction and feature expansion.PU-Net uses PointNet++ features, while EC-Net additionally targets sharp edges.
- 3D Upsampling: Existing upsampling methods are limited to small ratios such as 4×, motivating methods that address larger-scale densification.The passage identifies this limitation for PU-Net and EC-Net.
- 3D Completion: Shape-completion research covers voxel grids, meshes, distance fields, and point clouds, with earlier voxel and distance-field methods using 3D convolutions.The field has developed substantially across these representation formats.
3. Our Method
The method completes sparse, incomplete point clouds through a feature encoder, coarse reconstruction, and cascaded dense refinement. It combines partial-input details with global shape information, residual lifting, mean shape priors, and patch-level adversarial supervision.
- Overview: The generator extracts global features, reconstructs a coarse point cloud, then produces dense outputs through cascaded refinement.Feature extraction uses stacked PointNet components; coarse reconstruction uses fully connected layers, while dense reconstruction uses lifting modules.
- Coarse Reconstruction: Coarse reconstruction captures the complete object shape broadly but loses fine details recovered during dense reconstruction.The coarse network maps latent embedding f to Pcoarse, which provides global shape information for later refinement.
- Dense Reconstruction: The dense stage concatenates dynamically subsampled partial inputs with the coarse output to preserve observed details while synthesizing missing structure.Direct concatenation produced uneven point distributions, motivating dynamic subsampling before combining the two point sets.
- Lifting Module: Each lifting module doubles point count and predicts residual point features while refining positions through feature contraction and expansion.Shared MLPs consolidate local and global information in bottom-up and top-down operations, producing evenly distributed surface points.
- Shape Priors: Mean shape vectors from category-level latent embeddings provide an additional prior during iterative refinement.The latent embeddings are obtained from a pretrained PointNet auto-encoder across eight object categories.
- Optimization: Training jointly optimizes reconstruction and adversarial losses, while a patch discriminator evaluates local realism rather than only whole-object realism.The reconstruction loss uses Chamfer Distance for coarse and fine outputs; the discriminator combines patch selection, hierarchical feature integration, and confidence regression.
4. Experiments
The experiments compare the proposed method with 3D-EPN, PCN, and TopNet on ShapeNet and PCN-derived data using Chamfer Distance and Fréchet Point Cloud Distance.
- Evaluation setup: The method is compared with 3D-EPN, PCN, and TopNet using Chamfer Distance and Fréchet Point Cloud Distance.CD-P is used in Section 4.4, while CD-T is used elsewhere; FPD measures distance between real and fake point-cloud feature distributions.
- Datasets: The evaluation uses 30,974 objects across eight categories, with partial inputs generated by back-projecting 2.5D depth images into 3D.The categories are airplane, cabinet, car, chair, lamp, sofa, table, and vessel.
4.3. Implementation Details
The implementation trains the generator and discriminator with TTUR-based Adam optimization and evaluates qualitative outputs at multiple point resolutions.
- Optimization: Adam optimization uses learning rates of 0.0001 for the generator and 0.00005 for the discriminator under TTUR.Learning rates decay by 0.7 approximately every 40 epochs and are clipped by 10^-6.
- Architecture and losses: The model uses a coarse output size of 512 and a discriminator patch size of 256.The loss weights are λ=1 and β=200, while λf increases from 0.01 to 1 during the first 50,000 iterations.
- Evaluation resolutions: On the PCN dataset, qualitative comparisons use 16,384-point outputs and ground truth.These settings are reported for Figure 6.
- Evaluation resolutions: Qualitative ShapeNet comparisons use 2048-point partial and ground-truth inputs, with generated outputs of 2048 and 16,384 points.These settings are reported for Figure 5.
4.5. Point Completion on the Dataset of TopNet
On the TopNet training-data setting, the method is evaluated quantitatively on ShapeNet and qualitatively on PCN-derived data, with comparisons reported against existing methods.
- Quantitative comparison: The method achieves better quantitative results for all resolutions when trained on TopNet training data and tested on the created testing data.Random scaling augmentation uses values uniformly sampled between [1/1.5, 1] because training objects are larger than testing objects.
- Quantitative comparison: Table 1 reports quantitative point-cloud completion results across eight ShapeNet object categories.The supplied caption identifies the table as a comparison across eight categories.
- Qualitative comparison: Figure 6 compares qualitative completions on the PCN dataset at 16,384-point output and ground-truth resolution.The figure compares generated results from different methods.
4.6. Point Completion on Our Training Data
On the smaller training dataset, the method outperforms PCN and TopNet across resolutions while preserving detailed structures and producing more evenly distributed points.
- Quantitative results: 16.08%, 12.97%, 15.36%, and 15.56% are the relative improvements over PCN across all resolutions on the smaller training data.The authors interpret these improvements as evidence of robustness and generality.
- Quantitative results: Lower CD errors are obtained at 2048, 4096, 8192, and 16,384 resolutions using one model trained at 16,384 points.The comparison is against independently trained PCN and TopNet models.
- Qualitative results: The method generates details in both observed and missing regions at high and low resolutions.Examples include lampshades, car empennages, and airplane engines, while PCN and TopNet retain only general shapes.
- Qualitative results: Generated points are more evenly distributed across object surfaces than the compared PCN outputs in desk and chandelier examples.The cited examples contrast concentration on particular surfaces or corners with more uniform coverage.
4.7. Robustness to Occlusion
The method remains the best-performing approach as progressively larger portions of test inputs are occluded. Its generated completions also improve classification accuracy relative to PCN and TopNet.
- Occlusion robustness: The method achieves the best performance across occlusion rates from 20% to 70%, although error increases as more regions are occluded.The evaluation uses mean CD per point (10−4).
- Classification evaluation: Generated completions improve classification accuracy by 1.59% compared to PCN and TopNet.The comparison evaluates synthesized complete point clouds with a PointNet classifier.
- Classification evaluation: Complete ground-truth points provide higher classification accuracy than partial test points because partial data contain defects.The complete-point result is the upper bound, while the partial-point result is the lower bound.
4.9. Ablation Study
The ablation study finds that the full pipeline performs best, with every evaluated component contributing to the reported completion quality.
- Ablation results: The full pipeline performs best under the CD-P metric at 2048-point resolution.The baseline removes the discriminator, and the study evaluates the adversarial training, mean shape, contraction-expansion unit, mirror operation, and Chamfer Distance choices.
- Ablation results: Removing any evaluated component decreases performance relative to the complete model.The ablated components include the discriminator, mean shape features, contraction-expansion unit, mirror operation, and training-time Chamfer Distance calculations.
4.10. Shape Arithmetic for Feature Learning
Latent-vector interpolation produces smooth shape transitions and reasonable synthesized objects, while parameter sharing makes the model smaller than PCN and TopNet at 16,384 points.
- Shape arithmetic for feature learning: Smooth latent-vector transitions preserve critical geometric information and yield reasonable synthesized object shapes.The interpolation results are shown for chair and airplane shapes.
- Model size: At 16,384-point resolution, the model has fewer parameters and a smaller trained-model size than PCN and TopNet.The paper attributes this efficiency to sharing parameters across cascaded refinement steps.
5. Conclusion
The paper presents a point completion network that generates complete point sets from partial inputs using cascaded refinement and adversarial training. Experiments report state-of-the-art performance on completion datasets.
- Conclusion: The cascaded refinement generator preserves details in partial inputs while synthesizing missing parts with high quality.The generator uses a coarse-to-fine completion strategy.
- Conclusion: A patch discriminator learns accurate point distributions and penalizes generated objects that differ from ground truth.The discriminator operates through adversarial training.
- Conclusion: Experiments on completion datasets show that the method achieves state-of-the-art performance.The conclusion summarizes results across various experiments.