Source-linked AI summary
Detail Preserved Point Cloud Completion via Separated Feature Aggregation
Wenxiao Zhang, Qingan Yan, Chunxia Xiao
TL;DR
Point cloud completion must recover complete shapes from incomplete scans, while global-feature methods can lose geometric details. The paper separates multi-level features for known and missing parts using GLFA and RFA, then refines the reconstruction; experiments report improved detail preservation and shape prediction over prior methods, with failures when partial inputs are ambiguous.
Problem
Incomplete scans caused by viewpoint occlusion and low sensor resolution require shape completion, but global-feature methods can lose geometric details from original point clouds.
Method
The network aggregates multi-level features separately for known and missing parts through GLFA and RFA, then refines reconstructed points to improve uniformity and reduce outliers.
Results
The proposed strategies show significant improvements over previous methods in detail preservation and shape prediction, with qualitative and quantitative outperformance reported across evaluations.
Takeaways & Limitations
Separated feature aggregation supports the paper’s goal of preserving existing details while predicting missing shape in point cloud completion.
Takeaways & Limitations
The network may preserve discontinuities caused by viewpoint or fail to predict specific details when the partial model provides insufficient cues.
Abstract
from arXiv · showhide
Point cloud shape completion is a challenging problem in 3D vision and robotics. Existing learning-based frameworks leverage encoder-decoder architectures to recover the complete shape from a highly encoded global feature vector. Though the global feature can approximately represent the overall shape of 3D objects, it would lead to the loss of shape details during the completion process. In this work, instead of using a global feature to recover the whole complete surface, we explore the functionality of multi-level features and aggregate different features to represent the known part and the missing part separately. We propose two different feature aggregation strategies, named global \& local feature aggregation(GLFA) and residual feature aggregation(RFA), to express the two kinds of features and reconstruct coordinates from their combination. In addition, we also design a refinement component to prevent the generated point cloud from non-uniform distribution and outliers. Extensive experiments have been conducted on the ShapeNet dataset. Qualitative and quantitative evaluations demonstrate that our proposed network outperforms current state-of-the art methods especially on detail preservation.
1 Introduction
Point cloud completion addresses incomplete 3D scans, but global-feature encoder-decoder methods can lose geometric detail. This work separates known and missing-part representations through multi-level feature aggregation and adds refinement for improved completion.
- Incomplete 3D scans from occlusion and low sensor resolution limit applications including reconstruction, robotics, scene understanding, and autonomous driving.
- Volumetric methods incur high memory cost and low fidelity, while point-cloud methods are compact but harder to process because point sets are irregular.
- Encoder-decoder completion methods recover shapes from global feature vectors, but relying only on these vectors loses geometric details from the input.
- The proposed network extracts multi-level features and aggregates them separately for known and missing parts to preserve details and predict absent shape.
- GLFA and RFA are the proposed separated feature aggregation strategies, followed by refinement that promotes uniform coordinates and reduces noise and outliers.
- Experiments report qualitative and quantitative outperformance over state-of-the-art 3D point cloud completion methods.
2 Related Work
Point-cloud learning methods process irregular 3D data directly and have introduced several completion architectures. Existing completion systems generally decode complete shapes from encoded global feature vectors.
- PointNet processes point clouds directly with a symmetric function, while PointNet++ captures local structure from neighborhoods at multiple scales.
- FoldingNet deforms a canonical 2D grid onto a shape surface using a folding-based decoder.
- PCN uses coarse-to-fine completion, TopNet uses a hierarchical rooted-tree decoder, and RL-GAN-Net controls a GAN generator with reinforcement learning.
- These completion approaches generate complete point clouds from an encoded global feature vector.
3 Network Architecture
The network separates known-part and missing-part representations from multi-level features, then expands and reconstructs them before refinement. Its refinement pipeline targets uniform point distribution, fewer outliers, and smooth high-resolution surfaces.
- 3.1 Multi-level Features Extraction: Multi-level hierarchical features represent local and global properties, with the final-level global feature duplicated across input points.Features are progressively extracted with increasing grouping radii and interpolated to a common point number and feature size.
- 3.2 Separated Feature Aggregation: The method separately represents known and missing parts to preserve input details while providing cues for missing-shape prediction.Direct concatenation preserves existing details but lacks sufficient missing-part information and can tangle generated points with the partial input.
- 3.2 Separated Feature Aggregation: GLFA aggregates lower-level features for known parts and higher-level features for missing parts, while RFA derives missing features from differences between global and known features.A learnable transformation T(Corigin) supplies initial missing-part coordinates, and residual features are generated with shared fully connected layers.
- 3.3 Feature Expansion and Reconstruction: Expanded known and missing features are duplicated in a j:k ratio, processed by separated MLPs, and reconstructed into coarse point coordinates.The experiments use j:k = 1:1; the reconstructed output has rN×3 coordinates, with known- and missing-part outputs denoted Yknown and Ymissing.
- 3.4 Refinement Component: The refinement component uses FPS, attention-based point selection, repulsion loss, and local folding to improve distribution, remove incorrect points, and generate smooth high-resolution surfaces.FPS alone is random and does not remove noise or outliers; attention selects the top tN points, while local folding generates u^2-point patches around selected points.
- 3.5 Loss Function: The loss combines Chamfer distances for reconstructed and attended outputs, while repulsion is applied to attended points because local folding already models local geometry.The method chooses Chamfer distance for efficiency over Earth Movers Distance and sets α = 0.5 and β = 0.2 in experiments.
4 Experiments
Experiments on ShapeNet and KITTI evaluate the proposed separated feature aggregation and refinement strategies across known and novel categories. The methods preserve details, improve completion quality, and expose different strengths and limitations for known versus unseen categories.
- Experimental Setup: The ShapeNet experiments use 28,974 models from eight training categories, with separate test sets for eight known and eight novel categories.Novel categories are further divided into visually similar and dissimilar groups.
- ShapeNet Completion Results: NSFA-GLFA and NSFA-RFA achieve lower evaluation-metric values than the compared methods on both known and novel categories.The comparison includes FC, FoldingNet, PCN, and TopNet.
- ShapeNet Completion Results: Qualitative ShapeNet results show that NSFA-GLFA and NSFA-RFA preserve details and predict missing shapes better than FC, FoldingNet, PCN, and TopNet.The baseline methods recover overall shapes but lose many model details.
- KITTI Completion Results: On KITTI, NSFA-RFA obtains the lowest fidelity error, while all methods have very close MMD values and can resemble typical cars.Fidelity error measures preservation of input points; MMD compares outputs with the closest ShapeNet car point cloud.
- Reconstructed Coordinates: Visualized reconstructed coordinates place Yknown closer to the partial input and Ymissing closer to the missing shape, with stronger known-part completion for NSFA-RFA.Both parts still contain some completion or original-point effects because global features are aggregated into both feature groups.
- Feature Aggregation Strategy Evaluation: Feature aggregation reduces points gathering around the input; NSFA-RFA spreads points well on the surface, whereas NSFA-GLFA performs better on novel categories.The authors attribute NSFA-GLFA's novel-category advantage to its global-feature representation of missing parts, while residual aggregation struggles for totally unseen categories.
- Refinement Component: The refinement component combines FPS, attention, and local folding to improve uniformity, reduce outliers and noise, and produce a smooth high-resolution surface.Quantitative refinement results are reported for known and novel categories.
- Failure Cases: Failure cases arise when discontinuous observed parts resemble true details or when the partial input lacks enough cues to predict specific model details.In the latter case, the network can recognize the object category but not its particular geometry.
5 Conclusion
The paper proposes point cloud completion with two separated feature aggregation strategies that treat known and missing parts separately. RFA performs better on known categories, while GLFA has advantages on novel categories, with both improving detail preservation and shape prediction over prior methods.
- The proposed network uses GLFA and RFA to aggregate features for known and missing parts separately.
- RFA achieves overall better performance on known categories, whereas GLFA shows advantages on novel categories.
- Both aggregation strategies significantly improve detail preservation and shape prediction over previous methods.
B. Details of the network architecture
The architecture extracts multi-level features and reconstructs coordinates through level-specific and shared multilayer perceptrons. Its implementation uses separate MLPs for feature expansion, a shared coordinate-reconstruction MLP, attention scoring, and local folding.
- Multi-level feature extraction follows PointNet++ notation, representing levels with local-region counts, ball radii, and fully connected layer widths.
- Separated MLPs use widths [256, 128], while the shared MLP for coordinate reconstruction uses widths [64, 3].
- The RFA MLP shares the GLFA architecture, but its first fully connected layer transforms the computed residual difference.
- The attention module outputs a scalar score per point, and the local folding unit generates final coordinates with an MLP of widths [512, 512, 3].
C. Is our network just doing classification?
The paper argues that its network performs reconstruction rather than merely classification by producing model-specific details and outperforming a point-cloud retrieval baseline. The retrieval baseline selects the closest complete training shape by Chamfer Distance.
- Different input models receive different predicted details, which the authors present as evidence against classification-only behavior.
- The point-cloud retrieval baseline selects the complete training point cloud closest to the input under Chamfer Distance.
- The method significantly outperforms the retrieval baseline on F-score, while qualitative results show more accurate completions despite retrieval outputs having no outliers.
- The paper evaluates the retrieval baseline alongside other baselines and its NSFA-RFA method using F-score.
D. More results on real-scanned data.
Additional Structure Sensor experiments evaluate the network on real-scanned models. The reported results show recovery of complete shapes, although an unseen biplane type produces more noise.
- The authors scan additional models with a Structure Sensor and report that the network recovers complete shapes from the resulting partial scans.
- The biplane completion contains more noise because the training set apparently lacks that object type.
E. How we decide m in GLFA
GLFA forms known-part and missing-part features from early and late network levels, with the choice of m determining their overlap. Performance is best when the two feature groups share one overlapped layer, while no overlap or excessive overlap degrades results.
- GLFA aggregates the first m feature levels into fknown and the last m levels into fmissing.
- All tested networks achieve their best performance when m = ⌊n/2⌋ + 1, creating overlapped layers between fknown and fmissing.With five feature-extraction levels, m = 3 makes the third-level feature common to both groups.
- When m = ⌊n/2⌋, the absence of overlapped layers causes a dramatic performance drop across networks.The authors attribute this to the need for at least one overlapped layer to create correlation between fknown and fmissing.
- Increasing the number of overlapped layers beyond the best setting also reduces performance, possibly because excessive overlap obscures local-feature boundaries.
F. The choices of Cmissing
The study compares using Corigin directly with generating Cmissing through T-Net as the initial coordinates for missing-part features. T-Net enables faster network convergence and improves performance in the reported comparison.
- The missing-part features are concatenated with coordinates Cmissing, initialized either with Corigin or through T-Net.
- T-Net improves performance compared with using Corigin for Cmissing when both networks are trained for 25 epochs.
G. Evaluation of the components in loss function.
The loss function combines a repulsion term with intermediate and final LCD supervision. An ablation study evaluates the contribution of each term by removing them individually.
- The repulsion term Lrep(Ycoarse) encourages the generated results to be uniformly distributed.
- The other LCD components provide intermediate and final supervision to guarantee results at each step.
- Each loss term is evaluated through ablation by removing it and reporting the results in Table 4.