Source-linked AI summary
GRNet: Gridding Residual Network for Dense Point Cloud Completion
Haozhe Xie, Hongxun Yao, Shangchen Zhou, Jiageng Mao, Shengping Zhang, Wenxiu Sun
TL;DR
Point cloud completion must recover complete shapes from sparse observations, but existing processing can lose geometric structure or context. GRNet uses 3D grids with differentiable gridding, reverse conversion, feature sampling, and grid-based loss; it performs favorably against state-of-the-art methods across ShapeNet, Completion3D, and KITTI.
Problem
Point cloud completion seeks complete 3D shapes from incomplete observations, while existing voxelization and MLP approaches can lose geometric, structural, or contextual information.
Method
GRNet uses 3D grids as intermediate representations with Gridding, Gridding Reverse, Cubic Feature Sampling, and Gridding Loss for completion.
Results
GRNet performs favorably against state-of-the-art methods on ShapeNet, Completion3D, and KITTI.
Takeaways & Limitations
The proposed grid-based representation preserves point-cloud structure and context while supporting completion of missing parts and recovery of details.
Abstract
from arXiv · showhide
Estimating the complete 3D point cloud from an incomplete one is a key problem in many vision and robotics applications. Mainstream methods (e.g., PCN and TopNet) use Multi-layer Perceptrons (MLPs) to directly process point clouds, which may cause the loss of details because the structural and context of point clouds are not fully considered. To solve this problem, we introduce 3D grids as intermediate representations to regularize unordered point clouds. We therefore propose a novel Gridding Residual Network (GRNet) for point cloud completion. In particular, we devise two novel differentiable layers, named Gridding and Gridding Reverse, to convert between point clouds and 3D grids without losing structural information. We also present the differentiable Cubic Feature Sampling layer to extract features of neighboring points, which preserves context information. In addition, we design a new loss function, namely Gridding Loss, to calculate the L1 distance between the 3D grids of the predicted and ground truth point clouds, which is helpful to recover details. Experimental results indicate that the proposed GRNet performs favorably against state-of-the-art methods on the ShapeNet, Completion3D, and KITTI benchmarks.
1 Introduction
Point cloud completion addresses sparse, occluded observations by recovering complete 3D shapes. GRNet uses 3D grids and differentiable transformations to preserve structure and context, with favorable results across established benchmarks.
- Sparse and incomplete point clouds arise from limited sensor resolution and occlusion, yet complete shapes support applications such as semantic segmentation and SLAM.
- Voxelization enables 3D convolutions but irreversibly loses geometric information, while MLP-based methods may neglect structural and contextual relationships.
- GRNet introduces 3D grids as intermediate representations to regularize unordered point clouds and preserve structural and contextual information.
- Gridding, Gridding Reverse, and Cubic Feature Sampling are differentiable layers that convert between point clouds and grids, learn spatial context, and refine completed points.Gridding interpolates point contributions to eight cell vertices; Gridding Reverse produces a coarse point cloud, and Cubic Feature Sampling gathers corresponding vertex features.
- Gridding Loss computes an L1 distance between predicted and ground-truth 3D grids to address unorderedness and support recovery of fine-grained details.
- GRNet performs favorably against state-of-the-art methods on the ShapeNet, Completion3D, and KITTI benchmarks.
2 Related Work
Existing point cloud completion and reconstruction networks include MLP-based and graph-based approaches. MLPs process points independently before symmetric aggregation, whereas graph methods aggregate information from spatial neighbors.
- MLP-based Networks: MLP-based networks process points independently and aggregate them into global features using symmetric functions such as max pooling.
- MLP-based Networks: AtlasNet and MSN address structure loss in MLP-based processing by estimating collections of parametric surface elements.
- Graph-based Networks: Graph-based networks represent points as graph vertices, connect spatial neighbors with directed edges, and pool neighbor information into coarser graphs.
3 Gridding Residual Network
GRNet completes incomplete point clouds by processing them through regular 3D grids, then refining a coarse reconstruction with local cubic features and an MLP. Its Gridding Loss compares predicted and ground-truth grids to preserve geometric layout and details.
- Gridding: GRNet uses Gridding to convert an unordered point cloud into a regular 3D grid while preserving its spatial layout.
- 3D Convolutional Neural Network: A 3D CNN with skip connections processes grid values to complete missing parts of the incomplete point cloud.
- Gridding Reverse: Gridding Reverse converts the processed grid into a coarse point cloud by weighting the eight vertices of each grid cell.
- Cubic Feature Sampling: Cubic Feature Sampling concatenates features from the eight vertices surrounding each coarse point, providing local context for detail recovery.
- Multi-layer Perceptron: The MLP takes coarse points and their corresponding features as input and learns residual coordinate offsets to produce the final completed point cloud.
- Gridding Loss: Gridding Loss uses the L1 distance between value sets of predicted and ground-truth 3D grids to supervise completion.
4 Experiments
Experiments evaluate GRNet on ShapeNet, Completion3D, and KITTI using quantitative and qualitative completion metrics. Results show strong benchmark performance, while ablations examine grid resolution, Cubic Feature Sampling, and Gridding Loss.
- ShapeNet: ShapeNet evaluation compares point completion using Chamfer Distance and F-Score@1% on 16,384 points.The best results are highlighted in the corresponding tables.
- ShapeNet: GRNet outperforms competitive methods on ShapeNet in both Chamfer Distance and F-Score@1%.Qualitative results also indicate better recovery of object details, including chairs and lamps.
- Completion3D: 10.64 overall CD ranks GRNet first on the Completion3D benchmark after evaluating 1,184 recovered testing objects at 2,048 points.The reported CD uses the benchmark’s L2-norm protocol and is multiplied by 10^4.
- KITTI: KITTI evaluation uses Consistency and Uniformity because complete ground-truth point clouds are unavailable for the LiDAR scans.The cars are evaluated after transforming point clouds into bounding-box coordinates and back to the world frame.
- KITTI: GRNet outperforms competing methods on KITTI in Consistency and Uniformity, producing more reasonable shape completions from highly sparse scans.The paper attributes improved consistency to greater sensitivity to the spatial structure of input points.
- Ablation Study: Ablations show that higher grid resolution improves F-Score, Cubic Feature Sampling significantly improves completion, and removing Gridding Loss decreases CD and F-Score.Increasing Gridding Loss resolution from 64^3 to 128^3 yields reported improvements of 25.9% in CD and 5.4% in F-Score; the chosen Gridding resolution is 64^3.
5 Conclusion
The paper addresses completion of incomplete 3D point clouds by introducing GRNet, which uses 3D grids to preserve structural and contextual information. It combines three differentiable layers with Gridding Loss and evaluates the approach on three benchmarks.
- GRNet uses 3D grids as intermediate representations to regularize unordered point clouds.
- The network introduces differentiable Gridding, Gridding Reverse, and Cubic Feature Sampling layers.
- GRNet also introduces Gridding Loss for training point cloud completion.
- Experiments compare GRNet with state-of-the-art methods on ShapeNet, Completion3D, and KITTI.
A More Explanations on Gridding, Gridding Reverse, and Cubic Feature Sampling
The proposed layers convert between point clouds and grid representations while retaining spatial information and sampling contextual features from neighboring grid vertices. Their backward formulations support differentiable processing, with coordinate normalization used for generated points.
- Gridding: The Gridding layer computes each grid-vertex value from neighboring points and their interpolation weights.A vertex value is set to zero when it has no neighboring points.
- Differentiability: The proposed layers include backward computations for gridding, Gridding Reverse, and Cubic Feature Sampling.Cubic Feature Sampling treats floor and ceiling operations as nondifferentiable with respect to point coordinates.
- Gridding Reverse: Gridding Reverse generates a coarse point cloud by weighted combinations of the eight vertices in each 3D grid cell.The generated point coordinates are normalized for the completion pipeline.
- Cubic Feature Sampling: Cubic Feature Sampling concatenates features from the eight vertices of the grid cell containing each coarse point.The sampled feature representation has c channels for each of m coarse points.
B Additional Quantitative Results on ShapeNet
This appendix reports additional ShapeNet completion results using Chamfer Distance with the L1 norm on 16,384 points. It also records that the PCN values match those in the original PCN paper.
- PCN uses Chamfer Distance with the L1 norm as an evaluation metric.
- ShapeNet results are compared using Chamfer Distance with the L1 norm on 16,384 points.The reported values are multiplied by 10^3, and the best results are highlighted.
- The PCN values in Table 8 exactly match those reported in the original PCN paper.
C Quantitative Results on Completion3D
This appendix includes a screenshot of the online Completion3D leaderboard results for the point cloud completion benchmark.
- Figure 5 presents the online leaderboard results for the Completion3D benchmark.The leaderboard is available at completion3d.stanford.edu/results.
D Additional Quantitative Results on KITTI
KITTI evaluation compares methods using Fidelity Distance and Minimal Matching Distance, but the paper argues these metrics do not adequately reflect real-world LiDAR completion quality. Visual results highlight clutter removal as an important distinction.
- Fidelity Distance measures the average distance from each input point to its nearest neighbor in the output point cloud.
- Minimal Matching Distance is the Chamfer Distance between the output and the closest ShapeNet car point cloud.
- Both Fidelity Distance and Minimal Matching Distance are unsuitable for KITTI because real-world scans contain clutters and ShapeNet cars do not cover all real-world car types.
- Compared with MSN, GRNet recovers a complete point cloud while removing clutters from the input LiDAR scan.
E Additional Ablation Studies
The ablation study evaluates how many points should be sampled from the coarse point cloud generated by Gridding Reverse. Sampling 2,048 points provides the reported best balance for subsequent processing.
- Number of Sampling Points: Gridding Reverse produces a coarse point cloud, from which points are randomly sampled to create a fixed-size input for the following MLP.
- Number of Sampling Points: 2,048 sampled points achieve the best reported performance in terms of Chamfer Distance and F-Score.
- Number of Sampling Points: Table 10 reports Chamfer Distance and F-Score@1% for different sample counts on ShapeNet.
- Number of Sampling Points: Sampling 4,096 points introduces redundant information, whereas sampling 1,024 points may discard too much information for subsequent processing.
F Qualitative Comparisons
The qualitative comparison presents visual point-cloud completion results on ShapeNet against several state-of-the-art methods. The displayed comparisons include GRNet alongside AtlasNet, PCN, FoldingNet, TopNet, MSN, and ground truth.
- ShapeNet comparisons: The qualitative evaluation compares point-cloud completion methods on ShapeNet.
- Compared methods: The visual comparison includes AtlasNet, PCN, FoldingNet, TopNet, MSN, GRNet, and ground-truth point clouds.
- Comparison layout: The displayed rows use the sequence Input, AtlasNet, PCN, FoldingNet, TopNet, MSN, GRNet, and GT.