Source-linked AI summary
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing
Yongcheng Liu, Bin Fan, Gaofeng Meng, Jiwen Lu, Shiming Xiang, Chunhong Pan
TL;DR
Point-cloud shape recognition is difficult because irregular points can form diverse, indistinguishable shapes and existing approaches need sufficient context. DensePoint generalizes convolution to irregular points and densely aggregates multi-level, multi-scale semantics; experiments across four tasks report state-of-the-art performance. Its authors note limitations for highly non-uniform point distributions and very alike shapes.
Problem
Irregular point clouds can form diverse, hardly distinguishable shapes, creating a need for contextual semantics and a deep hierarchy for accurate shape recognition.
Method
DensePoint uses a permutation-invariant generalized convolution and dense connections to progressively aggregate multi-level and multi-scale semantics.
Results
DensePoint achieves state-of-the-art performance across shape classification, shape retrieval, part segmentation, and normal estimation benchmarks.
Takeaways & Limitations
DensePoint provides an efficient architecture for learning densely contextual representations directly from irregular point clouds.
Takeaways & Limitations
DensePoint does not consider local point-cloud density and does not evaluate the importance of each context level.
Abstract
from arXiv · showhide
Point cloud processing is very challenging, as the diverse shapes formed by irregular points are often indistinguishable. A thorough grasp of the elusive shape requires sufficiently contextual semantic information, yet few works devote to this. Here we propose DensePoint, a general architecture to learn densely contextual representation for point cloud processing. Technically, it extends regular grid CNN to irregular point configuration by generalizing a convolution operator, which holds the permutation invariance of points, and achieves efficient inductive learning of local patterns. Architecturally, it finds inspiration from dense connection mode, to repeatedly aggregate multi-level and multi-scale semantics in a deep hierarchy. As a result, densely contextual information along with rich semantics, can be acquired by DensePoint in an organic manner, making it highly effective. Extensive experiments on challenging benchmarks across four tasks, as well as thorough model analysis, verify DensePoint achieves the state of the arts.
1. Introduction
DensePoint addresses the difficulty of recognizing implicit 3D shapes by combining permutation-invariant local pattern learning with densely aggregated contextual semantics.
- Motivation: Irregular point clouds form diverse, often indistinguishable shapes, making high-level shape understanding challenging.Sufficient contextual semantic information is identified as essential for grasping these elusive shapes.
- Requirements: Point-cloud processing requires a permutation-invariant convolution operator for efficient local-pattern learning and a deep hierarchy for contextual semantics.These are presented as the two key requirements for effective CNN-style point-cloud learning.
- Approach: DensePoint generalizes convolution to irregular points while preserving permutation invariance, local connectivity, and weight sharing.The operator enables efficient inductive learning of local patterns and supports building a deep semantic hierarchy.
- Approach: DensePoint repeatedly aggregates multi-level and multi-scale semantics through dense connections, producing densely contextual representations.The architecture progressively combines context and semantics throughout its deep hierarchy.
- Evaluation: Experiments across shape classification, shape retrieval, part segmentation, and normal estimation report state-of-the-art performance.The paper also includes thorough model analysis.
2. Related Work
Prior 3D shape-learning methods use views, volumes, or direct point-cloud processing, each with limitations in geometric fidelity, local patterns, or contextual learning.
- View-based and volumetric methods: View-based methods apply image CNNs to 2D projections, but self-occlusions can substantially lose 3D shape information.These methods represent each shape as a collection of 2D views.
- View-based and volumetric methods: Volumetric methods enable 3D CNNs on regular grids, but low-resolution quantization causes information loss and still relies on bounding-volume subdivision.K-d trees and octrees partly address the issue while retaining subdivision-based representations.
- Deep learning on point cloud: Direct point-cloud methods avoid regular-space transformations but face challenges in learning local patterns and contextual semantics.PointNet processes points independently, while contextual methods such as PointNet++ and related approaches address context with varying limitations.
- DensePoint: DensePoint extends grid CNNs to irregular point configurations and aggregates multi-level, multi-scale semantics using dense connections.Its architecture is illustrated as an alternative to direct layer-by-layer connections.
3. Method
DensePoint extends convolution to irregular point clouds and builds dense hierarchical representations by repeatedly aggregating multi-level and multi-scale semantics.
- Convolution and Pooling on Point Cloud: The generalized point-cloud convolution decomposes processing into feature transformation and feature aggregation over local neighborhoods.A sampled centroid’s neighborhood is transformed pointwise, aggregated, and assigned to the centroid in the next layer.
- Convolution and Pooling on Point Cloud: Shared feature transformation and symmetric aggregation make the generalized convolution permutation invariant while preserving local connectivity and weight sharing.The operator therefore supports efficient inductive learning of local patterns on irregular points.
- DensePoint Architecture: In contrast to layer-by-layer CNN connections, DensePoint feeds each layer the outputs of all preceding layers through concatenation.This dense connection forces learning from multi-level representations rather than a single-level input.
- DensePoint Architecture: DensePoint progressively aggregates multi-level semantics with multi-scale shape information, increasing contextual scale as the network deepens.The resulting representation combines local-to-global shape information for point-cloud processing.
- DensePoint Architecture: DensePoint differs from traditional multi-scale learning by organically aggregating semantics across levels instead of gathering multi-scale information at one level.The architecture also avoids making each layer less contextual through a simple final concatenation of all layers.
- Narrow Architecture: DensePoint uses grouped feature transformation and a shared integration perceptron to widen intermediate outputs while maintaining efficiency.The grouped operator widens eφ, while ψ integrates information across groups; both include nonlinear activation.
4. Experiment
DensePoint is evaluated across four point-cloud tasks and detailed analyses, consistently outperforming competing methods while retaining competitive efficiency. Ablations and robustness studies support densely contextual semantics as the central design advantage, with limitations on non-uniform densities and very similar shapes.
- DensePoint for Point Cloud Processing: DensePoint is evaluated across shape classification, retrieval, part segmentation, and normal estimation benchmarks.The experiments include ModelNet40, ModelNet10, and ShapeNet part datasets.
- Shape classification: 26.9% lower error than PointNet++ on ModelNet40 classification, and 0.9% higher accuracy than SO-Net on ModelNet10 using only point inputs.DensePoint also outperforms all compared point-input methods.
- Shape retrieval: 18% higher mAP than PointNet on shape retrieval, with DensePoint accurately distinguishing some visually similar classes such as bottles and vases.The retrieval evaluation uses global features ranked by cosine distance.
- Shape part segmentation: DensePoint achieves the best ShapeNet part segmentation performance, improving over DGCNN by 1.9 class mIoU and 1.3 instance mIoU.It also reaches state-of-the-art performance in eight categories.
- Normal estimation: 48.6% lower estimation error than PointNet++ on ModelNet40 normal estimation, although intricate shapes such as curtains and plants remain difficult.DensePoint produces more decent predictions than PointNet and PointNet++ in the reported examples.
- DensePoint Analysis: DensePoint raises ablation accuracy from 88.6% for the baseline to 91.1%, 92.5% with ePConv, and 93.2% for the final voting model.A smaller network reaches 92.1% accuracy, while the model is reported as most efficient at depth L = 6 with 92.1% accuracy.
- DensePoint Analysis: The authors note that local point-cloud density is not considered and that context-level importance is not evaluated, limiting performance on highly non-uniform data and very alike shapes.The latter issue is also illustrated by residual confusion between benches and TV stands.
5. Conclusion
DensePoint extends CNN-style processing to irregular point clouds, builds a deep hierarchy that aggregates multi-level and multi-scale semantics, and achieves strong performance across four tasks.
- 5. Conclusion: DensePoint extends regular grid CNN to irregular point configurations through an efficient generalized convolution operator.The operator preserves point permutation invariance and convolutional properties including local connectivity and weight sharing.
- 5. Conclusion: DensePoint progressively aggregates multi-level and multi-scale semantics through a deep hierarchy.This design acquires contextual information together with rich semantics for implicit shape identification.
- 5. Conclusion: Extensive experiments across four tasks and thorough model analysis demonstrate that DensePoint achieves state-of-the-art performance.
- 5. Conclusion: DensePoint shows good robustness against noisy points, suggesting a direction for robust point cloud representation learning.
Supplementary Material
The supplementary material investigates DensePoint and provides additional retrieval examples, configuration details, and training information.
- Supplementary Material: It also provides additional shape retrieval examples and analysis, network configuration details, and training details.
- Supplementary Material: The supplementary material further investigates DensePoint, including neighborhood methods, dropout, network depth, memory, and runtime.All investigations are conducted on the ModelNet40 dataset.
B.1. Neighborhood Method
The study compares spherical random neighborhood sampling with k-nearest neighbors and reports better performance for Random-In-Sphere.
- B.1. Neighborhood Method: DensePoint compares Random-In-Sphere and k-NN neighborhood strategies using models with the same configuration.The main-paper local convolutional neighborhood is spherical, with a fixed number of randomly sampled neighbors for batch processing.
- B.1. Neighborhood Method: Random-In-Sphere performs better than k-NN, which the authors associate with improved coverage under inhomogeneous point distributions.The authors suggest k-NN can confine receptive fields to dense regions and overlook sparse points important for implicit-shape recognition.
B.2. Dropout on f N(x) in Eq. (4)
Dropout is evaluated at different ratios on f N(x), with a 20% ratio producing the best reported result.
- B.2. Dropout on f N(x) in Eq. (4): 20% dropout achieves the best reported result of 93.2% on DensePoint.Dropout is applied with different ratios on f N(x) in Eq. (4) to analyze its effect.
B.3. Network Depth
DensePoint’s classification accuracy improves with depth up to a moderate network, while excessive depth adds complexity without gains. The 11-layer model reaches the best reported result with acceptable complexity.
- B.3. Network Depth: 92.1% accuracy is achieved by a 6-layer DensePoint network with 0.53M parameters and 148M FLOPs/sample.This 6-layer model outperforms PointNet++ at lower computational cost.
- B.3. Network Depth: The study compares classification results across different network depths, with fully connected layers excluded from the depth count.Table III reports the accuracy of the depth variants.
- B.3. Network Depth: 15% lower error rate and one order of magnitude fewer FLOPs/sample distinguish the 6-layer DensePoint model from PointNet++.PointNet++ is reported at 90.7% accuracy, 1.48M parameters, and 1684M FLOPs/sample.
- B.3. Network Depth: 93.2% accuracy is reached by an 11-layer network with acceptable complexity.A 23-layer network increases complexity without bringing any gain.
B.4. Memory and runtime
DensePoint’s shallow design and efficient implementation reduce the memory and runtime costs associated with dense connections. The 6-layer model is the most efficient, while the 11-layer model remains competitive.
- B.4. Memory and runtime: The 6-layer DensePoint model is the best configuration in terms of efficiency, while the 11-layer model remains competitive.The comparison is summarized in the memory and runtime analysis.
- B.4. Memory and runtime: Dense connection costs are greatly alleviated by DensePoint’s shallow design and highly efficient implementation.Shared Memory Allocations can provide linear memory complexity for future very deep DensePoint networks.
- B.4. Memory and runtime: Feature propagation transforms features concatenated from the current interpolated layer and a long-range connected layer using an MLP.The feature propagation layer is used in the segmentation and normal-estimation configurations.
- B.4. Memory and runtime: The classification network contains 14 layers: 3 PPools, 2 DensePoints, and 3 fully connected layers.The first DensePoint has 3 layers and the second has 5 layers.
- B.4. Memory and runtime: The segmentation network has 23 layers comprising PPools, DensePoints, feature propagation layers, and fully connected layers.It uses 4 PPools, 3 DensePoints, 4 feature propagation layers, and 2 fully connected layers.
E. Training Details
DensePoint is trained with Adam using mini-batches of 32 and scheduled learning-rate and batch-normalization momentum decay. The supplementary configurations specify task-specific architectures and label encodings.
- E. Training Details: Adam optimization trains DensePoint with a mini-batch size of 32.Weights use He et al.’s initialization techniques.
- E. Training Details: The learning rate starts at 0.001 and decays by 0.7 every 20 epochs.Batch-normalization momentum starts at 0.9 and decays by 0.5 every 20 epochs.
- E. Training Details: The shape part segmentation configuration uses long-range connections and defines K as the number of classes.The configuration is summarized in Table V.
- E. Training Details: The shape classification configuration defines K as the number of classes.Its detailed architecture is presented in Table VI.