Source-linked AI summary
Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline
Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, Jia Deng
TL;DR
Point-cloud classification results may reflect protocol choices as well as network architecture, making architectural progress difficult to disentangle. The paper controls these factors and evaluates SimpleView, a simple projection-based baseline, finding competitive performance from PointNet++ and strong results from SimpleView across benchmarks.
Problem
Auxiliary evaluation schemes, data augmentation strategies, and loss functions affect point-cloud classification performance enough to make architectural contributions difficult to disentangle.
Method
The paper analyzes protocol effects, re-evaluates prior architectures under controlled protocols, and evaluates SimpleView, which projects points into depth maps, applies a lightweight CNN, and fuses features.
Results
SimpleView performs on par or better than state-of-the-art networks on ModelNet40, outperforms them on ScanObjectNN, and generalizes better across datasets; PointNet++ is competitive when protocols are controlled.
Takeaways & Limitations
Protocol control should accompany architecture comparisons, and SimpleView provides a strong projection-based baseline for point-cloud classification.
Takeaways & Limitations
The analysis is limited to point-cloud classification, with expansion to scene and part segmentation left as future work.
Abstract
from arXiv · showhide
Processing point cloud data is an important component of many real-world systems. As such, a wide variety of point-based approaches have been proposed, reporting steady benchmark improvements over time. We study the key ingredients of this progress and uncover two critical results. First, we find that auxiliary factors like different evaluation schemes, data augmentation strategies, and loss functions, which are independent of the model architecture, make a large difference in performance. The differences are large enough that they obscure the effect of architecture. When these factors are controlled for, PointNet++, a relatively older network, performs competitively with recent methods. Second, a very simple projection-based method, which we refer to as SimpleView, performs surprisingly well. It achieves on par or better results than sophisticated state-of-the-art methods on ModelNet40 while being half the size of PointNet++. It also outperforms state-of-the-art methods on ScanObjectNN, a real-world point cloud benchmark, and demonstrates better cross-dataset generalization. Code is available at https://github.com/princeton-vl/SimpleView.
1. Introduction
The paper argues that protocol choices can obscure architectural contributions in point-cloud classification and evaluates SimpleView, a simple projection-based baseline. It finds that controlling auxiliary factors makes PointNet++ competitive, while SimpleView performs strongly across benchmarks.
- Protocol and architecture: Protocol factors such as evaluation schemes, data augmentation, and loss functions can obscure improvements attributable to network architecture.The paper treats these architecture-independent choices as a method’s protocol when they can be controlled independently.
- Protocol and architecture: PointNet++ performance rises from 90.0±0.3 to 93.3±0.3 when switching from its original protocol to RSCNN’s protocol.The strongest protocols rely on test-set feedback, so the paper also re-evaluates architectures without such feedback.
- Protocol and architecture: With protocol factors controlled, PointNet++ performs competitively with more recent architectures across various settings.The paper uses strong augmentation and loss functions while avoiding test-set feedback in its protocol.
- SimpleView: SimpleView projects points onto depth maps from orthogonal views, processes them with a lightweight CNN, and fuses the resulting features.Unlike prior projection-based methods, it uses simple projections, a smaller CNN backbone, and no ImageNet pretraining.
- SimpleView: SimpleView performs on par or better than sophisticated state-of-the-art networks on ModelNet40 across multiple evaluation protocols.The comparison includes protocols used by prior methods and the paper’s own protocol.
- SimpleView: SimpleView outperforms state-of-the-art architectures on noisy, non-axis-aligned ScanObjectNN and shows better cross-dataset generalization.The method uses the same raw point-cloud input as point-based methods, making the comparison direct.
- Contribution: The paper evaluates a simple projection-based baseline rather than claiming architectural novelty.Its components have appeared previously; the contribution is demonstrating that this neglected baseline works surprisingly well.
2. Related Work
Related work spans point-based, projection-based, and voxel-based approaches for point-cloud analysis. The paper contrasts increasingly specialized prior methods with simpler projection choices.
- Point-Based Methods for Point-Cloud Analysis: Point-based methods process 3D points directly, progressing from PointNet’s pointwise MLPs and global max pooling to local-region and spatial-relation modeling.
- Projection-Based Methods for Point-Cloud Classification: Projection-based classification methods have used learned view selection, specialized rendering and feature merging, and ImageNet-pretrained CNN backbones.
- Projection-Based Methods for Point-Cloud Classification: SimpleView instead uses fixed orthogonal views, simpler rendering, a smaller network, and no ImageNet pretraining when protocols are controlled.
- Projection-Based Methods for Other Point-Cloud Analysis Tasks: Projection-based methods also support segmentation, reconstruction, and rendering, with prior view-based approaches aggregating features from multiple rendered views.
- Voxel-Based Methods: Voxel-based methods convert points into fixed 3D grids for 3D CNNs but are usually restricted to lower resolution because of the added dimension.
3. Method Overview
The method overview centers on controlling architecture-independent protocols before evaluating a simple projection baseline. SimpleView projects point clouds into six orthogonal depth images, extracts CNN features, and fuses them for classification.
- Protocol Analysis: The study transfers architecture-independent protocols across PointNet, PointNet++, DGCNN, and RSCNN to separate protocol effects from architectural effects.
- Input and Augmentation Choices: The analysis scopes inputs to fixed sets of 1024 points represented only by x, y, z coordinates, excluding normals and larger point sets.
- Input and Augmentation Choices: DGCNN-style random translation and scaling are used because rotation along the y-axis adversely affects performance on aligned ModelNet40 objects.
- Protocol Components: Smooth-loss improves the performance of all network architectures, while model selection can use validation-tuned final models instead of test-set feedback.
- Results: Under the controlled SimpleView protocol, SimpleView outperforms prior architectures while using the fewest parameters and comparable inference speed.
- SimpleView: SimpleView projects points onto six orthogonal planes as sparse depth images, applies a CNN, and fuses the extracted features for point-cloud classification.
- SimpleView: Perspective projection maps coordinates as x/z and y/z before discretization, while depth images use 128 X 128 resolution.
4. Experiments
Experiments show that protocol choices substantially affect ModelNet40 performance, while SimpleView remains competitive across datasets and training conditions. SimpleView also performs strongly on noisy, misaligned ScanObjectNN data and transfers better across datasets.
- Datasets: ModelNet40 contains 9,840 training objects and 2,468 test objects across 40 aligned categories.
- ModelNet40: Protocol changes raise PointNet++ mean performance from 89.8% to 93.3% and SimpleView mean performance from 90.7% to 93.6%.These results compare PointNet++ no Vote with RSCNN Vote for PointNet++, and PointNet++ no Vote with DGCNN Smooth for SimpleView.
- ModelNet40: SimpleView performs on par with or better than other methods under all evaluated prior protocols, although run variance limits claims about absolute ordering.RSCNN Vote selects the highest accuracy from 300 reshuffled, randomly augmented test evaluations.
- ModelNet40: Under the controlled SimpleView protocol, SimpleView outperforms prior architectures while using the fewest parameters and achieving competitive inference speed.Inference speed is averaged across 100 runs on an NVIDIA 2080Ti.
- Additional comparisons: SimpleView outperforms state-of-the-art methods across different ModelNet40 training-data sizes and remains competitive with additional recent methods.
- ScanObjectNN: On ScanObjectNN, SimpleView outperforms prior networks, reaches 80.5 ± 0.3 accuracy with the best test-model selection scheme, and transfers better across datasets.ScanObjectNN contains noisy, occluded, distorted, and non-axis-aligned real-world scans.
5. Discussion
The discussion argues that protocol factors can obscure architectural contributions and that SimpleView is a strong baseline. The authors recommend controlled comparisons while limiting the analysis to point-cloud classification.
- Findings: Auxiliary factors orthogonal to network architecture have a large effect on point-cloud classification performance.
- Findings: When these factors are controlled, PointNet++ performs competitively with more recent architectures, while a simple baseline performs on par with or better than state-of-the-art architectures.
- Implications: The authors recommend controlling protocols when comparing network architectures and encourage evaluating competing approaches under controlled conditions.
- Limitations: The analysis is limited to point-cloud classification, with scene and part segmentation identified as future directions.