Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook

• Point cloud classification assigns markers to points to identify overall or part properties, and this paper focuses on overall point-cloud attributes.Source: §I · p. 1
• Before PointNet, point-cloud processing relied on hand-designed feature rules and machine-learning classifiers because deep learning could not directly process unordered point clouds.Source: §I · p. 1
• PointNet directly processes point clouds, after which direct point-cloud methods gradually became dominant.Source: §I · p. 2
• Point clouds represent 3D scenes with rich information, but their irregularity, disorder, and sparsity make classification challenging.Source: §I · pp. 1–2
• The paper introduces 3D data and datasets, reviews recent methods, evaluates performance, and analyzes challenges and future trends.Source: §I · p. 2
• The survey classifies methods into multi-view-based, voxel-based, point-cloud-based, and polymorphic fusion-based categories, with further subdivision of point-cloud methods.Source: §I · p. 2

A. 3D data representation

• A point cloud is a collection of 3D points expressing spatial distribution and surface characteristics, with coordinates, color, and surface-normal information.Source: A. 3D data representation · p. 2
• A mesh represents 3D data as points with local relationships, commonly using triangular patches connected by edges.Source: A. 3D data representation · p. 2
• Voxels represent regularly sampled 3D space and can improve computing efficiency and reduce random-memory access after voxelization.Voxelization inevitably causes some information loss.Source: A. 3D data representation · pp. 2–3
• Multi-view representation renders a 3D object from multiple viewpoints, with perspective selection and viewpoint fusion as its main challenges.Source: A. 3D data representation · p. 3

B. Point cloud data storage format

• Point-cloud files differ primarily in ASCII versus binary storage; common formats include FLS, PCD, LAS, PLY, FBX, and E57.Source: B. Point cloud data storage format · p. 3
• OBJ stores geometry, normals, color, and texture in a primarily ASCII text format, while LAS mainly stores LIDAR data in binary form.Source: B. Point cloud data storage format · p. 3
• PLY supports ASCII and binary versions and stores polygonal 3D data with attributes such as color, transparency, normals, textures, and confidence.Source: B. Point cloud data storage format · p. 3
• E57 is vendor-neutral and stores point clouds, attributes, images, and metadata using both ASCII and binary codes.Source: B. Point cloud data storage format · p. 3

• ModelNet40 contains synthetic CAD objects in 40 categories, with 9843 training and 2468 testing shapes.Source: C. 3D point cloud public datasets · p. 3
• ModelNet-C contains 185,000 point clouds with 15 damage types and five severity levels for robustness benchmarking.Source: C. 3D point cloud public datasets · p. 3
• ModelNet10 is a ModelNet40 subset containing 10 classes, divided into 3991 training and 908 testing shapes.Source: C. 3D point cloud public datasets · p. 3
• ShapeNet provides large-scale CAD repositories, including ShapeNetCore with nearly 51,300 models across 55 annotated object categories.Source: C. 3D point cloud public datasets · p. 3
• Real-world datasets include Sydney Urban Objects, ScanNet, and ScanObjectNN, with ScanObjectNN challenging classification through background, missing parts, and deformations.Source: C. 3D point cloud public datasets · pp. 3–4

• The survey supplements recent research and divides classification methods into multi-view, voxel, point-cloud, and polymorphic fusion categories.Source: §III · p. 4

Multi-view-based methods

• Multi-view methods project 3D objects into multiple views, extract view features, and fuse them for classification.Source: A. Multi-view-based methods · p. 4
• MVCNN combines information from multiple rendered views into a compact shape descriptor, improving recognition over a single-view CNN.Source: A. Multi-view-based methods · p. 4
• Later methods address efficiency, view relationships, pooling information loss, and viewpoint selection through GPU acceleration, view grouping, clustering, and differentiable viewpoint prediction.Source: A. Multi-view-based methods · p. 4
• Multi-view methods outperform traditional manual feature extraction but struggle to fully exploit information, scale to large scenes, and model inherent 3D geometric relationships.Source: A. Multi-view-based methods · p. 4

• Voxel-based methods convert point clouds into voxel grids and use 3D CNNs to classify the resulting volumetric representation.Source: B. Voxel-based methods · p. 4
• VoxNet represents 3D information with a volumetric occupancy grid and is described as the earliest voxel-based 3DCNN model.Source: B. Voxel-based methods · p. 4
• OctNet and Kd-network reduce memory consumption and improve computational efficiency by exploiting sparse structure through octrees or Kd-trees.Source: B. Voxel-based methods · pp. 5–6
• Octree and Kd-tree methods cannot fully use local features because voxel boundary values affect representation, which reduces accuracy.Source: B. Voxel-based methods · p. 6
• Voxel-based models address point cloud disorder and unstructuredness, but sparsity and incomplete information limit efficiency and prevent full information use.Source: B. Voxel-based methods · p. 6

• Feature aggregation operators transfer information among discrete points and are divided into local and global aggregation.The survey organizes point-cloud-based methods from this feature-aggregation perspective.Source: C. Point cloud-based method · p. 6
• Point-cloud-based methods process the original points directly rather than converting them into alternate 3D representations.Source: C. Point cloud-based method · p. 6
• PointNet directly accepts point clouds, applies T-Net transformation, shares fully connected layers across points, and uses max pooling for global features.PointNet is described as a pioneering global feature-aggregation method.Source: C. Point cloud-based method · p. 6, Figure 7 · p. 6

Point-based methods

• PointNet++ hierarchically samples neighborhood centroids, groups local subsets, and applies PointNet to learn relationships among neighboring points.Its layers consist of sampling, grouping, and PointNet operations.Source: 1) Local feature aggregation · p. 6, Figure 8 · p. 6
• PointNeXt improves PointNet++ with revised training, separable MLPs, and inverted residual bottlenecks.Source: 1) Local feature aggregation · p. 6, Figure 9 · p. 6
• PointWeb connects neighboring points and uses Adaptive Feature Adjustment to model information transfer between points.The module adaptively adjusts local point features.Source: 1) Local feature aggregation · pp. 6–7
• RandLA-Net uses random sampling and efficient local aggregation to enlarge receptive fields while reducing memory and computational costs.The approach may discard important features from sparse points.Source: 1) Local feature aggregation · p. 7
• PointMLP omits complex local geometry extraction and instead uses a lightweight geometric affine module, improving inference speed.Source: 1) Local feature aggregation · p. 7

Specialized representations

• RepSurf represents point clouds with triangular or umbrella representative surfaces and can be inserted into classification or segmentation frameworks.Source: 1) Local feature aggregation · p. 7

Convolution-based methods

• Convolutional methods adapt CNNs to irregular point clouds through point convolutions, dynamic filters, continuous volume convolutions, and related operators.PointCNN highlights the importance of developing local structures for point-cloud classification networks.Source: 1) Local feature aggregation · pp. 7–8

Graph-based methods

• Graph-based methods represent points as graph vertices and use point relationships or spatially efficient pooling to aggregate features.ECC uses distances as edge weights but has high computational cost; Grid-GCN reduces theoretical time complexity through grid-space efficiency.Source: 1) Local feature aggregation · p. 8

Transformer-based methods

• Point Transformer processes unordered point sets, extracts local and global features, ranks local features, and links them through local-global attention.Source: 2) Global feature aggregation · p. 8, Figure 10 · p. 8
• Point TnT uses a two-stage design so a single point and a point set can attend to each other while addressing inefficient self-attention growth.Source: 2) Global feature aggregation · p. 8
• PCT introduces offset attention with an implicit Laplacian operator and normalization refinement while retaining permutation invariance.Source: 2) Global feature aggregation · p. 9
• Point-BERT masks local point-cloud blocks and applies a backbone Transformer, extending BERT-style learning to point clouds.Source: 2) Global feature aggregation · p. 9
• Point-MAE is a masked-autoencoder approach for point-cloud self-supervised learning that addresses location-information leakage and uneven information density.Source: 2) Global feature aggregation · p. 9
• VoxSeT combines voxel-set attention with Transformer processing and parallel point-cluster management at linear complexity.Source: 2) Global feature aggregation · p. 9

Global module-based methods

• PointASNL combines adaptive sampling with local-nonlocal processing to handle noisy point clouds and reduce learning sensitivity.Source: 2) Global feature aggregation · p. 9, Figure 11 · p. 9

RNN or LSTM-based methods

• Recurrent approaches use contextual sequence modeling, including PointNet extensions for large-scale scenes and CNN-DQN-RNN feature integration.Source: 2) Global feature aggregation · p. 9

• PointGrid mixes points and grids through convolution blocks, pooling, and fully connected layers for point-cloud representation.Source: D. Polymorphic Fusion-based methods · p. 10
• PVT combines point- and voxel-based modeling with sparse-window and relative-attention modules, and is reported to excel in classification accuracy.Source: D. Polymorphic Fusion-based methods · p. 10
• PointCLIP projects point clouds into multi-view depth maps without rendering and transfers pretrained CLIP knowledge for zero-shot recognition.An inter-view adapter extracts global features.Source: D. Polymorphic Fusion-based methods · p. 10, Figure 12 · p. 10
• CrossPoint learns 2D-to-3D correspondence by maximizing invariant-space agreement between point clouds and corresponding rendered 2D images.Source: D. Polymorphic Fusion-based methods · p. 10
• The survey identifies direct point-cloud processing as advantageous over multi-view and voxel methods because it can use point-cloud information fully.It also projects wider future use of Transformer-based methods.Source: D. Polymorphic Fusion-based methods · p. 10

• Accuracy, space complexity, and execution time evaluate point cloud classification methods, with accuracy identified as the key indicator.Source: §IV · p. 10
• Accuracy, precision, recall, and intersection-over-union are commonly used to evaluate classification accuracy.Source: §IV · p. 10
• Accuracy is the ratio of correctly predicted samples to total predicted samples.Source: §IV · p. 10
• Precision measures the proportion of true positive samples predicted as positive, while recall measures predicted positive samples relative to all true positive samples.Source: §IV · p. 10
• Intersection-over-union measures the ratio between the intersection and union of predicted and true values.Source: §IV · p. 10
• Overall accuracy, average accuracy, and mean intersection-over-union are used to measure current point cloud classification accuracy.Source: §IV · p. 10

• Future methods should achieve high accuracy while remaining efficient, because existing approaches tend to prioritize one objective over the other.Source: §V · p. 12
• Outdoor point cloud scenes have complex, changing information structures, so their classification efficiency and accuracy require further improvement.Source: §V · p. 12
• Original-point-cloud methods show performance advantages but use more complicated network models because their inputs preserve information integrity.Source: §V · p. 12
• Simple point-cloud-based methods are identified as a future research trend.Source: §V · p. 12
• Future research should pursue more innovative methods rather than mainly incremental improvements.Source: §V · p. 12
• Network architecture optimization should reduce computational complexity and memory usage while handling complex, irregular point clouds.Source: §V · p. 12

• The paper provides a comprehensive survey and discussion of recent deep learning-based point cloud classification methods.Source: §VI · p. 12
• It introduces point clouds, their applications, characteristics, and processing difficulties.Source: §VI · p. 12
• It summarizes 3D data representations, point cloud storage formats, and commonly used classification datasets.Source: §VI · p. 12
• Deep learning methods are classified into multi-view-based, voxel-based, point-cloud-based, and polymorphic fusion-based categories.Source: §VI · p. 12
• The paper compares existing methods, identifies current problems, and prospects future research directions.Source: §VI · p. 12