Source-linked AI summary
Semantic Classification of 3D Point Clouds with Multiscale Spherical Neighborhoods
Hugues Thomas, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, Yann Le Gall
TL;DR
The paper addresses the challenge of describing 3D point neighborhoods consistently for semantic classification. It proposes multiscale spherical neighborhoods with proportional subsampling, and reports stronger performance than state-of-the-art features while competing with complex methods, including deep learning. The approach remains point-wise and can produce unstructured patches without object coherence.
Problem
Point-wise 3D semantic classification needs expressive neighborhood features, but KNN neighborhoods do not preserve a consistent geometrical meaning across scales.
Method
The paper computes multiscale features from fixed-radius spherical neighborhoods with proportional grid subsampling and uses them for point-wise classification.
Results
The multiscale features outperform state-of-the-art features under the same conditions and compete with more elaborate classification schemes, including deep learning methods.
Takeaways & Limitations
Handcrafted multiscale neighborhood features can remain competitive with complex classification approaches across datasets acquired with different technologies.
Takeaways & Limitations
Point-wise classification has no object coherence, producing unstructured patches in some classified scenes.
Abstract
from arXiv · showhide
This paper introduces a new definition of multiscale neighborhoods in 3D point clouds. This definition, based on spherical neighborhoods and proportional subsampling, allows the computation of features with a consistent geometrical meaning, which is not the case when using k-nearest neighbors. With an appropriate learning strategy, the proposed features can be used in a random forest to classify 3D points. In this semantic classification task, we show that our multiscale features outperform state-of-the-art features using the same experimental conditions. Furthermore, their classification power competes with more elaborate classification approaches including Deep Learning methods.
1. Introduction
The paper targets point-wise 3D semantic classification by improving neighborhood-based geometric features. It introduces multiscale spherical neighborhoods and evaluates them against existing features and more elaborate methods.
- 1. Introduction: Point-wise classification avoids prior segmentation but depends on expressive descriptions of local neighborhood geometry.Segmentation-based approaches require a prior segmentation that does not use semantic information.
- 1. Introduction: Random Forest is identified as the most suitable classifier for geometric features, while spatial coherence can be added through Markov Random Fields.The paper focuses on the initial point-wise classification because subsequent processing benefits from stronger initial results.
- 1. Introduction: The method’s originality lies primarily in selecting spherical rather than KNN neighborhoods for multiscale feature computation.The paper distinguishes spherical neighborhoods by fixed radius from KNN neighborhoods by fixed point count.
- 1. Introduction: Multiscale spherical neighborhoods are introduced to compute geometric features for independently classified 3D points.The framework computes features for every point and then assigns labels independently.
- 1. Introduction: The study first compares multiscale features with state-of-the-art features under identical conditions, then compares classification with elaborate methods across larger datasets.Experiments cover two small outdoor datasets and three larger datasets, with parameter influence examined afterward.
2. Multiscale Spherical Neighborhoods
The proposed multiscale neighborhoods use fixed-radius spherical regions with scale-dependent subsampling, preserving geometric meaning while controlling density and computation. Multiple scales mitigate sparse-data problems and improve feature quality relative to KNN.
- 2. Multiscale Spherical Neighborhoods: Grid-based proportional subsampling controls the maximum neighborhood size, reducing the computational cost caused by many points at large scales or high densities.The grid retains the barycenter of points in each cell; the largest scales are the main computational bottleneck.
- 2. Multiscale Spherical Neighborhoods: A low subsampling ratio can leave too few points for discriminative features, whereas a higher ratio increases computation time.The parameter ρ controls this trade-off between feature sufficiency and computational expense.
- 2. Multiscale Spherical Neighborhoods: Multiple scales limit the effect of low density because larger neighborhoods can provide information when smaller spherical neighborhoods are insufficient.KNN can repeat the same information across scales while falsely describing the smaller scale without a reliability measure.
- 2. Multiscale Spherical Neighborhoods: Neighborhood scales are determined by the smallest radius r0, number of scales S, and consecutive-radius ratio ϕ, with cloud subsampling grid size rs/ρ.At scale s, the radius is rs = r0 ∗ϕs.
- 2. Multiscale Spherical Neighborhoods: Spherical neighborhoods describe the same-sized spatial region at each scale, unlike KNN, and neighborhood point count becomes an informative feature.This gives features a more consistent geometrical meaning and improves their reported performance.
3. Point-wise Semantic Classification
The section defines the feature sets and learning strategies used to classify 3D points, addressing large, imbalanced point clouds while evaluating generalization and feature descriptive power.
- Feature sets: Features are evaluated without intensity, color, or multispectral information in the first experiment, preserving prior experimental conditions.Additional color features are used in later experiments when available.
- Geometric features: The covariance-based feature set uses eigenvalue-derived geometry descriptors, verticality, and moments around eigenvectors.Features include eigenvalue statistics, shape measures, verticality for linear and planar structures, and first- and second-order moments.
- Geometric features: The neighborhood construction makes eigenvalue magnitudes density-consistent, so non-ratio features such as eigenvalue sums remain meaningful without normalization.The paper states that eigenvalue values do not vary with the original point-cloud density under these neighborhoods.
- Learning strategy: Small-cloud experiments balance classes by randomly selecting equal numbers of points per class and repeat trials with different training sets.This design supports comparison of feature descriptive power while accounting for variability from training-set selection.
- Evaluation: Larger separated training and test clouds provide a more relevant measure of generalization to unseen data than small test subsets.The paper distinguishes descriptive-power comparisons from evaluation of generalization performance.
- Learning strategy: Iterative training-point selection adds misclassified points to the training set before testing on larger unseen clouds.A classifier is repeatedly trained on points T, evaluated on training clouds U, and augmented with randomly selected misclassified points.
4. Experiments
The experiments compare multiscale spherical features with state-of-the-art features and elaborate classification methods across small and large 3D datasets. Results show strong benchmark performance, cross-city transfer, and a trade-off between feature quality and computation speed as subsampling increases.
- 4.1. State of the art features comparison: 24 mean IoU points on Rue Madame and 11 mean IoU points on Rue Cassette separate the multiscale features from previous state-of-the-art results.The comparison uses the same random-forest conditions as prior feature studies, and repeated trials produced standard deviations limited to a few percent.
- 4.1. State of the art features comparison: The multiscale features outperform state-of-the-art features, and the low standard deviation supports the validity of comparisons under random training-point selection.The reported experiments were reproduced 500 times to assess consistency despite randomness in selecting training points.
- 4.2. Results on large scale data: On Semantic3D, the classification method ranked second at submission time, exceeded Hackel et al., and was outperformed only by the same cutting-edge algorithm.The experiment uses the reduced-8 challenge and retains the previously used outdoor parameters.
- 4.2. Results on large scale data: The classifier transfers knowledge from Lille to Paris despite different architectural styles and is particularly efficient on buildings.Paris-Lille-3D uses a cross-city split, with training on Lille and testing on Paris.
- 4.2. Results on large scale data: The RF_MSSF classifier ranks among the best approaches on large datasets, beating nearly every elaborate method except Superpoint Graphs and competing with deep-learning methods.The classifier uses a smart choice of training points and is intended to isolate the point-wise descriptive power of the features without segmentation or regularization.
- 4.3. Parameter influence: Increasing ρ improves feature quality but slows computation because ρ controls the number of subsampled points in each neighborhood.The parameter study is conducted on the cross-city Paris-Lille-3D setting to focus on 3D descriptors and test generalization.
5. Conclusion
The proposed multiscale spherical features improve 3D point-cloud semantic classification and remain consistent across datasets acquired with different technologies and environments. They outperform recent complex schemes, including deep learning methods, while suggesting that handcrafted neighborhood ideas may benefit other frameworks.
- Its performance remains consistent across three datasets acquired with different technologies and in different environments.
- The method is built around new multiscale features for 3D point-cloud semantic classification.
- The approach outperforms recent and complex classification schemes, including deep learning methods, on large-scale datasets.
- The authors argue that handcrafted methods still have room for improvement despite deep learning becoming standard for several classification tasks.
- The multiscale neighborhood definition could benefit other frameworks, including deep learning.