Source-linked AI summary
Directionally Constrained Fully Convolutional Neural Network For Airborne Lidar Point Cloud Classification
Congcong Wen, Lina Yang, Ling Peng, Xiang Li, Tianhe Chi
TL;DR
Airborne LiDAR point-cloud classification is challenging because the data are unstructured, noisy, unevenly distributed, and incomplete. D-FCN labels raw point clouds using directionally constrained, orientation-aware convolution and multiscale learning. On the ISPRS benchmark, it achieves state-of-the-art average F1 and reported gains over state-of-the-art methods for powerline, car, and facade categories.
Problem
Airborne LiDAR point-cloud classification is challenging because unstructured data, limited resolution, sensor noise, uneven density, incompleteness, and scene complexity complicate classification.
Source: §1 · p. 1Method
D-FCN uses raw 3D coordinates and intensity with directionally constrained point convolution and a multiscale fully convolutional network to label all points end to end.
Source: §1 · p. 2, §3.1 · p. 3Results
8.4%, 3.4%, and 7.3% gains over state-of-the-art methods on powerline, car, and facade, respectively, accompanied a new state-of-the-art average F1 result on the ISPRS benchmark.
Source: §5 · p. 11Takeaways & Limitations
The reported benchmark results show that D-FCN classifies airborne LiDAR point clouds using only raw coordinates and intensity, without additional handcrafted geometry features.
Source: §5 · p. 11The model did not achieve the best overall accuracy because its initial parameters were selected using average F1 as the main indicator.
Source: §4.7 · p. 9
Abstract
from arXiv · showhide
Point cloud classification plays an important role in a wide range of airborne light detection and ranging (LiDAR) applications, such as topographic mapping, forest monitoring, power line detection, and road detection. However, due to the sensor noise, high redundancy, incompleteness, and complexity of airborne LiDAR systems, point cloud classification is challenging. In this paper, we proposed a directionally constrained fully convolutional neural network (D-FCN) that can take the original 3D coordinates and LiDAR intensity as input; thus, it can directly apply to unstructured 3D point clouds for semantic labeling. Specifically, we first introduce a novel directionally constrained point convolution (D-Conv) module to extract locally representative features of 3D point sets from the projected 2D receptive fields. To make full use of the orientation information of neighborhood points, the proposed D-Conv module performs convolution in an orientation-aware manner by using a directionally constrained nearest neighborhood search. Then, we designed a multiscale fully convolutional neural network with downsampling and upsampling blocks to enable multiscale point feature learning. The proposed D-FCN model can therefore process input point cloud with arbitrary sizes and directly predict the semantic labels for all the input points in an end-to-end manner. Without involving additional geometry features as input, the proposed method has demonstrated superior performance on the International Society for Photogrammetry and Remote Sensing (ISPRS) 3D labeling benchmark dataset. The results show that our model has achieved a new state-of-the-art level of performance with an average F1 score of 70.7%, and it has improved the performance by a large margin on categories with a small number of points (such as powerline, car, and facade).
1. INTRODUCTION
Airborne LiDAR point-cloud classification is difficult because point sets are unstructured and noisy, unevenly distributed, incomplete, and complex. The paper introduces D-FCN to label raw point clouds using orientation-aware convolution and multiscale feature learning.
1. INTRODUCTION
- Airborne LiDAR classification is challenging because point clouds are unstructured, have limited resolution and sensor noise, and vary in density, completeness, and scene complexity.Source: §1 · p. 1
- D-FCN takes raw 3D coordinates and LiDAR intensity as input and predicts semantic labels for every point end to end, including for arbitrary-sized point clouds.Source: §1 · p. 2
- D-Conv extracts local features from projected 2D receptive fields using directionally constrained nearest-neighbor search for orientation-aware convolution.Source: §1 · p. 2
- Downsampling and upsampling blocks form a multiscale fully convolutional network for learning point features at multiple scales.Source: §1 · p. 2
- The proposed method demonstrates superior performance on the ISPRS 3D labeling benchmark without additional geometry features as input.Source: §1 · p. 2
2. RELATED WORK
Earlier airborne LiDAR classification approaches rely on local handcrafted features, contextual models, or feature-image pipelines, each with stated limitations. Deep point-cloud methods learn directly from unstructured sets, but airborne LiDAR approaches still use handcrafted inputs; D-FCN is presented as a raw-input alternative.
2.1 Point-based feature classification method
- Single-point feature methods use handcrafted local geometry with classifiers, but uneven point distributions make feature estimates unstable and can produce noisy, inconsistent labels.Examples include density, curvature, and roughness descriptors used with methods such as SVMs, AdaBoost, and random forests.Source: §2.1 · p. 2
2.2 Context-based features classification method
- Context-based models incorporate neighborhood information, but manual context-feature extraction complicates classification and performance degrades on complex-scene point clouds.The AMN model is reported to risk oversmoothing small and large objects and error propagation across large regions.Source: §2.2 · p. 2
2.3 Deep learning-based classification method
- Deep-learning point-cloud classification is grouped into feature-image-based and point-cloud-based approaches.The shift toward learned 3D point signatures follows the broader uptake of deep learning.Source: §2.3 · p. 2
2.3.1 Feature image based classification method
- Feature-image methods transform point-cloud attributes into 2D images for CNN classification, but conversion can lose spatial information and still requires handcrafted feature engineering.The selected features and manually defined combinations can substantially influence final classification results.Source: §2.3.1 · p. 3
2.3.2 Point cloud-based classification method
- PointNet learns from unstructured point sets, and PointNet++ adds downsampling and upsampling modules; airborne LiDAR point-based methods still use handcrafted features, unlike D-FCN's raw-input approach.D-FCN is described as learning orientation-aware features and reaching state-of-the-art classification performance using raw coordinates and intensity without additional handcrafted features.Source: §2.3.2 · p. 3
3. METHODS
D-FCN combines directionally constrained local point convolution with an encoder-decoder network to produce semantic labels for input points. Its method defines projected neighborhood search, orientation-aware feature encoding, multiscale sampling, and evaluation metrics.
3. METHODS
- D-FCN uses an encode-decode framework, applying D-Conv before downsampling and interleaving it with feature propagation during upsampling.Skip connections carry features from the downsampling path to the upsampling path.Source: §3 · p. 3
3.1 Directionally constrained point convolution
- D-Conv constructs a local receptive field for each point, applies kernel-weighted convolution to its features, and outputs a new feature vector per point.For input P ∈ R^N×3 and feature matrix F ∈ R^N×d_in, it produces O ∈ R^N×d_out.Source: §3.1 · p. 3
3.1.1 Directionally Constrained Nearest Neighbor
- The neighborhood search projects points onto the xy plane, partitions it into N_d triangles, and selects K nearest points within radius R in each area.With no point in a triangle within R, the central point is duplicated there; the default N_d is 8.Source: §3.1.1 · pp. 3–4, Figure 1 · p. 4
3.1.2 Orientation-Aware Point Feature Encoding
- D-Conv preserves directional order by aggregating K neighbors within each triangle, convolving across the N_d directional features, and adding the output to the input.Each D-Conv module uses two convolution blocks.Source: §3.1.2 · pp. 4–5, Figure 3 · p. 4
3.2.1 Downsampling and Upsampling blocks
- Downsampling uses farthest-point sampling and neighborhood convolution; upsampling interpolates dropped-point features from nearest neighbors and concatenates same-scale features.The default neighborhood size is 32 for both modules.Source: §3.2.1 · p. 5
3.2.2 Overall architecture
- The network downsamples point sets to 1,024, 256, and 64 points, then upsamples to 256, 1,024, and 8,192 points for dense prediction.D-Conv precedes each sampling operation, and skip connections concatenate same-size features across the two paths.Source: §3.2.2 · p. 5, Figure 4 · p. 6
3.3 Evaluation metrics
- Evaluation uses precision, recall, F1 score, and overall accuracy; F1 accounts for precision and recall and is generally more suitable for uneven class distributions.Overall accuracy is the percentage of correctly classified points among all test points.Source: §3.3 · p. 5
4. EXPERIMENTS AND DISCUSSION
The experiments evaluate D-FCN for airborne LiDAR point-cloud classification, progressing from dataset and training strategies through module and hyperparameter analyses to final results and model comparisons.
4. EXPERIMENTS AND DISCUSSION
- The experiments assess D-FCN classification performance and compare the optimized model with other state-of-the-art airborne LiDAR classifiers.The paper also evaluates class balance, data augmentation, the D-Conv module, and hyperparameter configurations.Source: §4 · p. 6
- The study evaluates how the directionally constrained point convolution module affects classification performance.This is followed by an analysis of different hyperparameter configurations.Source: §4 · p. 6
- The experiments include patch sampling and class-balance strategies before reporting results with optimal hyperparameters.The section begins by introducing the experimental dataset and setup.Source: §4 · p. 6
4.1 Experimental dataset
The evaluation uses the ISPRS 3D labeling dataset, whose LiDAR scenes contain nine semantic categories. Under the contest split, Scene I is used for training and Scenes II and III for testing.
4.1 Experimental dataset
- The ISPRS dataset contains nine labeled categories, including powerline, low vegetation, impervious surface, car, fence/hedge, roof, facade, shrub, and tree.The airborne point clouds were captured with a Leica ALS50 at an average height of 500 meters and a 45-degree field of view.Source: §4.1 · p. 6
- The dataset stores XYZ coordinates, intensity, number of returns, and point labels in LiDAR point files.Source: §4.1 · p. 6
- Scene I, with 753,876 points, is used for training, while Scenes II and III together provide 411,722 test points.Test labels are withheld during training and used only for evaluation.Source: §4.1 · p. 6
4.2 Class balance and data augmentation strategies
The training data are highly imbalanced, so the method weights categories in its loss and augments training blocks. Test scenes are divided into horizontal blocks that D-FCN can process even when their point counts vary.
4.2 Class balance and data augmentation strategies
- Scene I contains 546 powerline points but 193,723 impervious-surface points, illustrating the severe category imbalance.The paper notes that direct training on the imbalanced data may harm overall deep-network performance.Source: §4.2 · p. 6
- Category-specific weights are added to the loss using the reciprocal of a logarithm of each category’s point-count percentage.The weights are intended to direct more attention toward minority categories; α is the class-balance coefficient.Source: §4.2 · p. 6
- Training samples are drawn from randomly selected 30 m × 30 m × 40 m cuboids, with 8,192 points selected per cuboid and random point dropout applied.The default dropout ratio is 12.5%.Source: §4.2 · pp. 6–7
- Figure 5 distinguishes the training Scene I from evaluation Scenes II and III, with point categories shown in contest colors.The category-distribution table reports point counts by scene, but the supplied caption gives no cell values.Source: Figure 5 · p. 7, Table 1 · p. 7
- Test scenes are split into 30 m × 30 m horizontal blocks, and variable-sized blocks can be input directly to the fully convolutional model.Small edge blocks are merged with surrounding larger blocks to preserve block integrity.Source: §4.2 · p. 7
4.3 Experimental setup
The model is trained in TensorFlow with Adam and a scheduled learning-rate reduction, then applied to test blocks in a forward pass whose predictions are merged into scene-level segmentation results.
4.3 Experimental setup
- Training uses batch size 6 and Adam with an initial learning rate of 0.01, halved every 3,000 steps.Training for 1,000 epochs to convergence took approximately 10 hours on a TESLA K80 GPU.Source: §4.3 · p. 7
- Each test block is classified in one network forward pass, and block predictions are merged into the final point-cloud segmentation.Source: §4.3 · p. 7
4.4 Effect of directionally constrained point convolutions
The experiments compare four neighborhood-partition strategies using identical network configurations. The authors select an eight-directional search in 2D space based on the reported classification comparisons.
4.4 Effect of directionally constrained point convolutions
- The four evaluated models used identical network configurations and varied whether directional search was used, its dimensionality, and its number of directions.The compared strategies were no directional convolution, eight directions in 3D, four directions in 2D, and eight directions in 2D.Source: §4.4 · p. 7, Table 2 · p. 8
- The qualitative classification results for the four neighborhood-partition strategies are shown in Figure 7.The circled regions highlight the most obvious differences among the models.Source: §4.4 · p. 8, Table 2 · p. 8
- The eight-directional 2D search outperformed the eight-directional 3D search and achieved better results than the four-directional 2D search.The authors attribute the 2D advantage probably to richer informative neighbors and use the eight-directional 2D configuration thereafter.Source: §4.4 · p. 8, Table 2 · p. 8
4.5 Model hyperparameters
The experiments tune sampling density, neighborhood size, and class balance, using average F1 score to select hyperparameters. The preferred settings vary with sampling density, while class balancing improves average F1 at the cost of the highest overall accuracy.
4.5 Model hyperparameters
- Average F1 score was used to select the best hyperparameters, including sampling-point count, neighbor-point count, and class-balance coefficient.The experiments varied N and K together, then compared class-balance coefficient settings.Source: §4.5 · p. 8
4.5.1 Number of sampling points and neighbor points
- 82.2% overall accuracy and 70.7% average F1 score were achieved with N = 8,192 and K = 2.With fewer sampled points, K = 1 performed better; with more points, increasing K to 2 helped, while K = 4 could reduce performance when N was sufficiently large.Source: §4.5.1 · p. 8, Table 3 · p. 9
4.5.2 Weight coefficient of data balance
- 84.6% overall accuracy was achieved without class balancing, while the highest average F1 score, 70.7%, came with class balancing at α = 1.2.The authors report that class balancing improved average F1, while no balancing could leave minority categories such as powerline and car with lower accuracy.Source: §4.5.2 · p. 9, Table 4 · p. 9
4.6 Classification results
On the ISPRS test scenes, D-FCN correctly labels most points, with strong F1 scores across six categories. Fence/hedge remains a weaker category, with many points confused with shrubs.
4.6 Classification results
- Most points in the test scenes received correct labels from D-FCN.The reported classification results and error maps are shown in Figures 8 and 9; Figure 8 marks incorrect points in red and correct points in green.Source: §4.6 · p. 9, Figure 8 · p. 9
- F1 scores exceeded 70% for powerline, low vegetation, impervious surfaces, car, roof, and tree.The authors also report reasonable classification performance on facade, but relatively lower performance on fence/hedge.Source: §4.6 · p. 9
- Many fence/hedge points were classified as shrubs, which the authors associate with the categories' similar spatial distribution and topology and the limited fence/hedge samples.The authors state these factors leave the model insufficiently trained to distinguish the categories.Source: §4.6 · p. 9
4.7 Comparisons with other methods
On the ISPRS benchmark, D-FCN achieved higher average F1 than the compared state-of-the-art models, with particularly large gains for powerline, car, and facade. It did not achieve the best overall classification accuracy, reflecting the study's emphasis on average F1.
4.7 Comparisons with other methods
- D-FCN did not achieve the best overall classification accuracy among the compared state-of-the-art models.The authors say they selected initial parameters primarily by average F1 rather than overall accuracy, because using overall accuracy as the main metric can cause minority categories to be ignored.Source: §4.7 · pp. 9, 11
- D-FCN outperformed the compared state-of-the-art models in average F1, with gains of 8.4% for powerline, 3.4% for car, and 7.3% for facade.The comparison covers the ISPRS benchmark; Table 6 reports per-category F1, overall accuracy, and average F1.Source: §4.7 · p. 9, Table 6 · p. 11
- In a selected complicated scene, D-FCN classified powerline points more accurately than RIT 1, NANJ2, and WhuY4, which misclassified most of them.The test-set powerline category contained 600 points, or 0.1% of the 411,722 points.Source: §4.7 · p. 11, Figure 10 · p. 11
5. CONCLUSIONS
The conclusion presents D-FCN as a multiscale model for directly labeling unstructured airborne LiDAR point clouds. It reports state-of-the-art average-F1 performance using only coordinates and intensity, with gains on powerline, car, and facade.
5. CONCLUSIONS
- D-Conv extracts locally representative features from projected 2D receptive fields, alongside a multiscale fully convolutional network for point-feature learning.Source: §5 · p. 11
- Using only original 3D coordinates and intensity, D-FCN achieved new state-of-the-art classification performance for average F1 score.The passage does not give a numeric average F1 value.Source: §5 · p. 11
- D-FCN outperformed state-of-the-art methods by 8.4% on powerline, 3.4% on car, and 7.3% on facade.Source: §5 · p. 11
- D-FCN processes input point clouds of arbitrary sizes and directly predicts semantic labels for all input points end to end.Source: §5 · p. 11