PaperlayerView exact revision on arXiv

Source-linked AI summary

Linked Dynamic Graph CNN: Learning on Point Cloud via Linking Hierarchical Features

Kuangen Zhang, Ming Hao, Jing Wang, Clarence W. de Silva, Chenglong Fu

arXiv:1904.10014v2cs.CV

TL;DR

Point clouds are sparse, unstructured, and unordered, making direct recognition difficult for conventional CNNs and RNNs. The paper proposes LDGCNN, which links hierarchical dynamic-graph features, removes the transformation network, and retrains the classifier; it reports state-of-the-art performance on ModelNet40 and ShapeNet.

  • Problem

    Point clouds are sparse, unstructured, and unordered, so conventional CNNs and RNNs are not suitable for recognizing them directly.

  • Method

    LDGCNN links hierarchical features from dynamic graphs, removes the transformation network, and freezes the feature extractor while retraining the classifier.

  • Results

    LDGCNN achieves state-of-the-art performance on ModelNet40 and ShapeNet, reaching 92.9% overall classification accuracy on ModelNet40.

  • Takeaways & Limitations

    The proposed architecture reduces model size relative to DGCNN while supporting direct point-cloud classification and segmentation.

Abstract

from arXiv · show

Learning on point cloud is eagerly in demand because the point cloud is a common type of geometric data and can aid robots to understand environments robustly. However, the point cloud is sparse, unstructured, and unordered, which cannot be recognized accurately by a traditional convolutional neural network (CNN) nor a recurrent neural network (RNN). Fortunately, a graph convolutional neural network (Graph CNN) can process sparse and unordered data. Hence, we propose a linked dynamic graph CNN (LDGCNN) to classify and segment point cloud directly in this paper. We remove the transformation network, link hierarchical features from dynamic graphs, freeze feature extractor, and retrain the classifier to increase the performance of LDGCNN. We explain our network using theoretical analysis and visualization. Through experiments, we show that the proposed LDGCNN achieves state-of-art performance on two standard datasets: ModelNet40 and ShapeNet.

I. INTRODUCTION

Point clouds provide robust 3D geometric information for robotic perception but are difficult for conventional CNNs and RNNs because they are sparse, unstructured, and unordered. The paper therefore focuses on directly classifying and segmenting point clouds using graph-based learning.

  • 3D sensors provide spatial information that is less affected by illumination than RGB images.The passage identifies depth cameras, LiDAR scanners, and radars as sources of 3D geometric data.
  • Point clouds are common representations of 3D geometric data and can also be obtained from meshes, voxels, and depth images.
  • The paper studies two tasks: classifying an entire point cloud and assigning a category to each point for segmentation.
  • Traditional CNNs and RNNs are unsuitable for direct point-cloud recognition because point clouds are sparse, unstructured, and unordered.
  • View-based methods can achieve high classification accuracy but are difficult to apply to point-cloud segmentation.
  • Voxel-based methods waste resources on sparse data and face high memory and computational costs, often requiring low grid resolution.

C. Geometric method

Geometric point-cloud methods learn directly from points and their local relationships. LDGCNN extends dynamic graph processing by linking hierarchical features, removing the transformation network, and retraining its classifier.

  • Geometric method: PointNet processes points independently, whereas PointNet++ adds local features through nested partitions and multiple scales.
  • Geometric method: Edge-convolution methods model relationships between a center point and its neighbors to extract local geometric features.
  • Geometric method: DGCNN is practical for sparse sensor data but relies on a transformation network, similar deep features, and many trainable parameters.
  • Geometric method: LDGCNN achieves state-of-the-art performance on ModelNet40 and ShapeNet for classification and segmentation.
  • Geometric method: LDGCNN links hierarchical features from different dynamic graphs to calculate informative edge vectors and avoid vanishing gradients.
  • Geometric method: LDGCNN removes DGCNN’s transformation network and uses MLPs to extract transformation-invariant features.
  • Geometric method: The network freezes its feature extractor and retrains the classifier to improve performance.

II. MATERIALS

The experiments use ModelNet40 for whole-point-cloud classification and ShapeNet parts for point-wise segmentation. The method constructs directed local graphs and maps point clouds or individual points to category distributions.

  • Datasets: ModelNet40 contains 12,311 CAD models across 40 categories, split into 9,843 training and 2,468 validation models.Each model is sampled to 1,024 normalized points.
  • Datasets: ShapeNet part contains 16,881 CAD models from 16 categories, with each point annotated as one of 50 parts.The cited passage states that models are sampled to 2,048 points.
  • Method: LDGCNN constructs a directed graph, extracts graph features, and uses them for point-cloud classification and segmentation.
  • Problem formulation: The method takes a point cloud as a set of 3D points, where each point has coordinates x_i, y_i, and z_i.
  • Problem formulation: For classification, a function f_c maps the whole point cloud to a probability distribution over categories.
  • Problem formulation: For segmentation, a function f_s computes a category probability distribution for each point.

1) Permutation invariance:

The problem formulation requires predictions to remain stable when point order changes and when the observed cloud undergoes translation or rotation. These invariance requirements reflect point clouds’ set-based representation and changing sensor poses.

  • Permutation invariance: Point-cloud classification must be invariant to permutations of the input points because a point cloud is a set rather than a sequence.
  • Transformation invariance: Classification and segmentation results should remain unchanged under the translation and rotation induced by changes in sensor-object pose.

3) Extracting local features:

The method represents a point cloud as a locally directed graph, using each point and its K nearest neighbors to capture local relationships while preserving order invariance.

  • Local features model relationships between each point and its neighbors rather than processing points individually.
  • A graph represents point-cloud data with vertices and edges, allowing node-wise propagation independent of input order.
  • K-NN constructs a locally directed graph because fully connected edges consume large memory.
  • Each point pi serves as a central node connected to its K nearest neighbors through directed edges.
  • K-NN is applied before each convolutional layer, constructing local graphs in both Euclidean and feature spaces.

C. Graph feature extraction

Graph feature extraction applies a shared edge-convolution function to each central point and its neighbors, then uses max pooling to obtain permutation-insensitive local features.

  • Edge convolution maps the local graph around central point pi to a local feature li.
  • The same feature-extraction function is applied to all points, using one central point and its K neighbors as the operation template.
  • The hidden feature h(pi, eij) represents the central point and one directed edge vector.
  • Max pooling selects the predominant edge feature without depending on neighbor order.
  • An MLP extracts hidden edge features from central-point and neighbor channels, transforming C input channels into C′ output channels.

D. Transformation invariant function

The transformation-invariant design replaces the transformation network with a shared MLP and training augmentation, using multiple channel-wise transformations to approximate rotation invariance.

  • The matrix R operates on each point’s coordinates xi, yi, and zi through its rotation entries rij.
  • The MLP can be written in matrix form, producing hidden feature vectors from point coordinates, weights, and biases.
  • Unlike the transformation network’s point-cloud-specific matrix, the MLP is static across point clouds but can apply channel-specific rotations and translations.
  • With at least 64 output channels, the MLP observes point clouds along at least 64 directions, which can make the network approximately rotation invariant.
  • Random rotations, shifts, scaling, and point noise augment training data so the network can learn rotation- and translation-invariant features.

E. LDGCNN architecture

LDGCNN differs from DGCNN by linking hierarchical features across dynamic graphs and removing the transformation network, enabling informative edge vectors and addressing deep-network optimization issues.

  • LDGCNN architecture: LDGCNN links hierarchical features from different layers of dynamic graphs.
  • LDGCNN architecture: The architecture removes DGCNN’s transformation network, whose rationale is discussed in the transformation-invariant design.
  • LDGCNN architecture: Hierarchical feature links can help avoid vanishing gradients in deep neural networks.
  • LDGCNN architecture: Current neighborhood indices can extract new edges from previous features, producing informative edge vectors when current neighbors are similar.

F. Freezing feature extractor and retraining classifier

LDGCNN improves classification by freezing the trained feature extractor and retraining only the classifier. This strategy reduces the optimization burden and is evaluated on ModelNet40 under the paper’s stated training setup.

  • F. Freezing feature extractor and retraining classifier: Freezing the feature extractor restricts backpropagation to the classifier’s three fully connected layers.The authors save global features after whole-network training and use them to train the classifier separately.
  • F. Freezing feature extractor and retraining classifier: The network is first optimized with Adam at a 0.001 learning rate, then the classifier is retrained using Momentum.Classification uses 1024 input points, batch size 32, momentum 0.9, K=20 neighbors, and dropout rate 0.5.
  • F. Freezing feature extractor and retraining classifier: Classification training uses one NVIDIA TITAN V GPU, while segmentation training uses two.
  • F. Freezing feature extractor and retraining classifier: 92.9% overall accuracy is achieved on ModelNet40 with 1024 input points, 0.7% higher than DGCNN and PointCNN.The paper reports that LDGCNN attains the highest accuracy on this dataset under the stated comparison.
  • F. Freezing feature extractor and retraining classifier: The authors compare their classification network on ModelNet40 while noting differences in how prior work uses testing data for model selection.They state that prior papers may report best testing accuracy, whereas last-epoch accuracy can be 1%-1.5% lower.

C. Point cloud segmentation

The paper applies LDGCNN to point-wise segmentation and evaluates it on ShapeNet using mean IoU. Its architecture supports both classification and segmentation, while experiments examine efficiency, feature separability, and component effects.

  • C. Point cloud segmentation: ShapeNet segmentation is evaluated with Intersection-over-Union, computed from predicted point labels and ground-truth point labels.IoU is the ratio of points in the intersection to points in the union.
  • C. Point cloud segmentation: LDGCNN’s segmentation architecture predicts labels for individual points, while its classification architecture outputs scores for m classes from an n-point cloud.The architecture uses shared MLPs and edge convolutional layers; the segmentation MLP has two layers with C′ channels.
  • D. Time and space complexity analysis: LDGCNN’s model size is smaller than PointNet, PointNet++, and DGCNN, while its forward time is shorter than DGCNN but longer than PointNet++ and PointNet.The longer forward time than PointNet++ and PointNet is attributed to K-NN neighbor searches in each edge convolutional layer.
  • E. Visualization and ablation experiments: T-SNE visualization is used to compare original point clouds with extracted features by reducing high-dimensional features to a visual representation.The paper reports that extracted features are more discriminative than the original point cloud.
  • E. Visualization and ablation experiments: Freezing the feature extractor and retraining the classifier increases overall classification accuracy from 91.8% to 92.9%.The transformation network does not increase the network’s performance in the reported ablation.
  • C. Point cloud segmentation: The paper compares segmentation outputs among ground truth, LDGCNN, and DGCNN in qualitative results.

V. CONCLUSIONS

The paper concludes that LDGCNN directly classifies and segments point clouds by linking hierarchical dynamic-graph features and refining the classifier after feature extraction. It reports state-of-the-art performance on ModelNet40 and ShapeNet and identifies broader evaluation as future work.

  • V. CONCLUSIONS: LDGCNN directly performs point-cloud classification and segmentation while linking hierarchical features from different dynamic graphs.
  • V. CONCLUSIONS: Removing the transformation network decreases model size, and classifier retraining raises ModelNet40 classification accuracy from 91.8% to 92.9%.
  • V. CONCLUSIONS: The network achieves state-of-the-art performance on the ModelNet40 and ShapeNet datasets.The paper also provides theoretical analysis and visualizations of classification and segmentation results.
  • V. CONCLUSIONS: Future work will evaluate LDGCNN on more semantic-segmentation datasets and apply it to real-time environmental understanding.
Loading 1904.10014v2…