Unsupervised Multi-Task Feature Learning on Point Clouds

• Point clouds are sparse, order-invariant sets sampled from object surfaces and used in sensing applications such as self-driving and robotics.Source: §1 · p. 1
• Deep models can perform many point-cloud tasks, but their feature learning typically requires large amounts of labeled data.Source: §1 · p. 1
• Existing unsupervised approaches use autoencoders or generative models, but often lack supervisory signals and target a single task.Source: §1 · p. 1
• The proposed model jointly uses self-supervision, autoencoding, and clustering to learn point and shape features while avoiding trivial solutions.Source: §1 · p. 1

• Point-cloud deep models include PointNet, hierarchical PointNet++, kd-tree architectures, RNNs, and graph-based convolutional models.Source: §2.1 · p. 2
• Generative approaches include multiresolution VAEs, autoregressive PointGrow, GANs, and GMMs trained on PointNet features.Source: §2.1 · p. 2
• Autoencoder methods reconstruct point clouds or point-pair feature spaces using architectures such as FoldingNet, PPF-FoldNet, AtlasNet, and SO-Net.Source: §2.1 · p. 2
• Self-supervised learning creates proxy tasks on unlabeled data, but its main challenge is designing tasks related to downstream objectives.Source: §2.2 · p. 2
• Multi-task learning shares features across relevant tasks, and prior work has applied self-supervised multi-task learning to high-level visual representation learning.Source: §2.3 · p. 2

Representation

• The encoder maps each point set to higher-dimensional point features and aggregates them into a shape feature.The point features encode local information, while aggregation produces a global representation.Source: §3 · p. 2

Training tasks

• Clustering maps latent shape features to K categories and supplies hard cluster assignments as pseudo-labels for self-supervised classification.The classifier predicts the assignments generated by the clustering function.Source: §3 · p. 3, §3.1 · p. 3
• The decoder reconstructs the original point set from the latent variable, while Chamfer distance measures differences between the original and reconstructed clouds.Chamfer distance compares nearest neighbors in both directions.Source: §3 · p. 3, §3.1 · p. 3

Optimization

• Centroid updates occur epoch-wise rather than batch-wise, and reconstruction is introduced to prevent convergence to trivial solutions.The model can empty some clusters and converge to a fixed number when initialized with an upper bound.Source: §3.1 · p. 3, §3 · p. 3
• The model minimizes a weighted linear combination of clustering, classification, and decoding objectives and trains the components jointly.The training procedure uses mini-batch stochastic gradient descent after centroid initialization.Source: §3.1 · p. 3

Architecture

• A multi-scale graph-based encoder applies graph convolutions, convolutions, and pooling to learn point and shape features from jittered point clouds.The architecture uses three neighborhood scales for graph convolution.Source: §3.2 · p. 3

Feature hierarchy

• Intermediate features pass through convolution and pooling layers to learn another representation level.The architecture progressively builds higher-level features before producing the final shape representation.Source: §20 · p. 4
• Point-wise features are pooled into a final shape feature and concatenated with it to form the final point features.This connects global shape information with local point representations.Source: §20 · p. 4

Multi-task learning

• Clustering, classification, and autoencoding are defined on shape features and trained jointly end-to-end.The classifier predicts cluster assignments, while the autoencoder reconstructs the original point cloud.Source: §20 · p. 4
• Because shape features aggregate corresponding point features, learning a good shape feature also pushes the model toward good point features.The paper explicitly links shape-level supervision to point-level representation learning.Source: §20 · p. 4

Experimental Protocol

• The model is pre-trained across 55 ShapeNet categories, then transferred to classification and part segmentation without fine-tuning its weights.ShapeNet pre-training uses 57,000 models, and the transferred features support downstream evaluation.Source: §4.2 · p. 4

Shape Classification

• NMI reaches 0.68 on the train set and 0.62 on validation, indicating that learned clusters encode category-level information without ground-truth labels.The model converges from 500 to 88 clusters, empties over 80% of them, and avoids collapsing to one cluster.Source: §4.2 · p. 5
• The learned shape features achieve state-of-the-art ModelNet40 accuracy among unsupervised methods and reach 68.88% accuracy in zero-shot clustered classification.The standard transfer result is obtained with a linear SVM and no fine-tuning; random hyper-parameter search raises accuracy to 89.5%.Source: §4.3 · pp. 5–6

Part Segmentation

• The learned point features achieve state-of-the-art accuracy and mIoU on ShapeNetPart in semi-supervised evaluation using 1% and 5% of the training set.A four-layer MLP is trained on sampled labels while the pre-trained feature extractor remains frozen.Source: §4.4 · p. 6
• HDBSCAN produces decent fully unsupervised segmentation, while co-clustering shows consistent learned point features within categories.The qualitative analyses include airplane co-clustering and comparisons with ground-truth labels, MLP predictions, and AHC.Source: §4.4 · p. 7

Ablation Study

• The graph-based encoder reaches 89.1% ModelNet40 accuracy versus 86.2% for PointNet, while a single reconstruction decoder reaches 85.7%.Masking cross-entropy reduces accuracy to 87.6%, masking k-means to 88.3%, and using all three losses yields 89.1%; reconstruction has the highest impact.Source: §4.5 · p. 7

• The model trains a multi-scale graph-based encoder with clustering, autoencoding, and self-supervised classification tasks.Source: §5 · p. 8
• 89.1% accuracy on ModelNet40 shape classification is reported as state-of-the-art among unsupervised models and competitive with supervised models.Source: §5 · p. 8
• 77.7 mIoU on ShapeNetPart segmentation is reported as only 8% below the state-of-the-art supervised model.Source: §5 · p. 8