PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Supervised Few-Shot Learning on Point Clouds

Charu Sharma, Manohar Kaul

arXiv:2009.14168v1cs.LGstat.ML

TL;DR

Point-cloud learning faces voxelization drawbacks and the burden of manual annotation. This paper uses cover-tree-derived self-supervised tasks on few-shot support sets to train embeddings for downstream networks. It reports improved few-shot performance across downstream tasks, with a support-set-only pretraining constraint.

  • Problem

    Point-cloud methods face voxelization costs and quantization artifacts, while supervised learning requires manual annotation of large datasets.

    Source: §1 · p. 1

  • Method

    The method derives two self-supervised tasks from a cover-tree hierarchy and pretrains on the few-shot support set to produce downstream point embeddings.

    Source: §1 · p. 2, §2.2.1 · p. 3

  • Results

    The authors report that their pretraining substantially improves state-of-the-art downstream methods in few-shot settings across classification and segmentation.

    Source: §1 · p. 2, §4 · p. 8

  • Takeaways & Limitations

    The reported gains support better learning from very limited labeled point-cloud examples in few-shot settings.

    Source: §4 · p. 8, Broader Impact · p. 9

    Self-supervised pretraining is restricted to the support set used for downstream few-shot learning.

    Source: §2.2.1 · p. 3

Abstract

from arXiv · show

The increased availability of massive point clouds coupled with their utility in a wide variety of applications such as robotics, shape synthesis, and self-driving cars has attracted increased attention from both industry and academia. Recently, deep neural networks operating on labeled point clouds have shown promising results on supervised learning tasks like classification and segmentation. However, supervised learning leads to the cumbersome task of annotating the point clouds. To combat this problem, we propose two novel self-supervised pre-training tasks that encode a hierarchical partitioning of the point clouds using a cover-tree, where point cloud subsets lie within balls of varying radii at each level of the cover-tree. Furthermore, our self-supervised learning network is restricted to pre-train on the support set (comprising of scarce training examples) used to train the downstream network in a few-shot learning (FSL) setting. Finally, the fully-trained self-supervised network's point embeddings are input to the downstream task's network. We present a comprehensive empirical evaluation of our method on both downstream classification and segmentation tasks and show that supervised methods pre-trained with our self-supervised learning method significantly improve the accuracy of state-of-the-art methods. Additionally, our method also outperforms previous unsupervised methods in downstream classification tasks.

1 Introduction

Point-cloud learning needs reliable geometric features, but voxelization and manual labeling bring costs and drawbacks. The paper proposes cover-tree-based self-supervised pretraining for few-shot learning and reports improved downstream performance.

1 Introduction

  • Point clouds support applications including navigation, self-driving vehicles, robotics, and shape modeling, which depend on reliable 3D geometric features.The paper contrasts learned semantic representations with edge-, corner-, and hand-crafted feature approaches.Source: §1 · p. 1
  • Voxelizing irregular point clouds adds memory costs and quantization artifacts, while supervised methods require manual annotation of large point-cloud datasets.These drawbacks motivate learning representations directly from raw points or through other means.Source: §1 · p. 1
  • Some unsupervised methods rely on reconstruction metrics such as Chamfer distance or EMD, which the paper describes as computationally inefficient and data-hungry.The cited methods may use raw clouds, voxels, or 2D point-cloud images.Source: §1 · p. 2
  • The method uses a cover-tree to partition point clouds at multiple scales and derive surrogate labels for two self-supervised pretext tasks.Figure 1 illustrates coarse and finer ball covers; the tasks encode the tree’s hierarchy.Source: §1 · p. 2, Figure 1 · p. 2
  • The authors report that few-shot downstream networks using their learned embeddings significantly outperform state-of-the-art supervised and unsupervised methods.Their evaluation covers classification and segmentation on dense and sparse datasets.Source: §1 · p. 2

2 Our Method

The method builds a multiscale cover-tree over point clouds, derives labels for two self-supervised tasks from its ball hierarchy, and uses the resulting embeddings to initialize few-shot downstream networks.

2.1 Preliminaries and Notation

  • Few-shot episodes train on m labeled examples from each of K′ sampled classes and test on disjoint query examples from those classes.A point cloud is represented as a set of points in R^d, typically with d=3.Source: §2.1 · p. 3

2.2.1 Preprocessing

  • The cover-tree represents point clouds as progressively finer ball coverings, with smaller radii at deeper levels to capture structure at multiple scales.At each non-root level, the balls form a non-disjoint covering of all points, and their centers are tree nodes.Source: §2 · p. 2, §2.2.1 · p. 3
  • Regression labels are the l2 distances between every distinct pair of ball centers at the same tree level.The distance task captures inter-ball relations without requiring parent-child pairs.Source: §2.2.1 · p. 3
  • The authors strictly restrict self-supervised pretraining to the same support set later used for few-shot learning.The cover-tree preprocessing defines balls and centers at successive levels for generating surrogate labels.Source: §2.2.1 · p. 3

2.2.2 Self-Supervised Learning Tasks

  • Classification labels identify which quadrant contains a child-ball center relative to its parent at the adjacent tree level.The regression and classification tasks are trained on same-level and valid parent-child ball pairs, respectively.Source: §2.2.1 · pp. 3–4, §2.2.2 · p. 4

2.2.3 Network Architecture

  • The trained feature extractor’s point embeddings initialize a separate downstream classification or segmentation network rather than joint training.The architecture supports downstream networks such as PointNet and DGCNN.Source: §2.2.3 · pp. 4–5
  • The self-supervised network branches into classification and regression heads, whose losses both backpropagate through the shared feature extractor.It represents each ball by the centroid of its point embeddings and concatenates paired ball vectors for task prediction.Source: §2.2.3 · p. 4, Figure 2 · p. 4
  • In the visualized feature spaces, points on semantically similar but distant object parts become closer under the proposed pretraining than with random initialization or VoxelSSL.The examples include corresponding wings of an aeroplane and parts of a table.Source: §2.2.3 · p. 5, Figure 3 · p. 5

3 Experiments

The experiments evaluate self-supervised point embeddings for few-shot classification and segmentation across four datasets. Results cover benchmark comparisons, pretext-task ablations, and the effect of the cover-tree expansion base.

3 Experiments

  • Experiments evaluate few-shot classification on ModelNet40 and Sydney and segmentation on ShapeNet and S3DIS, using sampled support examples and unseen query examples.ModelNet40 has 12,311 models with 1,024 points each; Sydney has 374 models with 100 points each.Source: §3 · p. 5

3.1 3D Object Classification

  • Our pretraining beats other methods in almost all classification settings and improves overall and majority-category ShapeNet IoU.The classification comparison includes dense ModelNet40 and sparse Sydney; part-segmentation comparisons include supervised and self-supervised baselines.Source: §3.1 · p. 6, Table 1 · p. 6, §3.3 · p. 7
  • ϵ=2.2 gives the best accuracy and silhouette coefficient on ModelNet40 and Sydney; large balls overlap labels, while tiny balls leave too few points for learning.The authors select the expansion base by grid search and cross-validation.Source: §3.1 · pp. 6–7, Figure 4 · p. 6

3.2 Ablation Study

  • Using both pretext tasks together improves performance most; regression alone boosts performance more than classification alone, which the authors relate to global versus local ball learning.The ablation fixes K′=10 and m∈{10,20}.Source: §3.2 · p. 7, Table 3 · p. 7

3.3 Part Segmentation

  • Figure 5 compares random-init DGCNN, VoxelSSL-pretrained DGCNN, and the proposed method; the proposed method separates the guitar into three parts and the headset headband from its pads.Source: §3.3 · p. 8, Figure 5 · p. 8

3.4 Semantic Segmentation

  • On S3DIS, our method improves most cases, with nearly 3% mIoU over VoxelSSL in Area 6 and 11% accuracy in Area 1.The evaluation trains on five areas and tests on the held-out area, rotating the held-out area across six areas.Source: §3.4 · p. 8, Table 4 · p. 8
  • On S3DIS, our method improves most cases, with nearly 2% mIoU over random-init DGCNN in Areas 3–5 and 5% accuracy in Area 4.Source: §3.4 · p. 8, Table 4 · p. 8

4 Conclusion

The paper represents point clouds as progressively finer cover-tree coverings and uses subset-derived surrogate labels for two pretext tasks. In few-shot downstream tasks, this pretraining substantially improves several state-of-the-art methods; geometry- and spectral-aware pretext tasks are proposed as future work.

4 Conclusion

  • Point clouds are represented as progressively finer cover-tree coverings, with surrogate labels from point subsets encoding the hierarchy for two pretext tasks.Source: §4 · p. 8
  • The authors report substantial performance improvements for several state-of-the-art downstream methods in few-shot learning.Source: §4 · p. 8
  • Future work could develop pretext tasks that capture point-cloud geometry and spectral information.Source: §4 · p. 8

Broader Impact

The paper frames its potential broader impact around learning from scarce labeled point-cloud examples, while noting possible effects on manual annotation work.

Broader Impact

  • The study targets better point-cloud learning with very limited labeled examples in few-shot settings.Source: Broader Impact · p. 9
  • Potential applications include 3D object detection, city-scale segmentation, and learning from rare organ disorders; the authors also note possible adverse effects on crowd-sourced annotation workers.Source: Broader Impact · p. 9

A Additional Experimental Results

Additional experiments examine cover-tree scales, point-density robustness, few-shot feature separation, and learned feature-space distances. The method remains competitive at reduced point counts and produces stronger class separation than VoxelNet pretraining in the reported visualizations.

Visualization of ball covers

Training

  • Pretraining uses up to three cover-tree levels, both R and C ball-pair tasks, and a three-layer MLP feature extractor, with 200 epochs and batch size 8.Classification and segmentation experiments use cover-trees with expansion constant ϵ = 2.0; downstream networks use default DGCNN and PointNet parameters.Source: §A.1 · p. 11

Effect of Point Cloud Density

  • At 128, 256, and 512 points, the method remains comparable and outperforms randomly initialized and VoxelNet-pretrained DGCNN on ModelNet40.The density experiment uses a 5-way 20-shot support set and 20 unseen query samples per class.Source: §A.1 · p. 11, Figure 7 · p. 13

T-SNE Visualization

  • In 10-class few-shot embeddings, the method separates classes better than VoxelNet on Sydney and ModelNet40, better than random DGCNN on Sydney, and nearly comparably on ModelNet40.The comparison is shown in t-SNE visualizations.Source: §A.1 · p. 11, Figure 8 · p. 13

Heatmap Visualization

  • Few-shot heatmaps show aeroplane wings close to one another and table legs close to one another in the learned feature space, unlike the compared baselines.The table top is far from a selected leg in the method's feature space; the passage cautions that few-shot learning is less pronounced than training with abundant point clouds.Source: §A.1 · p. 11

A.2 Part Segmentation

On ShapeNet part segmentation, the pretrained DGCNN and PointNet outperform baselines in overall IoU and most categories in several settings; in others, they are often best or second-best. Visual results are stronger for some complex objects, while Laptop and Bag segmentations are similar to random-init DGCNN.

A.2 Part Segmentation

  • Across ShapeNet Tables 5–8, DGCNN and PointNet with proposed pretraining outperform baselines in overall IoU and most categories.The reported segmentation uses 2048 points per cloud.Source: §A.2 · p. 13, Table 5 · p. 14, Table 6 · p. 14
  • In Tables 9–10, the method ranks best or second-best in most cases.The tables report mIoU by category and category mean.Source: §A.2 · p. 13, Table 9 · p. 15, Table 10 · p. 15
  • Evaluation uses 5- or 10-way settings with 5, 10, or 20 support examples per class and 20 query samples per class.The ShapeNet point clouds contain 2048 points each.Source: §A.2 · p. 13
  • Figures 13–14 compare segmentation outputs; Cars and Guitars are more properly segmented than with random-init DGCNN, while Laptop and Bag outputs are almost similar.The visual comparison also includes VoxelNet-pretrained DGCNN.Source: §A.2 · p. 14, Figure 13 · p. 20, Figure 14 · p. 21

Related papers

Loading 2009.14168v1…