PaperlayerView exact revision on arXiv

Source-linked AI summary

SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network

Mingmei Cheng, Le Hui, Jin Xie, Jian Yang

arXiv:2104.07861v3cs.CV

TL;DR

Large-scale point-cloud segmentation is costly to supervise, and reliably inferring labels for unlabeled points remains challenging. SSPC-Net combines superpoint graphs, pseudo-label propagation, dropout, and coupled attention; it reports better performance than current semi-supervised methods with fewer labels.

Abstract

from arXiv · show

Point cloud semantic segmentation is a crucial task in 3D scene understanding. Existing methods mainly focus on employing a large number of annotated labels for supervised semantic segmentation. Nonetheless, manually labeling such large point clouds for the supervised segmentation task is time-consuming. In order to reduce the number of annotated labels, we propose a semi-supervised semantic point cloud segmentation network, named SSPC-Net, where we train the semantic segmentation network by inferring the labels of unlabeled points from the few annotated 3D points. In our method, we first partition the whole point cloud into superpoints and build superpoint graphs to mine the long-range dependencies in point clouds. Based on the constructed superpoint graph, we then develop a dynamic label propagation method to generate the pseudo labels for the unsupervised superpoints. Particularly, we adopt a superpoint dropout strategy to dynamically select the generated pseudo labels. In order to fully exploit the generated pseudo labels of the unsupervised superpoints, we furthermore propose a coupled attention mechanism for superpoint feature embedding. Finally, we employ the cross-entropy loss to train the semantic segmentation network with the labels of the supervised superpoints and the pseudo labels of the unsupervised superpoints. Experiments on various datasets demonstrate that our semi-supervised segmentation method can achieve better performance than the current semi-supervised segmentation method with fewer annotated 3D points. Our code is available at https://github.com/MMCheng/SSPC-Net.

Introduction

Large point clouds make full annotation costly, while semi-supervised segmentation still faces difficulty inferring reliable labels from irregular geometry and uncertain pseudo-labels. SSPC-Net uses superpoints and graph-based label propagation, with dropout and coupled attention, and reports better performance with fewer labels.

Introduction

  • Full supervision is costly in labor and time because real-scene point clouds can contain millions of points, making accurate labels difficult to obtain.Source: Introduction · p. 1
  • Semi-supervised point-cloud segmentation remains challenging because irregular geometry complicates pseudo-label inference and uncertain pseudo-labels hinder discriminative feature learning.Source: Introduction · p. 1
  • SSPC-Net partitions point clouds into superpoints and builds a graph, converting point-level label prediction into superpoint-level prediction.The superpoints are sets of points with isotropically geometric features.Source: Introduction · p. 1
  • SSPC-Net combines dynamic superpoint label propagation and superpoint dropout with coupled attention, and reports better performance than current semi-supervised segmentation with fewer labels.Dropout prunes low-confidence extended superpoints; coupled attention alternates attention over supervised and extended superpoints to improve their feature discrimination.Source: Introduction · p. 2

Related Work

Prior point-cloud methods include voxel- and view-based approaches and methods that process unordered points directly. Semi- and weakly supervised work uses different annotation and contextual constraints, with some approaches tied to LiDAR sequences or requiring time-consuming subcloud labels.

Related Work

  • Voxel-based methods incur large memory costs at high resolution, while view-based methods are sensitive to 3D data density (Su et al. 2015; Boulch, Le Saux, and Audebert 2017; Tatarchenko et al. 2018).Source: Related Work · p. 2
  • PointNet processes unordered point clouds directly with multilayer perceptrons and max pooling, avoiding voxelization and additional preprocessing.Source: Related Work · p. 2
  • Mei et al. use LiDAR scan-sequence temporal constraints, while MPRM uses scene- and subcloud-level labels and Xu and Lee use block labels, augmentation, and smoothness constraints.The passage notes that LiDAR temporal constraints are not practicable for general point-cloud segmentation and that generating subcloud-level annotations remains time-consuming.Source: Related Work · p. 2

Our Method

SSPC-Net represents point clouds as superpoint graphs, propagates labels to selected unsupervised superpoints, and uses coupled attention to learn contextual features. Its graph embedding module supplies predictions at test time, while label propagation and coupled attention are used during training.

Our Method

  • The method combines the superpoint graph embedding, dynamic label propagation, and coupled attention in its training framework.Label propagation and coupled attention run only during training; test predictions come directly from the superpoint graph embedding module.Source: Our Method · p. 2, Framework · p. 5

Superpoint Graph Embedding

  • The graph embedding module partitions points into superpoints, builds their graph, and learns superpoint features with a lightweight PointNet-like structure and gated GNN.The method uses cross-entropy loss for the supervised superpoint segmentation objective.Source: Superpoint Graph Embedding · pp. 2–3

Dynamic Label Propagation

  • Label propagation progressively extends supervision over the superpoint graph, selecting high-confidence candidates and dropping the 5% farthest extended superpoints per cluster.Extensions occur every M epochs; the dropout rule retains the most reliable 95% of extended superpoints.Source: Dynamic Label Propagation · pp. 3–4

Coupled Attention for Feature Enhancement

  • Coupled attention alternates feature weighting between supervised and extended superpoints, using within- and across-cloud connections to model intra- and inter-relations.Superpoints with the same labels are encouraged to have more similar semantic embeddings than superpoints from different classes.Source: Coupled Attention for Feature… · pp. 4–5

Framework

  • The graph embedding module provides the test-time predictions, while dynamic label propagation and coupled attention are training-only components.The final objective combines Ls, Les, and Lese, with λ1 and λ2 set to 1.Source: Framework · p. 5

Experiments

SSPC-Net is evaluated on S3DIS, ScanNet, and vKITTI with sparse point annotations, alongside ablations of its components and training settings. The reported results include gains over a semi-supervised comparison and high accuracy for extended superpoints.

Implementation Details

  • Training uses Adam with a 0.01 base learning rate, dataset-specific batch sizes, and class-balanced random sampling of annotated points.Batch sizes are 4 for S3DIS and 8 for ScanNet and vKITTI; the supervision budget is distributed across classes.Source: Implementation Details · p. 5

Semi-supervised Semantic Segmentation

Ablation Study

  • Adding dynamic label propagation and coupled attention improves Area 5 S3DIS performance over the component configurations in the ablation study.The component comparison covers graph embedding, label propagation, and coupled attention at two annotation rates.Source: Ablation Study · pp. 6–7, Table 3 · p. 7
  • As supervision increases, supervised superpoints become more prevalent; with fewer supervised points, extended superpoints account for a larger proportion.The authors emphasize this larger extended-superpoint share as important when point annotations are extremely scarce.Source: Figure 5 · p. 7, Ablation Study · p. 7
  • Extended superpoints achieve around 90% overall accuracy, and the best tested propagation interval is M = 40 epochs.The reported extended-superpoint accuracy is computed at the point level; interval comparisons use a 0.01% supervision rate on Area 5 of S3DIS.Source: Ablation Study · p. 7, Table 4 · p. 7, Table 5 · p. 7

Conclusion

SSPC-Net combines superpoint graphs, dynamic pseudo-label propagation with superpoint dropout, and coupled attention for semi-supervised point-cloud segmentation. It reports better performance than current semi-supervised methods with fewer labels.

  • SSPC-Net reports better performance than current semi-supervised point-cloud segmentation methods with fewer labels.Source: Conclusion · p. 7
  • Superpoint graphs are used to explore long-range relations in the point cloud.Source: Conclusion · p. 7
  • Dynamic label propagation combined with superpoint dropout generates high-quality pseudo labels for unsupervised superpoints.Source: Conclusion · p. 7
  • A coupled attention module learns discriminative contextual features of superpoints and exploits the generated pseudo labels.Source: Conclusion · p. 7

Related papers

Loading 2104.07861v3…