PaperlayerView exact revision on arXiv

Source-linked AI summary

Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation

Li Jiang, Shaoshuai Shi, Zhuotao Tian, Xin Lai, Shu Liu, Chi-Wing Fu, Jiaya Jia

arXiv:2110.08188v1cs.CV

TL;DR

Point-cloud segmentation depends on costly point-level labels, motivating ways to use unlabeled clouds. This paper adds pseudo-label- and confidence-guided contrastive learning with category-balanced sampling; it reports improvements across three datasets, including relative mIoU gains with 5% labeled data.

  • Problem

    Point-cloud segmentation relies on large amounts of point-level labels, which are costly to prepare, while unlabeled clouds are less costly to collect.

    Source: §1 · p. 1

  • Method

    The method combines guided point contrastive learning using pseudo-label and confidence guidance with category-balanced sampling of points.

    Source: §1 · pp. 1–2

  • Results

    With 5% labeled data, relative mIoU improves by 13.9% on ScanNet V2, 17.8% on S3DIS, and 20.1% on SemanticKITTI.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    Experiments support the effectiveness of using unlabeled 3D data to improve prediction quality and model generalization ability.

    Source: §5 · p. 8

Abstract

from arXiv · show

Rapid progress in 3D semantic segmentation is inseparable from the advances of deep network models, which highly rely on large-scale annotated data for training. To address the high cost and challenges of 3D point-level labeling, we present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance. Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability in semi-supervised setting. Semantic predictions on unlabeled point clouds serve as pseudo-label guidance in our loss to avoid negative pairs in the same category. Also, we design the confidence guidance to ensure high-quality feature learning. Besides, a category-balanced sampling strategy is proposed to collect positive and negative samples to mitigate the class imbalance problem. Extensive experiments on three datasets (ScanNet V2, S3DIS, and SemanticKITTI) show the effectiveness of our semi-supervised method to improve the prediction quality with unlabeled data.

1. Introduction

Because point-level labels are costly, the introduction motivates using unlabeled point clouds for semi-supervised segmentation. It proposes guided contrastive learning and category-balanced sampling, reporting improvements over supervised-only models across three datasets.

  • Point-level labeling is time-consuming and labor-intensive, whereas collecting point clouds through 3D scanning and post-processing requires less effort.Source: §1 · p. 1
  • The introduction frames semi-supervised learning as a way to improve data efficiency and segmentation performance by incorporating unlabeled point clouds.Source: §1 · pp. 1–2
  • Guided point contrastive learning uses semantic predictions on unlabeled points to distinguish positive and negative pairs and enlarge inter-category feature distances.Pseudo-label guidance alleviates same-category negative pairs, while confidence guidance uses semantic scores to reduce the chance of worsening features.Source: §1 · pp. 1–2
  • On ScanNet V2 and S3DIS (indoor) and SemanticKITTI (outdoor), the semi-supervised method consistently surpasses supervised-only models by a large margin using 5%, 10%, 20%, 30%, and 40% labeled data.Source: §1 · p. 2
  • Category-balanced sampling uses pseudo labels to mitigate point-class imbalance and increase embedding diversity.Source: §1 · p. 2

2. Related Works

Related work covers voxel- and point-based 3D segmentation, image-focused semi-supervised learning, and point-level contrastive learning. The paper positions its approach as extending contrastive learning to semi-supervised point-cloud segmentation with point-wise pseudo labels.

  • 3D segmentation methods include voxel-based networks using regular grids and point-based approaches that learn directly from raw clouds.Other approaches use high-resolution voxel representations, sparse convolutions, kernel functions, or graph convolutions.Source: §2 · p. 2
  • Image semi-supervised learning uses consistency regularization, pseudo-label self-training, and generative adversarial networks for segmentation.Consistency can be encouraged through input or feature perturbations or by using different models.Source: §2 · p. 2
  • Semi-supervised learning for point-cloud scenes is described as underexplored, with cited 3D detection work using Mean Teacher or quality-aware pseudo labeling.Dense point-wise annotations for segmentation are more resource-intensive than 3D box annotations.Source: §2 · p. 2
  • Point-level contrastive learning separates positive and negative features, and this work extends it to semi-supervised segmentation using point-wise pseudo labels.PointContrast applies point-level contrastive learning to self-supervised pre-training; the proposed extension uses labeled and unlabeled clouds together.Source: §2 · pp. 2–3

3. Our Approach

The approach uses point-level feature learning on unlabeled, augmented point clouds, with semantic predictions guiding contrastive pairs and category-balanced sampling. A supervised branch and an unsupervised branch contribute cross-entropy and guided contrastive losses to the overall objective.

3.1. Preliminaries on Point-level Feature Learning

  • Point-level contrastive learning attracts matched points and repels negatives, but same-category negatives can harm features; an MSE consistency baseline also aligns matched points.The paper motivates using labeled data to guide feature learning from unlabeled point clouds.Source: §3.1 · p. 3

3.2. Pseudo Guidance on Contrastive Learning

  • The guided loss compares matched positive pairs against separately sampled negative points from the two augmented views.The negative points can include unmatched regions.Source: §3.2 · pp. 3–4
  • Pseudo-label guidance excludes negative pairs with matching predicted categories, while confidence guidance avoids pulling anchors toward low-confidence positive points.The guided contrastive loss is computed on two independently augmented views of an unlabeled point cloud.Source: §3.2 · pp. 3–4, Figure 3 · p. 5

3.3. Category-balanced Sampling

  • Category-Balanced Sampling evenly selects positive pairs by predicted category and draws negative embeddings from per-category memory banks across the training set.The negative banks use a first-in, first-out update strategy.Source: §3.3 · p. 5

3.4. Overall Architecture

  • The supervised branch applies cross-entropy to labeled point clouds, while the unsupervised branch uses two augmentations and a projector to form embeddings for guided contrastive learning.The framework shares the backbone, classifier, and projector weights across inputs.Source: Figure 2 · p. 4, §3.4 · p. 5
  • The overall objective combines supervised cross-entropy with the guided contrastive loss, weighted by a hyperparameter.The unlabeled branch supplies the guided contrastive loss and the labeled branch supplies cross-entropy.Source: §3.4 · p. 5

4. Experiments

The experiments evaluate the method on indoor and outdoor datasets across labeled-data ratios, comparing it with supervised-only training and other semi-supervised strategies. They report consistent gains from unlabeled data, with ablations examining guidance, sampling, and other design choices.

4.2. Main Results

  • At 5% labeled data, relative mIoU gains are 13.9% on ScanNet V2, 17.8% on S3DIS, and 20.1% on SemanticKITTI; semi-supervised models outperform supervised-only models across tested ratios.The main results cover mIoU and mAcc on the three datasets at labeled-data ratios from 5% to 40%.Source: §4.2 · pp. 6–7, Table 1 · p. 7
  • With 100% labeled data, the guided contrastive loss is used as an auxiliary constraint, and the method further improves prediction quality over its competitive supervised-only baseline.The labeled set is also fed to the unsupervised branch; the comparison with recent methods uses mIoU across three datasets.Source: Table 2 · p. 7, §4.2 · p. 7

4.3. Ablation Studies

  • On ScanNet V2 with 20% labeled data, pseudo-label guidance gives the largest component gain over vanilla point contrastive loss; confidence guidance and CBS raise the full-model mIoU to 66.7%.Confidence guidance moves mIoU from 65.9% to 66.4%; the full model with category-balanced sampling reaches 66.7%.Source: §4.3 · p. 7, Table 4 · p. 8

4.2. Main Results

  • In the transductive 100% setting, incorporating test data into the unlabeled training set raises performance; the SemanticKITTI CodaLab single-scan result is comparable to the SOTA’s at 70.2%.Table 3 reports mIoU for this setting.Source: §4.2 · p. 7, Table 3 · p. 8

4.3. Ablation Studies

  • CBS contributes more on SemanticKITTI, where category imbalance is greater, and increases the chance of sampling points from sparse categories.Six SemanticKITTI categories have fewer than 1‰ of training points, and motorcyclist has 0.04‰; the rarest ScanNet V2 category, sink, has 2.75‰. Replacing random crops with partial views lowers ScanNet V2 mIoU from 66.7% to 63.2%, while removing the projector lowers it to 65.0%.Source: §4.3 · p. 8, Table 5 · p. 8

4.4. Analysis on various Semi-supervised Strategies

  • Consistency regularization with MSE or cosine loss exceeds the supervised-only model by 1.0% and 0.9%, while the proposed method improves performance by 2.7%.The proposed strategy keeps matched points consistent and pushes apart points with different semantic predictions.Source: §4.4 · p. 8
  • Directly applying PointInfoNCE as the unsupervised loss performs poorly in this semi-supervised setting because same-category negative pairs can harm feature learning.The passage attributes the issue to the high probability of sampling negative pairs within the same category.Source: §4.4 · p. 8
  • In the reported comparison, self-training performs well at 65.5%, while the proposed method attains higher performance at 66.7%.Self-training uses a model trained on the labeled set to produce pseudo-labels for the unlabeled set, then constrains unsupervised-branch semantic predictions with cross-entropy loss.Source: §4.4 · p. 8

5. Conclusion

The paper presents a semi-supervised framework for 3D semantic segmentation that uses unlabeled point clouds through guided point contrastive learning and category-balanced sampling.

5. Conclusion

  • The framework uses pseudo-label and confidence guidance in its guided point contrastive loss to learn more discriminative features from unlabeled point clouds.Category-balanced sampling is also proposed to support more diverse feature embeddings.Source: §5 · p. 8
  • Experiments support the effectiveness of using unlabeled 3D data to improve model generalization ability.The conclusion describes the approach as a data-efficient framework for 3D semantic segmentation.Source: §5 · p. 8
  • The project was supported in part by a Research Grants Council of Hong Kong grant.The acknowledgment names Project no. CUHK 14206320.Source: §5 · p. 8
Loading 2110.08188v1…