Source-linked AI summary
Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels
Xun Xu, Gim Hee Lee
TL;DR
Point-cloud segmentation requires extensive point-level labels, which are costly to obtain. This paper trains with partial labels, using gradient approximation and additional constraints on unlabeled points. Across three public datasets, its results are comparable with full supervision using 10× fewer labeled points.
Problem
Point-cloud segmentation commonly assumes point-level labels, but exhaustive annotation is costly, especially for real indoor scenes.
Source: §1 · p. 1Method
The approach uses incomplete supervision and adds inexact supervision, Siamese self-supervision, and spatial/color smoothness constraints.
Source: §1 · p. 2Results
Takeaways & Limitations
Under a fixed labeling budget, the paper recommends labeling more samples with fewer points per sample rather than fewer, more densely labeled samples.
Source: §1 · p. 2The Siamese consistency branch assumes predictions remain invariant under in-plane rotation and mirror flipping.
Source: §3.4 · p. 4
Abstract
from arXiv · showhide
Point cloud analysis has received much attention recently; and segmentation is one of the most important tasks. The success of existing approaches is attributed to deep network design and large amount of labelled training data, where the latter is assumed to be always available. However, obtaining 3d point cloud segmentation labels is often very costly in practice. In this work, we propose a weakly supervised point cloud segmentation approach which requires only a tiny fraction of points to be labelled in the training stage. This is made possible by learning gradient approximation and exploitation of additional spatial and color smoothness constraints. Experiments are done on three public datasets with different degrees of weak supervision. In particular, our proposed method can produce results that are close to and sometimes even better than its fully supervised counterpart with 10$\times$ fewer labels.
1. Introduction
Point-cloud segmentation depends on extensive point-level labels, making annotation a costly bottleneck. This work investigates segmentation from partial labels, using gradient approximation and additional constraints on unlabeled points.
1. Introduction
- Exhaustive point-level labeling is costly: a 3D shape may have 1k–10k points, while a real indoor scene can involve millions.The paper notes that datasets may require labels for billions of points.Source: §1 · p. 1
- Prior weakly supervised approaches address related problems, but one uses structured optimization without spatial and color context, while another is limited to 3D LiDAR scans.The paper frames its goal as supporting contextual modeling on generic 3D point clouds.Source: §1 · p. 1
- The incomplete-supervision strategy can succeed with 10× fewer labels, because its learning gradient approximates the fully supervised gradient.The paper says the approximation converges in distribution, with variance inversely proportional to the number of sampled points.Source: §1 · p. 2
- The method adds inexact supervision, Siamese consistency under rotation and flipping, and spatial/color smoothness to constrain unlabeled points.The smoothness constraint can also refine predictions at inference through graph-like label propagation.Source: §1 · p. 2
- The paper analyzes annotation strategy under a fixed labeling budget and evaluates its approach on three public datasets.Its stated contributions also include a statistical explanation for weak supervision and three additional losses.Source: §1 · p. 2
2. Related Work
Related work spans incomplete and inexact supervision, alongside point-cloud methods that typically rely on exhaustive point-level labels. The paper positions its approach as combining deep models with spatial constraints under weak supervision.
2. Related Work
- Incomplete supervision uses partially labeled instances and exploits assumptions such as graph structure, spatial/color continuity, or consistency under data transformation.The paper uses “semi-supervised,” “weakly supervised,” and “weak supervision” interchangeably for this type.Source: §2 · p. 2
- Prior point-cloud methods either do not learn directly from point clouds or use optimization-based inference; the paper instead combines deep networks with spatial constraints.The paper contrasts its weak-supervision setting with spatial-smoothness and CRF refinement methods that require full supervision.Source: §2 · pp. 2–3
- Inexact supervision infers pixel-level predictions from image-level labels and can complement incomplete supervision in image segmentation.The paper introduces it as a complement for point-cloud segmentation.Source: §2 · pp. 2–3
- Point-cloud segmentation methods learn point-level classifiers, but exhaustive point labeling does not scale well.The paper proposes using only a fraction of points as labels.Source: §2 · p. 3
3. Methodology
The method builds on a point-cloud encoder and trains with sparse point labels, using gradient approximation and additional constraints to incorporate unlabeled points. It also applies spatial and color smoothness during training and inference.
3.1. Point Cloud Encoder Network
- The encoder f(X; Θ) maps each point cloud to an N×K feature embedding, with K matching the number of segmentation categories.Inputs contain N points with xyz coordinates and possibly additional features such as RGB values.Source: §3.1 · p. 3
3.2. Incomplete Supervision Branch
- The labelled-point loss uses a binary mask to restrict softmax cross-entropy supervision to points with ground-truth labels.The mask is 1 for labelled points and 0 otherwise.Source: §3.2 · p. 3
- With 10× fewer labelled points, the method yields results comparable to full supervision, while its gradient difference has variance σ^2/||M||1.The analysis also favors labelling points across each sample over labelling all points in only a subset of samples under a fixed budget.Source: §3.2 · pp. 3–4
3.4. Siamese Self-Supervision
- A Siamese branch applies random in-plane rotations and mirroring, then minimizes the L2 distance between predictions for original and augmented samples.The authors report that L2 distance performed better than KL-Divergence.Source: §3.4 · p. 4
3.5. Spatial & Color Smoothness Constraint
- Spatial and color graphs define a Laplacian regularizer for training embeddings, while label propagation refines predictions to comply with the manifold without straying too far from network predictions.The method combines xyz and RGB graph weights when both channels are available and adds must-link or must-not-link constraints when the manifold conflicts with labelled ground truth.Source: §3.5 · pp. 4–5
3.6. Training
- The final training objective sums segmentation, inexact-supervision, Siamese self-supervision, and smoothness losses, each weighted by λ1, λ2, and λ3 set to 1.Training uses segmentation loss alone for 100 epochs, followed by the combined objective for another 100 epochs.Source: §3.6 · p. 5
4. Experiment
Experiments evaluate weakly supervised segmentation on ShapeNet, PartNet, and S3DIS under sparse-label settings. The method is competitive with full supervision, and fixed-budget results favor labeling more samples with fewer points each.
4.1. Dataset
- Evaluation covers ShapeNet, PartNet, and S3DIS, using ShapeNet's 1pt, 1%, and 10% settings and a uniformly labelled 10% setting for S3DIS.PartNet evaluation uses level 1 annotations; S3DIS uses Area 5 as the holdout, and results report sample- and category-averaged mIoU.Source: §4.1 · p. 5, §4.2 · pp. 5–6
4.2. Weakly Supervised Segmentation
- The comparisons include fully supervised, alternative weakly supervised, and unsupervised methods, with mIoU averaged over test samples and categories.Unsupervised predictions are matched to ground truth using the best permutation before calculating mIoU.Source: §4.2 · p. 6
4.3. Qualitative Examples
- With 10% labelled training points, S3DIS examples show cleaner noisy regions than the baseline, while some object-boundary errors remain.ShapeNet plane and car predictions are described as close to fully supervised results.Source: §4.3 · p. 6
4.4. Label More Points Or More Samples
- At a fixed 10% annotation budget on ShapeNet, mIoU improves as labels cover more samples rather than concentrating on fewer samples.The tested strategies vary sample coverage from 10% to 100% while keeping x% samples times y% labelled points equal to 1000.Source: §4.4 · p. 6, Table 3 · p. 7
4.2. Weakly Supervised Segmentation
- With 10% labelled points, the method is comparable to full supervision on ShapeNet and slightly superior on S3DIS; its 1pt results are strong across the three datasets.On S3DIS, the 1pt multi-task model is 1% below full supervision; PartNet shows the same result pattern, with the proposed variant outperforming alternative weakly supervised and unsupervised methods.Source: §4.2 · p. 6, Table 2 · p. 7, Table 1 · p. 7
5. Ablation Study
The ablations find benefits from Siamese consistency, inexact supervision, and spatial/color smoothness, with Siamese supervision especially helpful on S3DIS. Performance approaches full supervision with 10% labelled points, and the reported loss patterns hold across PointNet and DGCNN.
Importance of Individual Components
- Siamese self-supervision helps S3DIS most, while inexact supervision and training- and inference-stage smoothness add further gains.The authors attribute S3DIS's larger Siamese benefit to diverse object orientations and layouts; consistency constraints outperform augmentation alone.Source: §5 · p. 7, Table 4 · p. 7
Compatibility with Encoder Network
- PointNet and DGCNN exhibit the same patterns for the proposed losses and inference label propagation.Table 4 reports the compatibility study alongside the component ablations.Source: §5 · p. 7, Table 4 · p. 7
Amount of Labelled Data
- Performance on all three datasets approaches full supervision after 10% labelled points with the baseline weak-supervision method.The study varies the labelled-point fraction from 1% to 100%.Source: §5 · p. 8, Figure 6 · p. 8
Point Feature Embedding
- Full supervision and 10% weak supervision produce similar point-feature embedding patterns in the T-SNE visualization.The authors take this as evidence that a few labelled points can yield competitive performance.Source: §5 · p. 8, Figure 7 · p. 8
6. Conclusion
The paper concludes that existing point-cloud encoders can achieve very competitive segmentation with few labelled points. Across three public datasets, the proposed approach is comparable with full supervision using 10× fewer labels.
6. Conclusion
- Existing point-cloud encoder networks can produce very competitive segmentation performance with only a few labelled points.The conclusion reports experiments on three public datasets and comparable results to full supervision with 10× fewer labelled points.Source: §6 · p. 8
- The method adds inexact-supervision, Siamese self-supervision, and spatial- and color-smoothness losses to regularize the model.Source: §6 · p. 8
- The paper provides statistical analysis and annotation-strategy insights for a fixed labelling budget.Source: §6 · p. 8
A. Compatibility with Alternative Encoder Networks
The ShapeNet evaluation reports robust shape segmentation with PointNet++ under very sparse annotation.
A. Compatibility with Alternative Encoder Networks
- PointNet++ shape segmentation remains robust on ShapeNet with one point per category or 10% labelled points, compared with full supervision.The three settings are evaluated using the final model.Source: A. Compatibility with Alternative… · p. 10
B. Additional Details on Datasets
The paper provides further details on its PartNet weakly supervised segmentation experiments in Table 6.
B. Additional Details on Datasets
- Additional details on the PartNet weakly supervised segmentation experiments are presented in Table 6.Source: B. Additional Details on Datasets · p. 10
C. Additional Qualitative Examples
Additional qualitative examples compare full and weak supervision on S3DIS and ShapeNet; the reported gap between weak and full supervision is smaller on ShapeNet part segmentation.
C. Additional Qualitative Examples
- On S3DIS, 10% weak supervision is comparable to full supervision and consistently improves over the weak-supervision baseline across selected rooms.The examples also show results using one labelled point per category, described as surprisingly good.Source: C. Additional Qualitative Examples · p. 10
- Figure 8 provides additional S3DIS examples comparing full and weak supervision.Source: Figure 8 · p. 11
- On ShapeNet part segmentation, the gap between weak and full supervision is smaller than in the S3DIS examples.Figure 7 presents additional full- versus weak-supervision examples for ShapeNet.Source: C. Additional Qualitative Examples · p. 10, Figure 7 · p. 14