PaperlayerView exact revision on arXiv

Source-linked AI summary

LaserMix for Semi-Supervised LiDAR Semantic Segmentation

Lingdong Kong, Jiawei Ren, Liang Pan, Ziwei Liu

arXiv:2207.00026v4cs.CVcs.LGcs.RO

TL;DR

Dense LiDAR annotation is costly, and semi-supervised LiDAR segmentation remains underexplored. LaserMix mixes laser beams from different scans and encourages confident, consistent predictions using LiDAR spatial cues. The framework reports competitive results against fully supervised counterparts with 2× to 5× fewer labels, while its conclusion identifies label bias and out-of-domain data as unaddressed issues.

  • Problem

    Dense LiDAR point-cloud annotation is costly, and semi-supervised LiDAR segmentation remains underexplored, with prior approaches not accounting for LiDAR-specific spatial properties.

    Source: §1 · pp. 1–2

  • Method

    LaserMix mixes laser-beam areas from different LiDAR scans and uses a teacher-student pipeline to encourage confident, consistent predictions based on spatial cues.

    Source: §1 · p. 2, §3.2 · p. 4, §3.3 · p. 5

  • Results

    With 2× to 5× fewer labels, the framework achieves competitive results over fully supervised counterparts and outperforms prevailing semi-supervised methods on nuScenes and SemanticKITTI.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experiments support the importance of LiDAR spatial prior and the approach's superiority across three popular benchmarks.

    Source: §5 · p. 8

    Label bias and out-of-domain data are not addressed or discussed, despite their potential safety-critical relevance in real-world autonomous driving applications.

    Source: §5 · p. 9

Abstract

from arXiv · show

Densely annotating LiDAR point clouds is costly, which restrains the scalability of fully-supervised learning methods. In this work, we study the underexplored semi-supervised learning (SSL) in LiDAR segmentation. Our core idea is to leverage the strong spatial cues of LiDAR point clouds to better exploit unlabeled data. We propose LaserMix to mix laser beams from different LiDAR scans, and then encourage the model to make consistent and confident predictions before and after mixing. Our framework has three appealing properties: 1) Generic: LaserMix is agnostic to LiDAR representations (e.g., range view and voxel), and hence our SSL framework can be universally applied. 2) Statistically grounded: We provide a detailed analysis to theoretically explain the applicability of the proposed framework. 3) Effective: Comprehensive experimental analysis on popular LiDAR segmentation datasets (nuScenes, SemanticKITTI, and ScribbleKITTI) demonstrates our effectiveness and superiority. Notably, we achieve competitive results over fully-supervised counterparts with 2x to 5x fewer labels and improve the supervised-only baseline significantly by 10.8% on average. We hope this concise yet high-performing framework could facilitate future research in semi-supervised LiDAR segmentation. Code is publicly available.

1. Introduction

Densely annotating LiDAR point clouds limits the scalability of fully supervised segmentation, while semi-supervised approaches remain underexplored and existing SSL methods do not fully account for LiDAR-specific spatial structure. LaserMix uses that structure to exploit unlabeled scans and reports competitive results with substantially fewer labels.

1. Introduction

  • Semi-supervised LiDAR segmentation remains underexplored, and prior SSL approaches do not account for intrinsic LiDAR properties, with a cited point-cloud method not differentiating indoor and outdoor scenes.The text describes dense annotation as costly and labor-intensive, restraining fully supervised scalability.Source: §1 · pp. 1–2
  • LaserMix partitions scans into low-variation laser-beam areas, mixes areas from different scans, and encourages confident, consistent predictions before and after mixing.The proposed mixing-based teacher-student pipeline is intended to leverage LiDAR's strong spatial prior.Source: §1 · p. 2
  • With 2× to 5× fewer labels, the pipeline achieves competitive results over the fully supervised counterpart and outperforms prevailing semi-supervised methods by up to +5.7% mIoU on nuScenes and +3.5% mIoU on SemanticKITTI.On SemanticKITTI, it also reaches 54.4% mIoU using 0.8% labels, on-par with named methods using 100% labels.Source: §1 · p. 2
  • The framework is presented as statistically grounded, representation-agnostic, and superior to previous state-of-the-art methods in both low- and high-data regimes.The stated representation examples include range view and voxel.Source: §1 · p. 2

2. Related Work

Related work spans fully supervised LiDAR segmentation, weak and other reduced-label strategies, and SSL for 2D and 3D tasks. The paper positions its approach around unlabeled outdoor LiDAR data, whose spatial structure and scale differ from much of the prior SSL literature.

2. Related Work

  • Fully supervised LiDAR methods across range-view, bird’s-eye-view, voxel, and multi-view representations rely on large annotated datasets, and their performance degrades severely in low-data regimes.Other annotation-cost strategies include weak, scribble, box, and active-learning supervision.Source: §2 · p. 2
  • Image SSL methods use input perturbations, consistency between perturbed networks, or offline pseudo-labeling, but cited perturbations can be inapplicable or sub-par in 3D and offline labels require extra storage.The paper states its framework encourages consistency regularization and entropy minimization without extra overhead.Source: §2 · p. 2
  • Much 3D SSL work targets object-centric point clouds or indoor scenes, while cited point-cloud segmentation work focuses mainly on indoor scenes and does not distinguish their uniform distributions from spatially structured LiDAR.The paper describes itself as among the first to explore SSL for LiDAR segmentation and says it establishes benchmarks on autonomous-driving databases.Source: §2 · p. 3

3. Approach

The framework uses LiDAR’s spatial structure to encourage confident, consistent predictions within predefined areas, then operationalizes this objective by interleaving laser-partitioned scans. Its pipeline combines supervised, mixed-data, and teacher-consistency signals.

3.1. Leveraging the Spatial Prior for SSL

  • The spatial-prior objective favors confident, consistent predictions within a predefined area regardless of data outside it.It is grounded in the premise that points and labels within a spatial area have relatively low variation; using the full point cloud reduces the framework to classic entropy minimization.Source: §3.1 · p. 3

3.2. LaserMix

  • LaserMix partitions scans by laser-beam inclination and interleaves alternating areas from two scans, while operating directly on point clouds across LiDAR representations.The method uses odd-indexed areas from one scan and even-indexed areas from the other; the labels are mixed in the same way.Source: §3.2 · p. 4
  • Two predictions reduce the marginalization cost from |A|·|Xout| to 2, with 2× batch memory and no training-speed impact.The two data fillings are the original scan and the mixed scan.Source: §3.2 · p. 4

3.3. Overall Pipeline

  • The pipeline trains a Student with labeled and mixed-data losses, uses confidence-thresholded Teacher predictions as pseudo-labels, and adds a Student–Teacher consistency loss.The Teacher is updated by exponential moving average; the framework uses it at inference without extra inference overhead.Source: §3.3 · pp. 4–5

4. Experiments

Across the three LiDAR datasets and multiple data splits, LaserMix improves over supervised-only baselines and outperforms prior SSL methods. Ablations indicate that spatially informed laser partitioning, teacher supervision, and appropriate mixing and pseudo-label settings contribute to the framework’s performance.

4.1. Settings

  • Experiments use nuScenes, SemanticKITTI, and ScribbleKITTI with 1%, 10%, 20%, and 50% labeled scans, range-view and voxel backbones, and a unified evaluation protocol.The benchmarks use shared backbones, codebase, configurations, and augmentations; ScribbleKITTI uses line-scribble annotations.Source: §4.1 · pp. 5–6

4.2. Comparative Study

  • Range-view gains reach 11.2% mIoU, while voxel results average 4.1% mIoU gains over supervised-only baselines across splits and datasets.The benchmark covers nuScenes, SemanticKITTI, and ScribbleKITTI; Table 1 compares SSL methods using range-view and voxel representations.Source: §4.2 · p. 6, Table 1 · p. 6
  • LaserMix outperforms GPC on SemanticKITTI, especially with few annotations, and is competitive with fully supervised methods using 2× to 5× fewer annotations.Image-segmentation SSL methods yield sub-par performance in the LiDAR benchmark; qualitative results report broader error reduction around the ego-vehicle for LaserMix than for prior methods.Source: §4.2 · pp. 6–7, Table 2 · p. 6

4.3. Ablation Study

  • LaserMix beats CutMix by up to 3.3% mIoU; reversing or shuffling beam order reduces results from 68.2% to 64.4% or 63.8%.Random-point MixUp does not improve on baseline on average, while CutOut drops performance from CutMix; the authors attribute LaserMix’s advantage to using spatial position as well as area structure.Source: §4.3 · p. 7
  • Teacher-generated pseudo-labels outperform Student-generated ones, and Lmix contributes significantly; EMA decay rates from 0.9 to 0.99 typically work best.The ablation uses a 10% labeled budget and range-view backbones; a rate of 0.999 tends to hurt consistency between Teacher and Student.Source: §4.3 · pp. 7–8, Figure 4 · p. 7
  • Azimuth-only splitting does not improve performance, while finer mixing granularity helps until it becomes too fine, when scores begin to drop.The authors attribute the azimuth result to weak semantic correlation in that direction and conjecture that overly fine partitions hurt semantic coherence; Table 4 encodes azimuth and inclination partitions.Source: Table 4 · p. 7, §4.3 · p. 8
  • Mixing two unlabeled scans with pseudo-labels drops the score from 68.2% to 66.9% but still outperforms existing methods.The authors interpret this as evidence that LaserMix provides a consistency-regularization effect during SSL.Source: §4.3 · p. 8
  • A confidence threshold around 0.9 gives the best results on the three tested datasets; thresholds that are too low or too high can reduce performance.Low thresholds force consistency on low-quality pseudo-labels, while high thresholds can diminish mixing benefits.Source: Figure 4 · p. 7, §4.3 · p. 8

5. Conclusion

The conclusion presents a statistically principled SSL pipeline that uses LiDAR spatial priors through LaserMix and reports its superiority across three benchmarks. It also identifies label bias and out-of-domain data as unaddressed issues.

5. Conclusion

  • LaserMix intertwines laser beams from different scans to exploit LiDAR spatial priors in a statistically principled semi-supervised segmentation pipeline.The authors report superiority on three popular benchmarks and say the results demonstrate the importance of spatial priors.Source: §5 · p. 8
  • The authors identify finer-grained spatial partitions, newer SSL techniques, and extensions to 3D detection and tracking as future directions.Source: §5 · p. 8
  • Label bias and out-of-domain data are not addressed, although the authors note these could be safety-critical in real-world autonomous-driving deployment.Source: §5 · p. 9

Appendix

The appendix supplements the main paper with representation details, a spatial-prior case study, implementation information, additional results, and acknowledgments.

6. LiDAR Representation

  • The appendix adds technical details on LiDAR range-view and voxel representations.Source: Appendix · p. 9

7. Case Study: Spatial Prior in LiDAR Data

  • A case study examines the strong spatial prior in outdoor LiDAR data.Source: Appendix · p. 9

8. Implementation Details

  • Additional implementation details cover the SSL algorithms used in the experiments.Source: Appendix · p. 9

9. Additional Experimental Results

  • Supplementary experiments provide class-wise IoU scores and visual comparisons.Source: Appendix · p. 9

10. Acknowledgments

  • The appendix acknowledges the public resources used in the work.Source: Appendix · p. 9

6. LiDAR Representation

This section describes range-view projection and cylindrical voxel partitioning as LiDAR representations, including how points and labels are mapped for segmentation.

6. LiDAR Representation

  • The section reviews range-view and cylindrical voxel representations, described as efficient and strong-performing options for LiDAR data.It situates them among raw-point, bird’s-eye-view, and other voxel representations.Source: §6 · p. 9

6.1. Range View Projection

  • Range-view projection maps each 3D LiDAR point to grid coordinates in an h × w range image, with h based on beam count and w on horizontal angular resolution.Labels are projected in the same way, and the resulting image is used as input to range-view segmentation networks.Source: §6.1 · p. 9

6.2. Cylindrical Partition

  • Cylindrical partition transforms Cartesian points into cylinder coordinates, then assigns points to predefined voxel cells and gives each cell a majority-voted semantic label.The passage attributes cylinder voxels’ better segmentation performance than conventional cubic voxels to LiDAR point-cloud density decreasing with range.Source: §6.2 · p. 9

7. Case Study: Spatial Prior in LiDAR Data

The case study analyzes spatial patterns in SemanticKITTI by partitioning scans into laser-beam areas and examining how semantic classes distribute across them.

7. Case Study: Spatial Prior in LiDAR Data

  • The SemanticKITTI case study investigates whether LiDAR scans contain strong spatial patterns in laser-beam partitions.Source: §7 · p. 9

7.1. Laser Partition

  • Each 64-beam SemanticKITTI scan is split into eight non-overlapping areas, each containing points from eight consecutive beams.Source: §7.1 · p. 10

7.2. Spatial Prior

  • Road points concentrate in the first four areas, vegetation and buildings in long-distance areas, and dynamic classes in middle-distance areas.The authors connect these distributions to real-world street-scene layout and report that spatial priors better encourage consistency regularization under annotation scarcity.Source: §7.2 · p. 10

8. Additional Implementation Detail

This appendix documents the three datasets and implementation choices used for the semi-supervised LiDAR segmentation benchmark. It spans different sensor configurations and annotation proportions, and reports improvements across all three datasets.

8.1. Dataset

  • nuScenes has 40,000 scans from 1,000 20-second scenes; SemanticKITTI uses 22 sequences, while ScribbleKITTI annotates SemanticKITTI with scribbles.The datasets use 16 classes for nuScenes and 19 for SemanticKITTI; ScribbleKITTI uses the SemanticKITTI configuration and Seq. 08 for validation.Source: §8.1 · p. 10
  • The framework improves across nuScenes, SemanticKITTI, and ScribbleKITTI despite differences in beam count, inclination range, and annotation proportion.nuScenes uses 32 beams and 100% labels; SemanticKITTI and ScribbleKITTI use 64 beams, with ScribbleKITTI using 8.06% semantic labels.Source: §8.1 · p. 10

8.2. Model Configuration

  • The experiments use a compact ResNet34-point FIDNet range-view backbone and a reduced-resolution, compact Cylinder3D voxel backbone.The smaller Cylinder3D configuration uses about 4× less memory, while nuScenes validation mIoU falls from 76.1% to 74.1%.Source: §8.2 · p. 10

8.1. Dataset

  • The dataset configuration table summarizes class and scan counts, input resolutions, sensor ranges, label proportions, and example inputs and labels.A separate table describes semantic-class occupation and distribution across eight laser-beam groups.Source: Table 5 · p. 11, Table 6 · p. 12

8.2. Model Configuration

  • Input augmentations differ by representation, while LaserMix and baseline loss weights are specified across the three datasets.Range-view inputs use jittering, scaling, flipping or shifting; voxel inputs use rotation and flipping. LaserMix samples 1–6 spatial areas, with λmix=1.Source: §8.2 · p. 12
  • The GPC comparison uses scores reported in its paper because the model is not open-sourced, so backbones and data splits may not align.The authors also provide a sequential benchmark track intended to better align settings with LiDAR data collection.Source: §8.2 · p. 12

9. Additional Experimental Result

Additional experiments report class-wise comparisons, granularity studies, and a Cityscapes extension. The appendix also describes qualitative error-map demos on SemanticKITTI.

Extension to RGB Data

  • On all four Cityscapes splits, the proposed approaches improve performance on top of MeanTeacher and CPS.The experiments partition image areas along the vertical direction, adapting the range-view mixing strategy to structural RGB images.Source: Table 7 · p. 13, §9 · p. 13

Comparative Study

  • LaserMix shows overt class-wise IoU improvements for almost all semantic classes across nuScenes, SemanticKITTI, and ScribbleKITTI.The authors attribute these improvements to the SSL framework’s consistency regularization.Source: §9 · p. 13

Ablation Study

  • Increasing granularity along inclination tends to improve segmentation, and uniformly sampling spatial-area counts further increases diversity and scores.The granularity studies cover range-view and voxel representations on nuScenes under the 10% split.Source: §9 · p. 13

Video Demos

  • On SemanticKITTI validation sets [1], models trained with 1% labeled data compare the sup.-only model with MeanTeacher, visualizing error maps from LiDAR bird’s-eye and range views.Source: §9 · p. 13

10. Public Resources Used

The paper acknowledges public datasets, toolkits, and model resources used in the work, alongside listed licenses for several resources.

Public Resources Used

  • Acknowledged resources include nuScenes, nuScenes-devkit, SemanticKITTI, its API, ScribbleKITTI, and FIDNet.The listed licenses are CC BY-NC-SA 4.0 for nuScenes, Apache 2.0 for the devkit, and MIT for the SemanticKITTI API; other entries are marked unknown.Source: §10 · p. 13
  • Cylinder3D and TorchSemiSeg are also acknowledged among the public resources used.Cylinder3D is listed with Apache License 2.0 and TorchSemiSeg with MIT License.Source: §10 · p. 13
  • The acknowledgment also lists Mix3D, MixUp, CutMix, CutMix-Seg, CBST, and MeanTeacher.Mix3D is marked unknown; MixUp and CBST are Attribution-NonCommercial 4.0; CutMix and CutMix-Seg are MIT; MeanTeacher is Attribution-NonCommercial 4.0.Source: §10 · p. 13

Related papers

Loading 2207.00026v4…