PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Range View Representation for LiDAR Segmentation

Lingdong Kong, Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou, Yu Qiao, Ziwei Liu

arXiv:2303.05367v3cs.CVcs.RO

TL;DR

Projection-based LiDAR methods are efficient, but range-view segmentation has faced accuracy limitations from representation and processing challenges. RangeFormer combines self-attention, tailored augmentation, supervised post-processing, and STR scalable training to address them. It surpasses competing point, voxel, and fusion approaches across semantic and panoptic benchmarks while improving efficiency.

  • Problem

    Projection-based methods offer efficient 2D processing, but range-view segmentation accuracy has remained behind point-, voxel-, and fusion-based approaches.

  • Method

    RangeFormer uses a Transformer-based full-cycle framework with MLP decoding, tailored training and augmentation designs, supervised RangePost processing, and STR group-wise scalable training.

  • Results

    RangeFormer and STR surpass competing modalities across SemanticKITTI, nuScenes, and ScribbleKITTI semantic and panoptic segmentation benchmarks, achieving 73.3% mIoU and 64.2% PQ on SemanticKITTI.

  • Takeaways & Limitations

    RangeFormer demonstrates that range view can achieve superior LiDAR segmentation performance while providing a better accuracy-efficiency trade-off.

  • Takeaways & Limitations

    The reported rasterization setting can retain only approximately 27.3% of grids relative to 120,000 3D points, corresponding to over 70% information loss.

Abstract

from arXiv · show

LiDAR segmentation is crucial for autonomous driving perception. Recent trends favor point- or voxel-based methods as they often yield better performance than the traditional range view representation. In this work, we unveil several key factors in building powerful range view models. We observe that the "many-to-one" mapping, semantic incoherence, and shape deformation are possible impediments against effective learning from range view projections. We present RangeFormer -- a full-cycle framework comprising novel designs across network architecture, data augmentation, and post-processing -- that better handles the learning and processing of LiDAR point clouds from the range view. We further introduce a Scalable Training from Range view (STR) strategy that trains on arbitrary low-resolution 2D range images, while still maintaining satisfactory 3D segmentation accuracy. We show that, for the first time, a range view method is able to surpass the point, voxel, and multi-view fusion counterparts in the competing LiDAR semantic and panoptic segmentation benchmarks, i.e., SemanticKITTI, nuScenes, and ScribbleKITTI.

1. Introduction

Range view offers efficient LiDAR processing but suffers information loss, empty grids, and distorted or semantically incoherent projections. RangeFormer addresses these issues with Transformer-based modeling, augmentation, supervised post-processing, and scalable low-resolution training, achieving strong accuracy and efficiency.

  • Motivation: Range view is computationally tractable, but its segmentation accuracy has lagged behind point-, voxel-, and fusion-based approaches.2D rasterization and mature 2D operators enable fast processing, while competing representations often require neighborhood search, 3D convolutions, or multi-branch networks.
  • Challenges: The representation suffers from many-to-one point mappings, empty grids caused by sparsity or sensor disruptions, and shape distortions during rasterization.Limited horizontal angular resolution causes adjacent points to collide, while sparse observations create holes in the range image.
  • RangeFormer: RangeFormer formulates range-view segmentation as seq2seq prediction and uses self-attention to capture global contextual dependencies omitted by limited-receptive-field FCNs.Its hierarchical features are decoded through MLP heads.
  • STR: STR divides each scan along azimuth into groups and trains on randomly selected low-resolution views, reducing training complexity while largely preserving convergence and segmentation accuracy.During inference, all groups are processed in one stacked batch and recombined into the complete scan.
  • Results: RangeFormer and STR achieve 73.3% mIoU and 64.2% PQ on SemanticKITTI while running 2× to 5× faster than recent voxel and fusion methods.The framework also reports superiority on nuScenes and ScribbleKITTI and can operate at sensor frame rate.

2. Related Work

Prior LiDAR methods use point, voxel, BEV, range-view, or multi-view representations, each balancing accuracy, efficiency, and computational cost. The paper builds on range-view processing, Transformer architectures, and tailored augmentation to improve scalable segmentation.

  • LiDAR Representation: Point and sparse voxel methods are prevalent but have O(N · d) complexity, while fusion methods often use multiple networks with high training and inference overhead.BEV is efficient but has sub-par performance, whereas range view directly reflects the LiDAR sampling process and retains compact semantic and structural cues.
  • Architecture: Prior range-view models primarily use fully convolutional architectures, including encoder-decoder, dilated-convolution, and lightweight designs.RangeFormer instead uses a self-attention-based architecture with Transformer blocks and MLP heads.
  • Data Augmentation: Existing LiDAR augmentation methods address class balance, overfitting, out-of-context mixing, or semi-supervised learning across different representations and axes.The paper introduces a range-view-specific combination of mixing, shifting, union, and copy-paste operations.

3. Technical Approach

The approach revisits range-view rasterization and introduces RangeFormer and STR to address information loss, deformation, semantic incoherence, and training overhead. It combines self-attention architecture, range-specific augmentation, and multi-view scalable training.

  • 3.1. Preliminaries: Range-view rasterization maps LiDAR points into an H × W cylindrical range image and projects grid predictions back to points.Each grid contains rasterized coordinates, depth, intensity, and existence features, while labels share the same rasterization index.
  • 3.2. RangeFormer: A Full-Cycle Framework: Typical 64 × 512 range images represent only about 27.3% of approximately 120,000 LiDAR points, creating a many-to-one conflict and over 70% information loss.Lower horizontal resolution also worsens shape deformation and semantic incoherence.
  • 3.2. RangeFormer: A Full-Cycle Framework: RangeFormer uses a self-attention-based network with Transformer blocks, MLP heads, hierarchical features, and lightweight decoding to model global contextual dependencies.Its semantic head unifies feature channels and spatial resolutions before producing class probabilities; bilinear interpolation is used during decoding.
  • 3.2. RangeFormer: A Full-Cycle Framework: RangeAug combines RangeMix, RangeUnion, and RangePaste to mix scans, fill empty grids, and paste rare classes while preserving projection structure.RangeUnion samples 50% of candidate empty grids for point filling, and RangePaste maintains corresponding positions in the range image.
  • 3.3. STR: Scalable Training from Range View: The occupancy analysis identifies width 1920 as the most informative representation, but its memory and computation costs motivate scalable multi-view training.STR splits the panorama into Z azimuthal views, rasterizes each at high horizontal resolution, and reduces effective training resolution by Z times.

4. Experimental Analysis

Experiments show that RangeFormer improves range-view LiDAR segmentation across semantic, panoptic, weakly supervised, and efficiency settings. STR further reduces training resource demands while preserving competitive accuracy.

  • Experimental settings: Experiments use SemanticKITTI, nuScenes, and ScribbleKITTI, with SemanticKITTI providing 19 classes from a 64-beam LiDAR sensor and nuScenes 16 classes from a sparser 32-beam sensor.
  • Semantic segmentation: RangeFormer improves SemanticKITTI semantic segmentation by 9.3%, 9.8%, and 8.6% mIoU over CENet at 512, 1024, and 2048 resolutions, respectively.It also surpasses 2DPASS by 0.4% mIoU and GASN by 2.9% mIoU.
  • Panoptic segmentation: Panoptic-RangeFormer outperforms Panoptic-PHNet on PQ, PQ†, and RQ, including under STR, with especially strong gains for stuff classes.
  • Weakly-supervised segmentation: RangeFormer achieves 63.0% mIoU under weak supervision, while range-view methods outperform voxel-based methods on ScribbleKITTI.
  • Efficiency: Projection-based methods are 2× to 5× faster than voxel and fusion counterparts, and RangeFormer offers the best reported accuracy-efficiency trade-off.

5. Conclusion

The paper proposes RangeFormer and STR to strengthen range-view LiDAR segmentation. It reports superior semantic and panoptic performance with improved accuracy-efficiency trade-offs.

  • RangeFormer achieves superior performance to other modalities in semantic and panoptic LiDAR segmentation.
  • STR provides a scalable way to handle LiDAR learning and processing while yielding better accuracy-efficiency trade-offs.

Appendix

The appendix supplements the main paper with implementation details, additional quantitative results, qualitative comparisons, and acknowledgements.

  • Section 6 provides additional implementation details for the proposed methods and experiments.
  • Section 7 reports additional quantitative results, including class-wise IoU scores for comparative and ablation studies.
  • Section 8 provides additional qualitative results through visual comparisons and video demonstrations.
  • Section 9 acknowledges the public resources used in the work.

6. Additional Implementation Detail

The implementation details cover datasets, model configuration, augmentation operations, and the STR and RangePost procedures used to improve efficiency and range-view processing.

  • The supplementary implementation section describes datasets, network configurations, training details, and testing details.
  • RangeFormer uses GPU-assisted RangeAug, RangePost, and STR partitioning during data preparation to avoid extra training overhead.The common augmentation set includes scaling, rotation, jittering, flipping, and random dropping.
  • Common augmentations transform point coordinates or remove points through scaling, global rotation, jittering, flipping, and random dropping.The listed operations include random coordinate perturbations, axis flips, and 10% point dropping.
  • RangeAug combines RangeMix, RangeUnion, RangePaste, and RangeShift to alter or supplement range-view inputs.These operations mix scans, fill empty grids, paste tail-distribution classes, and shift grids along the azimuth direction.
  • STR trains on arbitrary low-resolution 2D range images while maintaining satisfactory 3D segmentation accuracy and improving the accuracy-efficiency trade-off.RangePost instead reduces aliasing by sub-sampling the point cloud into equal-interval sub-clouds, inferring them, and stitching predictions back.

7. Additional Quantitative Result

Additional experiments evaluate comparative performance, panoptic segmentation, and STR resolution choices across three LiDAR benchmarks.

  • The quantitative studies cover comparative and ablation experiments on SemanticKITTI, nuScenes, and ScribbleKITTI.
  • Comparative Study: RangeFormer and STR outperform competing representations and fusion methods on SemanticKITTI semantic segmentation while using only range-view inputs.The comparisons include raw point, BEV, range-view, voxel, and multi-view fusion methods.
  • Comparative Study: RangeFormer and STR show advantages over a recent LiDAR panoptic segmentation method across PQ, RQ, SQ, and IoU on SemanticKITTI.
  • Ablation Study: 64.3% mIoU is achieved by RangeFormer with W = 192 under STR, while performance remains relatively stable at small range-image resolutions.Higher horizontal resolutions tend to improve segmentation performance while requiring more memory.

8. Additional Qualitative Result

Qualitative visualizations compare RangeFormer with prior range-view methods and report improved predictions, while also identifying recurring failure cases.

  • RangeFormer produces better qualitative segmentation than prior range-view methods and eliminates erroneous predictions around the ego vehicle.The improvement is especially visible in complex regions containing multiple clustered classes.
  • Errors remain likely at object-background boundaries, for rare classes, and in long-distance regions.The paper suggests that designs explicitly considering these cases could improve segmentation further.

9. Public Resources Used

The paper acknowledges public datasets, benchmarks, models, and software resources used during development and evaluation.

  • The acknowledged public resources include SemanticKITTI, nuScenes, ScribbleKITTI, and their associated APIs or development kits.
Loading 2303.05367v3…