Source-linked AI summary
Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
Hui Zhou, Xinge Zhu, Xiao Song, Yuexin Ma, Zhe Wang, Hongsheng Li, Dahua Lin
TL;DR
Driving-scene LiDAR segmentation methods often project point clouds into 2D, which alters 3D topology and geometric relations. Cylinder3D instead studies 3D representations and backbones and proposes cylinder partitioning with 3D convolution and context modeling. It achieves state-of-the-art performance on SemanticKITTI, with a reported 6% mIoU gain over existing methods.
Problem
3D-to-2D projections used in LiDAR segmentation can compress 3D topology and fail to model geometric information.
Method
Cylinder3D retains a 3D representation through cylinder partitioning and 3D convolution, using asymmetric residual blocks and dimension-decomposition context modeling.
Results
6% mIoU gain over state-of-the-art algorithms is reported on driving-scene semantic-segmentation benchmarks.
Takeaways & Limitations
3D partitioning with 3D convolution performs better than the compared alternatives for driving-scene LiDAR segmentation.
Takeaways & Limitations
The related-work discussion identifies projection-based methods’ geometric-information limitation but does not state a limitation of Cylinder3D itself.
Abstract
from arXiv · showhide
State-of-the-art methods for large-scale driving-scene LiDAR semantic segmentation often project and process the point clouds in the 2D space. The projection methods includes spherical projection, bird-eye view projection, etc. Although this process makes the point cloud suitable for the 2D CNN-based networks, it inevitably alters and abandons the 3D topology and geometric relations. A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space. In this work, we first perform an in-depth analysis for different representations and backbones in 2D and 3D spaces, and reveal the effectiveness of 3D representations and networks on LiDAR segmentation. Then, we develop a 3D cylinder partition and a 3D cylinder convolution based framework, termed as Cylinder3D, which exploits the 3D topology relations and structures of driving-scene point clouds. Moreover, a dimension-decomposition based context modeling module is introduced to explore the high-rank context information in point clouds in a progressive manner. We evaluate the proposed model on a large-scale driving-scene dataset, i.e. SematicKITTI. Our method achieves state-of-the-art performance and outperforms existing methods by 6% in terms of mIoU.
1 Introduction
Driving-scene LiDAR segmentation must handle sparse, distance-dependent point densities and preserve 3D geometric structure. Cylinder3D studies representations and backbones, then proposes a 3D cylinder-based framework with specialized modules.
- Motivation: Outdoor driving-scene LiDAR has varying point densities that challenge methods developed for dense, uniformly sampled indoor point clouds.Nearby regions contain much denser points than distant regions.
- Motivation: Range-image and bird-view representations simplify 3D point clouds into 2D grids but may lose accurate geometric information.Range images use spherical projection, while bird-view images compress point height into a global height feature.
- Method: Cylinder3D uses cylinder partitioning to balance the distribution of driving-scene points and combines it with 3D convolution, asymmetric residual blocks, and dimension-decomposition context modeling.The framework is designed to retain 3D topology, match cuboid objects, and progressively exploit context using low-rank convolution kernels.
- Findings: Experiments show that directly processing point clouds without 3D-to-2D projection is crucial for superior segmentation performance.The study compares point representations and network architectures across 2D, 2.5D, and 3D settings.
- Results: 6% mIoU gain over state-of-the-art algorithms is reported on driving-scene semantic-segmentation benchmarks.The contribution statement characterizes this as a large-margin improvement.
2 Related Work
Prior outdoor LiDAR methods commonly project 3D point clouds into 2D grids for efficient 2D convolution, while indoor methods often operate directly on points. These approaches face different density and geometric-structure challenges, motivating Cylinder3D’s 3D design.
- Indoor-scene Point Cloud Segmentation: Indoor point-cloud methods commonly process raw points or use clustering because indoor scenes have uniform density and limited spatial range.PointNet, PointNet++, and clustering-based methods are cited as examples.
- Indoor-scene Point Cloud Segmentation: Indoor segmentation methods can be computationally costly and do not account for the varying sparsity characteristic of outdoor LiDAR.This contrasts indoor-scene assumptions with autonomous-driving point clouds.
- Outdoor-scene Point Cloud Segmentation: Outdoor LiDAR methods commonly use spherical or bird-view projection followed by 2D convolution on the resulting grids.SqueezeSeg, Darknet, SqueezeSegv2, RangeNet++, and PolarNet are representative methods.
- Outdoor-scene Point Cloud Segmentation: 3D-to-2D projection inevitably compresses 3D topology and fails to model geometric information.The limitation applies to projection-based outdoor-scene segmentation approaches.
- Network Architectures for Segmentation: Existing 3D U-Net-style approaches adapt segmentation networks to 3D but often fail to explore driving-scene LiDAR distributions and properties.Cylinder3D addresses this with asymmetric residual blocks and dimension-decomposition context modeling.
3 Methodology
Cylinder3D preserves 3D structure by combining cylinder partitioning with a 3D U-Net tailored to sparse, unevenly distributed outdoor LiDAR points. Its asymmetric residual blocks and dimension-decomposition context modeling adapt computation and context aggregation to driving-scene geometry.
- 3 Methodology: Outdoor driving-scene clouds cover large areas, contain over 100,000 points, and vary substantially in density, limiting direct adaptation of indoor segmentation methods.Nearby regions are denser than distant regions, creating a distinctive representation challenge.
- 3 Methodology: 3D partitioning and 3D convolution preserve geometric structures that projection-based 2D representations may abandon.The framework is motivated by experiments comparing 2D, 2.5D, and 3D partitions and networks.
- 3 Methodology: The network combines 3D cylinder partitioning with a 3D U-Net, asymmetric residual blocks for cuboid objects, and dimension-decomposition context modeling.The context module decomposes high-rank context into lower-rank representations across spatial dimensions.
- 3 Methodology: Cylinder partition converts Cartesian points (x, y, z) to (ρ, θ, z), uses uniformly split cylinder dimensions, and produces a 3D representation R ∈ C × H × W × L.The resulting farther-away regions use larger voxels, more evenly distributing points while retaining a 3D grid.
- 3 Methodology: Asymmetric residual blocks use 3 × 1 × 3 and 1 × 3 × 3 convolutions, matching a 3 × 3 × 3 receptive field with 33% lower computational cost.They form the basic components of the downsample and upsample blocks.
- 3 Methodology: The implementation uses sparse 3D convolutions, a 4-layer MLP-based cylinder partition, a lightweight 3 × 3 × 3 segmentation head, and weighted cross-entropy plus Lovasz-softmax losses.The total optimization loss is ζall = ζiou + ζacc, optimized with Adam at an initial learning rate of 0.001.
4 Experiments
Experiments compare 2D and 3D representations and backbones on SemanticKITTI, then evaluate Cylinder3D, its components, and qualitative predictions.
- 4.1 Dataset and Metric: SemanticKITTI contains 23,201 training and 20,351 testing point clouds, with 19 classes used for training and evaluation.Sequence 08 serves as validation, and performance is measured by mean intersection-over-union across classes.
- 4.2 Backbone and Representation Odyssey from 2D to 3D: Cylinder 3D voxelization divides point clouds in cylindrical coordinates to match varying sparsity and balance point distribution.Cuboid voxelization uses Cartesian coordinates but can incur high computational costs because of large voxel resolutions and 3D convolutions.
- 4.2 Backbone and Representation Odyssey from 2D to 3D: 1.7% gain follows replacing polar projection with cylinder voxelization, which retains 3D topology.The result supports the effectiveness of the proposed 3D cylinder partition.
- 4.2 Backbone and Representation Odyssey from 2D to 3D: 4.2% gain brings Cylinder3D to 64.3% mIoU on the validation set after converting the 2D backbone to a 3D backbone.The experiment attributes the improvement to the cooperation of 3D cylinder partition and 3D convolution.
- 4.3 Results on SemanticKitti: At least 6% mIoU separates the proposed method from other state-of-the-art methods on the SemanticKITTI test set.The comparison includes RangeNet++, PolarNet, SqueezeSegv3, and RandLA-Net, among others.
- 4.4 Effects of network components: 1.5% mIoU comes from the asymmetric residual block, while dimension-decomposition context modeling reaches 64.3% mIoU and flip testing adds 0.9%.The ablation uses cylinder partition and a 3D U-Net with common residual blocks and no dimension-decomposition context modeling as its baseline.
- 4.5 Visualization: Cylinder3D mainly achieves decent accuracy and separates nearby objects in validation visualizations.The paper attributes this behavior to maintaining 3D topology and using geometric information.
5 Conclusion
Cylinder3D keeps LiDAR segmentation in 3D through a cylinder partition and 3D convolutional network, while adding modules for efficiency and high-rank context.
- 5 Conclusion: Cylinder partition suits the varying sparsity of driving-scene LiDAR point clouds.It is paired with a 3D convolution-based network designed around asymmetric residual blocks and dimension-decomposition context modeling.
- 5 Conclusion: The cooperation of cylinder partition and 3D convolution networks achieves state-of-the-art performance on the SemanticKITTI test set.The framework is presented as following the 3D nature of LiDAR point clouds.