Source-linked AI summary
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, Dahua Lin
TL;DR
2D projection loses 3D topology, while regular 3D voxel methods gain little on sparse, varying-density outdoor LiDAR. The paper uses cylindrical partition, asymmetrical 3D convolutions, and point-wise refinement, achieving leading segmentation results and generalizing to other LiDAR tasks.
Problem
2D projection alters 3D geometry, while regular 3D voxelization and convolution provide limited gains because outdoor LiDAR is sparse and varies in density.
Method
The framework combines distance-aware cylindrical partition, asymmetrical 3D convolution networks, and point-wise refinement to preserve geometry while addressing outdoor LiDAR properties and voxel-label information loss.
Results
The method achieves the 1st place on SemanticKITTI, state-of-the-art performance on nuScenes, and strong generalization to LiDAR panoptic segmentation and 3D detection.
Takeaways & Limitations
The proposed 3D framework maintains geometric relations while handling outdoor point-cloud sparsity and varying density across segmentation and other LiDAR tasks.
Abstract
from arXiv · showhide
State-of-the-art methods for large-scale driving-scene LiDAR segmentation often project the point clouds to 2D space and then process them via 2D convolution. Although this corporation shows the competitiveness in the point cloud, it inevitably alters and abandons the 3D topology and geometric relations. A natural remedy is to utilize the3D voxelization and 3D convolution network. However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited. An important reason is the property of the outdoor point cloud, namely sparsity and varying density. Motivated by this investigation, we propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pat-tern while maintaining these inherent properties. Moreover, a point-wise refinement module is introduced to alleviate the interference of lossy voxel-based label encoding. We evaluate the proposed model on two large-scale datasets, i.e., SemanticKITTI and nuScenes. Our method achieves the 1st place in the leaderboard of SemanticKITTI and outperforms existing methods on nuScenes with a noticeable margin, about 4%. Furthermore, the proposed 3D framework also generalizes well to LiDAR panoptic segmentation and LiDAR 3D detection.
1. Introduction
Outdoor LiDAR segmentation must preserve 3D geometry while addressing sparsity and varying density, which limit both 2D projection and regular 3D voxel approaches. The proposed framework combines cylindrical partition, asymmetrical 3D convolutions, and point-wise refinement, achieving strong benchmark and cross-task results.
- 2D projection methods lose and alter accurate 3D geometric information, motivating segmentation directly on 3D structure.
- Regular 3D voxelization and convolution provide limited gains outdoors because point clouds are sparse and vary in density.
- Cylindrical partition dynamically assigns larger cells to farther, sparser regions, producing a more balanced point distribution than cubic partition.
- Asymmetrical 3D convolutions strengthen horizontal and vertical kernels to match driving-scene object distributions and improve robustness to sparsity.
- A point-wise refinement module reduces information loss when differently labeled points share a voxel and voxel labels encode them jointly.
2. Related Work
Related work mainly follows either 2D projection with 2D convolutions or 3D voxel partition with 3D networks. The paper positions its approach as a 3D alternative while adapting convolutional architecture to LiDAR segmentation.
- Indoor point-cloud methods commonly exploit raw point features, sampling, grouping, ordering, or clustering for generally uniform-density scenes.
- Outdoor methods often project point clouds into spherical or bird’s-eye-view 2D grids before applying 2D convolutional networks.
- Projection methods inevitably lose and alter 3D topology, whereas 3D voxel partition mainly retains geometric information.
- Segmentation architectures build on fully convolutional networks through dilated convolution, multi-scale context, attention, neural architecture search, and U-Net structures.
- The proposed network follows U-Net’s 3D adaptation while using an asymmetrical architecture for LiDAR segmentation.
3. Methodology
The method combines cylindrical partition with asymmetrical 3D convolutions to model outdoor LiDAR geometry under sparsity and varying density, then refines point-wise predictions to reduce cell-label information loss.
- The framework uses cylindrical partition and asymmetrical 3D convolution networks to preserve 3D geometry while addressing outdoor point-cloud sparsity and varying density.Cylindrical partition handles partitioning, while the convolution design addresses the resulting point-distribution challenges.
- Cylindrical Partition: Cylinder partition produces a more balanced point distribution by increasing grid size with distance, reducing the imbalance caused by uniform cells.The figure compares the proportion of non-empty cells across distances for cylindrical and cubic partitions.
- Cylindrical Partition: Cylindrical partition transforms Cartesian points (x, y, z) into cylinder coordinates (ρ, θ, z), using larger cells for farther-away regions.Point-wise MLP features are reassigned to the resulting cylindrical grid, which is unrolled into a 3D representation R ∈ C × H × W × L.
- Asymmetrical 3D Convolution Network: Asymmetrical residual blocks strengthen horizontal and vertical kernels to match driving-scene object shapes and improve robustness to outdoor sparsity.Asymmetrical downsample and upsample blocks are built by incorporating these residual blocks; the design also reduces computation and memory cost versus regular square-kernel 3D convolutions.
- Asymmetrical 3D Convolution Network: Dimension-decomposition context modeling constructs high-rank global context by combining low-rank features obtained from three rank-1 kernels.The resulting context is formed through a decompose-aggregate strategy.
- Point-wise Refinement Module: Because different categories can share one cell, the method adds a point-wise refinement module that fuses features before and after 3D convolution.Training combines voxel-wise and point-wise losses: L = L_voxel + L_point.
4. Experiments
Experiments evaluate the method on SemanticKITTI and nuScenes, quantify component effects, and test generalization to panoptic segmentation and 3D detection.
- Experimental setup: The method is evaluated on SemanticKITTI and nuScenes, with additional experiments on LiDAR panoptic segmentation and 3D detection.
- Results on SemanticKITTI: SemanticKITTI test results show 8% ∼17% mIoU gains over projection-based methods and better performance than compared voxel-based 3D convolution methods.
- Results on nuScenes: About 4% ∼7% performance gains over projection-based methods are reported on nuScenes, with especially strong results for sparse bicycle and pedestrian categories.
- Ablation studies: Cylindrical partition and asymmetrical 3D convolution each improve SemanticKITTI validation mIoU by about 3% over their corresponding alternatives.
- Ablation studies: The point-wise refinement module adds about 0.7% performance on the strong model, while dimension-decomposition context modeling adds 1.4%.
5. Conclusion
The proposed framework maintains 3D geometric relations while addressing outdoor LiDAR sparsity and varying density. It achieves leading results on SemanticKITTI and nuScenes and generalizes to related LiDAR tasks.
- The framework maintains 3D geometric relations through cylindrical partition and asymmetrical 3D convolution networks designed for outdoor LiDAR.These components target sparsity and varying density in outdoor point clouds.
- The model achieves 1st place in SemanticKITTI and state-of-the-art performance in nuScenes.
- The framework generalizes well to LiDAR panoptic segmentation and LiDAR 3D detection.