Source-linked AI summary
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Wei Li, Yuexin Ma, Hongsheng Li, Ruigang Yang, Dahua Lin
TL;DR
Outdoor LiDAR methods face a trade-off: 2D projections alter 3D geometry, while regular 3D voxel methods make limited gains on sparse, varying-density clouds. The paper combines cylindrical partitioning with asymmetrical 3D convolution and reports strong results across semantic segmentation, panoptic segmentation, and 3D detection.
Problem
2D projection loses or alters 3D geometric relations, while regular 3D voxelization yields limited outdoor gains amid sparse, varying-density point clouds.
Source: §1 · p. 1Method
The framework combines distance-adaptive cylindrical partitioning with asymmetrical 3D convolution to retain geometric information and address outdoor point-cloud sparsity and density variation.
Source: §1 · p. 1, §3.2 · p. 4Results
SemanticKITTI reaches leaderboard state of the art for single- and multi-scan challenges; nuScenes and A2D2 outperform existing methods, with strong performance and generalization on panoptic segmentation and 3D detection.
Source: §1 · p. 2, §6 · p. 13Takeaways & Limitations
The authors report that retaining 3D structure and balancing point distribution could benefit segmentation results.
Source: §4.2 · p. 8
Abstract
from arXiv · showhide
State-of-the-art methods for driving-scene LiDAR-based perception (including point cloud semantic segmentation, panoptic segmentation and 3D detection, \etc) often project the point clouds to 2D space and then process them via 2D convolution. Although this cooperation shows the competitiveness in the point cloud, it inevitably alters and abandons the 3D topology and geometric relations. A natural remedy is to utilize the 3D voxelization and 3D convolution network. However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited. An important reason is the property of the outdoor point cloud, namely sparsity and varying density. Motivated by this investigation, we propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties. The proposed model acts as a backbone and the learned features from this model can be used for downstream tasks such as point cloud semantic and panoptic segmentation or 3D detection. In this paper, we benchmark our model on these three tasks. For semantic segmentation, we evaluate the proposed model on several large-scale datasets, \ie, SemanticKITTI, nuScenes and A2D2. Our method achieves the state-of-the-art on the leaderboard of SemanticKITTI (both single-scan and multi-scan challenge), and significantly outperforms existing methods on nuScenes and A2D2 dataset. Furthermore, the proposed 3D framework also shows strong performance and good generalization on LiDAR panoptic segmentation and LiDAR 3D detection.
1 INTRODUCTION
The paper argues that 2D projection loses LiDAR’s 3D geometric relations, while uniform 3D voxelization has limited benefit for outdoor clouds with sparse, varying-density points. It proposes cylindrical partitioning and asymmetrical 3D convolution to address these properties, then evaluates the framework across semantic, panoptic, and detection tasks.
1 INTRODUCTION
- 2D projection alters or loses 3D geometric information, while direct 3D voxelization yields limited gains because outdoor point clouds are sparse and vary in density.Uniform cubic partitioning neglects that varying-density property.Source: §1 · p. 1
- Cylindrical partitioning assigns larger cells farther from the origin to balance point distribution, while asymmetrical 3D convolutions strengthen horizontal and vertical responses to match driving-scene objects.The framework’s two components address outdoor point-cloud properties through partitioning and network design, respectively.Source: §1 · p. 1
- 89% versus 61% of cells contain points under cylindrical versus cubic partitioning, respectively, showing the more balanced distribution reported for cylindrical cells.This comparison is presented in Fig. 1a.Source: Figure 1 · p. 2
- A point-wise module refines voxel-network features to alleviate information loss from cell-label encoding when points from different categories share a voxel.The paper presents this as an additional component for segmentation.Source: §1 · p. 2
- The method reaches SemanticKITTI leaderboard state of the art for single- and multi-scan challenges, outperforms existing methods on nuScenes and A2D2, and performs strongly on panoptic segmentation and 3D detection.The panoptic and detection experiments also show good generalization capability.Source: §1 · p. 2
2 RELATED WORK
Related work covers point-based indoor methods, 2D projections and 3D voxel encodings for outdoor clouds, and convolutional architectures adapted for feature extraction. The paper positions its approach against projection-related geometric loss and voxel methods’ limited outdoor gains.
2 RELATED WORK
- Indoor point-cloud methods learn directly from points or use clustering, but their assumptions and computational demands limit adaptation to large-range, varying-density outdoor scenes.The passage characterizes indoor clouds as generally uniform-density, small in point count, and limited in scene range.Source: §2 · p. 2
- Outdoor projection methods use range-image or bird’s-eye-view representations, but lose or alter 3D topology and geometric information.The cited examples include spherical projections, and polar-coordinate bird’s-eye-view projection.Source: §2 · p. 2
- 3D voxel partition retains geometric information, yet outdoor gains remain limited when methods neglect sparsity and varying density.The cited voxel approaches use regular 3D convolutions,,.Source: §2 · p. 3
- Feature-extraction architectures build on FCN, dilated and context modeling, attention, and U-Net’s symmetric skip-feature structure; this work adapts U-Net to 3D.The passage cites DeepLab,, PSP, and U-Net.Source: §2 · p. 3
3 METHODOLOGY
The methodology combines cylindrical partitioning with asymmetrical 3D convolution to represent outdoor LiDAR geometry while addressing varying density and sparsity. Segmentation adds point-wise refinement to reduce the effects of lossy cell-label encoding.
3.2 Cylindrical Partition
- About 6 times less non-empty-cell proportion occurs in farther regions with cubic than cylindrical partitioning.Cylinder-coordinate cells grow larger farther from the origin, maintaining a more balanced distribution while retaining a 3D grid representation.Source: Figure 3 · p. 4, §3.2 · p. 4
3.1 Framework Overview
- Cylindrical partitioning and asymmetrical 3D convolution generate voxel-wise outputs, with point-wise refinement added for segmentation.The partition produces a more balanced point distribution, while the convolution is designed to match driving-scene point distributions and handle sparsity.Source: §3.1 · p. 4
3.3 Asymmetrical 3D Convolution Network
- Asymmetrical residual blocks strengthen horizontal and vertical kernels; three rank-1 kernels are aggregated to form high-rank global context.The blocks are used to build downsample and upsample blocks; the asymmetrical residual block is reported to save computation and memory compared with regular square-kernel 3D convolution.Source: §3.3 · pp. 4–5
3.4 Sparse Activation Visualization
- Asymmetrical networks with cylindrical partition show sparser, more region-focused filter activations than regular 3D convolution networks.The authors interpret this visualization as evidence that the model can adapt to sparse point-cloud input and focus on certain regions.Source: §3.4 · p. 5, Figure 5 · p. 5
3.5 Point-wise Refinement Module
- Mixed-category points within one cell cause lossy label encoding, so point-wise refinement fuses features before and after 3D convolution.Both majority and minority cell-label encodings retain information loss rather than reaching ideal 100 percent mIoU.Source: §3.5 · p. 6, Figure 7 · p. 6
3.6 Objective Function
- Semantic segmentation uses voxel-wise weighted cross-entropy and Lovasz-softmax losses plus point-wise weighted cross-entropy; panoptic segmentation adds instance loss, while detection uses focal and smooth L1 losses.The point-wise refinement output is the final semantic segmentation output during inference.Source: §3.6 · p. 6
4 EXPERIMENTS
The experiments evaluate semantic segmentation on SemanticKITTI, nuScenes, and A2D2, panoptic segmentation on SemanticKITTI, and 3D detection on nuScenes. They report strong results across these tasks and use ablations to assess the proposed components and design choices.
4.1 Dataset and Metric
- SemanticKITTI evaluation covers single-frame and multiple-frame segmentation; the paper uses mIoU, averaged across classes, as its evaluation metric.The multi-scan challenge includes moving-status categories.Source: §4.1 · pp. 6–7
4.2 LiDAR-based Semantic Segmentation
- The method achieves consistent state-of-the-art semantic segmentation across SemanticKITTI, nuScenes, and A2D2, including single- and multi-scan settings.On SemanticKITTI multi-scan, it gains 0.3% over KPConv and 8.4% over SpSeqnet in mIoU.Source: §4.2 · pp. 8–9
4.3 LiDAR-based Panoptic Segmentation
- On SemanticKITTI panoptic segmentation, the method improves PQ by about 4.7 percentage points and PQTh and RQTh by 10 points over existing methods.The authors report that PointGroup and LPASD perform poorly on this outdoor LiDAR task.Source: §4.3 · p. 10, Table 7 · p. 11
4.4 LiDAR-based 3D Detection
- For nuScenes 3D detection, cylindrical partition and asymmetrical 3D convolution improve the baselines, including the strong SSNv2 baseline.Against SECOND, the two components improve NDS by 2.8 and 1.5 percentage points, respectively.Source: §4.4 · p. 10, Table 8 · p. 11
4.5 Ablation Studies
- On SemanticKITTI validation, cylindrical partition and asymmetrical 3D convolution each improve mIoU by about 3% over the cubic-partition, regular-convolution baseline.Dimension-decomposition context modeling adds 1.4%, and point-wise refinement adds 0.7%.Source: §4.5 · p. 11, Table 9 · p. 11
- With cylindrical partition fixed, the proposed height-aware asymmetrical residual block improves mIoU by about 3%; the 1D variant gains 1.3%.The proposed block reports about 10% gains on bicycle, person, other-vehicle, and motorcycle categories.Source: §4.5 · p. 12, Table 10 · p. 12
- The 480×360×32 representation outperforms the tested alternatives by 2 percentage points; all tested cylindrical partitions outperform the cubic partition.The authors associate smaller cells with larger representations and larger cells with possible category mixing.Source: §4.5 · pp. 12–13, Table 11 · p. 13
5 DISCUSSION
The discussion compares inference efficiency with 2D and 3D methods, then evaluates the method against approaches designed to handle sparsity. It reports improved performance and efficiency in both comparisons.
5.1 Comparison of Inference Time
- Inference time is measured in evaluation mode with the setting unchanged; existing-method timings are taken from.The study compares against methods addressing sparsity, including SPVNAS [47] and MinkowskiNet.Source: §5.1 · p. 13, §5.2 · p. 13
- About 5.0× speedup and 14% performance improvement are reported over RandLA.The comparison includes computation and post-processing time for RandLA.Source: §5.1 · p. 13
- A cylindrical partition creates a more compact representation, while asymmetrical 3D convolutions reduce computation by 33% versus a 3 × 3 × 3 convolution with the same output filters.The cited example says a regular cubic grid of 1000 × 1000 × 45 is more than four times larger than the cylindrical representation.Source: §5.1 · p. 13
5.2 Comparison with other methods dealing with sparsity issue
- On SemanticKITTI validation, the method reports better performance and efficiency than other methods designed to handle sparsity.The comparison is presented in Table 13.Source: §5.2 · p. 13, Table 13 · p. 13
6 CONCLUSION
The paper concludes that its cylindrical partition and asymmetrical 3D convolution networks maintain 3D geometric relations while addressing sparsity and varying density in outdoor LiDAR point clouds. It reports state-of-the-art semantic segmentation results on SemanticKITTI, A2D2, and nuScenes, with generalization to panoptic segmentation and 3D detection.
6 CONCLUSION
- The model achieves state-of-the-art results on SemanticKITTI, A2D2, and nuScenes, and shows good generalization to LiDAR panoptic segmentation and 3D detection.The conclusion describes the framework as maintaining 3D geometric relations while addressing sparsity and varying density in outdoor LiDAR point clouds.Source: §6 · p. 13