PaperlayerView exact revision on arXiv

Source-linked AI summary

Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images

Zhi Tian, Xiangxiang Chu, Xiaoming Wang, Xiaolin Wei, Chunhua Shen

arXiv:2205.13764v2cs.CV

TL;DR

LiDAR 3D detection must handle unordered point clouds, while BEV methods typically require voxelization and sparse convolutions and prior range-view methods largely use single frames. FCOS-LiDAR detects in compact range images with standard convolutions and introduces MRV projection for multi-frame fusion. It achieves favorable comparisons with BEV detectors while being simpler and faster in the reported settings.

  • Problem

    Unordered LiDAR points make it difficult to apply grid-based 2D detection techniques, and prior range-view detectors largely lacked multi-frame fusion.

  • Method

    FCOS-LiDAR is a fully convolutional one-stage detector using LiDAR range images, with MRV projection to support multi-frame point-cloud fusion.

  • Results

    FCOS-LiDAR outperforms CenterPoint while being significantly faster for single-frame inputs and achieves competitive performance with multi-frame BEV detectors.

  • Takeaways & Limitations

    The results provide evidence that range-view 3D detectors using standard convolutions can compare favorably with mainstream BEV-based detectors.

  • Takeaways & Limitations

    The paper notes that its LiDAR-based object-localization technique might be abused for military purposes, including lethal autonomous weapons.

Abstract

from arXiv · show

We present a simple yet effective fully convolutional one-stage 3D object detector for LiDAR point clouds of autonomous driving scenes, termed FCOS-LiDAR. Unlike the dominant methods that use the bird-eye view (BEV), our proposed detector detects objects from the range view (RV, a.k.a. range image) of the LiDAR points. Due to the range view's compactness and compatibility with the LiDAR sensors' sampling process on self-driving cars, the range view-based object detector can be realized by solely exploiting the vanilla 2D convolutions, departing from the BEV-based methods which often involve complicated voxelization operations and sparse convolutions. For the first time, we show that an RV-based 3D detector with standard 2D convolutions alone can achieve comparable performance to state-of-the-art BEV-based detectors while being significantly faster and simpler. More importantly, almost all previous range view-based detectors only focus on single-frame point clouds, since it is challenging to fuse multi-frame point clouds into a single range view. In this work, we tackle this challenging issue with a novel range view projection mechanism, and for the first time demonstrate the benefits of fusing multi-frame point clouds for a range-view based detector. Extensive experiments on nuScenes show the superiority of our proposed method and we believe that our work can be strong evidence that an RV-based 3D detector can compare favourably with the current mainstream BEV-based detectors.

1 Introduction

FCOS-LiDAR addresses the difficulty of applying grid-based 2D detection techniques to unordered LiDAR point clouds by detecting objects in a compact range view. It uses standard convolutions and MRV projection to support multi-frame fusion while remaining simpler than BEV-based detectors.

  • Motivation: Point-cloud 3D detection is difficult because LiDAR points are unordered, whereas modern CNNs assume grid-organized inputs.Existing methods therefore commonly structure points into voxels or pillars before detection.
  • Range-view representation: Range views project spherical-coordinate LiDAR points into compact 2D images compatible with sensor sampling and standard 2D detection methods.This representation avoids sparse convolutions and alleviates point loss compared with BEV representations.
  • Results: MRV projection enables range-view detectors to benefit from multi-frame fusion and achieve competitive performance with multi-frame BEV detectors.The authors identify this as their first approach to range-view multi-frame fusion.
  • Proposed detector: FCOS-LiDAR is a fully convolutional one-stage detector that operates on LiDAR range images and is substantially simpler than mainstream BEV-based detectors.Its range-view design minimizes the gap between 3D and 2D detectors.
  • Results: In single-frame settings, FCOS-LiDAR outperforms CenterPoint while being significantly faster.The comparison is reported against the state-of-the-art BEV-based detector CenterPoint.
  • Implications: The reported results are presented as evidence that range-view detectors can compare favorably with mainstream BEV-based detectors.The authors use these results to encourage further attention to the range-view direction.

2 Related Work

Prior LiDAR detectors predominantly use BEV representations built from voxels or pillars, while range-view methods apply 2D convolutions to compact range images. Existing range-view approaches improve detection quality but generally remain limited to single-frame inputs.

  • Bird-view based 3D Detection: Most top-performing BEV detectors convert point clouds into voxel or pillar representations before applying detection pipelines.CenterPoint builds on voxel-based or pillar-based BEV representations with an anchor-free pipeline.
  • Range-view based 3D Detection: Range-view detectors transform point clouds into range images and apply 2D convolutions to detect 3D objects.Representative methods address uncertainty, range-dependent dilation, scale variation, and geometric feature extraction.
  • Range-view based 3D Detection: Existing range-view detectors described here focus on single-frame point clouds and neglect substantial multi-frame information.This limitation motivates the paper’s multi-frame range-view projection mechanism.

3 Our Approach

FCOS-LiDAR represents LiDAR point clouds as range images and processes them with fully convolutional 2D networks. Its approach combines multi-frame range-view projection, modality-wise convolutions, and FCOS-style multi-level detection heads.

  • 3.1 Range View Representation: A LiDAR point is transformed from Cartesian to spherical coordinates and discretized into an m × n range image using azimuth and inclination bins.Each pixel stores nine channels: Cartesian and spherical coordinates, intensity, point existence, and relative timestamp.
  • 3.2 Multi-round Range View Projection (MRV): Multi-frame projection suffers severe bin collisions: on nuScenes, about 90% of roughly 240K input points are discarded, leaving approximately 28K points.This decimation leaves multi-frame range views with nearly the same number of valid points as single-frame inputs.
  • 3.2 Multi-round Range View Projection (MRV): MRV repeatedly projects rejected points into additional nine-channel groups until enough points are retained; nuScenes uses five projection rounds.The mechanism is designed to preserve more multi-frame information instead of discarding colliding points.
  • 3.3 Modality-wise Convolutions: Modality-wise convolutions process the range image’s distinct channel types separately before merging their features with a 1 × 1 convolution.Grouped convolutions reflect the weaker relationships among coordinate, spherical, intensity, existence, and timestamp modalities.
  • 3.4 Overall Architecture: FCOS-LiDAR uses a LiDAR-Net backbone, an FPN with six pyramid levels, and separate classification and regression heads to predict 3D boxes.The backbone inserts modality-wise branches with dilation rates 1, 3, and 6, while the detector follows an anchor-free FCOS-style design.
  • 3.4 Overall Architecture: Training assigns each range-image pixel targets when its original 3D point lies inside an object box, with regression targets defined relative to that pixel’s coordinates.Classification labels are dynamically reassigned using the top K lowest-cost pixels for each ground-truth box.

4 Experiments

Experiments on nuScenes evaluate FCOS-LiDAR’s multi-frame range-view projection, convolution design, detection heads, inference efficiency, and comparison with state-of-the-art methods. The results support competitive detection performance with efficient standard-convolution processing.

  • Experimental Setup: The nuScenes experiments use 1000 scenes and evaluate 3D detection with mAP and NDS.The dataset includes 700 training, 150 validation, and 150 test scenes; mAP uses bird-eye-view center distances, while NDS combines mAP with several error measures.
  • 4.1 Multi-round Range View Projection: Prioritizing current-frame points during collisions preserves multi-frame performance at 57.08% mAP, versus 54.29% without prioritization.The single-frame counterpart achieves 53.42% mAP, making current-frame priority important for effective range-view fusion.
  • 4.2 Modality-wise Convolutions: Multiple dilation rates improve mAP from 55.87% to 57.08%, while untying detection-head weights improves mAP from 56.44% to 57.08%.The dilation experiment evaluates multi-scale context aggregation, and the head-weight result contrasts with common image-based detector behavior.
  • 4.4 Inference Time Comparisons: FCOS-LiDAR uses deterministic, efficient MRV and standard convolutions, and achieves competitive performance with state-of-the-art methods on the nuScenes test set.Compared with CenterPoint, MRV is faster than voxelization and the network avoids sparse convolutions; the test-set model uses wider detection heads and modified late-training augmentation.

5 Conclusion

FCOS-LiDAR is an efficient range-view 3D detector using standard convolutions alone. The paper also demonstrates multi-frame fusion for range-view detection and notes a potential military misuse.

  • FCOS-LiDAR performs range-view 3D detection using standard convolutions alone, without requiring voxelization or sparse convolutions.
  • The proposed detector demonstrates that range-view 3D detection can benefit from multi-frame fusion through MRV.
  • The method's stated societal-impact concern is potential misuse for military purposes, including lethal autonomous weapons.

A More Experiments

Additional experiments examine detection-head capacity, convolutional depth, and comparisons with popular 3D detectors. Larger detection heads improve mAP with added latency, while two modality-wise convolution layers perform best in the reported study.

  • 57.71% mAP is achieved with 128 detection-head channels, up from 57.08%, at the cost of 6ms additional latency.
  • Two convolutional layers achieve the best performance among the tested modality-wise convolution configurations.
  • Table 9 reports latency, mAP, NDS, and per-class results while varying the number of modality-wise convolution layers.

B Visualization

The paper presents visualization results for FCOS-LiDAR on the nuScenes validation set and reports reliable operation across varied challenging circumstances.

  • FCOS-LiDAR is reported to work reliably under a wide variety of challenging circumstances.
  • The comparison context uses methods trained for 40 epochs with MMDetection3D to support fair validation-set comparisons.
  • Figure 4 visualizes FCOS-LiDAR results on the nuScenes validation set.

C Rejoinder

The rejoinder explains the comparison choices, reports controlled validation and test-set settings, and clarifies the feature-map resolutions used by the FPN.

  • Previous range-view methods are difficult to compare directly because they do not release full code and differ in training, architecture, and processing settings.
  • The paper mainly compares FCOS-LiDAR with mainstream BEV methods such as CenterPoint and PointPillar, including in the multi-frame setting.
  • The validation model uses 64 detection-head channels, whereas the test model uses FCOS-LiDAR with 128 channels and a fade strategy that improves performance by about 2% mAP.
  • Only the first FPN feature level matches the original image size; subsequent levels are down-sampled by powers of 2.

Q. Does random scale augmentation cause object artifacts?

The augmentation is designed to avoid changing range-view projections and applies only a small global scale perturbation. Therefore, it is presented as unlikely to create object artifacts.

  • Global scaling proportionally scales all points in a cloud by the same factor.This preserves the points’ azimuth and inclination angles.
  • Because the spherical angles remain unchanged, the range-view projections also remain unchanged.
  • The scale factor is restricted to 0.95–1.05, limiting the perturbation to a small change in the point cloud.

Q. MRV’s performance in multi-frame settings still falls behind BEV’s.

MRV enables range-view detectors to use multi-frame fusion and substantially improves velocity estimation, but multi-frame performance still remains below BEV’s.

  • Multi-frame range-view fusion remains an open question, with further work needed to match BEV performance.
  • MRV is the first range-view approach reported to demonstrate positive benefits from multi-frame fusion.Earlier range-view detectors had not shown positive multi-frame results.
  • On nuScenes validation, the best model reduces averaged velocity error from 1.08 for single-frame input to 0.301.

Q. The ablation study of Table 3.

Table 3 compares multi-frame models that differ in channel grouping, while holding the fusion setting constant. The reported inference-time differences are mostly too small to attribute confidently to architecture.

  • All Table 3 entries use multi-frame fusion, with channel grouping as the distinguishing design choice.
  • Inference time is measured with batch size 1 on a 3090Ti GPU without a sliding temporal window.
  • Except for the first row, inference-time differences are ≤1ms and may reflect implementation effects rather than network architecture.
  • The selected model is the one with the best mAP and NDS, with performance prioritized over the small timing differences.

Q. Comparisons with the nuScenes leaderboard results.

The paper cautions that leaderboard comparisons are affected by ensembles, test-time augmentation, and other techniques. Its own method is described as simpler to deploy, while target assignment uses first-round projection points.

  • Leaderboard methods often use ensembles, test-time augmentation, and techniques such as PointPainting to improve results.
  • The authors mainly compare against published papers because leaderboard results are not peer-reviewed.
  • For multi-round projection, target assignments use only first-round points rather than all projected rounds.Using all rounds would require an additional branch to predict the active round at inference.
  • The reported first-round and all-round target-assignment variants achieve similar performance.
Loading 2205.13764v2…