PaperlayerView exact revision on arXiv

Source-linked AI summary

SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving

Eren Erdal Aksoy, Saimir Baci, Selcuk Cavdar

arXiv:1909.08291v1cs.CVcs.RO

TL;DR

SalsaNet tackles joint road-and-vehicle segmentation in sparsely annotated 3D LiDAR data. It uses an encoder-decoder network with camera-to-LiDAR auto-labeling and compares BEV with SFV projections. On KITTI, it reports projection-agnostic performance, state-of-the-art accuracy, and lower computation time.

  • Problem

    Limited annotated LiDAR data, especially for road segments, hinders joint segmentation of roads and vehicles needed for autonomous-driving scene understanding.

  • Method

    SalsaNet is an encoder-decoder network that segments road and vehicle points from LiDAR, using transferred camera labels and comparing BEV and SFV inputs.

  • Results

    SalsaNet outperforms other state-of-the-art semantic-segmentation methods in pixel-wise accuracy while requiring much less computation time and performs well in both BEV and SFV.

  • Takeaways & Limitations

    Camera-to-LiDAR label transfer supplies training annotations, while SalsaNet’s comparable BEV and SFV performance indicates projection-agnostic operation.

Abstract

from arXiv · show

In this paper, we introduce a deep encoder-decoder network, named SalsaNet, for efficient semantic segmentation of 3D LiDAR point clouds. SalsaNet segments the road, i.e. drivable free-space, and vehicles in the scene by employing the Bird-Eye-View (BEV) image projection of the point cloud. To overcome the lack of annotated point cloud data, in particular for the road segments, we introduce an auto-labeling process which transfers automatically generated labels from the camera to LiDAR. We also explore the role of imagelike projection of LiDAR data in semantic segmentation by comparing BEV with spherical-front-view projection and show that SalsaNet is projection-agnostic. We perform quantitative and qualitative evaluations on the KITTI dataset, which demonstrate that the proposed SalsaNet outperforms other state-of-the-art semantic segmentation networks in terms of accuracy and computation time. Our code and data are publicly available at https://gitlab.com/aksoyeren/salsanet.git.

I. INTRODUCTION

SalsaNet addresses joint road-and-vehicle segmentation in LiDAR point clouds despite limited annotations and irregular point structure. It uses automatic camera-to-LiDAR labeling and compares BEV and SFV projections, achieving strong accuracy with lower computation time.

  • Semantic segmentation of drivable space and vehicles supports autonomous-vehicle scene understanding, maneuver planning, and decision making.
  • LiDAR offers reliable distance measurement and broad field of view, but sparse, irregular, non-uniform points make direct processing expensive.
  • SalsaNet jointly segments road and vehicle points using a LiDAR-only encoder-decoder with residual blocks and skip connections.
  • Automatic camera-to-LiDAR label transfer expands KITTI annotations from approximately 300 annotated road images to approximately 11K point clouds.
  • SalsaNet uses BEV inputs with statistical point-cloud channels and is separately evaluated with SFV to compare projection effects.
  • Quantitative and qualitative KITTI experiments report projection-agnostic performance, higher pixel-wise accuracy than competing methods, and substantially lower computation time.

II. RELATED WORK

Prior LiDAR segmentation work spans heuristic, point-wise, voxel, and image-projection approaches, often addressing roads or obstacles separately. SalsaNet builds on encoder-decoder and projection-based methods while jointly labeling roads and vehicles using automatically expanded annotations.

  • Deep-learning LiDAR segmentation methods differ in architecture and representation, including fully convolutional, encoder-decoder, and multi-branch networks.
  • Voxel grids can waste computation on empty regions, while point-wise methods struggle with large LiDAR sets; 2D projections are compact and suited to real-time processing.
  • BEV and SFV are common projections, but SFV distortion and deformation can affect small objects such as vehicles.
  • Nearby methods often segment only road objects, whereas SalsaNet focuses on road and vehicle segments and excludes sparse pedestrian and cyclist classes.
  • Existing road-segmentation methods omit road objects and rely on approximately 300 labeled samples, motivating automatic labeling of approximately 11K KITTI point clouds.

III. METHOD

The method section organizes SalsaNet around automatic LiDAR labeling, point-cloud representation, network architecture, and training details.

  • SalsaNet’s method description begins with automatic labeling before covering point-cloud representation, network architecture, and training details.

A. Data Labeling

The labeling pipeline uses mature image-based segmentation to annotate LiDAR point clouds, then represents the resulting data through compact 2D projections. BEV is formed from a defined spatial region and statistical grid-cell features.

  • Automatic labeling addresses the difficulty of manually annotating large LiDAR point clouds by transferring image-based road and object segmentation to 3D data.
  • The pipeline generates network inputs in both BEV and SFV formats from automatically labeled point clouds.
  • Projecting LiDAR onto a 2D grid preserves height as an additional channel while enabling compact representations and standard convolutions.
  • BEV rasterizes a 50 m by 18 m front-facing region into a 256×64 grid with 0.2 m and 0.3 m cell sizes along the two axes.
  • Each BEV cell stores mean and maximum elevation, average intensity, and point count as normalized image channels.

2) Spherical-Front-View (SFV):

SalsaNet also uses a spherical-front-view projection that maps LiDAR points into a panoramic 2D grid using angular coordinates. The representation stores geometric, intensity, range, and occupancy information, but introduces distortion and occlusion effects.

  • SFV projects the 3D point cloud onto a sphere to create a dense panoramic grid representation.
  • Each point is represented by azimuth and zenith angles plus intensity, then discretized into grid coordinates (u, v).
  • The SFV region of interest covers a 90° front-view area.
  • Each grid cell stores Cartesian coordinates, intensity, range, and occupancy, producing a six-channel 64 × 512 image.
  • SFV is denser than BEV but can distort small objects and increase mutual occlusion, motivating BEV as the main representation.

C. Network Architecture

SalsaNet uses a 256×64×4 BEV input and an encoder-decoder architecture built from ResNet blocks, pooling, and upsampling. Its class-balanced loss reweights under-represented classes to address severe class imbalance.

  • Network architecture: SalsaNet receives a 256×64×4 BEV projection of the LiDAR point cloud.
  • Network architecture: The encoder uses successive ResNet blocks with pooling and reaches a total downsampling factor of 16, while the decoder upsamples features.
  • Network architecture: Convolutional layers use leaky-ReLU activations and batch normalization, with dropout placed after batch normalization.
  • Network architecture: Skip connections combine decoder features with outputs from corresponding early residual blocks.
  • Class-balanced loss: Class imbalance can bias segmentation toward frequently occurring road and background classes, harming vehicle segmentation.
  • Class-balanced loss: The loss weights classes by the inverse square root of their frequency to strengthen under-represented classes.

E. Optimizer And Regularization

SalsaNet is trained with Adam and regularized through learning-rate decay, dropout, data augmentation, and extended training.

  • Adam training starts with a 0.01 learning rate, which is decayed by 0.1 every 20K iterations.
  • The model uses dropout probability 0.5 and batch size 32.
  • Training runs for 500 epochs.
  • Input augmentation applies horizontal flips, random pixel noise with probability 0.5, and random z-axis rotations between −5° and 5°.

IV. EXPERIMENTS

Experiments evaluate SalsaNet on automatically annotated KITTI point clouds using class-wise segmentation metrics and comparisons with SqueezeSeg baselines. SalsaNet achieves the highest IoU scores in both BEV and SFV, while border points remain a source of minor mislabeling.

  • Evaluation setup: 10,848 KITTI point clouds are generated with point labels for road, vehicle, and background classes.
  • Evaluation metrics: Performance is evaluated with per-class precision, recall, and intersection-over-union, alongside average IoU across all three classes.
  • Baselines: SalsaNet is compared with SqueezeSeg and SqueezeSegV2 using their reported configurations with adjusted training procedures.
  • Quantitative results: SalsaNet records the highest IoU scores in all evaluated cases.
  • Quantitative results: In BEV, competing methods trail SalsaNet by 6.3% in vehicle IoU, whereas the SFV margin is 0.6%.
  • Error analysis: Minor mislabeling occurs mainly near road and vehicle borders and may be reduced through more precise training annotations.

C. Qualitative Results

SalsaNet’s qualitative evaluation on KITTI shows strong separation of road, vehicle, and background points, while also exposing failures caused by incomplete automatically generated road labels.

  • Qualitative Results: Camera images overlaid with predictions are visualization aids only and were not used to train SalsaNet.
  • Qualitative Results: SalsaNet distinguishes road, vehicle, and background points across qualitative KITTI examples.Excluded classes such as cyclists are generally segmented as background.
  • Qualitative Results: The final example contains an incomplete road segment because MultiNet produced missing road labels under strong sunlight.The authors identify improved camera-image road segmentation as a potential remedy for training-label quality.
  • Qualitative Results: Additional qualitative results cover varied KITTI scenarios in the supplementary video.
  • Qualitative Results: Figure 4 uses green and red points for road and vehicle segments, while dark- and light-gray points indicate points inside and outside the BEV region.

D. Ablation Study

The ablations show that weighted loss improves vehicle segmentation, BEV channels contribute similarly, and SFV channels behave inconsistently; runtime evaluation reports high-speed BEV inference.

  • Ablation Study: Removing loss weights decreases vehicle IoU by 2.5% in the BEV ablation, although background IoU slightly increases.The authors associate this with road and vehicle points being under-represented in training data.
  • Ablation Study: BEV channels have almost equal contributions to segmentation accuracy, indicating no redundant information among the employed BEV features.The analyzed channels encode mean elevation, maximum elevation, reflectivity, and point density.
  • Ablation Study: SFV channel effects are inconsistent: adding the mask raises average accuracy by 4.5%, while the first two coordinate channels reduce it by 0.04% and 0.34%.The authors interpret these results as evidence that SFV contains redundant information that can mislead feature learning.
  • Ablation Study: The authors select BEV as SalsaNet’s main representation because SFV shows inconsistent channel effects and deformation effects discussed elsewhere.
  • Runtime Evaluation: U-Net is slightly faster, but its lower accuracy creates a speed–accuracy trade-off relative to SalsaNet.SalsaNet also has lower runtime variability than the other methods.
  • Runtime Evaluation: SalsaNet reaches up to 160 Hz for a single forward pass while providing the highest accuracies in BEV.Runtime measurements were repeated ten times on all test data using the same NVIDIA Tesla V100-DGXS-32GB GPU.

V. CONCLUSION

SalsaNet is a real-time LiDAR-only network for jointly segmenting drivable road and vehicle points, with strong performance across BEV and SFV inputs and automatically generated training labels.

  • Conclusion: SalsaNet performs real-time semantic segmentation of drivable road and vehicle points using 3D LiDAR data only.
  • Conclusion: SalsaNet performs equivalently well with BEV and SFV projections, unlike other cited networks that struggle to extract local features in BEV.
  • Conclusion: Transferring image-based point-wise semantic information to LiDAR automatically generates a large annotated dataset for training.
  • Conclusion: The authors characterize SalsaNet as simple, fast, and state-of-the-art, and plan to apply it to bootstrap detection and tracking.
Loading 1909.08291v1…