Source-linked AI summary
SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud
Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, Kurt Keutzer
TL;DR
The paper addresses limited robustness and costly labels for LiDAR point-cloud segmentation, including poor transfer from synthetic to real data. It introduces SqueezeSegV2 and a three-part unsupervised domain-adaptation pipeline, reporting improved real-data segmentation and synthetic-to-real transfer.
Problem
LiDAR segmentation models need higher accuracy, while labeled point-cloud collection is expensive and synthetic-data training is hindered by domain shift.
Method
SqueezeSegV2 combines CAM, focal loss, batch normalization, and a LiDAR mask channel with learned intensity rendering, geodesic correlation alignment, and progressive domain calibration.
Results
The model improves segmentation accuracy by 6.0%-8.6% over SqueezeSeg on real data, while the adaptation pipeline raises real-world test accuracy from 29.0% to 57.4% after synthetic training.
Takeaways & Limitations
The proposed model and pipeline improve robustness and transferability for road-object segmentation from LiDAR point clouds.
Abstract
from arXiv · showhide
Earlier work demonstrates the promise of deep-learning-based approaches for point cloud segmentation; however, these approaches need to be improved to be practically useful. To this end, we introduce a new model SqueezeSegV2 that is more robust to dropout noise in LiDAR point clouds. With improved model structure, training loss, batch normalization and additional input channel, SqueezeSegV2 achieves significant accuracy improvement when trained on real data. Training models for point cloud segmentation requires large amounts of labeled point-cloud data, which is expensive to obtain. To sidestep the cost of collection and annotation, simulators such as GTA-V can be used to create unlimited amounts of labeled, synthetic data. However, due to domain shift, models trained on synthetic data often do not generalize well to the real world. We address this problem with a domain-adaptation training pipeline consisting of three major components: 1) learned intensity rendering, 2) geodesic correlation alignment, and 3) progressive domain calibration. When trained on real data, our new model exhibits segmentation accuracy improvements of 6.0-8.6% over the original SqueezeSeg. When training our new model on synthetic data using the proposed domain adaptation pipeline, we nearly double test accuracy on real-world data, from 29.0% to 57.4%. Our source code and synthetic dataset will be open-sourced.
I. INTRODUCTION
SqueezeSeg enabled efficient LiDAR point-cloud segmentation but remained limited by dropout noise, costly labeled data, and synthetic-to-real domain shift. SqueezeSegV2 addresses these challenges with a more robust model and domain-adaptation pipeline.
- SqueezeSeg projects 3D LiDAR point clouds onto a spherical surface and uses a 2D CNN for point-wise labeling at over 100 frames per second.
- Dropout noise from limited sensing range, mirror diffusion, and incident-angle jitter can corrupt early layers and reduce segmentation accuracy.
- Collecting and annotating tens of thousands of labeled point clouds is time consuming and expensive, motivating synthetic data from GTA-V.
- Synthetic-data training suffers from domain discrepancy because observed LiDAR and road-object distributions differ between synthetic and real domains.
- SqueezeSegV2 mitigates dropout noise with Context Aggregation Module and combines focal loss, batch normalization, and a LiDAR mask channel with domain adaptation.
III. IMPROVING THE MODEL STRUCTURE
SqueezeSegV2 improves SqueezeSeg by adding contextual aggregation to early layers, where small receptive fields make the network sensitive to missing LiDAR points. CAM aggregates a larger neighborhood to reduce this sensitivity.
- SqueezeSegV2 adds Context Aggregation Modules to the base SqueezeSeg architecture for road-object segmentation from 3D LiDAR point clouds.
- Dropout noise arises from limited sensor range, mirror reflection, and incident-angle jitter, and it can strongly corrupt filters with small early-layer receptive fields.
- CAM uses large-kernel max pooling followed by two convolution layers with an intermediate ReLU to aggregate context efficiently and reduce sensitivity to missing data.
- CAM is inserted after the first three modules, where receptive fields are small, improving real-data accuracy and reducing the synthetic-to-real domain gap.
B. Focal Loss
LiDAR segmentation is dominated by background points, causing insufficient attention to foreground objects. SqueezeSegV2 replaces cross-entropy with focal loss to emphasize difficult examples.
- LiDAR point categories are highly imbalanced, with many more background points than foreground objects such as cars and pedestrians.
- This imbalance makes the model focus on easy background points while inadequately addressing foreground objects during training.
- Focal loss replaces the original cross-entropy loss and modulates each pixel’s contribution to focus training on hard examples.
- For pixel probability p_t, focal loss multiplies cross entropy by (1 − p_t)^γ, down-weighting well-classified pixels as p_t approaches 1.
C. Other Improvements
SqueezeSegV2 adds a binary LiDAR mask channel and batch normalization, while the broader adaptation framework uses synthetic GTA-LiDAR and real KITTI data. These changes improve selected segmentation outcomes.
- The LiDAR mask channel indicates whether each pixel is missing or present, supplementing the original x, y, z, intensity, and depth channels.
- Batch normalization is added after every convolution layer to alleviate internal covariate shift during training.
- The domain-adaptation framework trains SqueezeSegV2 from labeled synthetic GTA-LiDAR data toward unlabeled real-world KITTI data.
- Batch normalization improves car segmentation, while the mask channel significantly improves cyclist segmentation.
IV. DOMAIN ADAPTATION TRAINING
The paper’s unsupervised domain-adaptation pipeline trains SqueezeSegV2 on synthetic LiDAR data while addressing domain shift through three strategies.
- The pipeline uses learned intensity rendering, geodesic correlation alignment, and progressive domain calibration to address domain shift.
- The pipeline trains SqueezeSegV2 on synthetic GTA-LiDAR data for improved real-world performance.
- GTA-LiDAR is a large-scale 3D LiDAR point-cloud dataset constructed for this adaptation setting.
A. The GTA-LiDAR Dataset
GTA-LiDAR supplies diverse synthetic scans, while learned intensity rendering and alignment procedures address missing intensity information and synthetic-to-real discrepancies.
- The GTA-LiDAR Dataset: 100,000 LiDAR scans are simulated in GTA-V across diverse scenes, car types, and traffic conditions.
- The GTA-LiDAR Dataset: Synthetic data lacks intensity, an important signal whose absence can cause serious accuracy loss.
- The GTA-LiDAR Dataset: Learned intensity rendering predicts intensity from the point cloud’s x, y, z, and depth channels using unlabeled LiDAR data for self-supervision.
- The GTA-LiDAR Dataset: 3X lower MSE, from 0.033 to 0.011, is achieved by the hybrid intensity loss compared with ℓ2 loss.
- The GTA-LiDAR Dataset: Geodesic correlation alignment combines focal loss on labeled synthetic data with geodesic loss between synthetic and real batch output distributions.
D. Progressive Domain Calibration
Progressive domain calibration addresses accumulated distribution shift by calibrating network layers sequentially with unlabeled real data.
- D. Progressive Domain Calibration: Distribution discrepancies can accumulate or amplify across network layers and degrade performance.
- D. Progressive Domain Calibration: Progressive domain calibration breaks distribution-shift propagation through layer-wise calibration.
- D. Progressive Domain Calibration: The procedure computes each layer’s real-data output statistics, re-normalizes them, and updates that layer’s BatchNorm parameters sequentially.
V. EXPERIMENTS
The experiments evaluate SqueezeSegV2 on converted KITTI data and test synthetic-to-real generalization using GTA-LiDAR for training and KITTI for testing.
- V. EXPERIMENTS: SqueezeSegV2 is trained and tested on a converted KITTI dataset.
- V. EXPERIMENTS: Synthetic-to-real generalization is evaluated by training on GTA-LiDAR and testing on real-world KITTI data.
- V. EXPERIMENTS: The experimental setup is used to verify the model’s generalization ability.
A. Experimental Settings
The evaluation uses KITTI point-wise labels derived from 3D bounding boxes, with IoU measuring predicted-versus-ground-truth point-set overlap.
- KITTI provides 10,848 point-wise labeled samples, split into 8,057 training samples and 2,791 testing samples for SqueezeSegV2.
- Performance is evaluated by point-wise comparison of predicted labels with ground-truth labels on class-level segmentation tasks.
- IoU_c is defined as the intersection cardinality divided by the union cardinality of predicted and ground-truth point sets for class c.
B. Improved Model Structure
SqueezeSegV2 improves SqueezeSeg with normalization, focal loss, a LiDAR mask channel, and CAM, producing better class-specific segmentation and greater robustness to dropout noise.
- Figure 7 shows more accurate cyclist segmentation and avoidance of a falsely detected distant car with SqueezeSegV2.
- SqueezeSegV2 combines batch normalization, a LiDAR mask channel, focal loss, and CAM in its proposed architecture.
- Batch normalization improves car segmentation, while the mask channel improves cyclist segmentation.
- Focal loss improves pedestrian and cyclist segmentation by addressing class imbalance caused by their relatively few points.
- CAM improves all classes by reducing the network's sensitivity to dropout noise.
C. Domain Adaptation Pipeline
The domain-adaptation pipeline addresses poor transfer from synthetic GTA-LiDAR to real KITTI data using learned intensity rendering, geodesic correlation alignment, and progressive domain calibration.
- Models trained on the source domain without adaptation transfer poorly because observed LiDAR and road-object distributions differ between domains.
- The combined adaptation pipeline performs best among the evaluated adaptation methods.
- Adding CAM boosts real-data performance and supports dropout noise as a significant source of domain discrepancy.
- An obvious performance gap remains relative to SqueezeSegV2 trained on real KITTI data, leaving synthetic-LiDAR adaptation challenging.
- The proposed pipeline combines learned intensity rendering, geodesic correlation alignment, and progressive domain calibration for synthetic-to-real adaptation.
- The pipeline improves real-world accuracy for synthetic-data training by 28.4%, outperforming a baseline trained on real data.