Source-linked AI summary
SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances
Yancheng Pan, Biao Gao, Jilin Mei, Sibo Geng, Chengkun Li, Huijing Zhao
TL;DR
3D semantic segmentation needs large point-wise labeled datasets with diverse scenes and dynamic objects, but existing resources remain limited. SemanticPOSS addresses this gap with 2988 varied LiDAR scans, point-wise and instance-level annotations, and SemanticKITTI-compatible interfaces. Experiments report stronger dynamic-object labeling when training data contains more dynamic instances, with improvements for people and cars to some degree.
Problem
Existing 3D semantic segmentation datasets lack sufficient point-wise labels, scene diversity, and dynamic objects needed by deep learning models.
Method
The paper constructs SemanticPOSS with varied LiDAR scans, point-wise and instance-level annotations, and a SemanticKITTI-compatible data format and interface.
Results
Training sets containing more dynamic instances produce stronger ability to label dynamic objects, with SemanticPOSS improving prediction accuracy for people, cars, and others to some degree.
Takeaways & Limitations
SemanticPOSS provides varied scenes and abundant dynamic instances that support evaluation and training for 3D semantic segmentation of dynamic objects.
Abstract
from arXiv · showhide
3D semantic segmentation is one of the key tasks for autonomous driving system. Recently, deep learning models for 3D semantic segmentation task have been widely researched, but they usually require large amounts of training data. However, the present datasets for 3D semantic segmentation are lack of point-wise annotation, diversiform scenes and dynamic objects. In this paper, we propose the SemanticPOSS dataset, which contains 2988 various and complicated LiDAR scans with large quantity of dynamic instances. The data is collected in Peking University and uses the same data format as SemanticKITTI. In addition, we evaluate several typical 3D semantic segmentation models on our SemanticPOSS dataset. Experimental results show that SemanticPOSS can help to improve the prediction accuracy of dynamic objects as people, car in some degree. SemanticPOSS will be published at \url{www.poss.pku.edu.cn}.
I. INTRODUCTION
3D semantic segmentation datasets remain limited in point-wise labels, scene diversity, and dynamic objects, motivating SemanticPOSS. The dataset provides varied LiDAR data in a SemanticKITTI-compatible format and evaluates representative models.
- Deep learning models for 3D semantic segmentation require large, diverse manually labeled datasets, but existing datasets remain insufficient.
- Point-wise labeled 3D data is scarce because annotating point clouds manually is difficult and time-consuming.
- Limited scene diversity creates selection bias and can reduce performance on new scenes.
- Existing datasets contain few dynamic objects, potentially weakening recognition of important pedestrians and cars.
- SemanticPOSS contains 2988 LiDAR scans with point-wise and instance-level labels across varied campus scenes with many dynamic instances.
- SemanticPOSS follows SemanticKITTI’s data format and interface, and the paper evaluates typical 3D semantic segmentation models on it.
II. RELATED WORKS
The related work spans datasets built from RGB-D cameras, static point clouds, and sequential LiDAR data. These categories differ in sensing conditions, density, scene type, and annotation characteristics.
- 3D point cloud datasets support tasks including classification, detection, and semantic segmentation, with dataset types differing in density and point-cloud features.
- RGB-D datasets combine color images with per-pixel depth and generally focus on indoor scenes because outdoor depth quality is difficult to obtain.
B. Static Point Clouds Datasets
Static, sequential, and synthetic point-cloud datasets address annotation and coverage trade-offs differently. SemanticPOSS extends sequential point-cloud data with point-wise labels, varied scenes, and many dynamic instances.
- Static Point Clouds Datasets: Static point-cloud datasets are independent scans without dynamic objects, typically offering many points but few frames.
- Sequential Point Clouds: Sequential point clouds from moving LiDAR provide many frames and points, but each frame is sparse and difficult to annotate.
- Sequential Point Clouds: SemanticKITTI provides 4549M labeled points across 22 KITTI Vision Odometry Benchmark sequences.
- Synthetic Datasets: Synthetic datasets can rapidly generate large point clouds with accurate point-wise labels, but differences from realistic scenes remain.
- Dataset Positioning: SemanticPOSS increases sequential point-cloud data with point-wise labels and more complicated scenes containing many dynamic instances.
III. DATASET CONSTRUCTION
SemanticPOSS data was collected with a vehicle-mounted Pandora sensor module and GPS/IMU system around Peking University. The route includes varied, dynamic campus environments and multiple sensing modalities.
- Sensors and Data Collection: The collection vehicle used a Pandora sensor module together with GPS/IMU localization to capture point-cloud data.
- Sensors and Data Collection: Pandora integrates cameras, LiDAR, data processing, synchronization, and calibration in one sensor module.
- Sensors and Data Collection: The system includes a 40-channel LiDAR, a forward-facing color camera, and four wide-angle monocular cameras covering 360 degrees.
- Collection Route: Data collection at Peking University covered teaching buildings, a school gate, main road, and parking lot over approximately 1.5 kilometers.
- Collection Route: The collected scenes include many walking or riding students and moving vehicles, making them dynamic, varied, and complicated.
B. Data Annotation
SemanticPOSS uses a hybrid annotation workflow that combines automated preprocessing with human correction to produce point-wise labels. The process is designed to reduce manual effort while preserving annotation quality.
- Annotation scope: SemanticPOSS assigns every point a semantic label and gives each dynamic object a unique instance label.Dynamic objects include people, cars, and riders.
- Automated preprocessing: A segmentation program projects point clouds into range images, divides them into segments, and tracks corresponding segments across frames.The segmentation uses region growing and simple data association.
- Track-wise annotation: A pre-annotation program assigns semantic labels to tracked segments using manually defined geometric rules and matched camera images.Human annotators correct wrong labels, wrong segments, and selected unlabeled segments.
3) Point-wise Annotation:
Point-wise annotation combines registered multi-scan visualization with automatic instance-label generation. Human annotators correct semantic labels and bounding boxes where needed.
- Point-wise annotation: Multiple point-cloud scans are overlapped using GPS/IMU localization, calibration, and additional SLAM refinement.The registered scans are manually annotated with the SemanticKITTI point-cloud application.
- Instance generation: After semantic labels are corrected, a computer program generates bounding boxes for dynamic instances and repeats data association.This produces nearly completely correct instance labels without manual intervention.
- Quality correction: Annotators modify only some malformed bounding boxes after automatic instance-label generation.The workflow therefore limits manual intervention to residual instance-label errors.
C. Dataset Analysis
SemanticPOSS is analyzed through its dynamic-instance distribution, location-based data split, evaluation metrics, and two complementary experiments. The dataset emphasizes varied scenes with many people and cars.
- Dataset statistics: 2988 frames contain an average of 8.29 people instances and 15.02 car instances per frame.Most frames contain 3-12 people, while car counts vary across frames.
- Dataset split: The dataset is split by vehicle location, with part 3 used for testing and the other five parts used for training.This location-based split addresses repeated passes through the same positions and uses six parts of about 500 frames each.
- Evaluation metrics: IoU measures class-level intersection over union, while mIoU is the arithmetic mean of IoU across the evaluated classes.TP_c, FP_c, and FN_c denote true-positive, false-positive, and false-negative predictions for class c.
- Experiments: Two experiments evaluate model performance within SemanticPOSS and cross-dataset generalization between SemanticPOSS and SemanticKITTI.The within-dataset evaluation uses selected semantic labels and excludes confusing labels such as unlabeled.
B. Baseline Models
The baseline study compares PointNet++, which processes raw point clouds, with SqueezeSegV2, which processes range images. It also uses PointNet++ for cross-dataset evaluation between SemanticKITTI and SemanticPOSS.
- Baseline selection: PointNet++ and SqueezeSegV2 are selected as representative baselines for the two main 3D semantic-segmentation input forms.PointNet++ uses raw unordered point clouds, whereas SqueezeSegV2 uses range images.
- PointNet++: PointNet++ hierarchically captures local features through recursive partitioning and improves contextual feature learning over PointNet.The passage describes improved robustness and generalization performance relative to PointNet.
- SqueezeSegV2: SqueezeSegV2 combines CNNs, CRF, and a Context Aggregation Module for point-wise prediction from range images.Its design also targets loss, structure, and robustness to point-cloud dropout noise.
C. Results and Discussion
Experiments compare models trained and tested across SemanticKITTI and SemanticPOSS, examining class scale, scene selection bias, and dynamic-object labeling. SemanticPOSS improves dynamic-object performance while retaining similar static-object labeling in the shown examples.
- Experimental setup: The experiments compare models trained and tested on SemanticKITTI and SemanticPOSS, including PointNet++ models labeled PN-KITTI and PN-POSS.The second experiment evaluates whether scene variety affects model performance.
- Class scale: IoU is strongly correlated with the scale of the corresponding class, with road scoring higher than people across the evaluated models.The paper attributes this pattern to the greater sufficiency of data for learning features of larger classes.
- Scene diversity: Scene selection bias affects cross-dataset generalization: PN-KITTI drops on SemanticPOSS, while PN-POSS also drops when tested on SemanticKITTI.The paper states that combining multiple datasets can alleviate this bias.
- Dynamic-object labeling: PN-POSS performs better at labeling dynamic objects, while its static-object labeling is not much different from PN-KITTI in the illustrated results.The figure highlights improved dynamic-object labeling in boxes A–D but shows some static-object mistakes in box E.
V. CONCLUSION AND FUTURE WORK
The paper presents SemanticPOSS as a point-wise labeled dataset containing varied scenes and many dynamic instances. Its experiments associate greater dynamic-instance representation with stronger labeling of people and cars, while future work may add matched camera images.
- Conclusion: SemanticPOSS is a large-scale 3D point-cloud dataset with point-wise labels for semantic segmentation.The dataset is presented as the paper’s central contribution.
- Conclusion: The dataset contains various and complicated scenes with a large quantity of dynamic instances.The reported conclusion focuses on dataset composition as a distinguishing characteristic.
- Conclusion: Training sets containing more dynamic instances are associated with stronger labeling of people, cars, and other dynamic objects.The paper reports this improvement as occurring to some degree.
- Future work: Future work may extend SemanticPOSS by adding images matched with its LiDAR point clouds for multi-sensor-fusion research.The authors state that their Pandora sensor provides accurate LiDAR–image matching.