Source-linked AI summary
Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways
Weikai Tan, Nannan Qin, Lingfei Ma, Ying Li, Jing Du, Guorong Cai, Ke Yang, Jonathan Li
TL;DR
Large-scale labeled datasets for outdoor point-cloud semantic segmentation remain limited despite the importance of urban scene perception. This paper introduces Toronto-3D, evaluates deep-learning baselines and MS-TGNet, and reports a highest mIoU of 60.96% with competitive OA of 91.69%.
Problem
Publicly accessible large-scale labeled point-cloud datasets for developing urban semantic-segmentation models remain limited, although urban scene perception supports applications such as HD mapping and autonomous driving.
Method
The paper constructs Toronto-3D from mobile LiDAR data and evaluates six point-based deep-learning semantic-segmentation models, including the proposed MS-TGNet.
Results
60.96% highest mIoU and 91.69% competitive OA were achieved by MS-TGNet on Toronto-3D.
Takeaways & Limitations
Toronto-3D provides a large-scale labeled urban outdoor benchmark for comparing point-cloud semantic-segmentation methods.
Takeaways & Limitations
Performance of PointNet++-backbone models was probably limited by RAM constraints on the number of points they could process.
Abstract
from arXiv · showhide
Semantic segmentation of large-scale outdoor point clouds is essential for urban scene understanding in various applications, especially autonomous driving and urban high-definition (HD) mapping. With rapid developments of mobile laser scanning (MLS) systems, massive point clouds are available for scene understanding, but publicly accessible large-scale labeled datasets, which are essential for developing learning-based methods, are still limited. This paper introduces Toronto-3D, a large-scale urban outdoor point cloud dataset acquired by a MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of point clouds and consists of about 78.3 million points with 8 labeled object classes. Baseline experiments for semantic segmentation were conducted and the results confirmed the capability of this dataset to train deep learning models effectively. Toronto-3D is released to encourage new research, and the labels will be improved and updated with feedback from the research community.
1. Introduction
Urban applications need accurate 3D scene perception, but large-scale labeled point-cloud datasets remain limited. The paper addresses this gap with a dataset, an integrated segmentation network, and comparisons of state-of-the-art methods.
- 3D LiDAR point clouds provide georeferenced information desirable for urban studies and scene understanding.
- Point clouds are unstructured, unordered, and typically large, creating challenges for processing.
- Mobile platforms integrating MLS, positioning sensors, and cameras offer flexible data collection for urban mapping and autonomous driving.
- The paper presents a large-scale point-wise labeled urban outdoor dataset for semantic segmentation.
- The study investigates an integrated segmentation network and compares state-of-the-art deep-learning methods on the proposed dataset.
2. Available point cloud datasets for 3D Semantic Segmentation
Existing outdoor point-cloud datasets vary substantially in size, labeling scope, point density, and public availability. The reviewed datasets support semantic-segmentation research but also have constraints such as limited scale, restricted subsets, or classification-quality issues.
- Oakland 3-D contains about 1.6 million points across 44 labeled classes, but literature evaluates only five classes.
- The Oakland 3-D dataset uses a mono-fiber LiDAR with relatively low point density and is considered suitable for lightweight networks.
- The reviewed datasets illustrate the range of scale and labeling configurations available for urban outdoor semantic segmentation.
- The publicly available iQmulus contest subset contains 12 million points in a 200 m range with eight valid classes, versus over 300 million points in the full dataset.
- Paris-Lille-3D covers close to 2 km with over 140 million points and provides detailed labels for 50 classes, using nine classes for segmentation benchmarks.
3. New dataset: Toronto-3D
Toronto-3D is a vehicle-mounted MLS dataset covering approximately 1 km and 78.3 million points, manually labeled into eight object categories. Its untrimmed measurement range and resulting density variation are intended to reflect challenging real-world collection conditions.
- 3.1. Data acquisition: Toronto-3D was acquired with a vehicle-mounted Teledyne Optech Maverick2 system combining a 32-line LiDAR, panoramic camera, GNSS, and SLAM.
- 3.2. Description of the dataset: The dataset covers approximately 1 km of road in Toronto and contains approximately 78.3 million points.
- 3.3. Challenges of Toronto-3D: All points within approximately 100 m of the road centerline were retained with limited post-processing, producing realistic density, noise, and distant-object variation.
- 3.2. Description of the dataset: Points retain coordinates, RGB reflectance, LiDAR intensity, GPS time, scan angle, and an object-class label.
- 3.2. Description of the dataset: Toronto-3D is manually labeled into eight categories, including road, road marking, natural, building, utility line, pole, car, and fence.
- 3.3. Challenges of Toronto-3D: Point density can vary by more than 10 times because of full-range coverage and repeated scans, challenging segmentation robustness.
4. Methods
The paper frames semantic segmentation as point-wise labeling of unstructured, unordered point clouds and evaluates point-based deep-learning models on this task. It uses standard IoU, OA, and mIoU metrics and compares several state-of-the-art architectures.
- Semantic segmentation assigns a semantic label to each point in a point cloud.
- Point-cloud models include view-based, voxel-based, and point-based approaches, with point-based methods processing 3D points directly.
- Six point-based deep-learning models were tested, including PointNet++, DGCNN, KPFCNN, MS-PCNN, and TGNet.
- Evaluation uses per-class IoU, overall accuracy, and mean IoU to measure class-level and overall segmentation quality.IoU uses true positives, false positives, and false negatives for each class.
- The experimental split uses L002 for testing and the other three sections for training and validation, with only point coordinates as input.The authors note that indoor-scene parameter settings for PointNet++ and DGCNN may limit their performance.
5. Results and discussions
Baseline experiments show that KPFCNN and MS-TGNet perform best overall, while several classes remain difficult. The discussion attributes errors to varying point density, limited context, object ambiguity, and processing constraints.
- Performance of baseline approaches: MS-TGNet achieved the highest mIoU of 60.96% and second-highest OA of 91.69% among the baseline approaches.KPFCNN achieved the highest OA and second-highest mIoU.
- Areas for improvements: Concrete barriers, tree trunks, and partially scanned trucks were misclassified because of similar structures, limited object context, or incomplete scans.The examples compare errors from KPFCNN and MS-TGNet and indicate possible benefits from edge features for natural objects.
- Performance of baseline approaches: KPFCNN uses more points and outperformed MS-TGNet in most categories, while MS-TGNet was stronger on natural and road marking classification.
- Areas for improvements: PointNet++-backbone methods, including MS-TGNet, were probably limited by RAM constraints on the number of points they could process.The authors suggest that methods processing larger point sets may have advantages in outdoor scenes.
6. Conclusions
Toronto-3D is a large-scale MLS dataset for urban outdoor semantic segmentation, accompanied by baseline evaluations and the proposed MS-TGNet model. The dataset provides challenging labels and is intended to support further research and community-driven label updates.
- Toronto-3D covers approximately 1 km of road in Toronto and contains over 78 million manually labeled points across eight categories.The categories are road, road marking, natural, building, utility line, pole, car, and fence.
- Five state-of-the-art end-to-end segmentation algorithms and MS-TGNet were evaluated on the dataset.
- MS-TGNet achieved the highest mIoU of 60.96% and competitive OA of 91.69% on Toronto-3D.
- All tested methods need improvement on road markings and fences, while the dataset introduces labels for these classes and utility lines.
- The dataset was released to encourage new deep-learning models, with labels intended to be improved through research-community feedback.