Source-linked AI summary
IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments
Girish Varma, Anbumani Subramanian, Anoop Namboodiri, Manmohan Chandraker, C V Jawahar
TL;DR
Existing autonomous-navigation datasets largely emphasize structured roads, leaving generalization to diverse and unstructured conditions insufficiently tested. IDD provides a finely annotated Indian-road dataset with expanded labels and a four-level hierarchy, and its evaluations show substantially lower semantic-segmentation performance than Cityscapes while motivating domain adaptation, few-shot learning, and behavior prediction.
Problem
Algorithms are largely untested on road conditions that are significantly more diverse and unstructured.
Method
IDD is a 10,004-frame dataset from 182 drive sequences with expanded labels, fine annotations, and a four-level hierarchy for unstructured Indian road scenes.
Results
Semantic segmentation performances are consistently far lower on IDD than on Cityscapes using identical models and larger-scale IDD training data.
Takeaways & Limitations
IDD provides a benchmark for unstructured road-scene understanding and motivates domain adaptation, few-shot learning, and behavior prediction.
Takeaways & Limitations
Domain adaptation, few-shot learning, and behavior prediction are highlighted as future problems but are not considered in this paper.
Abstract
from arXiv · showhide
While several datasets for autonomous navigation have become available in recent years, they tend to focus on structured driving environments. This usually corresponds to well-delineated infrastructure such as lanes, a small number of well-defined categories for traffic participants, low variation in object or background appearance and strict adherence to traffic rules. We propose IDD, a novel dataset for road scene understanding in unstructured environments where the above assumptions are largely not satisfied. It consists of 10,004 images, finely annotated with 34 classes collected from 182 drive sequences on Indian roads. The label set is expanded in comparison to popular benchmarks such as Cityscapes, to account for new classes. It also reflects label distributions of road scenes significantly different from existing datasets, with most classes displaying greater within-class diversity. Consistent with real driving behaviours, it also identifies new classes such as drivable areas besides the road. We propose a new four-level label hierarchy, which allows varying degrees of complexity and opens up possibilities for new training methods. Our empirical study provides an in-depth analysis of the label characteristics. State-of-the-art methods for semantic segmentation achieve much lower accuracies on our dataset, demonstrating its distinction compared to Cityscapes. Finally, we propose that our dataset is an ideal opportunity for new problems such as domain adaptation, few-shot learning and behaviour prediction in road scenes.
1. Introduction
IDD addresses the limited coverage of structured driving datasets by providing a large, diverse benchmark for unstructured Indian road scenes. Its expanded labels, hierarchy, and evaluations expose challenges for segmentation and motivate broader learning problems.
- Algorithms remain largely untested on road conditions that are significantly more diverse and unstructured.
- IDD contains 10,004 labeled images with fine instance-level boundaries, collected across cities, cameras, sensors, and lighting conditions.
- IDD uses a four-level label hierarchy with 7, 16, 26, and 30 labels to support different training complexity levels.
- Indian roads contribute novel traffic participants, greater within-class appearance diversity, different class distributions, and broader weather, lighting, air-quality, and dust variation.
- Semantic segmentation performances are consistently far lower on IDD than on Cityscapes using identical models and larger-scale IDD training data.
- IDD supports problems beyond segmentation, including domain adaptation with nonoverlapping labels, few-shot learning, and behavior prediction in less rule-constrained scenes.
2. Challenges in Unstructured Environments
IDD targets road scenes where boundaries, traffic behavior, participants, signage, and ambient conditions are substantially less structured than conventional autonomous-driving datasets.
- Ambiguous Road Boundaries: Cityscapes-trained models degrade on Indian road scenes with ambiguous drivable boundaries, muddy terrain, and unsafe flat areas beside roads.Road edges may be partly drivable, while flat regions beside roads may be incorrectly recognized as road.
- Diversity of Vehicles and Pedestrians: Indian roads contain diverse vehicles and pedestrians, including autorickshaws, heavily worn cars, crowded motorbikes, jaywalkers, and participants that follow traffic rules less consistently.These conditions weaken correlations between traffic participants and road signage such as lanes or traffic lights.
- Extensive Use of Information Boards: Billboards are widespread and can indicate shops, landmarks, special vehicles, or driving-school and delivery vehicles.Their presence adds visually and semantically varied information to road scenes.
- Diversity of Ambient Conditions: Lighting and atmospheric conditions vary across times of day, shadows, cloud cover, fog, dust, and smog, producing substantial appearance variation.Cityscapes-pretrained models yield lower accuracies in these settings.
3. Dataset
IDD is a finely annotated dataset of unstructured Indian road scenes, designed with broad labels and a hierarchy that accommodates ambiguity and class imbalance.
- 3.1. Acquisition: 182 drive sequences from Bangalore, Hyderabad, and their outskirts yield 10,004 finely annotated frames spanning urban, rural, highway, and narrow-road conditions.The locations include construction, poorly defined boundaries, many pedestrians, motorbikes, trucks, and diverse vehicle models.
- 3.2. Annotation: IDD uses 34 fine-annotation labels and a four-level hierarchy containing 7, 16, 26, and 30 labels across progressively less ambiguous categories.New labels include autorickshaw, billboard, animal, and curb.
- 3.4. Statistical Analysis and Dataset Splits: Level-4 labels are highly imbalanced, so drive-sequence splitting was carefully designed to approximate 70%, 10%, and 20% train, validation, and test proportions.Lower hierarchy levels reduce imbalance, while rare labels such as parking, animal, caravan, and traffic light remain challenging.
- 3.5. Comparison with Other Datasets: The dataset includes calibrated-camera comparisons with Cityscapes and differs from other datasets in camera setup, imagery, and road-scene coverage.The comparison considers datasets including KITTI, CamVid, Daimler, Mapillary Vistas, and BDD100K.
- 3.4. Statistical Analysis and Dataset Splits: IDD has more pixels for several vehicle and road-scene classes, high counts for new labels, and almost twice as many traffic-participant instances as Cityscapes.For most vehicle classes, pixel counts are 5–10 times higher than Cityscapes.
4. Benchmarks
IDD is benchmarked for semantic and instance segmentation, and experiments show substantial domain and class-specific difficulty alongside useful cross-dataset transfer behavior.
- 4.1. Semantic Segmentation: Control experiments estimate IoU upper bounds at different input resolutions and show that low-resolution processing contributes significantly to segmentation degradation.The bounds are evaluated across multiple levels of the label hierarchy.
- 4.2. Domain Discrepancy: IDD is harder than Cityscapes but has a similar hardness level to Mapillary Vistas, while IDD-trained models predict Cityscapes and BDD100K labels better than the converse.The comparison uses common labels and DRN-D-38 models.
- 4.3. Semantic Segmentation Benchmark: Semantic segmentation is evaluated with mIoU at multiple hierarchy levels using DRN-D-38, ERFNet, and challenge submissions.Level-4 results include rare classes, while levels 1 and 2 support evaluation of real-time models.
- 4.4. Class IoUs and Confusion Matrix: IoU is below 25% for bicycle, traffic light, vehicle fall-back, and fence, with bicycle and traffic-light errors linked to low pixel counts.The confusion matrix reports confusion among motorcycle and bicycle, billboard and traffic sign, and drivable and non-drivable classes.
- 4.4. Class IoUs and Confusion Matrix: Qualitative predictions identify muddy drivable areas and new classes such as autorickshaw, curb, and billboard in unstructured scenes.The instance-segmentation benchmark evaluates per-instance masks and average precision across overlap thresholds from 0.5 to 0.95.
5. Conclusion
IDD establishes a benchmark for autonomous navigation in unstructured driving conditions, with greater traffic-participant and background diversity than existing datasets. Its unconstrained scenes also motivate future work on higher-level reasoning, few-shot learning, and domain adaptation.
- Dataset scope: IDD targets autonomous navigation in unstructured driving conditions and includes safe or unsafe drivable areas beside the road, additional vehicle labels, and a label hierarchy.The dataset also analyzes label statistics, class imbalance, and domain discrepancy with other semantic segmentation datasets.
- Dataset scope: India-based acquisition introduces greater diversity in traffic-participant appearance and background categories than existing semantic segmentation datasets.The paper identifies this as a challenge for state-of-the-art semantic segmentation and as an effort focused beyond North America and Europe.
- Future directions: Unconstrained traffic behavior, missing lane cues, rare categories, varied common-category appearance, and changing weather, time, and air quality create settings for scene understanding and path planning.The paper also identifies few-shot learning and domain adaptation as future problems for this dataset.