Source-linked AI summary
The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections
Julian Bock, Robert Krajewski, Tobias Moers, Steffen Runde, Lennart Vater, Lutz Eckstein
TL;DR
Automated-driving research needs large, diverse urban-intersection trajectory datasets, but existing public datasets are limited in size, road-user coverage, or public-road realism. The paper constructs inD from drone recordings processed with computer vision and reports a dataset of more than 11,500 road users from German intersections. It concludes that the dataset provides large, accurate, varied intersection trajectories for research applications.
Problem
Public trajectory datasets do not adequately provide the size, quality, variety, and public-road coverage needed for data-driven automated-driving research at urban intersections.
Method
The paper records naturalistic traffic at German intersections with camera-equipped drones and processes the videos through automated computer-vision detection, tracking, classification, and trajectory extraction.
Results
inD contains trajectories of more than 11 500 road users, including more than 5000 vulnerable road users, collected from 10 hours of video at four German intersection locations.
Takeaways & Limitations
inD supplies naturalistic intersection trajectories covering vehicles, pedestrians, and bicyclists for safety validation, traffic simulation, traffic analysis, driver modeling, and road-user prediction research.
Abstract
from arXiv · showhide
Automated vehicles rely heavily on data-driven methods, especially for complex urban environments. Large datasets of real world measurement data in the form of road user trajectories are crucial for several tasks like road user prediction models or scenario-based safety validation. So far, though, this demand is unmet as no public dataset of urban road user trajectories is available in an appropriate size, quality and variety. By contrast, the highway drone dataset (highD) has recently shown that drones are an efficient method for acquiring naturalistic road user trajectories. Compared to driving studies or ground-level infrastructure sensors, one major advantage of using a drone is the possibility to record naturalistic behavior, as road users do not notice measurements taking place. Due to the ideal viewing angle, an entire intersection scenario can be measured with significantly less occlusion than with sensors at ground level. Both the class and the trajectory of each road user can be extracted from the video recordings with high precision using state-of-the-art deep neural networks. Therefore, we propose the creation of a comprehensive, large-scale urban intersection dataset with naturalistic road user behavior using camera-equipped drones as successor of the highD dataset. The resulting dataset contains more than 11500 road users including vehicles, bicyclists and pedestrians at intersections in Germany and is called inD. The dataset consists of 10 hours of measurement data from four intersections and is available online for non-commercial research at: http://www.inD-dataset.com
I. INTRODUCTION
Urban-intersection trajectory data are needed for data-driven automated-driving research, but existing public datasets lack sufficient size, road-user diversity, or public-road realism. The paper proposes drone-based recording and introduces inD to address these gaps.
- Motivation: Intersections challenge automated driving because their scenarios are complex and varied, while data-driven methods require large measurement datasets.Such data support road-user behavior models for prediction or simulation and safety-validation approaches.
- Motivation: Ground-level and on-board sensors can occlude road users and alter behavior because people notice conspicuous measurement equipment.The proposed drone approach is intended to reduce both problems.
- Approach: A bird’s-eye drone perspective provides accurate longitudinal and lateral positions, avoids road-user occlusions, and typically preserves natural behavior.The drone can hover up to 100 meters and is typically not perceived by road users.
- Contribution: The paper creates inD, a naturalistic road-user trajectory dataset for German intersections, using drone recordings and deep-learning-based extraction.The dataset is presented as a successor to the highD highway dataset and is intended for research using intersection trajectories.
- Related work: Existing datasets are often small, pedestrian-only, campus-based, controlled, or lacking vehicle trajectories, limiting their suitability for automated driving on public roads.The Stanford Drone Dataset contains 10,300 trajectories but 94.5% are pedestrians, bicyclists, and skaters; highD contains no vulnerable road users.
III. REQUIREMENTS FOR A TRAJECTORY DATASET
The proposed dataset is specified to support data-driven automated-driving research through naturalistic, sufficiently large, diverse, comprehensive, accurate, and infrastructure-aware trajectory data.
- Dataset requirements: Road users must retain naturalistic behavior by remaining uninfluenced by the measurement method.Visible sensors that resemble traffic-surveillance cameras are identified as a possible source of influence.
- Dataset requirements: The dataset must contain trajectories of several thousand road users because size and variety are crucial for data-driven algorithms.
- Dataset requirements: Measurements must span multiple recording sites and times to represent varied layouts, traffic rules, and traffic densities.Measurements should mainly occur on public roads rather than private grounds for public-road applicability.
- Dataset requirements: All road-user types must be detected and tracked because they influence one another.
- Dataset requirements: Trajectory positioning error must be less than 0.1 meters independently of road-user type.
- Dataset requirements: The dataset must precisely record and provide infrastructure because road-user behavior depends strongly on road layout and local traffic rules.
IV. METHOD
Creating a drone-based trajectory dataset requires video preprocessing and automated computer-vision processing to detect, track, and classify road users accurately.
- Processing pipeline: The processing pipeline includes video preprocessing and automated detection, tracking, and classification of all road users.Manual frame-by-frame annotation is considered infeasible for extracting the trajectories.
A. Selection of Recording Sites and Flight Approval
Four recording locations were selected based on traffic volume and composition, with a separate safety concept developed for each site before recording.
- Site selection and flight approval: Four locations were selected using traffic volume and traffic composition, followed by site-specific safety planning.Planning covered the landing site, drone position, recording area, legal safety distances, and no-fly zones.
B. Recordings and Pre-processing
The recordings were designed to maximize image quality and cover complete intersection areas using a drone positioned up to 100 meters high.
- Recordings were conducted in sufficient lighting and low winds to improve image sharpness and steadiness.
- The drone hovered at a predefined position to completely cover the relevant intersection area.The maximum altitude was 100 meters.
- Videos were recorded with a DJI Phantom 4 Pro in 4K resolution.The stated resolution was 4096x2160 pixels.
C. Detection and Classification
Road users were detected and classified from video using deep-neural-network-based computer vision, segmentation, geometric processing, and limited manual annotation.
- 2D road-user positions were extracted automatically from each video frame because frame-by-frame manual annotation was infeasible.The approach used modern computer vision algorithms based on deep neural networks.
- 400 images from all recordings were polygon-annotated to train the detection network.Flipping, rotations, deformations, and GAN-synthesized samples augmented the training data.
- Semantic segmentation was preferred over bounding-box detection because bird’s-eye-view objects can have imprecise box descriptions.Pixel-level class information allows a polygon to be derived for each detected object.
- Pedestrian and bicyclist detection was challenging because they occupy few pixels, while road-user sizes vary greatly.At up to 100 meters altitude and 4K resolution, each pixel covered about 4x4 centimeters.
- Thresholding and morphological operations removed outliers, while size, speed, and position classified pedestrians, cyclists, cars, trucks, and buses.Ambiguous cases represented less than 1% and were manually annotated.
D. Tracking and Post-processing
A tracking algorithm connected frame-level detections into road-user tracks, removed false positives, and predicted positions through brief detection gaps.
- Detections in consecutive frames were matched by distance to form continuous tracks.
- False-positive detections were completely removed during tracking.
- When occlusion caused a road user to disappear for a few frames, future positions were predicted until detection resumed.The passage gives traffic-sign occlusion as an example.
E. Dataset Format and Tools
The dataset combines images, metadata, trajectory CSV files, and supporting Matlab and Python tools for data handling, visualization, and maneuver extraction.
- Each recording is distributed as one drone-view image and three CSV files.The files include recording metadata, tracked-road-user metadata, and trajectories.
- Table I compares existing road-user trajectory datasets.
- Recording and road-user metadata include locations, recording time and duration, road-user type, track duration, and average speed.
- Trajectory files provide position, orientation, speed, and acceleration for every frame.
- Matlab and Python source code supports importing, handling, visualizing, and extracting maneuvers from the data.The code repository is described as constantly updated.
V. DATASET STATISTICS AND EVALUATION
The inD dataset contains more than 11,500 road-user trajectories collected at four diverse German intersections, including vehicles and vulnerable road users.
- More than 11,500 road users are represented, including cars, trucks, buses, pedestrians, and bicyclists.
- Recordings were collected at four Aachen intersections for around 20 minutes per location, covering areas from 80x40 to 140x70 meters.
- Bendplatz: Bendplatz is a university-adjacent, four-armed intersection where turning vehicles commonly interact with pedestrians, cyclists, and buses.
- Frankenburg: Frankenburg is a four-armed urban intersection with a zebra crossing, parking lots, and frequent cyclist and pedestrian activity.
- Heckstrasse: Heckstrasse is a suburban T-junction with cycle paths and a pedestrian crossing, where interactions arise from vehicle turns.
- Neukoellner Strasse: Neukoellner Strasse is an industrial T-junction with transit traffic, bus movements, and pedestrian interactions near bus stops.
B. Comparison with Existing Datasets
Compared with existing public trajectory datasets, inD combines many trajectories with a more representative mix of road-user types and higher stated tracking accuracy.
- inD and the Stanford Drone Dataset contain significantly more road-user trajectories than most compared datasets.
- The Stanford Drone Dataset is dominated by pedestrians and contains few vehicles, limiting vehicle–VRU and vehicle–vehicle interactions.
- inD provides a more representative distribution of road-user types on measured public urban intersections than the Stanford Drone Dataset.
- inD tracks use pixel-accurate segmentation from 4K images, whereas Stanford Drone tracks use bounding boxes from 595 x 326 pixel images.
VI. CONCLUSION
The paper presents inD as a large, accurate, and varied dataset of naturalistic road-user trajectories from public urban intersections.
- The processing pipeline extracted vehicle and pedestrian trajectories with pixel accuracy from 10 hours of video at four recording locations.
- The authors state that inD surpasses comparable datasets in size and accuracy while covering varied public intersections rather than a university campus.