Source-linked AI summary
DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving
Tianqi Wang, Sukmin Kim, Wenxuan Ji, Enze Xie, Chongjian Ge, Junsong Chen, Zhenguo Li, Ping Luo
TL;DR
Autonomous-driving datasets lack direct, explainable safety evaluation and often omit realistic accident scenarios or practical end-to-end V2X prediction. DeepAccident addresses this with a large simulated V2X benchmark, an accident-prediction task, and V2XFormer, which outperforms the single-vehicle model across perception and prediction tasks.
Problem
Existing datasets lack safety-critical accident scenarios and practical end-to-end V2X motion and accident prediction evaluation.
Method
DeepAccident uses CARLA-generated collision scenarios with four vehicles and one infrastructure, and introduces end-to-end accident prediction with V2XFormer.
Results
V2XFormer outperforms the single-vehicle model in perception and prediction tasks on DeepAccident, while four-vehicle-plus-infrastructure V2X achieves 44.0 VPQ versus 31.6 for the single-vehicle model on validation.
Takeaways & Limitations
DeepAccident provides a direct safety benchmark and supports V2X research across perception, motion prediction, and accident prediction.
Abstract
from arXiv · showhide
Safety is the primary priority of autonomous driving. Nevertheless, no published dataset currently supports the direct and explainable safety evaluation for autonomous driving. In this work, we propose DeepAccident, a large-scale dataset generated via a realistic simulator containing diverse accident scenarios that frequently occur in real-world driving. The proposed DeepAccident dataset includes 57K annotated frames and 285K annotated samples, approximately 7 times more than the large-scale nuScenes dataset with 40k annotated samples. In addition, we propose a new task, end-to-end motion and accident prediction, which can be used to directly evaluate the accident prediction ability for different autonomous driving algorithms. Furthermore, for each scenario, we set four vehicles along with one infrastructure to record data, thus providing diverse viewpoints for accident scenarios and enabling V2X (vehicle-to-everything) research on perception and prediction tasks. Finally, we present a baseline V2X model named V2XFormer that demonstrates superior performance for motion and accident prediction and 3D object detection compared to the single-vehicle model.
Introduction
Existing autonomous-driving datasets support progress in single-vehicle perception and prediction but lack practical, safety-critical V2X motion and accident evaluation. DeepAccident addresses these gaps with diverse collision scenarios, multi-agent sensing, a new prediction task, and V2XFormer.
- Existing V2X datasets primarily support perception, while motion-prediction datasets may require ground-truth positions, maps, and traffic-light states.
- Accident datasets commonly provide low-resolution, single-camera data with coarse labels, limiting detailed safety evaluation.
- DeepAccident reconstructs diverse real-world accidents in CARLA using four vehicles and one infrastructure with multi-view cameras, LiDAR, and task labels.
- The benchmark introduces end-to-end accident prediction for collision occurrence, timing, location, and involved vehicles or pedestrians.
- V2XFormer is introduced as a baseline for perception and prediction, with superior performance to a single-vehicle model on DeepAccident.
Related Work
Prior V2X datasets emphasize perception, while existing motion and accident datasets impose restrictive inputs or limited sensing and annotation. DeepAccident combines end-to-end prediction, detailed accident labels, diverse V2X sensing, and larger scale.
- V2X datasets: Existing V2X datasets mainly target perception, and V2X-seq supports motion prediction only with perfect perception and ground-truth scene inputs.
- V2X datasets: DeepAccident uses raw multi-view cameras and LiDAR for common perception tasks plus end-to-end motion and accident prediction.
- Accident datasets: Unlike prior accident datasets, DeepAccident provides detailed accident vehicle identities and future colliding trajectories in V2X scenarios.
- Dataset comparisons: Table 1 compares dataset attributes, including end-to-end motion prediction and whether required motion labels are officially available.
- Dataset comparisons: Table 2 compares the scale of existing autonomous-driving datasets with DeepAccident.
DeepAccident Dataset
DeepAccident constructs diverse collision scenarios at signalized and unsignalized intersections, capturing multiple viewpoints with four vehicles and comprehensive sensors. Randomized environments and broad task labels increase dataset diversity and utility.
- Dataset Generation: DeepAccident designs 12 intersection accident types based on NHTSA pre-crash reports, covering signalized and unsignalized settings.
- Dataset Generation: Each scenario contains two vehicles with overlapping trajectories plus two following vehicles to capture diverse viewpoints of the same accident.
- Dataset Generation: Rule-based reactions alter accident vehicles’ arrival times, producing varied collision positions, angles, and sometimes additional collisions.
- Dataset Statistics: Randomized surrounding traffic, pedestrians, weather, and time of day enhance scenario diversity, while the dataset supports detection, tracking, BEV segmentation, motion, and accident prediction.
- Dataset Statistics: DeepAccident contains 285k annotated samples and 57k annotated V2X frames at 10 Hz, split into 203k training, 41k validation, and 41k testing samples.
End-to-End Motion and Accident Prediction
The proposed pipeline predicts scene motion from multi-view camera histories, then derives accident occurrence and attributes from future BEV outputs. V2XFormer fuses aligned BEV features from multiple agents, and APA evaluates accident prediction across position thresholds.
- End-to-End Motion and Accident Prediction: Multi-view camera streams generate scene-wide motion predictions, which are post-processed into accident occurrence, identities, positions, and timing.
- Network Structure: V2XFormer extracts shared-weight BEV features for each agent, wraps them into ego coordinates, concatenates them, and fuses the result.
- Network Structure: The model jointly predicts the ego vehicle’s future motion because the ego vehicle can collide with other vehicles or pedestrians.
- Accident Prediction: Accident prediction combines centerness, segmentation, offset, and future-flow outputs to obtain current and future BEV instance segmentation.
- Accident Prediction: APA averages accident prediction accuracy over position-difference thresholds D = {5,10,15} meters.
- Accident Prediction: True-positive metrics separately quantify accident identity, position, and timing errors, using a 10-meter position threshold for matching.
Experiment
Experiments evaluate V2XFormer on motion, accident prediction, and 3D detection, including fusion choices, visibility conditions, robustness, and cross-dataset generalization. V2X configurations outperform the single-vehicle baseline, with larger gains when accident visibility is limited.
- Robustness and generalization: The experiments additionally assess longer prediction horizons, pose error, latency, and model transfer using synthesized DeepAccident data with nuScenes.These evaluations extend beyond the default validation-split comparison.
- Overall performance: V2X models outperform the single-vehicle baseline by 12.4 mIOU, 7.6 APA, and 14.3 detection mAP with four vehicles and infrastructure.The comparison covers motion prediction, accident prediction, and 3D object detection.
- V2X fusion module: CoBEVT performs best among the evaluated V2X fusion modules across all three tasks and is selected for subsequent experiments.The comparison includes average pooling, DiscoNet, V2X-ViT, and CoBEVT under the five-agent setting.
- Accident visibility: For invisible accident scenarios, V2X-5agent gains 15.9 mIOU and 12.2 APA over the single-vehicle model, compared with 11.7 and 9.1 for visible scenarios.Accident invisibility is defined using observation-frame visibility from the ego vehicle’s perspective.
- Real-world generalization: Training with DeepAccident and nuScenes improves nuScenes validation performance by 1.9 mAP and 0.8 VPQ over the original model.The result supports the usefulness of synthesized DeepAccident data for real-world scenarios.
Conclusion
DeepAccident is presented as a large-scale V2X dataset and direct safety benchmark for collision scenarios. The paper introduces end-to-end accident prediction and V2XFormer, which outperforms the single-vehicle model across perception and prediction tasks.
- Conclusion: DeepAccident provides collision scenarios, end-to-end motion and accident prediction, and sensor data from four vehicles plus infrastructure.The dataset supports V2X research on perception and prediction.
- Conclusion: V2XFormer outperforms the single-vehicle model in both perception and prediction tasks.The model is offered as a baseline for future research.
Ablation Experiments
The validation and testing splits show similar model behavior and consistent relative performance gaps. The four-vehicle-plus-infrastructure V2X model substantially outperforms the single-vehicle model, including on VPQ.
- Ablation Experiments: The four-vehicle-plus-infrastructure V2X model achieves 44.0 VPQ on validation and 43.2 VPQ on testing, exceeding the single-vehicle model by 12.4 and 12.7.The consistent gaps across splits support the validity of the dataset split.
- Ablation Experiments: Qualitative comparisons show stronger V2X prediction and perception when ego-vehicle visibility is reduced by occlusion, weather, or lighting.Figures 9 and 10 compare V2X-5agents with the single-vehicle baseline.
The DeepAccident Dataset
DeepAccident collects multi-view sensor data for each scenario using four vehicles and one infrastructure viewpoint. Its scenarios and annotations span diverse vehicles, conditions, viewpoints, and accident settings for V2X research.
- The DeepAccident Dataset: Each scenario uses four vehicles and one infrastructure viewpoint to provide diverse sensor perspectives for V2X research.Vehicles 1 and 2 collide, vehicles 3 and 4 follow as V2X agents, and infrastructure data is also collected.
- The DeepAccident Dataset: DeepAccident includes six annotated object classes and diverse instance sizes, distances, velocities, and ego-vehicle statistics.The classes are car, van, truck, motorcycle, cyclist, and pedestrian.
- The DeepAccident Dataset: The dataset contains 691 scenarios split into 483 training, 104 validation, and 104 testing scenarios, with 285k annotated samples distributed across the splits.The corresponding sample counts are 203k, 41k, and 41k.
- The DeepAccident Dataset: V2X-5agents qualitatively improves motion and accident prediction over the single-vehicle model across diverse scenarios.The comparison includes accident locations and motion-prediction outcomes.
- The DeepAccident Dataset: The dataset provides vehicle and infrastructure camera images plus LiDAR point clouds during collisions, with examples spanning road types, weather, and time of day.Red boxes mark colliding vehicles, while green boxes mark additional V2X vehicles collecting data.