PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Traffic Accident Detection in First-Person Videos

Yu Yao, Mingze Xu, Yuchen Wang, David J. Crandall, Ella M. Atkins

arXiv:1903.00618v4cs.CV

TL;DR

Traffic anomaly detection in moving-camera videos is challenged by rare events, static-camera assumptions, and reliance on hand-labeled anomaly categories. The paper predicts future traffic-participant locations and ego-motion from normal driving videos, then monitors prediction deviations, outperforming published baselines and the state of the art on two accident datasets.

Abstract

from arXiv · show

Recognizing abnormal events such as traffic violations and accidents in natural driving scenes is essential for successful autonomous driving and advanced driver assistance systems. However, most work on video anomaly detection suffers from two crucial drawbacks. First, they assume cameras are fixed and videos have static backgrounds, which is reasonable for surveillance applications but not for vehicle-mounted cameras. Second, they pose the problem as one-class classification, relying on arduously hand-labeled training datasets that limit recognition to anomaly categories that have been explicitly trained. This paper proposes an unsupervised approach for traffic accident detection in first-person (dashboard-mounted camera) videos. Our major novelty is to detect anomalies by predicting the future locations of traffic participants and then monitoring the prediction accuracy and consistency metrics with three different strategies. We evaluate our approach using a new dataset of diverse traffic accidents, AnAn Accident Detection (A3D), as well as another publicly-available dataset. Experimental results show that our approach outperforms the state-of-the-art.

I. INTRODUCTION

The paper targets rare and difficult-to-anticipate roadway anomalies in moving-camera videos, where supervised labeling and whole-frame prediction are impractical. It proposes unsupervised detection from predicted traffic-participant trajectories and ego-motion, reporting improvement over baselines and prior state of the art.

Motivation

  • Rare driving events are difficult to collect and anticipate, limiting supervised recognition of unusual roadway situations.Such events may occur too infrequently for sufficient training data, while safety measurement may require billions of test miles.Source: §I · p. 1

Proposed approach

  • The approach learns normal roadway conditions and signals anomalies when observations do not fit the learned model.Training uses massive normal dashboard-video collections without manual labeling.Source: §I · p. 1
  • Future object locations are predicted from dashboard-camera video, then compared with locations observed in subsequent frames.Major deviations between predicted and actual locations are used as anomaly evidence.Source: §I · p. 1

Design rationale

  • The method predicts object trajectories instead of whole RGB frames and explicitly models future ego-vehicle odometry for moving-camera scenes.Predicted and real ego-motion also help distinguish accidents involving the ego-vehicle from accidents merely observed by it.Source: §I · pp. 1–2

Evaluation

  • Experiments use three datasets, including approximately 1,500 dashboard-camera traffic accidents collected from YouTube, and outperform published baselines and state of the art.The reported evaluation includes a newly collected labeled accident dataset.Source: §I · p. 2

II. RELATED WORK

Prior anomaly-detection methods largely reconstruct or compare video frames and often assume static-camera settings, while road-accident approaches commonly require supervision. This paper instead detects deviations in predicted object trajectories and ego-motion for first-person driving videos.

Trajectory Prediction

  • Trajectory-prediction research has mainly addressed static-camera, third-person views, with more recent work extending prediction to first-person videos.Examples include pedestrian and vehicle trajectory prediction from vehicle-mounted cameras.Source: §II · p. 2

Video Anomaly Detection

  • Video anomaly detection commonly models normal training data through reconstruction using convolutional autoencoders and related temporal architectures.These methods target regular visual and motion patterns in surveillance-style settings.Source: §II · p. 2
  • Whole-frame reconstruction is difficult in dynamic autonomous-driving scenes because intense ego-motion disrupts prediction of current or future RGB frames.The proposed alternative uses differences between predicted and actual trajectories of other vehicles.Source: §II · p. 2, §III · p. 2

Road Accident Detection

  • Road-anomaly methods detect violations and collisions, but supervised models require arduous annotations and training examples for anomaly categories.This motivates an unsupervised formulation based on normal driving data.Source: §II · p. 2
  • The model learns normal object and ego motions from large-scale non-anomalous driving videos, then applies three anomaly-detection strategies at test time.The training data does not require hand labeling or explicit examples of every possible anomaly.Source: §III · pp. 2–3

A. Future Object Localization (FOL)

The future object localization model predicts each visible object’s bounding-box trajectory for the next δ frames from current visual evidence, location, and historical information. Its encoders combine box information with pixel-level spatiotemporal features.

Bounding Box Prediction

  • An observed object bounding box is represented by its center location, width, and height in pixels.The notation defines the current object state used by the localization model.Source: 1) Bounding Box Prediction · p. 3
  • Given image evidence, a visible object’s current location, and historical information, the model predicts its bounding-box trajectory for δ future frames.The trajectory contains one parameterized bounding box for each future frame.Source: 1) Bounding Box Prediction · p. 3
  • Two GRU-based encoders process the object’s current bounding box and pixel-level spatiotemporal features to update hidden states.The spatiotemporal features are extracted using region-of-interest pooling with bilinear interpolation.Source: 1) Bounding Box Prediction · p. 3

2) Ego-Motion Cue:

The method predicts future ego-vehicle motion to account for the moving camera and feeds that information into future object localization. Tracking maintains predictions through missed detections, reducing the effect of temporary occlusion and detection failures.

2) Ego-Motion Cue:

  • Ego-motion is modeled by representing the ego-vehicle pose with yaw and ground-plane position, then predicting future odometry changes with an RNN encoder-decoder.The predicted sequence contains future ego-position changes relative to the current pose.Source: 2) Ego-Motion Cue · p. 3
  • The predicted ego-motion output is combined with the future-object-localization decoder state as input to the next time step.Using changes in ego-position avoids accumulated odometry errors.Source: 2) Ego-Motion Cue · p. 3

Tracking

  • Trackers store current boxes, predicted future boxes, and tracker age for observed and missed objects.New trackers are initialized for newly detected objects, while existing trackers are updated each step.Source: 3) Missed Objects · p. 3
  • Temporarily missed objects use previously predicted boxes as estimated current locations for continued future localization.This mechanism is intended to reduce the impact of failed detection or tracking in individual frames.Source: 3) Missed Objects · p. 3
  • Trackers older than the maximum age A are removed from the tracker list.The algorithm takes observed boxes and image evidence as input and outputs updated trackers.Source: 3) Missed Objects · p. 3, §1 · p. 3

B. Traffic Accident Detection

The paper detects traffic accidents by monitoring whether predicted object trajectories and locations remain accurate and consistent. It uses three strategies to derive anomaly scores from deviations in future-location predictions.

1) Predicted Bounding Boxes - Accuracy:

The predicted bounding-box accuracy method compares predicted and observed object locations using averaged future boxes and IoU. Its effectiveness depends on accurate object tracking, which can fail during severe accidents and ego-motion.

  • The method averages δ predicted bounding boxes and computes IoU with each observed object’s bounding box.The FOL model predicts boxes for the next δ future frames from preceding frames.Source: 1) Predicted Bounding Boxes - Accuracy · p. 4
  • The aggregate anomaly score averages IoU values over all observed objects.Source: IoU · p. 4
  • Accurate object tracking is required to match predicted and observed bounding boxes.Source: IoU · p. 4
  • Severe accidents and ego-motion can cause tracking errors that increase false negatives by ignoring unsuccessfully tracked objects.Source: IoU · p. 4

2) Predicted Box Mask - Accuracy: (

The predicted box mask accuracy method compares binary masks formed from predicted and observed bounding boxes. Unlike object-by-object box comparison, it evaluates all objects simultaneously using mask IoU.

3) Predicted Bounding Boxes - Consistency:

The consistency strategy detects anomalies from disagreement among multiple future bounding-box predictions, while the evaluation uses diverse accident and normal-driving datasets. High prediction variability indicates abnormal motion, and anomaly scores are normalized for evaluation.

Predicted Bounding Boxes - Consistency

Evaluation

Evaluation datasets

  • A3D contains 1,500 dashboard-camera clips covering 18 traffic-accident types, 128,175 frames, varied conditions, locations, and participant types.Source: §IV · p. 5
  • HEV-I provides 230 normal on-road intersection videos for training, with no overlap between its collection setting and A3D testing data.Source: §IV · p. 5

A. Implementation Details

The evaluation uses frame-level ROC/AUC for accident detection and compares the proposed approach with K-NN, Conv-AE, and a future-frame GAN baseline. FOL prediction errors are reported in pixels.

  • Accident detection is evaluated with frame-level ROC curves and AUC, where higher AUC indicates better performance.Source: B. Evaluation Metrics · p. 5
  • The K-NN baseline scores video chunks by their normalized distance to nearby normal chunks in pretrained I3D feature space.Videos are segmented into 16-frame chunks labeled using the eighth frame, with K = 1 and K = 5 evaluated.Source: C. Video Anomaly Detection Baselines · p. 5
  • Conv-AE reconstructs input images using convolutional, pooling, deconvolutional, and upsampling layers to derive anomaly scores.The model is trained for 20 epochs on a mixture of SA and HEV-I data, selecting the best model.Source: C. Video Anomaly Detection Baselines · pp. 5–6
  • FOL prediction performance is reported using final displacement error, average displacement error, and final intersection-over-union, with errors measured in pixels.Source: Table II · p. 6
  • The state-of-the-art baseline predicts future RGB frames with a GAN and detects anomalies from differences between predicted and observed frames.Its publicly available implementation is fine-tuned on the same dataset as Conv-AE for a fair comparison.Source: C. Video Anomaly Detection Baselines · p. 6

D. FOL Results

FOL prediction accuracy decreases on the accident datasets relative to HEV-I, and the proposed methods outperform the evaluated baselines on A3D. The comparison also shows how tracking, object detection, and ego-motion affect robustness.

  • Accident videos produce higher FDE and ADE and lower FIOU than HEV-I, indicating reduced future-object-location prediction accuracy.The difference is attributed to different cameras and scenarios, while A3D is slightly harder than SA because it contains more diverse accident types.Source: D. FOL Results · p. 6
  • The proposed method outperforms K-NN, Conv-AE, and the state-of-the-art baseline on A3D.Source: E. Accident Detection Results on A3D… · p. 6
  • FOL-Mask significantly outperforms FOL-AvgIoU and FOL-MinIoU because it does not rely on accurate tracking, though missed normal detections can create false alarms.Source: E. Accident Detection Results on A3D… · p. 6
  • FOL-MinIoU can perform worse than average-IoU scoring because one failed prediction for a normal object can dominate the minimum over many objects.Source: E. Accident Detection Results on A3D… · p. 6
  • Removing ego-involved anomalies improves FOL-AvgIoU and FOL-MinIoU, while the other methods remain relatively robust to ego-motion.The results motivate reducing dependence on accurate object detection and tracking during anomalies.Source: E. Accident Detection Results on A3D… · p. 6
  • Qualitative failures include false alarms from inconsistent predictions of occluded objects and false negatives when participants are fully occluded or appear accidentally normal.Source: E. Accident Detection Results on A3D… · p. 6

F. Results on the SA Dataset

On the Street Accident dataset, the proposed method was tested using training only on HEV-I and outperformed all baselines and published state-of-the-art methods.

Loading 1903.00618v4…