PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Traffic Accident Detection in First-Person Videos

Yu Yao, Mingze Xu, Yuchen Wang, David J. Crandall, Ella M. Atkins

arXiv:1903.00618v4cs.CV

TL;DR

Traffic anomaly detection in moving-camera videos is challenged by rare events, static-camera assumptions, and reliance on hand-labeled anomaly categories. The paper predicts future traffic-participant locations and ego-motion from normal driving videos, then monitors prediction deviations, outperforming published baselines and the state of the art on two accident datasets.

  • Problem

    Existing anomaly-detection methods assume fixed cameras or require hand-labeled examples of explicitly trained anomaly categories, while rare roadway events are difficult to collect.

  • Method

    An unsupervised model predicts future traffic-participant trajectories and ego-vehicle motion from normal dashboard-camera videos, then detects anomalies using prediction accuracy and consistency.

  • Results

    The method significantly outperforms published baselines and the state of the art on the A3D and SA accident datasets.

  • Takeaways & Limitations

    Prediction deviations provide signals for detecting traffic accidents without explicitly training on examples of every possible anomaly.

Abstract

from arXiv · show

Recognizing abnormal events such as traffic violations and accidents in natural driving scenes is essential for successful autonomous driving and advanced driver assistance systems. However, most work on video anomaly detection suffers from two crucial drawbacks. First, they assume cameras are fixed and videos have static backgrounds, which is reasonable for surveillance applications but not for vehicle-mounted cameras. Second, they pose the problem as one-class classification, relying on arduously hand-labeled training datasets that limit recognition to anomaly categories that have been explicitly trained. This paper proposes an unsupervised approach for traffic accident detection in first-person (dashboard-mounted camera) videos. Our major novelty is to detect anomalies by predicting the future locations of traffic participants and then monitoring the prediction accuracy and consistency metrics with three different strategies. We evaluate our approach using a new dataset of diverse traffic accidents, AnAn Accident Detection (A3D), as well as another publicly-available dataset. Experimental results show that our approach outperforms the state-of-the-art.

I. INTRODUCTION

The paper targets rare and difficult-to-anticipate roadway anomalies in moving-camera videos, where supervised labeling and whole-frame prediction are impractical. It proposes unsupervised detection from predicted traffic-participant trajectories and ego-motion, reporting improvement over baselines and prior state of the art.

  • Motivation: Rare driving events are difficult to collect and anticipate, limiting supervised recognition of unusual roadway situations.Such events may occur too infrequently for sufficient training data, while safety measurement may require billions of test miles.
  • Proposed approach: The approach learns normal roadway conditions and signals anomalies when observations do not fit the learned model.Training uses massive normal dashboard-video collections without manual labeling.
  • Proposed approach: Future object locations are predicted from dashboard-camera video, then compared with locations observed in subsequent frames.Major deviations between predicted and actual locations are used as anomaly evidence.
  • Design rationale: The method predicts object trajectories instead of whole RGB frames and explicitly models future ego-vehicle odometry for moving-camera scenes.Predicted and real ego-motion also help distinguish accidents involving the ego-vehicle from accidents merely observed by it.
  • Evaluation: Experiments use three datasets, including approximately 1,500 dashboard-camera traffic accidents collected from YouTube, and outperform published baselines and state of the art.The reported evaluation includes a newly collected labeled accident dataset.

II. RELATED WORK

Prior anomaly-detection methods largely reconstruct or compare video frames and often assume static-camera settings, while road-accident approaches commonly require supervision. This paper instead detects deviations in predicted object trajectories and ego-motion for first-person driving videos.

  • Trajectory Prediction: Trajectory-prediction research has mainly addressed static-camera, third-person views, with more recent work extending prediction to first-person videos.Examples include pedestrian and vehicle trajectory prediction from vehicle-mounted cameras.
  • Video Anomaly Detection: Video anomaly detection commonly models normal training data through reconstruction using convolutional autoencoders and related temporal architectures.These methods target regular visual and motion patterns in surveillance-style settings.
  • Video Anomaly Detection: Whole-frame reconstruction is difficult in dynamic autonomous-driving scenes because intense ego-motion disrupts prediction of current or future RGB frames.The proposed alternative uses differences between predicted and actual trajectories of other vehicles.
  • Road Accident Detection: Road-anomaly methods detect violations and collisions, but supervised models require arduous annotations and training examples for anomaly categories.This motivates an unsupervised formulation based on normal driving data.
  • Road Accident Detection: The model learns normal object and ego motions from large-scale non-anomalous driving videos, then applies three anomaly-detection strategies at test time.The training data does not require hand labeling or explicit examples of every possible anomaly.

A. Future Object Localization (FOL)

The future object localization model predicts each visible object’s bounding-box trajectory for the next δ frames from current visual evidence, location, and historical information. Its encoders combine box information with pixel-level spatiotemporal features.

  • Bounding Box Prediction: An observed object bounding box is represented by its center location, width, and height in pixels.The notation defines the current object state used by the localization model.
  • Bounding Box Prediction: Given image evidence, a visible object’s current location, and historical information, the model predicts its bounding-box trajectory for δ future frames.The trajectory contains one parameterized bounding box for each future frame.
  • Bounding Box Prediction: Two GRU-based encoders process the object’s current bounding box and pixel-level spatiotemporal features to update hidden states.The spatiotemporal features are extracted using region-of-interest pooling with bilinear interpolation.

2) Ego-Motion Cue:

The method predicts future ego-vehicle motion to account for the moving camera and feeds that information into future object localization. Tracking maintains predictions through missed detections, reducing the effect of temporary occlusion and detection failures.

  • 2) Ego-Motion Cue:: Ego-motion is modeled by representing the ego-vehicle pose with yaw and ground-plane position, then predicting future odometry changes with an RNN encoder-decoder.The predicted sequence contains future ego-position changes relative to the current pose.
  • 2) Ego-Motion Cue:: The predicted ego-motion output is combined with the future-object-localization decoder state as input to the next time step.Using changes in ego-position avoids accumulated odometry errors.
  • Tracking: Trackers store current boxes, predicted future boxes, and tracker age for observed and missed objects.New trackers are initialized for newly detected objects, while existing trackers are updated each step.
  • Tracking: Temporarily missed objects use previously predicted boxes as estimated current locations for continued future localization.This mechanism is intended to reduce the impact of failed detection or tracking in individual frames.
  • Tracking: Trackers older than the maximum age A are removed from the tracker list.The algorithm takes observed boxes and image evidence as input and outputs updated trackers.

B. Traffic Accident Detection

The paper detects traffic accidents by monitoring whether predicted object trajectories and locations remain accurate and consistent. It uses three strategies to derive anomaly scores from deviations in future-location predictions.

  • Three strategies monitor prediction accuracy and consistency of objects’ future locations for traffic accident detection.
  • All three methods use multiple previous FOL outputs to compute anomaly scores.
  • The methods assume non-anomalous object trajectories and locations can be precisely predicted, while deviations suggest anomalies.

1) Predicted Bounding Boxes - Accuracy:

The predicted bounding-box accuracy method compares predicted and observed object locations using averaged future boxes and IoU. Its effectiveness depends on accurate object tracking, which can fail during severe accidents and ego-motion.

  • The method averages δ predicted bounding boxes and computes IoU with each observed object’s bounding box.The FOL model predicts boxes for the next δ future frames from preceding frames.
  • The aggregate anomaly score averages IoU values over all observed objects.
  • Accurate object tracking is required to match predicted and observed bounding boxes.
  • Severe accidents and ego-motion can cause tracking errors that increase false negatives by ignoring unsuccessfully tracked objects.

2) Predicted Box Mask - Accuracy: (

The predicted box mask accuracy method compares binary masks formed from predicted and observed bounding boxes. Unlike object-by-object box comparison, it evaluates all objects simultaneously using mask IoU.

  • Predicted and observed bounding-box regions are converted into binary masks before computing IoU.Pixels inside boxes receive value 1 and background pixels receive value 0.
  • The mask metric compares bounding boxes of all objects simultaneously rather than matching them object by object.
  • Accurate predictions produce relatively large IoU values compared with the ground-truth observation.

3) Predicted Bounding Boxes - Consistency:

The consistency strategy detects anomalies from disagreement among multiple future bounding-box predictions, while the evaluation uses diverse accident and normal-driving datasets. High prediction variability indicates abnormal motion, and anomaly scores are normalized for evaluation.

  • Predicted Bounding Boxes - Consistency: The method measures similarity among δ future bounding-box predictions using their standard deviation.
  • Predicted Bounding Boxes - Consistency: It uses the maximum STD across box center coordinates and dimensions because anomalies can affect different bounding-box components.Suddenly stopped cross traffic may produce large horizontal-axis variability.
  • Predicted Bounding Boxes - Consistency: Low STD indicates stable predictions and normal movement, whereas high standard deviation suggests abnormal motion.
  • Evaluation: All three methods normalize computed anomaly scores for evaluation.
  • Evaluation datasets: A3D contains 1,500 dashboard-camera clips covering 18 traffic-accident types, 128,175 frames, varied conditions, locations, and participant types.
  • Evaluation datasets: HEV-I provides 230 normal on-road intersection videos for training, with no overlap between its collection setting and A3D testing data.

A. Implementation Details

The evaluation uses frame-level ROC/AUC for accident detection and compares the proposed approach with K-NN, Conv-AE, and a future-frame GAN baseline. FOL prediction errors are reported in pixels.

  • Accident detection is evaluated with frame-level ROC curves and AUC, where higher AUC indicates better performance.
  • The K-NN baseline scores video chunks by their normalized distance to nearby normal chunks in pretrained I3D feature space.Videos are segmented into 16-frame chunks labeled using the eighth frame, with K = 1 and K = 5 evaluated.
  • Conv-AE reconstructs input images using convolutional, pooling, deconvolutional, and upsampling layers to derive anomaly scores.The model is trained for 20 epochs on a mixture of SA and HEV-I data, selecting the best model.
  • FOL prediction performance is reported using final displacement error, average displacement error, and final intersection-over-union, with errors measured in pixels.
  • The state-of-the-art baseline predicts future RGB frames with a GAN and detects anomalies from differences between predicted and observed frames.Its publicly available implementation is fine-tuned on the same dataset as Conv-AE for a fair comparison.

D. FOL Results

FOL prediction accuracy decreases on the accident datasets relative to HEV-I, and the proposed methods outperform the evaluated baselines on A3D. The comparison also shows how tracking, object detection, and ego-motion affect robustness.

  • Accident videos produce higher FDE and ADE and lower FIOU than HEV-I, indicating reduced future-object-location prediction accuracy.The difference is attributed to different cameras and scenarios, while A3D is slightly harder than SA because it contains more diverse accident types.
  • The proposed method outperforms K-NN, Conv-AE, and the state-of-the-art baseline on A3D.
  • FOL-Mask significantly outperforms FOL-AvgIoU and FOL-MinIoU because it does not rely on accurate tracking, though missed normal detections can create false alarms.
  • FOL-MinIoU can perform worse than average-IoU scoring because one failed prediction for a normal object can dominate the minimum over many objects.
  • Removing ego-involved anomalies improves FOL-AvgIoU and FOL-MinIoU, while the other methods remain relatively robust to ego-motion.The results motivate reducing dependence on accurate object detection and tracking during anomalies.
  • Qualitative failures include false alarms from inconsistent predictions of occluded objects and false negatives when participants are fully occluded or appear accidentally normal.

F. Results on the SA Dataset

On the Street Accident dataset, the proposed method was tested using training only on HEV-I and outperformed all baselines and published state-of-the-art methods.

  • The model was trained on HEV-I and evaluated on SA, which contains 165 test videos with different anomalies.SA videos were collected from dashboard cameras in Taiwan at 720p resolution.
  • The best proposed method outperformed every baseline and the published state-of-the-art on SA.
  • SA is much smaller than A3D and appears biased toward anomalies involving bikes.
  • Some SA videos were recorded from cyclist head cameras, producing irregular camera angles and large vibrations.
Loading 1903.00618v4…