Source-linked AI summary
A Discriminative Framework for Anomaly Detection in Large Videos
Allison Del Giorno, J. Andrew Bagnell, Martial Hebert
TL;DR
The paper addresses anomaly detection when separate training sequences are unavailable and anomaly scores should be independent of temporal order. It defines anomalies as frames distinguishable from other frames in the same video and uses permutation-based discriminative learning. On standard datasets adjusted to remove training data, the method performs as well as state of the art.
Problem
Anomaly detection remains difficult when no separate training sequences are available and the order of anomaly occurrences should not affect scores.
Method
The method estimates frame discriminability within the test video using simple classifiers, change detection, and shuffled permutations of frames.
Results
The algorithm performs as well as state of the art on standard datasets after training data are removed.
Takeaways & Limitations
Permutation-based discriminative learning supports anomaly detection without separate training data or assumptions about anomaly order.
Takeaways & Limitations
The method can produce false positives when operating only on the test sequence, including events that occur uniquely in the foreground.
Abstract
from arXiv · showhide
We address an anomaly detection setting in which training sequences are unavailable and anomalies are scored independently of temporal ordering. Current algorithms in anomaly detection are based on the classical density estimation approach of learning high-dimensional models and finding low-probability events. These algorithms are sensitive to the order in which anomalies appear and require either training data or early context assumptions that do not hold for longer, more complex videos. By defining anomalies as examples that can be distinguished from other examples in the same video, our definition inspires a shift in approaches from classical density estimation to simple discriminative learning. Our contributions include a novel framework for anomaly detection that is (1) independent of temporal ordering of anomalies, and (2) unsupervised, requiring no separate training sequences. We show that our algorithm can achieve state-of-the-art results even when we adjust the setting by removing training sequences from standard datasets.
1 Introduction
The paper targets anomaly detection when no separate training sequences exist and anomaly scores should not depend on temporal order. It proposes defining anomalies as frames distinguishable from other frames in the same video, using a permutation-based discriminative framework.
- Problem: The setting requires detecting anomalies from the test video itself while scoring instances independently of their order.This excludes separate training sequences and temporal assumptions about which event appears first.
- Our approach: The proposed definition labels frames anomalous when they are easily distinguished from other frames in the same video.The approach avoids requiring a representative anomaly class or a precomputed distribution of familiar events.
- Previous approaches: Scanning methods learn familiarity models from training data or early frames, so they can miss repeated anomalies and favor events occurring earlier.The paper identifies both training requirements and temporal ordering as violations of its target setting.
- Our approach: The framework uses simple classifiers and shuffled permutations before change detection to make frame comparisons independent of temporal order.Shuffling also lets the method compare each frame against many permutations of other frames while reducing overfitting.
- Contributions: The contributions include a permutation-based framework without training data, supporting theory for parameter choices, and evaluation showing performance similar to methods requiring training data.The listed contributions also include experimental evaluation of the framework.
- Motivating example: In the MNIST toy example, shuffling makes the method classify only the rare 4’s and 5’s as anomalous rather than producing order-dependent false positives on 2’s.The unshuffled method has temporal disadvantages similar to online one-class SVM, whereas shuffling avoids those pitfalls.
2 Method
The framework detects anomalous frames by discriminatively comparing sliding-window descriptors with other frames, repeating this process over shuffled orders and averaging the resulting scores.
- Taking the direct approach: Density-ratio estimation provides a direct alternative to separately modeling the distributions underlying anomaly detection.The approach avoids unnecessary independent distribution modeling by computing a ratio between distributions.
- System overview: The framework accepts frame descriptors, supports flexible feature choices, and outputs anomalousness estimates for each frame.Features may be selected or aggregated according to domain knowledge, computational cost, and desired detection resolution.
- No shuffles - change detection: Without shuffling, sliding-window change detection compares each window with preceding frames assumed to be familiar.The window advances through the video, while classifier confidence indicates how distinguishable each window frame is from earlier points.
- Adding in shuffles - full anomaly detection: For each shuffle, classifiers evaluate sliding-window frames, and the algorithm combines their outputs into final per-frame anomaly estimates.The procedure iterates over windows and returns {a1, ..., aT}.
- No shuffles - change detection: A sliding window regularizes classification and reduces computation relative to point-by-point processing.The number of required splits is inversely proportional to the window size tw.
- Adding in shuffles - full anomaly detection: Shuffling frame order K times reduces temporal-order effects by repeating distinguishability scoring across permutations.The resulting scores are averaged, then converted to log-odds as the final anomaly score.
3 Supporting Theory
The framework analyzes how shuffles, window size, and classifier complexity affect discriminative anomaly scoring. Its theory shows that sufficient shuffling removes temporal dependence, while window and classifier choices trade off computational cost, anomaly separation, and overfitting.
- Classifier complexity: Classifier complexity must balance anomaly–familiar separability against the risk of distinguishing familiar points from one another.Higher Rademacher complexity makes random-label distinctions easier, while smaller training subsets increase classification error; the framework therefore chooses l2-regularized logistic regression.
- Shuffle count: The probability that an anomaly appears first in fewer than 1/A − ϵ shuffles decreases exponentially as the number of shuffles increases.The bound uses a relative tolerance because deviations matter more when the expected first-occurrence fraction 1/A is small.
- Shuffle count: Increasing the number of shuffles K removes temporal dependencies by giving each anomaly repeated opportunities to appear first.The analysis considers the worst case in which all A anomalies have identical feature vectors, so one anomaly can suppress another when split across labels.
- Shuffle count: O(A log A) shuffles are required, for fixed δ and ϵp, to reorder A anomalies enough times to score them equally.This bound follows from extending a single-anomaly concentration bound to all anomalies with a union bound.
- Window size: Larger windows reduce classifier overfitting and computational load, but can cause different anomaly types to interfere with one another’s scores.Smaller windows make anomalies easier to distinguish but require more splits and can increase the risk of misleadingly easy classifications.
- Parameter choice: Window size is chosen empirically: decrease tw when familiar-event variance is high and increase it when no anomalies are visible.This heuristic follows the intended balance between suppressing familiar-versus-familiar distinctions and retaining anomaly separability.
4 Experiments
The experiments evaluate the method on multiple video anomaly-detection datasets using only testing videos, and report near-state-of-the-art performance despite removing separate training data. Results also examine shuffling, parameter robustness, and practical evaluation limitations.
- The algorithm was tested on the Avenue, Subway, Personal Vacation, and UMN Unusual Activity datasets.
- On Avenue, the method uses only individual testing videos and achieves near-state-of-the-art performance without training sequences or assumptions about familiar video segments.
- Performance can be limited by feature representations, false positives relative to training-based ground truth, and detections occurring up to 15 frames early.
- On UMN, the method outperforms on all but one scene, with average AUC = 0.91, although reported approaches in achieve higher average AUC.
- All K = 10 settings outperform their K = 0 counterparts on frame-based AUC averaged across Avenue’s 21 test videos.
5 Discussion
The discussion presents permutation testing as an anomaly-detection method that is independent of anomaly order and requires no separate training sequences. It reports state-of-the-art performance on standard datasets after removing training data.
- The method identifies anomalies independently of their order and requires no separate training sequences.
- Permutation testing requires no assumptions about descriptor content and allows users to plug in optimal features when anomalous frames are distinguishable in that feature space.
- The algorithm performs as well as state of the art on standard datasets even when training data are removed.