Source-linked AI summary
Few-shot Scene-adaptive Anomaly Detection
Yiwei Lu, Frank Yu, Mahesh Kumar Krishna Reddy, Yang Wang
TL;DR
Video anomaly detection must learn from normal data while handling unseen deployment scenes, where standard models may not generalize and target-scene data are costly to collect. The paper formulates few-shot scene-adaptive anomaly detection and uses meta-learning to adapt from a few frames; experiments report significant gains over alternatives, including over 20% improvements in some cross-dataset cases.
Problem
Existing anomaly detection models learn from normal videos but may not generalize to unseen target scenes, while collecting extensive target-scene training data is impractical.
Method
The paper uses optimization-based meta-learning across scene-mimicking tasks to adapt a future-frame prediction anomaly detector to a new scene from a few frames.
Results
The proposed approach significantly outperforms alternative methods, with cross-dataset gains exceeding 20% in some cases.
Takeaways & Limitations
Few-shot scene adaptation targets anomaly detection deployment in a new camera scene without requiring extensive videos from that scene.
Takeaways & Limitations
The experiments assume training videos contain only normal events, while test videos may contain anomalies.
Abstract
from arXiv · showhide
We address the problem of anomaly detection in videos. The goal is to identify unusual behaviours automatically by learning exclusively from normal videos. Most existing approaches are usually data-hungry and have limited generalization abilities. They usually need to be trained on a large number of videos from a target scene to achieve good results in that scene. In this paper, we propose a novel few-shot scene-adaptive anomaly detection problem to address the limitations of previous approaches. Our goal is to learn to detect anomalies in a previously unseen scene with only a few frames. A reliable solution for this new problem will have huge potential in real-world applications since it is expensive to collect a massive amount of data for each target scene. We propose a meta-learning based approach for solving this new problem; extensive experimental results demonstrate the effectiveness of our proposed method.
1 Introduction
Video anomaly detection learns normal behaviour because abnormal events are too diverse to enumerate, but scene changes undermine direct model transfer. The paper proposes adapting a model to an unseen target scene from only a few frames.
- Motivation: Anomaly detection identifies abnormal video frames without exhaustive abnormal-event labels, typically using distance from a learned normal distribution.Reconstruction error and future-frame prediction error are common indicators of anomaly likelihood.
- Motivation: Models trained in one scene can lose performance in a different scene, while broad scene generalization may require large models unsuitable for edge devices.The paper motivates scene-specific adaptation rather than relying on direct transfer or very high-capacity models.
- Proposed Problem: Few-shot scene-adaptive anomaly detection uses videos from multiple training scenes and a few frames from an unseen target scene to produce a scene-specific detector.The target scene is not observed during training, and the adaptation is intended for the remaining videos from that scene.
- Proposed Approach: Meta-learning trains the model across tasks that mimic few-shot adaptation, enabling rapid adjustment to a deployment camera.The proposed setting reflects fixed-camera applications where performance on the deployed scene is the primary concern.
- Results: The proposed approach significantly outperforms alternative methods on several benchmark datasets.This is the paper’s reported overall experimental conclusion.
2 Related Work
Prior video anomaly detection methods mainly reconstruct frames or predict future frames, while few-shot meta-learning provides mechanisms for rapid adaptation. The paper applies optimization-based meta-learning to the more challenging anomaly-detection setting.
- Anomaly Detection: Reconstruction-based methods detect anomalies from reconstruction error, whereas prediction-based methods compare predicted and actual future frames.Both families are trained to model normal video behaviour.
- Few-Shot and Meta Learning: Few-shot learning aims to adapt quickly to new tasks using only a few training samples.Meta-learning is presented as a general strategy for achieving this rapid adaptation.
- Few-Shot and Meta Learning: Meta-learning approaches are commonly grouped into metric-based, model-based, and optimization-based methods.The categories differ in whether they learn a metric, use architectural adaptation, or modify optimization for rapid updating.
- Few-Shot and Meta Learning: The paper follows an optimization-based meta-learning approach and applies it to video anomaly detection rather than simple image-classification tasks.The related-work discussion characterizes anomaly detection as a more challenging application domain.
3 Problem Setup
Standard anomaly detection trains on normal videos and assumes the learned model generalizes to testing videos, but target-scene data are often unavailable. The proposed setup instead adapts a model to an unseen scene using only a few initial frames.
- Standard Anomaly Detection: Reconstruction- and prediction-based detectors compare reconstructed or predicted frames with actual frames and flag differences above a threshold.Training uses normal videos because abnormal events are difficult to collect.
- Problem Motivation: The standard framework assumes that a model learned from training videos generalizes well to test videos, including videos from the deployment scene.Collecting many training videos from the target scene is described as unrealistic in practical applications.
- Few-Shot Scene-Adaptive Anomaly Detection: Few-shot scene-adaptive anomaly detection trains on videos from different scenes and tests on a target scene absent during training.The model adapts to the target scene from only a few initial frames.
- Few-Shot Scene-Adaptive Anomaly Detection: The adapted model is expected to perform well specifically in the target scene rather than requiring broad performance across all scenes.This follows the fixed-camera deployment motivation described in the problem setup.
4 Our Approach: MAML for Scene-Adaptive Anomaly Detection
The approach uses MAML to learn model parameters that quickly adapt to a previously unseen scene from a few frames. Meta-training optimizes adaptation across scene-specific tasks, while meta-testing applies the adapted model to remaining frames in the new scene.
- Backbone architecture: The framework is backbone-independent and can be paired with anomaly detection models, including the paper’s future-frame prediction r-GAN architecture.The proposed backbone combines a U-Net, ConvLSTM, generator, discriminator, and adversarial training to model spatial-temporal video information.
- Task construction: Tasks mimic deployment by sampling K input/output pairs from one video for adaptation and separate K pairs from that video for validation.For future-frame prediction, consecutive video segments provide t input frames and the next frame as the target.
- Meta-training: The inner update adapts parameters from θ to θ′ using gradient descent on each task’s training set.The adaptation step uses the task training loss, with α controlling the step size.
- Training objective: The training objective combines L1, multi-scale structural similarity, and gradient-difference losses for future-frame prediction.The coefficients λ1, λ2, and λ3 weight the three loss terms.
- Meta-training: MAML meta-training learns initial parameters that minimize validation loss after task-specific adaptation across multiple scene tasks.Each task contains training and validation data, and the outer update accounts for the validation performance of adapted parameters.
- Meta-testing: During meta-testing, a few frames from a new scene produce adapted parameters, which are then evaluated on the scene’s remaining frames.This setup uses the first several frames of one video for adaptation and the remaining frames for testing.
5 Experiments
The experiments repurpose multiple anomaly-detection datasets for scene-disjoint and cross-dataset evaluation, comparing the proposed method with baselines and backbone alternatives. Results show stronger performance from r-GAN and especially from scene-adaptive meta-learning in both within-dataset and cross-dataset settings.
- Datasets and Setup: The experimental setup uses training videos from multiple scenes and test videos from different scenes, requiring repurposed datasets.Shanghai Tech is split into six training scenes and seven testing scenes; UCF crime supplies normal videos for meta-training, while several other datasets are used for testing.
- Datasets and Setup: Table 1 evaluates r-GAN, its variants, and existing methods using AUC (%) under the standard no-adaptation setup.The comparison covers UCSD Ped1, UCSD Ped2, CUHK Avenue, and Shanghai Tech.
- Ablation Study: Using five sampled tasks per meta-training epoch produces a better model than using one task, while one-task training still exceeds fine-tuning.The ablation compares N = 1 and N = 5 sampled tasks.
- Sanity Check on Backbone Architecture: The proposed r-GAN backbone outperforms its variants and existing state-of-the-art methods on almost all datasets.This sanity check motivates using r-GAN as the backbone for the few-shot scene-adaptive experiments.
- Experimental Results: On Shanghai Tech, the proposed scene-adaptive model outperforms the pre-trained and fine-tuned baselines across a split with six training scenes and seven testing scenes.The split is designed to evaluate generalization to scenes distinct from those used for training.
- Experimental Results: In cross-dataset testing, the proposed approach improves over the baselines by more than 20% in some cases and adapts to the substantially different UR Fall dataset.Meta-training uses Shanghai Tech or UCF crime, while meta-testing uses UCSD Ped1, UCSD Ped2, CUHK Avenue, and UR Fall.
6 Conclusion
The paper introduces few-shot scene-adaptive anomaly detection, adapting an anomaly detector to a new scene from only a few frames. Its meta-learning approach trains across multiple scenes and significantly outperforms alternative methods.
- Conclusion: Few-shot scene-adaptive anomaly detection adapts an anomaly detector to a new scene using only a few frames from that scene.The target scene is unseen during training, while meta-training uses videos from multiple scenes to construct adaptation tasks.
- Conclusion: The proposed meta-learning approach learns to adapt effectively to new scene-specific tasks.The model is trained on multiple scene-based tasks and then adapted using the corresponding scene's few frames.
- Conclusion: Experimental results show that the proposed approach significantly outperforms alternative methods.