Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection

Jia-Xing Zhong; Nannan Li; Weijie Kong; Shan Liu; Thomas H. Li; Ge Li

Source-linked AI summary

Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection

Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H. Li, Ge Li

arXiv:1903.07256v1Mar 18, 2019cs.CV

TL;DR

Weakly supervised video anomaly detection is reframed as supervised learning with noisy labels, where normal snippets inside anomalous videos create incorrect annotations. The paper uses a GCN-based cleaner with alternate cleaning and classification to enable action classifiers, achieving state-of-the-art performance across three datasets and two classifier types.

Problem
Normal snippets inside anomaly-labeled videos create incorrect annotations, motivating a noisy-label formulation of weakly supervised anomaly detection.
Method
A graph convolutional cleaner propagates information from high-confidence to low-confidence predictions through alternate cleaning and classifier-retraining stages.
Results
State-of-the-art performance was reported across 3 different-scale datasets with two types of action classifiers, including an AUC of 82.12% on UCF-Crime.
Takeaways & Limitations
The noisy-label perspective preserves the strengths of fully supervised action classifiers and enables end-to-end anomaly detection without test-time post-processing.
Takeaways & Limitations
The classifier failed to detect an anomaly when the testing scene differed from scenes represented in training data.

Abstract

from arXiv · show

Video anomaly detection under weak labels is formulated as a typical multiple-instance learning problem in previous works. In this paper, we provide a new perspective, i.e., a supervised learning task under noisy labels. In such a viewpoint, as long as cleaning away label noise, we can directly apply fully supervised action classifiers to weakly supervised anomaly detection, and take maximum advantage of these well-developed classifiers. For this purpose, we devise a graph convolutional network to correct noisy labels. Based upon feature similarity and temporal consistency, our network propagates supervisory signals from high-confidence snippets to low-confidence ones. In this manner, the network is capable of providing cleaned supervision for action classifiers. During the test phase, we only need to obtain snippet-wise predictions from the action classifier without any extra post-processing. Extensive experiments on 3 datasets at different scales with 2 types of action classifiers demonstrate the efficacy of our method. Remarkably, we obtain the frame-level AUC score of 82.12% on UCF-Crime.

1. Introduction

The paper reframes weakly supervised anomaly detection as supervised learning with noisy snippet labels, then alternates label cleaning and action-classifier optimization. A GCN exploits video structure to clean labels, enabling end-to-end action classification and strong results across datasets and classifiers.

Weakly supervised anomaly detection uses only video-level labels, without trimmed anomalous segments or temporal annotations.
The paper reformulates anomaly detection as supervised learning under noisy labels because anomalous videos can contain normal snippets.
The noise-labeled perspective lets fully supervised action classifiers participate throughout training and produce snippet predictions without test-time post-processing.
Alternate optimization repeatedly cleans classifier predictions with a label-noise cleaner, then retrains the action classifier using the refined labels.
The proposed GCN cleans noisy labels by propagating supervisory information from high-confidence predictions to low-confidence ones.
Experiments on 3 different-scale datasets with two action-classifier types report state-of-the-art performance validating the approach.

2. Related Work

Related work spans conventional video anomaly detection, deep action classification, noisy-label learning, and graph convolutional networks. The paper connects these areas by using action classifiers for frame-level anomaly detection under weak supervision.

Anomaly detection: Video anomaly detection has commonly modeled anomalies as deviations from regular patterns using statistical and structured models.
Action analysis: Modern action analysis relies heavily on deep architectures, including two-stream networks and C3D variants that achieve state-of-the-art classification performance.
Action analysis: Temporal action detection seeks intervals overlapping ground truth, whereas anomaly detection targets robust frame-level performance across discrimination thresholds.
Learning under noisy labels: Noisy-label research mainly addresses noise reduction or loss correction, including explicit noise models and loss-adjustment methods.
Graph convolutional neural network: Graph convolutional networks use graph-structured signals and parameterized convolutions, including approaches based on spectral graph theory.

3. Problem Statement

The problem uses video-level anomaly labels to infer snippet-level abnormalities without temporal annotations. The paper recasts this setting as one-sided label noise, allowing action classifiers after appropriate label cleaning.

For a video containing N snippets, the video label Y indicates whether any anomalous clips occur, while temporal annotations are unavailable during training.
MIL formulation: Under the MIL formulation, snippets are instances in positive or negative bags, and the goal is to identify positive anomalous instances from bag-level supervision.
Noisy-labeled learning formulation: Negative video labels are noiseless because they indicate that all snippets are normal.
Noisy-labeled learning formulation: Positive video labels are noisy because anomalous videos may contain both anomalous and normal clips.
Noisy-labeled learning formulation: This setting is one-sided label noise: noise occurs only with positive video labels, Y = 1.
Noisy-labeled learning formulation: Handling the positive-label noise permits applying well-developed action classifiers to anomaly detection.

4. Graph Convolutional Label Noise Cleaner

The cleaner alternates between correcting noisy snippet predictions and retraining an action classifier. It uses feature-similarity and temporal-consistency graphs to propagate supervision from high-confidence snippets, enabling direct classifier predictions at test time.

Alternate Optimization: The cleaner alternates noise correction with classifier retraining, using refined snippet labels to produce increasingly reliable predictions.The classifier is retrained with labels generated by the cleaner, and this cycle repeats before final testing.
Graph Modules: Feature embeddings are compressed before entering graph modules that model similarity among snippets and temporal consistency across their positions.The two graph structures provide complementary relations for label propagation.
Feature Similarity Graph Module: Feature-similarity propagation connects snippets according to normalized similarity, encouraging related snippets to receive related anomaly labels through graph convolution.The module uses renormalized adjacency and differentiable trainable layers.
Temporal Consistency Graph Module: Temporal consistency uses a kernel based on snippet distance, with nearby snippets connected more closely before graph convolution.The resulting temporal graph is incorporated through renormalized adjacency and trainable layers.
Loss Function: The fused graph outputs produce snippet anomaly probabilities, while direct supervision uses high-confidence snippets selected through predictive uncertainty.Lower predictive variance indicates higher confidence, and temporal ensembling smooths predictions across training steps.
Deployment: The final trained classifier directly predicts anomaly scores without additional post-processing, preserving the standard fully supervised classifier update and deployment process.The classifier requires almost no change during training or testing beyond using cleaner-generated snippet labels.

5. Experiments

Experiments evaluate the approach across three anomaly-detection datasets and two action-classifier architectures, using frame-level AUC and complementary comparisons. Alternate training improves classifiers across datasets, while ablations support the roles of temporal consistency, feature similarity, and graph convolution.

Datasets: Experiments use UCF-Crime, ShanghaiTech, and UCSD-Peds2, spanning large-, medium-, and small-scale anomaly-detection datasets.ShanghaiTech and UCSD-Peds2 use reorganized or repeated splits because their default training data contain no anomaly videos.
Evaluation: Frame-level AUC is the primary metric, with larger values indicating stronger diagnostic ability and more robust performance across discrimination thresholds.The study also compares ROC curves and false alarm rates on UCF-Crime.
Experimental setup: Two mainstream action classifiers, C3D and TSN, verify the proposed cleaner across different classifier structures and input modalities.C3D is a 3D-convolutional network, while TSN uses a two-stream architecture.
UCF-Crime: 10.2% relative gain raises the optical-flow branch AUC from 70.87% to 78.08% during alternate training on UCF-Crime.The following training steps significantly improve detection performance for all action classifiers.
Ablation studies: 72.93% AUC is reached by the independent temporal-consistency module, while the feature-similarity module raises AUC from 70.87% to 72.44%.Removing the feature-similarity graph lowers AUC to 67.23%, and removing temporal information performs worse than Step-1.
UCF-Crime: 82.12% is the highest reported UCF-Crime AUC, while TSN’s RGB branch reduces the false alarm rate to 0.1%.The proposed ROC curves nearly enclose competing curves across thresholds; C3D is slightly inferior to Sultani et al. on false alarm rate at score 0.5.
ShanghaiTech and UCSD-Peds2: Alternate training improves all action classifiers on ShanghaiTech and stabilizes repeated UCSD-Peds2 experiments, although excessive iterations slightly reduce optical-flow performance.On UCSD-Peds2, the method outperforms other methods with both input modalities.

6. Conclusion

The paper reframes weakly supervised anomaly detection as supervised learning with noisy labels and uses a GCN to clean those labels for action-classifier training.

6. Conclusion: Weakly supervised anomaly detection is recast as supervised learning under noisy labels.The noisy-label perspective contrasts with the multiple-instance learning formulation used in previous works.
6. Conclusion: The method uses a graph convolutional network to clean labels for training an action classifier.During alternate optimization, the GCN reduces noise by propagating anomaly information.
6. Conclusion: The approach directly inherits strengths of well-developed action classifiers and provides an integral end-to-end anomaly-detection model.These strengths are identified as a key merit of the proposed perspective.

Supplementary Materials (Appendix)

The supplementary material lists Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H. Li, and Ge Li as paper authors.

Supplementary Materials (Appendix): Jia-Xing Zhong and Nannan Li are listed among the paper authors.
Supplementary Materials (Appendix): Weijie Kong and Shan Liu are listed among the paper authors.
Supplementary Materials (Appendix): Thomas H. Li and Ge Li are listed among the paper authors.

1. Processing Speed in the Test Phase

The method uses action classifiers directly at test time and reports their speed with preprocessing included, achieving real-time or super-real-time performance.

2. Processing Speed in the Test Phase: The reported FPS measurements include preprocessing operations such as resizing, 10-crop oversampling, and zero-centering.
2. Processing Speed in the Test Phase: Action classifiers directly generate anomaly predictions during testing without a separate post-processing stage.The approach is described as computationally efficient because it directly utilizes action classifiers.
2. Processing Speed in the Test Phase: The three action classifiers retain real-time or super-real-time performance under this measurement protocol.

2. Implementation of Label Noise Cleaner

The label-noise cleaner selects high-confidence snippets, expands their selection across steps, and uses a bounded neighborhood and specified optimization settings.

2. Implementation of Label Noise Cleaner: The cleaner initially selects the highest-confidence 30% or 60% of snippets as H for two-stream or C3D networks, respectively.The cardinality of H increases by 30% at each step.
2. Implementation of Label Noise Cleaner: Normal videos are included to help learn an unbiased model, while classifier probabilities are min-max normalized into a single anomaly category.
2. Implementation of Label Noise Cleaner: The cleaner samples at most 1,600 high-confidence snippets with no more than eight neighbors per video and uses three alternate-optimization steps.The implementation uses learning rate 0.0001, momentum 0.9, and weight decay 0.0005.

3. More Comparisons on UCSD-Peds

The UCSD-Peds comparisons account for differing data splits by reproducing selected open-source methods on the paper’s split while retaining original reported results for reference.

Comparability: Different data splits make the default implementations of several 2018 unary-classification works not directly comparable with the paper’s results.The comparison concerns experiments on UCSD-Peds.
Reproduction: Selected open-source works are re-evaluated using the data split adopted by the paper.This reproduction is intended to align the comparison conditions.
Reporting: Original-paper results are also reported in square brackets for reference.These values are distinguished from the reproduced results.

4. Vectorized Feature Similarity Module

The feature similarity module represents snippets as a graph, propagates anomaly-label information through graph operations, and supports end-to-end differentiable training. The section also describes temporal ensembling details and the reorganized ShanghaiTech split.

Graph representation: The feature similarity graph F = (V, E, X) models a video as vertices, snippet-similarity edges, and d-dimensional snippet attributes.X ∈ R^N×d contains the features of N snippets.
Graph representation: Each adjacency entry A(i,j) measures the feature similarity between the ith and jth snippets.The adjacency matrix is defined over the N snippets.
Graph propagation: The graph-Laplacian operation drives nearby vertices toward the same anomaly label using a renormalization trick.The operation uses a self-loop adjacency matrix eA = A + I_N and its corresponding degree matrix eD.
Optimization: The differentiable computational procedure allows single or multiple stacked graph modules to be trained end-to-end.The implementation uses vectorized operations, including summation and broadcasting diagonal functions.
Graph propagation: A graph-module layer produces output H using a trainable matrix W and activation function σ.The corresponding temporal similarity module can be vectorized similarly.
Optimization: The indirect loss term follows a temporal ensembling strategy with γ = 0.5, without cool-start initialization or bias correction.The rough predictions from the action classifier make those original temporal-ensembling components unnecessary.