PaperlayerView exact revision on arXiv

Source-linked AI summary

Recognition of Abnormal Events in Surveillance Videos using Weakly Supervised Dual-Encoder Models

Noam Tsfaty, Avishai Weizman, Liav Cohen, Moshe Tshuva, Yehudit Aperstein

arXiv:2511.13276v1cs.CV

TL;DR

Rare anomalies and the cost and ambiguity of precise temporal labels motivate video-level weak supervision for surveillance anomaly detection. The paper combines I3D and TimeSformer representations with top-k pooling, and reports the highest AUC among compared methods on UCF-Crime.

  • Problem

    Rare surveillance anomalies are difficult to detect, while precise temporal annotations are costly, ambiguous, and inconsistent, motivating video-level weak supervision.

    Source: §I · p. 1

  • Method

    The method combines I3D and TimeSformer segment representations, scores them with fully connected layers, and aggregates the top-k scores into video-level predictions.

    Source: §I · p. 1, §II · p. 2

  • Results

    The dual-backbone fusion model achieves the highest AUC among compared methods on UCF-Crime.

    Source: §III · p. 2

  • Takeaways & Limitations

    The framework demonstrates strong UCF-Crime performance despite uniform frame sampling, under the authors’ assumption that long continuous videos are rare.

    Source: §IV · p. 2

    Uniform frame sampling is used because of computational limitations, with its sufficiency premised on UCF-Crime rarely containing long continuous videos.

    Source: §IV · p. 2

Abstract

from arXiv · show

We address the challenge of detecting rare and diverse anomalies in surveillance videos using only video-level supervision. Our dual-backbone framework combines convolutional and transformer representations through top-k pooling, achieving 90.7% area under the curve (AUC) on the UCF-Crime dataset.

I. INTRODUCTION

Rare anomalies and the cost and ambiguity of precise temporal labeling make video-level weak supervision a practical alternative for surveillance anomaly detection. The paper proposes a dual-backbone MIL network combining I3D and TimeSformer features with top-k pooling.

I. INTRODUCTION

  • Precise temporal annotations are costly, ambiguous, and inconsistent, motivating video-level weak supervision for rare surveillance anomalies.In MIL, a video is a bag of temporal segments, and an anomalous video label implies that at least one segment is abnormal.Source: §I · p. 1
  • Prior MIL approaches include temporal smoothness and attention, feature-magnitude separation with multi-scale modeling, and transformer-based sequence self-training.Clustering in encourages compact normal-pattern groups and greater separation from anomalous ones.Source: §I · p. 1
  • An I3D feature-embedding and threshold-scoring pipeline is described as a simpler but less discriminative surveillance-data baseline.Source: §I · p. 1
  • The proposed dual-backbone MIL network combines I3D for spatiotemporal motion analysis with TimeSformer to enrich video representations.Lightweight fully connected layers process the fused features, and top-k pooling produces video-level predictions.Source: §I · p. 1

II. PROPOSED METHOD

The method divides each video into temporal segments, samples frames, and encodes them with I3D and TimeSformer. It scores segment features and uses top-k pooling to produce video-level anomaly predictions.

II. PROPOSED METHOD

  • Each video is divided into 32 temporal segments, with 16 uniformly sampled frames per segment; shorter segments are padded by repeating the last frame.The paper notes that about 6% of UCF-Crime video durations exceed ten minutes and most last only a few minutes.Source: §II · pp. 1–2
  • I3D and TimeSformer encode each sampled segment, producing 768- and 1024-dimensional features that are normalized and concatenated into a 1792-dimensional representation.Four fully connected layers with LeakyReLU between layers output a scalar anomaly score for each segment.Source: §II · p. 2, Figure 1 · p. 2
  • The k highest segment scores are aggregated by top-k pooling into a video-level anomaly logit and sigmoid-predicted anomaly probability.The model is jointly optimized with binary cross-entropy loss.Source: §II · p. 2

III. EXPERIMENTS AND RESULTS

The experiments evaluate anomaly detection on UCF-Crime, which contains untrimmed surveillance videos spanning 13 anomaly categories and normal events. The dual-backbone fusion model is reported to achieve the highest AUC among the compared approaches.

III. EXPERIMENTS AND RESULTS

  • UCF-Crime anomalies are irregular or criminal activities in untrimmed surveillance videos spanning 13 categories, alongside normal events.Source: §III · p. 2
  • The dual-backbone fusion model achieves the highest AUC among the compared methods on UCF-Crime.The comparison includes transformer-based, CLIP-based, I3D-with-clustering, and graph-based approaches.Source: Table I · p. 2, §III · p. 2

IV. CONCLUSIONS AND FUTURE WORK

The conclusion presents a framework combining I3D and TimeSformer with lightweight fully connected layers, reporting strong UCF-Crime performance despite uniform frame sampling. The sampling choice is guided by the authors’ assumption that long continuous videos are rare in that dataset.

IV. CONCLUSIONS AND FUTURE WORK

  • The conclusion presents an architecture integrating I3D and TimeSformer encoders with lightweight fully connected layers.Source: §IV · p. 2
  • The framework demonstrates strong UCF-Crime performance despite uniform frame sampling, which is attributed to computational limitations.Source: §IV · p. 2
  • The sampling design assumes UCF-Crime rarely contains long continuous videos; future work proposes multiclass detection and advanced sampling strategies.Source: §IV · p. 2
Loading 2511.13276v1…