Source-linked AI summary

Anomaly Detection using Edge Computing in Video Surveillance System: Review

Devashree R. Patrikar, Mayur Rajram Parate

arXiv:2107.02778v2cs.CV

TL;DR

Because surveillance cannot be continuously watched by people and anomaly detection is delay-sensitive, the paper reviews methods for intelligent video surveillance with an emphasis on edge computing. It concludes that edge-based anomaly detection remains less explored and needs attention.

  • Problem

    Surveillance cannot be watched continuously by people, while anomaly detection is delay-sensitive and cloud processing entails bandwidth needs and longer response times.

    Source: §1 · p. 2, §4 · p. 5
  • Method

    The paper surveys anomaly-detection techniques across surveillance scenarios and reviews edge-computing techniques, challenges, and opportunities.

    Source: §1 · p. 3, §8 · p. 19
  • Results

    The survey concludes that anomaly detection using edge computing in video surveillance is less explored and needs attention.

    Source: §8 · p. 19
  • Takeaways & Limitations

    Benchmark comparisons may not represent all real-life situations, and performance depends on crowd density, decreasing as crowds increase and working best when crowds are sparse.

    Source: §7 · p. 19

    Migrating machine-learning surveillance methods to edge devices is challenging because of their restricted resources.

    Source: §6 · p. 16

Abstract

from arXiv · show

The current concept of Smart Cities influences urban planners and researchers to provide modern, secured and sustainable infrastructure and give a decent quality of life to its residents. To fulfill this need video surveillance cameras have been deployed to enhance the safety and well-being of the citizens. Despite technical developments in modern science, abnormal event detection in surveillance video systems is challenging and requires exhaustive human efforts. In this paper, we surveyed various methodologies developed to detect anomalies in intelligent video surveillance. Firstly, we revisit the surveys on anomaly detection in the last decade. We then present a systematic categorization of methodologies developed for ease of understanding. Considering the notion of anomaly depends on context, we identify different objects-of-interest and publicly available datasets in anomaly detection. Since anomaly detection is considered a time-critical application of computer vision, our emphasis is on anomaly detection using edge devices and approaches explicitly designed for them. Further, we discuss the challenges and opportunities involved in anomaly detection at the edge.

1 Introduction

The introduction frames anomaly detection as a time-sensitive surveillance task that motivates edge computing, and presents a review to connect scattered methods and address the underexplored edge-computing context.

1 Introduction

  • Cloud processing requires substantial bandwidth and longer response times, while surveillance anomaly detection is delay-sensitive; combining cloud and edge computing is presented as a better real-time solution.The review describes sensors collecting data, followed by preprocessing and feature extraction, then modeling to classify behavior as abnormal or normal.Source: §1 · pp. 2–3
  • The review brings together scattered learning methods and approaches, from handcrafted spatio-temporal features to object-level information and machine-learning methods.It aims to provide a clearer view of anomaly-detection schemes.Source: §1 · p. 3
  • The review categorizes surveillance targets by application context, including automobiles, individuals, crowds, objects, and events.Its scenarios include vehicle parking, traffic, public places, industrial and home surveillance, with attention to targets and their interactions.Source: §1 · p. 3
  • The review addresses a gap in prior surveys by focusing on anomaly detection in video surveillance using edge computing and discussing its challenges and opportunities.It also aims to connect disparities between problem formulations and proposed solutions and to assess technique suitability across contexts.Source: §1 · p. 3

2 Related Work

Prior reviews cover road-traffic and human or crowd behavior from different angles, while surveillance targets and settings shape which anomaly-detection methods are applicable.

2 Related Work

  • Road-traffic anomaly methods are categorized by features, object representation, approaches, and models because they cannot be applied universally across traffic scenarios.Prior surveys address vehicle detection and tracking, vehicle attributes, and traffic anomaly detection [20].Source: §2 · p. 4
  • The reviewed literature treats road traffic and human or crowd behavior as distinct anomaly contexts, with methods and surveillance targets varying accordingly.The survey notes that recent anomaly-detection and automated-video-surveillance reviews are listed in Table 1.Source: §2 · p. 4
  • Human and crowd anomalies are more complex, and occlusion and visual disparity in crowded scenes reduce detection accuracy.Prior reviews examine tracking, interactions, activity, crowd motion patterns, and trajectory-based methods.Source: §2 · p. 4

3 Surveillance Targets

The review organizes surveillance anomaly detection by the target being monitored, including people, crowds, traffic, events or objects, and human-object interactions.

3 Surveillance Targets

  • Surveillance targets are the entities among which an anomaly detection method aims to detect anomalies.Source: §3 · p. 4

3.1 Individual

  • Individual-focused methods address behaviors such as security breaches, running, robbery, and falls among elderly people.Source: §3.1 · p. 5

3.2 Crowd

  • Crowd anomalies include changes in motion, density, or kinetic energy, while occlusion can make high-density crowd counting difficult.Source: §3.2 · p. 5

3.3 Automobiles and Traffic

  • Automobile and traffic surveillance covers traffic flow and density, law violations, safety events, and parking occupancy.Source: §3.3 · p. 5

3.4 Inanimate objects or events

  • Inanimate-object and event targets include fires, smoke, and manufacturing defects.Source: §3.4 · p. 5

3.5 Interaction between humans and objects

  • Interaction-focused schemes detect anomalies such as suspicious baggage or pedestrians and cyclists using inappropriate parts of a roadway.Source: §3.5 · p. 5

4 Anomaly Detection Methodologies in Video Surveillance

Continuous manual monitoring of surveillance video is impractical, motivating intelligent systems that detect anomalous events. The review distinguishes anomaly types and organizes detection methods into learning-based and modeling-based approaches.

  • Because people cannot watch surveillance video continuously and track anomalies, the paper identifies a need for intelligent surveillance systems.Source: §4 · p. 5
  • Anomalies are categorized as contextual, point, or collective according to whether abnormality depends on context, a single instance, or a group of instances.Source: §4 · p. 6
  • The review groups detection schemes into learning-based and modeling-based approaches, with further subcategories for clarity.Source: §4 · p. 6

4.1 Learning

The review groups learning methods into supervised, unsupervised, and semi-supervised approaches. Their distinctions concern the labels available in training and how normality or anomalous events are identified.

4.1 Learning

  • Learning methods are classified as supervised, unsupervised, or semi-supervised according to the training data and methodology.Source: §4.1 · p. 6

4.1.1 Supervised learning

  • Supervised methods train on labeled data, using single-, two-, or multi-class formulations; multi-class methods may misclassify events outside the learned domain.Source: §4.1.1 · pp. 6–7

4.1.2 Unsupervised Learning

  • Unsupervised methods use unlabeled data and commonly treat rare events as anomalous, but nondeterministic anomalies and insufficient training data make detection challenging.Source: §4.1.2 · p. 7
  • Unsupervised approaches include trajectory clustering and self-supervised or clustered spatiotemporal representations for detecting abnormal patterns.Source: §4.1.2 · p. 7
  • An incremental spatiotemporal learner uses periodic human observation and fuzzy aggregation to update normal behavior as anomalies evolve.Source: §4.1.2 · p. 8

4.1.3 Semi-supervised Learning

  • Semi-supervised learning combines a small labeled set with a larger unlabeled set, and the review reports improved learning accuracy from using unlabeled data with limited labels.Source: §4.1.3 · p. 8

4.2 Modeling Algorithms for Anomaly Detection

The review categorizes modeling algorithms by how they represent normal activity or distinguish anomalies, including statistical, proximity, classification, reconstruction, and prediction-based methods. It also covers additional approaches such as fuzzy traffic models and sparsity-based methods.

4.2.1 Statistical Based

  • Statistical methods model normal-activity distributions, with normal events assigned higher likelihood; parametric models assume a probability density function, while non-parametric models adapt to data.Source: §4.2.1 · p. 8

4.2.2 Proximity Based

  • Proximity methods identify anomalies through distance from neighbors or sparse density, and dense crowds make irregularities harder to detect.Source: §4.2.2 · p. 12

4.2.3 Classification Based

  • Classification methods separate events using learned margins, including SVM-based features, object trajectories, and trajectory descriptors for motion information.Source: §4.2.3 · p. 12

4.2.4 Reconstruction Based

  • Reconstruction methods treat large reconstruction error as evidence of abnormality, while low-rank methods use sparse reconstruction to describe abnormal test samples.Source: §4.2.4 · p. 12

4.2.5 Prediction based

  • Prediction-based methods detect anomalies from differences between predicted and actual spatiotemporal features, and also use temporal models such as LSTMs and ISTA.Source: §4.2.5 · p. 12

4.2.6 Other Approaches

  • Other approaches include fuzzy traffic models based on flow, density, and motion, sparsity-based background subtraction, and high-frequency correlation sensors for vehicle anomalies.Source: §4.2.6 · p. 13

5 Edge Computing

Edge computing processes and stores much of its data on-device, offering an alternative to centralized or cloud processing when network latency and bandwidth are problematic. The review notes both emerging edge anomaly-detection approaches and a software–hardware implementation gap.

5 Edge Computing

  • Centralized and cloud processing require video transmission, whose latency and bandwidth demands make them unsuitable for time-critical anomaly detection.Edge computing is proposed where latency and privacy are concerns.Source: §5 · p. 13
  • Edge computing is decentralized, with most device-generated data processed and stored on the device itself.The review describes federated autoencoders on edge devices, with a central server updating and redistributing local models.Source: §5 · p. 13
  • The review identifies a considerable gap between software and hardware implementations for computer-vision learning methods and edge devices.It notes that only a few contributions address anomaly detection directly on edge or terminal devices.Source: §5 · p. 13
  • The described edge-computing architecture places fog nodes between cloud infrastructure and terminal devices such as sensors and smartphones.Fog nodes are devices with computing capability, memory, and network connectivity.Source: §5 · p. 13

6 Anomaly Detection using Edge Devices

The review surveys edge-based surveillance methods and datasets, then identifies challenges involving edge-node discovery, security and accessibility, and quality of service. It describes approaches spanning traditional vision methods, learning models, distributed systems, and application-specific anomaly detection.

6 Anomaly Detection using Edge Devices

  • For response-sensitive surveillance, reviewed edge approaches range from Haar-Cascade and HOG+SVM methods to Kerman, which outperforms Haar-cascade, HOG+SVM, and SSD Google Net on FPR, FPS, and speed.The review also notes Kerman can track and re-find human objects in real time.Source: §6 · p. 14
  • SaliencyGAN is comparable to supervised approaches with 30% labeled data, while edge–cloud models and an LBP-BNN target reduced delay and real-time passenger emotion detection.The LBP-BNN classifies expressions into six categories; the review describes it as faster and improved.Source: §6 · p. 14
  • For UAV surveillance, cloud-based tracking faces large response times from data transfer, while the edge-based AGSSN is proposed to address latency and communication overhead.The review identifies task offloading across edge nodes as an option for distributing computationally expensive tasks.Source: §6 · p. 14
  • Edge surveillance examples include distributed traffic monitoring, vehicle anomaly detection, and parking-occupancy tracking, whose network latency increases with camera count.The parking experiment uses edge devices with SSD-Mobilenet and TensorFlow Lite across varied weather and time conditions.Source: §6 · pp. 14, 16

6.1 Datasets

  • Public anomaly-detection datasets cover varied targets and contexts, including crowds, pedestrians, vehicles, and human-object interactions.Examples include CUHK events such as littering and walking on grass, and UCSD crowd scenes spanning sparse to dense conditions.Source: §6.1 · pp. 16–17

6.2.1 Challenge 1: Locating Edge Nodes

  • Locating edge nodes requires efficient benchmarking of device capability and availability across devices from different generations.The review notes that machine-learning tasks are now used for anomaly detection setups, unlike in the past.Source: §6.2.1 · p. 17

6.2.2 Challenge 2: Security and accessibility

  • Edge processing distributes tasks across many nodes, making user-access encryption and securing offloaded data challenges.The passage notes that data is handled by many devices during offloading.Source: §6.2.2 · p. 17

6.2.3 Challenge 3: Quality of Service

  • Quality of service depends on throughput, avoiding overloaded nodes, and partitioning and scheduling tasks during peak-hour overload.The review frames these as requirements for the overall framework.Source: §6.2.3 · p. 17

6.2.4 Opportunity 1: Distributed computing

The review presents distributed computing, low-latency processing, and smart computation as opportunities for edge-based anomaly detection. These approaches distribute workloads and processing to address response time and resource constraints.

6.2.4 Opportunity 1: Distributed computing

  • Computation offloading divides intensive tasks among available network nodes according to their capabilities, supporting workload distribution and reduced response time.The review cites video-surveillance offloading in Internet of Vehicles and fog-enabled traffic management.Source: §6.2.4 · p. 19

6.2.5 Opportunity 2: Low Latency Computing and Decentralized Cloud

  • Edge processing handles data where it is produced, reducing latency relative to cloud processing, while service delay is described as reducing both transmission and processing delay.The review notes that geographically distant cloud centers further hamper response time.Source: §6.2.5 · p. 19

6.2.6 Opportunity 3: Smart Computation

  • Smart computation hierarchically distributes computation and may use volunteer nodes when edge resources are limited.The review associates it with accuracy, efficiency, lower computation, and latency.Source: §6.2.6 · p. 19

7 Observations

The review cautions that benchmark-dataset comparisons may not represent all real-life situations, and notes that model performance decreases as crowd density increases. It argues that background information may help model weather conditions, while edge computing and task offloading are promising for delay-sensitive applications.

7 Observations

  • Benchmark-dataset comparisons may not capture all real-life scenarios, and anomaly-detection performance decreases as crowd density increases.The review says models work best when crowds are sparse and suggests background information could help model rainy, sunny, or snowy conditions.Source: §7 · p. 19
  • For delay-sensitive surveillance, the review presents edge computing as promising because it processes data on-device, and says task offloading divides workload to improve overall efficiency.The passage also associates on-device processing with greater privacy and security.Source: §7 · p. 19

8 Conclusion

The survey covers anomaly-detection techniques across surveillance scenarios and emphasizes learning techniques, models, and approaches. It identifies edge-based anomaly detection as less explored and in need of attention.

8 Conclusion

  • The survey spans vehicular, pedestrian, crowd, traffic, industrial, and public-place anomaly-detection scenarios.It examines the learning techniques, models, and approaches used across them.Source: §8 · p. 19
  • The survey emphasizes learning techniques, models, approaches, and scenario diversity in anomaly detection.Source: §8 · p. 19
  • Anomaly detection using edge computing is less explored and needs attention.The authors identify further work toward state-of-the-art anomaly detection and intelligent surveillance on edge devices.Source: §8 · p. 19
Loading 2107.02778v2…