PaperlayerView exact revision on arXiv

Source-linked AI summary

IoT Anomaly Detection Methods and Applications: A Survey

Ayan Chatterjee, Bestoun S. Ahmed

arXiv:2207.09092v1cs.SE

TL;DR

IoT anomaly-detection research is expanding across application domains while facing gaps involving heterogeneous sensors, changing data distributions, and limited ground-truth data. This survey categorizes detection methods, analyzes recent publications and application trends, and reviews 64 papers published from January 2019 to July 2021. It concludes that no single generic algorithm fits every application and identifies directions for further research.

  • Problem

    IoT anomaly detection spans diverse applications, but the literature has methodological shortages involving multi-sensor integration, data and concept drift, and augmented data with limited ground-truth information.

  • Method

    The survey categorizes anomaly-detection algorithms, extracts keyword trends, searches multiple scholarly databases, and filters recent publications to examine methods and application domains.

  • Results

    The review discusses 64 selected articles from January 2019 to July 2021 and identifies distinct IoT application domains and recent developments.

  • Takeaways & Limitations

    No single best generic algorithm exists; detection methods are specific to particular applications, and several application domains require further research.

  • Takeaways & Limitations

    IoT anomaly detection still needs approaches for multi-representation traffic, drift adaptation, and learning or testing with augmented data when reliable class data are insufficient.

Abstract

from arXiv · show

Ongoing research on anomaly detection for the Internet of Things (IoT) is a rapidly expanding field. This growth necessitates an examination of application trends and current gaps. The vast majority of those publications are in areas such as network and infrastructure security, sensor monitoring, smart home, and smart city applications and are extending into even more sectors. Recent advancements in the field have increased the necessity to study the many IoT anomaly detection applications. This paper begins with a summary of the detection methods and applications, accompanied by a discussion of the categorization of IoT anomaly detection algorithms. We then discuss the current publications to identify distinct application domains, examining papers chosen based on our search criteria. The survey considers 64 papers among recent publications published between January 2019 and July 2021. In recent publications, we observed a shortage of IoT anomaly detection methodologies, for example, when dealing with the integration of systems with various sensors, data and concept drifts, and data augmentation where there is a shortage of Ground Truth data. Finally, we discuss the present such challenges and offer new perspectives where further research is required.

1 Introduction

IoT anomaly detection addresses unusual system states in environments requiring near-real-time processing, but automated detection remains difficult because labels, normal behavior, signal quality, and data heterogeneity vary. This survey updates the literature and examines recent methods, applications, and challenges.

  • IoT anomaly detection identifies unusual system states and supports data cleaning and classification across network and data-center stages.Anomaly detection algorithms act as checkpoints for incoming traffic at multiple stages.
  • Anomalies can provide actionable information in medicine, manufacturing, finance, traffic management, energy, betting, and industrial safety.
  • Automated detection is difficult because anomalous data are hard to define and categorize when labeled training data are partial or unavailable.
  • Changing normal behavior, noisy measurements, low signal-to-noise ratios, interconnected systems, and varied input types increase IoT detection complexity.Household occupancy changes can alter electricity demand, while noise may resemble true anomalies.
  • The survey provides an updated view of recent IoT anomaly-detection developments, algorithm categorization, application domains, and keyword trends.

2 Background and related work

The paper defines anomalies and categorizes IoT detection methods by approach, application, anomaly type, and latency. It situates this framework within prior surveys while emphasizing recent, diverse IoT applications and unresolved methodological needs.

  • Definition and categorization: An anomaly is a data point that deviates significantly from predicted or conventional behavior in a point, context, time slice, or entire dataset.The paper describes anomalies as potentially arising from sensor failure or external attack.
  • Definition and categorization: IoT anomaly-detection methods are categorized by method, application, anomaly type, and algorithm latency.The four-category framework combines classifications from earlier publications.
  • By Method: Geometrical methods classify observations using distance or density thresholds, while statistical methods model normal data with mathematical distributions.A static or dynamic threshold t separates normal observations below the threshold from anomalies above it.
  • By Method: Machine-learning model selection depends on data structure: LSTM and transformer models favor sequential inputs, whereas CNN and autoencoder models favor non-sequential data.
  • By Application, Anomaly Type, and Latency: Anomaly categories include constructive applications, point and contextual anomaly types, and online or later-stage processing distinguished by latency.The passages describe constructive applications such as elderly fall prevention and online algorithms that process individual points or windows.
  • Related work and contribution: Prior IoT anomaly-detection surveys commonly focus on particular data, applications, or methods, whereas this paper examines recent methods, unique applications, and present-day challenges.The review uses a bubble chart and search strategy to address developments from 2019 onward that earlier reviews did not sufficiently cover.

3 Search strategy

The review extracted keyword trends and searched major scholarly databases for recent IoT anomaly-detection research. After filtering for experimental evidence and relevance, it selected 64 articles for discussion.

  • 3.1 Extraction of keyword trends: Keywords from 8,348 unique article titles were extracted, cleaned, and grouped into methodology/platform and application categories.Larger bubbles represented more frequent keywords, while smaller bubbles indicated potential for increased contributions.
  • 3.2 Papers’ selection criteria: The database search collected 2,242 articles published between January 2019 and July 2021 using IoT, anomaly-detection, and trending keywords.Searches covered IEEE Xplore, Elsevier ScienceDirect, MDPI, Google Scholar, and arXiv; duplicates and review publications were removed.
  • 3.2 Papers’ selection criteria: The remaining papers were filtered for experiments, evaluation, and results before abstracts, discussions, and conclusions were examined for relevance.Relevance included notable contributions and unique application domains, with repeated domains narrowed to selected recent and highly ranked papers.
  • 3.2 Papers’ selection criteria: 64 articles published during the previous three years were selected after the filtering process.The review therefore focuses on recent publications rather than the full initial search set.

4 Recent advancements

Recent IoT anomaly-detection applications span network and application data across personal, residential, industrial, and urban settings. The review organizes these applications into sixteen categories and describes methods for time-series and high-dimensional data.

  • 4 Recent advancements: IoT anomaly-detection applications use network data or application data, including streamed time-series data and processed tabular data.Application data comes from sensors serving practical or business uses, while network telemetry includes CPU and memory monitoring for network health.
  • 4 Recent advancements: The reviewed applications cover individual health monitoring, smart homes, manufacturing equipment, sensor-transfer quality, and smart-city traffic monitoring.These applications require data security, user privacy, and reliable data transfer.
  • 4 Recent advancements: The survey combines the applications into sixteen categories based on the two main data types.The categories and associated papers are listed in Table 1.
  • 4.1 IoT anomaly detection applications with time-series data: Hierarchical clustering groups linked sensors so that one lightweight detection method can be applied across each cluster.For general time-series or streaming data, the review also describes robust statistical M-estimators paired with an LSTM neural network.
  • 4.1 IoT anomaly detection applications with time-series data: WAAD applies a discrete wavelet transform and autoencoder reconstruction to detect anomalies in non-stationary, non-periodic univariate time series.It computes reconstruction error for each sliding time window.
  • 4.1 IoT anomaly detection applications with time-series data: CNN- and LSTM-based autoencoder architectures extract spatial and temporal features from sensor data, while HDoutliers assigns probability-based labels to high-dimensional outliers.The reviewed methods also address local and global anomalies, dimensionality, scalability, categorical variables, and non-normal distributions.

4.2 Monitoring the health of machinery

Machinery-health monitoring applies application-data anomaly detection to identify mechanical-condition changes and equipment faults. The reviewed approaches combine NB-IoT sensing, server-side diagnosis, and vibration-signal feature extraction.

  • 4.2 Monitoring the health of machinery: The Mahalanobis–Taguchi method detects changes in machinery mechanical condition and can monitor fatigue in logistics systems.It is presented for application data from IoT sensors attached to industrial machinery.
  • 4.2 Monitoring the health of machinery: Li et al.’s NB-IoT condition-monitoring system sends machinery sensor data to a server that locates anomaly causes and estimates damage.Wavelet packet decomposition extracts features from the vibration signal.

4.3 Robotics or manufacturing applications

Robotics, manufacturing, smart-city, restaurant, and environmental applications use diverse anomaly-detection methods for faults, attacks, sensor defects, and changing conditions. The reviewed systems include adaptive, deep-learning, clustering, and distance-based approaches.

  • 4.3 Robotics or manufacturing applications: A VAE-based real-time system detects spatial and temporal anomalies in industrial robots after injected faults are observed.The model is designed to handle multiple time-series data.
  • 4.3 Robotics or manufacturing applications: IoT-CAD adaptively monitors correlated oxygen, ammonia, nitrate, temperature, and pressure signals to detect environmentally unexpected changes.The system can self-update and identify anomalous nitrate rises in water tanks.
  • 4.3 Robotics or manufacturing applications: Combining edge-device electricity data with weather data creates a sparse anomaly-perception model that labels anomalous data quickly and efficiently.The reviewed applications also address navigation and planning errors affecting connected autonomous vehicles.
  • 4.3 Robotics or manufacturing applications: I-LSTM applies concept-drift adaptation and deep learning to detect and classify anomalies in smart-city data.Its smooth activation function supports multiclass anomaly detection and is described as improving integrated smart-home quality of service.
  • 4.3 Robotics or manufacturing applications: Restaurant edge systems use smart-camera and temperature-meter data to detect sensor defects, security attacks, and increased foot traffic.K-means clustering is followed by Bhattacharyya-distance analysis to discover anomalies.

4.7 Surveillance and video IoT applications

Surveillance and video IoT applications use cameras, sensors, and drones to detect safety and security anomalies, including personnel, smoke, and hostile-environment threats.

  • Deep learning analyzes camera and remote-sensing data to detect personnel and fire smoke around power equipment.The method addresses inadequate monitoring of electrical equipment and its potential economic and societal impacts.
  • READ-IoT monitors heterogeneous IoT objects in hostile environments and automatically detects threats while executing appropriate responses.Its cascaded detection components funnel subsystem outputs into a centralized source for field decisions.
  • A CNN plus bidirectional LSTM pipeline extracts features from successive video frames to distinguish normal from abnormal incidents.CNN features feed a multilayer LSTM for real-world anomaly identification and classification.

4.8 General-purpose frameworks

General-purpose IoT anomaly-detection frameworks combine multiple learning methods or distributed architectures to handle unreliable data, heterogeneous deployments, and privacy-sensitive processing.

  • The RAD Framework combines SVM, KNN, random forest, and nearest centroid classifiers for less reliable IoT data.The authors claim accuracy of up to 98% for attacks on IoT devices and cluster failure prediction.
  • AnoML-IoT integrates wireless protocols, anomaly-detection algorithms, and deployment across edge, fog, and cloud platforms with little user input.Its pipeline covers data intake, training, deployment, inference, and maintenance.
  • An edge-intelligence framework uses sequential clustering and classification to reduce network congestion and centralized-CPU pressure.The design shifts computational work away from centralized servers.
  • An on-device federated-learning framework keeps personal data local while edge devices train a global attention-based neural network.Devices send gradients to a cloud aggregator until convergence.

4.11 Other frameworks

Other IoT anomaly-detection frameworks address electricity-use monitoring, online trend changes, sensor tampering, and infrastructure monitoring across distributed systems.

  • Blockchain-based sensor processing detects electricity-use anomalies and can warn factories and residents to reduce consumption.The framework uses smart-meter readings alongside sensor processing and blockchain.
  • TONTA provides online anomaly and trend-change detection for forwarding nodes in ad hoc IoT networks.The method targets missed unexpected occurrences or departures during time-consuming ML-based network training, although jitters can occur.
  • Security applications dominate IoT anomaly-detection research, spanning devices and infrastructure, data transport, and miscellaneous applications.The survey identifies these three categories because the security literature is too extensive to cover fully.
  • Isolation forests detect sensor tampering from packet length, daily packet-length counts, and hash values in industrial and office environments.The approach evaluates clustering goodness using the Silhouette coefficient.
  • A distributed GNN system monitors network infrastructure, including nodes and connected devices, using the underlying graph structure.A separate PTA-based approach models smart-device activity to protect private home-device data from intruders.

4.14 Security application for IoT data transport

IoT data-transport security frameworks detect cyberattacks and abnormal network behavior using deep representation learning, statistical models, ensembles, feature selection, and geometric methods.

  • An adaptive hybrid strategy integrates deep autoencoders and feature extraction for near-real-time IoT cyberattack detection.The approach targets secure data transport and attacks including man-in-the-middle and trojans.
  • Network data can identify whether unexpected container shutdowns in MLOps reflects a security attack, while DDoS attacks bottleneck networks.The miscellaneous security category covers effects on IoT infrastructure, applications, networks, and ML models.
  • VAE-based models map nonlinear normal and attack data into linear, isolated latent representations.The approach inherits supervised learning strengths to identify known threats to VAEs.
  • A Gaussian Mixture-based Correntropy ensemble one-class model profiles new edge-network flows while targeting zero-day attacks.The protected setting includes routers and switches at airports and gateways.
  • Stacked ensemble meta-learning improves predictions by combining low-accuracy base-classifier outputs in a higher-level model.The design is proposed for IoT device anomaly detection.
  • Ensemble feature selection removes individual-method bias while retaining non-redundant and relevant features for easier debugging and explanation.
  • Geometric Entropy Minimization learns a nominal submanifold and detects persistent deviations from it in sequentially acquired data.The method first extracts univariate summary statistics from high-dimensional observations.

4.16 Miscellaneous IoT applications

Miscellaneous IoT anomaly-detection applications remain at an early stage, including safety monitoring based on data from multiple sensors. The section highlights a selected set of highly cited papers organized by distinct application domains and publication year.

  • Miscellaneous IoT applications: Miscellaneous IoT applications are still in the early stages of development.The reviewed examples include mining-sensor safety warnings and construction safety monitoring for inconsistencies in collected data.
  • Miscellaneous IoT applications: Mining sensors support real-time tracking of safety warnings and construction-safety monitoring.These applications search for inconsistencies in sensor-gathered data.
  • Miscellaneous IoT applications: Fifteen highly cited papers from 2019 through July 2021 are grouped by year and distinct application domain.The selection contains five papers for each year.

5 Conclusions and new perspectives

The review concludes that IoT anomaly detection remains an early-stage field without one generic best algorithm, while identifying research needs in learning with limited labels, multi-representation traffic, drift adaptation, and augmented data. These perspectives span univariate, multivariate, and high-dimensional data.

  • Conclusions and new perspectives: IoT anomaly detection remains early-stage, with no single best generic algorithm across applications.The review instead identifies methods specific to particular applications.
  • Conclusions and new perspectives: Labeled training data are costly, time-consuming, sometimes unavailable, and may not represent all anomalies or class distributions.The review therefore highlights unsupervised, semi-supervised, and self-supervised approaches, especially online methods.
  • Conclusions and new perspectives: Multi-representation traffic requires detection methods that handle data from multiple sensor types at scale with precision and recall.The review gives urban building profiling integrating RGB, hyperspectral, LiDAR, and thermal sensors as an example.
  • Conclusions and new perspectives: Data drift can make incoming traffic differ from training data, motivating proactive adaptation rather than routine retraining alone.The review identifies drift adaptation as a research perspective for changing data distributions.
  • Conclusions and new perspectives: Data augmentation is needed when one or more anomaly classes lack sufficient accurate and reliable samples.The review identifies GANs and other statistical or machine-learning methods as ways to generate additional training samples.
Loading 2207.09092v1…