Source-linked AI summary
Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art
Mohammad Braei, Sebastian Wagner
TL;DR
Because it was unclear which statistical, machine learning, or deep learning approaches perform best for univariate time-series anomaly detection, this paper quantitatively compares 20 methods across multiple datasets. The experiments found that statistical approaches generally achieved the best accuracy for point and collective anomalies while requiring less computation time.
Problem
Broad comparative evidence across statistical, machine learning, and deep learning anomaly detection methods for time-series data remains limited.
Method
The paper quantitatively evaluates 20 statistical, classical machine learning, and deep learning methods on five univariate time-series datasets, using validation to optimize models and hyperparameters.
Results
Statistical approaches performed best on the univariate benchmarks for point and collective anomalies and required less computation time than machine learning and deep learning approaches.
Takeaways & Limitations
For these univariate benchmarks, statistical methods provide the strongest combination of detection accuracy and computational efficiency.
Takeaways & Limitations
Clustering time-series subsequences is constrained because prior work found K-means cluster centers unstable across repeated runs on the same dataset.
Abstract
from arXiv · showhide
Anomaly detection for time-series data has been an important research field for a long time. Seminal work on anomaly detection methods has been focussing on statistical approaches. In recent years an increasing number of machine learning algorithms have been developed to detect anomalies on time-series. Subsequently, researchers tried to improve these techniques using (deep) neural networks. In the light of the increasing number of anomaly detection methods, the body of research lacks a broad comparative evaluation of statistical, machine learning and deep learning methods. This paper studies 20 univariate anomaly detection methods from the all three categories. The evaluation is conducted on publicly available datasets, which serve as benchmarks for time-series anomaly detection. By analyzing the accuracy of each method as well as the computation time of the algorithms, we provide a thorough insight about the performance of these anomaly detection approaches, alongside some general notion of which method is suited for a certain type of data.
1 Introduction
The paper addresses the difficulty of choosing among statistical, machine learning, and deep learning methods for time-series anomaly detection. It compares methods from all three categories quantitatively across multiple time-series datasets.
- The paper focuses on anomaly detection methods for time-series data.
- Deep learning methods, including MLPs, CNNs, and LSTMs, have been proposed for anomaly detection.
- It is unclear which statistical, machine learning, or deep learning category is most appropriate for detecting time-series anomalies.
- The paper quantitatively compares multiple approaches from each category.
- The methods are evaluated on multiple time-series datasets to provide a reliable comparison.
- The paper introduces anomaly-detection concepts, selected approaches, the evaluation approach, and experimental setup.
2 Foundations
This section defines basic concepts that are fundamental to the anomaly-detection process and support understanding of later algorithms.
- The section defines basic concepts for anomaly detection.
- These concepts are fundamental to the anomaly-detection process.
- The concepts support understanding of algorithms introduced in later sections.
2.1 Anomalies and outliers
The paper distinguishes anomalies from noise and novelty while treating outliers and anomalies interchangeably for its time-series evaluation. Anomalies deviate from the general data distribution and form a small minority, but their detectability depends on the dataset.
- The paper reviews multiple historical definitions of outliers and anomalies, reflecting disagreement about their boundaries.
- The paper treats outlier and anomaly as interchangeable terms when evaluating time-series.
- Anomalies constitute only a very small part of the dataset, while most data points are normal.
- Anomalies are observations or sequences that deviate remarkably from the general distribution of data.
- Noise is unwanted variation or attribute error, whereas an anomaly is of interest to the analyst.
- Anomalies may be obvious in noiseless data but difficult to distinguish from other points in sparse, noisy data.
- Novelty patterns are initially unseen points considered normal after detection, but this paper treats novelty and anomaly detection equally.
2.2 Types of Anomalies
Time-series anomalies are commonly categorized as point, collective, or contextual anomalies. Knowing the likely type helps analysts select methods, since some approaches detect point anomalies but miss collective or contextual ones.
- Point anomalies are individual values that differ significantly from surrounding data.
- Collective anomalies are sequences whose combined behavior is anomalous even when individual points appear normal.
- Contextual anomalies are points judged anomalous in one context but normal in another.
- Knowing the likely anomaly type assists analysts in selecting an appropriate detection method.
- Some approaches that detect point anomalies fail to identify collective or contextual anomalies.
2.3 Stochastic Processes and Time-series
The paper defines stochastic processes as time-indexed random variables and time-series as observed sequences over time. It distinguishes univariate from multivariate series and emphasizes temporal dependence in time-series analysis.
- A stochastic process is a family of time-indexed random variables over a sample space and index set.
- A time-series is a sequence of observations collected through continuous measurements over time, generally at equidistant intervals.
- Univariate time-series use one source or dimension, whereas multivariate series combine information from more than one sensor.
- Unlike independent random-variable datasets, time-series observations depend on both component dimensions and temporal position.
2.4 Time-series patterns
Time-series patterns include trend, seasonality, cycles, and stationarity, each describing how statistical characteristics or fluctuations behave over time. These properties are important for the statistical anomaly-detection methods discussed later.
- Trend: A trend occurs when the mean changes over time, either linearly or non-linearly.The cited example has a positive trend from 2005 to 2008 and a negative trend afterwards.
- Seasonality: Seasonality is the periodic recurrence of fluctuations caused by factors such as the time of year or day of the week.The monthly home sales index for 20 major US cities from 2000 to 2019 is presented as seasonal.
- Cycles: Cyclic time-series are influenced by time factors with non-fixed periods longer than a year.The cited example has an approximate 12-year cycle.
- Stationarity: A stationary time-series has the same distributional characteristics over every time interval.Its stated characteristics include constant mean, variance, autocorrelation, and absence of seasonality.
- White noise: White noise is stationary because it is uncorrelated over time and has a fixed distribution with constant mean and finite variance.Its autocorrelation and partial autocorrelation functions are zero.
2.5 Anomaly detection
Anomaly detection identifies rare observations that deviate from expected behavior, with methods tailored to temporal structure, data dimensionality, labeling, and anomaly type. For univariate time-series, prediction-based methods compare forecasts with observations, while other approaches search for unusual shapes.
- Definition and scope: Anomaly detection targets rare data points that deviate markedly from the surrounding data.The literature uses related terms including event detection, novelty detection, change point detection, and fault detection.
- Detection rule: Anomaly scores are converted into binary normal-versus-anomaly labels by marking points above threshold δ as anomalies.The score is denoted γ, and the thresholding rule defines the binary detector.
- Detection challenge: Because anomalies often comprise less than 1% of a dataset, ordinary classifiers can exceed 99% accuracy by labeling every point normal.This class imbalance makes anomaly detection more difficult than ordinary binary classification.
- Data scope: The paper focuses on univariate temporal datasets containing labeled normal and anomalous points.This scope supports discussion of supervised anomaly-detection methods for these data.
- Temporal structure: Temporal dependence means sudden sequence changes can be anomalous even when independent-point methods would identify only separate clusters.The engine-temperature example illustrates a sudden transition from the mid-30s to the high-80s.
- Method families: Univariate time-series detection commonly fits a forecasting model, uses sliding windows for predictions, and scores deviations from observed values.The error e_i = d(x_i, x̂_i) is proportional to the anomaly score; unusual-shape methods provide an alternative.
3 Selected Anomaly detection approaches for time series
The paper groups univariate time-series anomaly detection methods into statistical, classical machine learning, and neural-network approaches. These approaches address issues including non-stationarity, parameter selection, sliding-window representation, and distinguishing anomalies from normal patterns.
- The approaches are divided into statistical, classical machine learning, and neural-network categories.
- Statistical approaches: Statistical methods include AR, MA, ARMA, and ARIMA models, alongside methods from forecasting competitions and recent research.
- Statistical approaches: AR models estimate current values from preceding observations and use the error value to determine the anomaly score.
- Statistical approaches: ARMA model selection must balance overfitting from large p and q values against underfitting from values that are too small.
- Statistical approaches: ARIMA extends ARMA with d, the number of differencing operations used to address non-stationary time-series.Differencing can remove trends and seasonality; non-linear trends may require d > 1.
- Classical machine learning approaches: Sliding-window clustering computes distances from analyzed sequences to cluster representations, but its effectiveness is challenged by evidence that subsequence clustering can resemble random-walk clustering.The method uses a threshold on window error values, with complexity O(kNrw).
- Classical machine learning approaches: Classical machine-learning approaches include sliding-window Isolation Forest, which isolates anomalous points closer to the root of randomly generated trees, and One-Class SVM.One-Class SVM is trained only on normal data and classifies test data by similarity to that normal class.
- Neural networks: Neural-network approaches include fully connected MLPs and CNNs, with CNNs using convolution and pooling layers to focus on local patterns while reducing parameters.In MLP time-series prediction, the lagged input p also represents the sliding-window size and input-layer width.
4 Approach
The paper frames a gap in broad comparisons of statistical, machine-learning, and deep-learning anomaly detection methods for time-series data. It evaluates methods across multiple benchmark datasets using accuracy and computation time.
- Related work: Existing studies often compare methods within one data type, one category, or a single dataset, leaving cross-category evidence limited.
- Datasets: The evaluation uses multiple publicly available time-series datasets selected as anomaly-detection benchmarks.
- Datasets: The selected datasets include Yahoo traffic series, synthetic point-anomaly and seasonal series, changepoint series, and NYC taxi demand.
- Datasets: The NYC taxi dataset contains 10,320 half-hourly observations and five collective anomalies associated with major events and weather.
- Metrics: Evaluation considers F-Score, ROC-AUC, and computation time, reflecting both detection quality and suitability for rapidly arriving data.
5 Experiments
The experiments section establishes the settings needed to evaluate the different anomaly-detection approaches.
- Experimental settings: The experiments list the settings used to evaluate the different anomaly-detection approaches.
5.1 Datasets
The evaluation divides each dataset into training and test data, standardizes the data, and computes AUC on the test portion.
- Data split and evaluation: Datasets are split into 30% training data and 70% test data, with standardized inputs and AUC computed on the test data.
5.2 Experimental Setup
The experimental setup tunes models and forecasting components with validation data, uses a naive forecasting baseline, and specifies implementations and hyperparameters across statistical, machine-learning, and deep-learning methods.
- Training and tuning: A validation set is used to optimize model and forecasting hyperparameters and prevent overfitting.
- Forecasting baseline: The naive model provides a lower-bound baseline, and the target model is tuned toward an NMM below 1.
- Implementation: Implementations use Statsmodels and custom code for statistical methods, Scikit-learn for classical machine learning, and TensorFlow with Keras for deep learning.
- Implementation: All computations run in a single process and single thread on the hardware specified in Table 1.
- Hyperparameters: The setup defines general and method-specific hyperparameters for statistical, machine-learning, and deep-learning approaches.
- Hyperparameters: The machine-learning configurations specify LOF with k=10, isolation forest with 10 trees, and an RBF-kernel one-class SVM.
- Hyperparameters: Deep-learning configurations include MLP, CNN, batch-normalized CNN, and residual CNN architectures trained with Adam and MSE.
6 Results
Across the univariate benchmarks, statistical methods generally achieved the strongest AUC results and lowest computation times, while deep learning methods often performed poorly or required substantial runtime. Dataset anomaly type affected this pattern, with deep learning performing much better on NYCT contextual anomalies.
- 6.1 AUC-Values: Statistical models achieved the best average AUC results across UD1–UD4, while deep learning methods generally performed poorly.Four of the five best-performing algorithms were statistical, whereas four of the five worst-performing algorithms were deep learning approaches.
- 6.1 AUC-Values: NYCT reversed this pattern: deep learning methods performed much better, while statistical approaches achieved very low AUC values.The NYCT anomalies were contextual, unlike the predominantly point or collective anomalies in UD1–UD4.
- 6.1 AUC-Values: Statistical models overfit NYCT data because anomalous values resembled normal values but were anomalous in context.Deep learning approaches were described as more flexible because their hyperparameters can be tuned to the anomaly type.
- 6.2 Computation Time: AR and MA were the fastest algorithms, whereas deep learning methods generally required much more computation time because of training.LSTM and GRU had particularly high runtime, and statistical methods such as SES and ES were not uniformly fast.
- 6.2 Computation Time: On NYCT, most deep learning methods had low inference time, but LSTM and GRU remained unusually slow during inference.Although LSTM achieved the best NYCT results, its inference time was still considered critical.
- 6.3 Computation Time vs AUC-Value: The accuracy–runtime comparison favored statistical methods, especially AR and MA, while residual-block CNN performed poorly on both measures.The preferred region combines high AUC with low computation time.
7 Conclusion and Future Work
The paper compares 20 statistical, classical machine learning, and deep learning methods for univariate anomaly detection. Statistical approaches perform best overall for point and collective anomalies, while some neural networks excel on contextual anomalies despite higher computation time.
- 20 methods spanning statistical, classical machine learning, and deep learning approaches were evaluated on five datasets comprising 368 univariate time series.
- Statistical approaches perform best on univariate time series for detecting point and collective anomalies.
- Statistical approaches require less computation time than classical machine learning and deep learning approaches.
- Neural networks can outperform statistical methods when a univariate dataset mainly consists of contextual anomalies.
- Some deep learning methods achieve higher AUC-rates than statistical approaches but still require more computation time.