Source-linked AI summary
LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection
Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, Gautam Shroff
TL;DR
Unpredictable sensor behavior makes prediction-error anomaly detection difficult. EncDec-AD reconstructs normal multivariate time-series with an LSTM encoder-decoder and detects anomalies from reconstruction errors, working across predictable and unpredictable datasets.
Problem
Uncaptured external factors, loads, and manual controls can make machine sensor time-series difficult to predict, limiting prediction-based anomaly detection.
Method
EncDec-AD trains an LSTM encoder-decoder on normal multivariate sequences, then uses reconstruction errors to assign anomaly scores.
Results
EncDec-AD detects anomalies across predictable, unpredictable, periodic, aperiodic, and quasi-periodic time-series, including sequences from length 30 to 500.
Takeaways & Limitations
Normal-only reconstruction provides a viable anomaly-detection approach for both predictable and unpredictable time-series.
Takeaways & Limitations
The threshold and model size are selected using validation sequences, with β < 1 assumed because anomalous points may be a small fraction of anomalous-labeled sequences.
Abstract
from arXiv · showhide
Mechanical devices such as engines, vehicles, aircrafts, etc., are typically instrumented with numerous sensors to capture the behavior and health of the machine. However, there are often external factors or variables which are not captured by sensors leading to time-series which are inherently unpredictable. For instance, manual controls and/or unmonitored environmental conditions or load may lead to inherently unpredictable time-series. Detecting anomalies in such scenarios becomes challenging using standard approaches based on mathematical models that rely on stationarity, or prediction models that utilize prediction errors to detect anomalies. We propose a Long Short Term Memory Networks based Encoder-Decoder scheme for Anomaly Detection (EncDec-AD) that learns to reconstruct 'normal' time-series behavior, and thereafter uses reconstruction error to detect anomalies. We experiment with three publicly available quasi predictable time-series datasets: power demand, space shuttle, and ECG, and two real-world engine datasets with both predictive and unpredictable behavior. We show that EncDec-AD is robust and can detect anomalies from predictable, unpredictable, periodic, aperiodic, and quasi-periodic time-series. Further, we show that EncDec-AD is able to detect anomalies from short time-series (length as small as 30) as well as long time-series (length as large as 500).
1. Introduction
Uncaptured external factors and manual controls can make machine sensor time-series difficult to predict, weakening prediction-error anomaly detectors. EncDec-AD instead reconstructs normal multi-sensor sequences and uses reconstruction error to identify anomalies, including when anomalous data is unavailable or sparse.
- Motivation: Unmonitored load, environmental conditions, and manual controls can make machine behavior inherently difficult to predict from sensor data.A machine’s load may be unknown or change abruptly, and some manual controls may not be captured.
- Motivation: Prediction-based anomaly detectors become ineffective when near-future sensor values cannot be predicted reliably.The paper specifically cites EWMA, SVR, and LSTM-based prediction models as affected approaches.
- EncDec-AD: EncDec-AD trains an LSTM encoder-decoder on normal sequences to reconstruct the input time-series and uses reconstruction error to estimate anomaly likelihood.The decoder reconstructs the time-series, while higher reconstruction error indicates a possible anomaly.
- EncDec-AD: Training only on normal sequences is useful when anomalous data is unavailable or sparse, as in machines serviced before anomalies appear in sensor readings.This setting makes supervised classification over normal and anomalous sequences difficult.
2. EncDec-AD
EncDec-AD reconstructs multivariate normal windows with an LSTM encoder-decoder, then converts reconstruction errors into point-level anomaly likelihoods. The method reverses decoder targets during training and supports supervised threshold selection when labeled anomalous sequences are available.
- EncDec-AD: Each time-series point is an m-dimensional sensor-reading vector, and the model trains on normal windows before assigning anomaly scores to test points.A higher anomaly score indicates a higher likelihood that the point is anomalous.
- LSTM Encoder-Decoder as reconstruction model: The encoder maps the input sequence to a fixed-length representation, while the decoder reconstructs the sequence from that representation.The encoder’s final hidden state initializes the decoder, and a linear layer predicts the target values.
- LSTM Encoder-Decoder as reconstruction model: During training, the target sequence is presented in reverse order, whereas inference feeds each predicted value back into the decoder for the next prediction.Figure 2 illustrates this reconstruction process for a sequence with L = 3.
- Computing likelihood of anomaly: Reconstruction-error vectors from validation normal sequences estimate a Normal distribution used to compute anomaly likelihoods.The method partitions normal data for training and early stopping, then estimates distribution parameters from validation errors.
- Computing likelihood of anomaly: When anomalous sequences are available, a threshold τ is selected on validation data to maximize Fβ, and points above τ are labeled anomalous.The paper also labels an entire window anomalous when it contains an anomalous pattern.
3. Experiments
The experiments evaluate EncDec-AD across public and proprietary datasets spanning periodic, quasi-periodic, quasi-predictable, and unpredictable behavior. The method detects anomalies across these settings, while prediction-based LSTM-AD performs better on predictable datasets and EncDec-AD performs better on Engine-NP.
- Datasets: The evaluation covers power demand, space shuttle valve, ECG, and two engine datasets, including quasi-predictable Engine-P and unpredictable Engine-NP.The first three datasets are public, while the engine dataset is proprietary and represents two applications.
- Experimental Setup: The experiments use single-hidden-layer encoder and decoder architectures with c LSTM units, trained using mini-batch Adam optimization.Table 2 reports EncDec-AD performance across all datasets.
- Visualization: Figure 3 compares normal and anomalous original sequences with reconstructed sequences and log-scale anomaly scores.Blue denotes original sequences, green reconstructed sequences, and red anomaly scores; red regions mark anomaly locations when available.
- Observations: EncDec-AD detects anomalies across periodic, quasi-periodic, and varying-window scenarios, including windows equal to and longer than one cycle.The study evaluates power demand, space shuttle, and ECG time-series under these conditions.
4. Related Work
Related work includes prediction-based anomaly detection using prediction errors and non-temporal reconstruction models. EncDec-AD instead reconstructs full time-series from sequence representations using an LSTM encoder-decoder.
- Prediction-Based Detection: Prediction-based anomaly detection models use prediction error or a function of prediction error to measure anomaly severity.Deep LSTM prediction models learn from normal time-series and predict future points.
- Reconstruction-Based Detection: EncDec-AD learns a representation from the entire sequence and reconstructs that sequence, distinguishing it from prediction-based models.The paper presents the LSTM encoder-decoder as a temporal extension of reconstruction approaches such as denoising autoencoders and Deep Belief Nets.
5. Discussion
The discussion presents normal-sequence reconstruction with an LSTM encoder-decoder as a viable anomaly-detection approach. It reports effectiveness on both predictable and unpredictable time-series, including sequences as long as 500.
- Robustness: EncDec-AD is shown to detect anomalies in both predictable and unpredictable time-series, unlike approaches relying on predictability.The authors characterize it as potentially more robust than such models within the evaluated settings.
- Sequence Length: EncDec-AD detects anomalies in time-series with lengths as large as 500.The authors interpret this result as evidence that the LSTM encoder-decoder learns a robust model of normal behavior.