Source-linked AI summary
Collective Anomaly Detection based on Long Short Term Memory Recurrent Neural Network
Loic Bontemps, Van Loi Cao, James McDermott, Nhien-An Le-Khac
TL;DR
Network anomaly detectors often overlook relationships among recent events, limiting detection of attacks expressed as collective anomalies. The paper trains an LSTM RNN on normal time-series data and uses recent thresholded prediction errors for real-time detection. Experiments on the time-series KDD 1999 dataset indicate efficient collective-anomaly detection, with a trade-off between detection rate and false alarms.
Problem
Existing anomaly-detection approaches often evaluate points separately without incorporating previous events, although some attacks occur as collections of related points.
Method
The model trains an LSTM RNN on normal time-series data, predicts each time step, and evaluates recent prediction errors in a circular array against thresholds.
Results
100% collective anomaly detection is possible, but reducing false alarms results in fewer correctly detected alarms.
Takeaways & Limitations
The experiments suggest the model can efficiently detect collective anomalies in the time-series KDD 1999 dataset.
Takeaways & Limitations
The training data must be organized coherently to guarantee system stability.
Abstract
from arXiv · showhide
Intrusion detection for computer network systems becomes one of the most critical tasks for network administrators today. It has an important role for organizations, governments and our society due to its valuable resources on computer networks. Traditional misuse detection strategies are unable to detect new and unknown intrusion. Besides, anomaly detection in network security is aim to distinguish between illegal or malicious events and normal behavior of network systems. Anomaly detection can be considered as a classification problem where it builds models of normal network behavior, which it uses to detect new patterns that significantly deviate from the model. Most of the cur- rent research on anomaly detection is based on the learning of normally and anomaly behaviors. They do not take into account the previous, re- cent events to detect the new incoming one. In this paper, we propose a real time collective anomaly detection model based on neural network learning and feature operating. Normally a Long Short Term Memory Recurrent Neural Network (LSTM RNN) is trained only on normal data and it is capable of predicting several time steps ahead of an input. In our approach, a LSTM RNN is trained with normal time series data before performing a live prediction for each time step. Instead of considering each time step separately, the observation of prediction errors from a certain number of time steps is now proposed as a new idea for detecting collective anomalies. The prediction errors from a number of the latest time steps above a threshold will indicate a collective anomaly. The model is built on a time series version of the KDD 1999 dataset. The experiments demonstrate that it is possible to offer reliable and efficient for collective anomaly detection.
1 Introduction
The paper frames network anomaly detection as identifying malicious behavior against expected network behavior, while highlighting that point-wise methods miss attacks expressed collectively over time. It proposes an LSTM RNN model that uses recent prediction errors to detect such collective anomalies.
- Network anomaly detection identifies illegal or malicious activities by comparing them with normal or expected network behavior.
- Existing approaches often treat anomalies as isolated points and do not represent previous events when evaluating the current point.
- Collective anomalies consist of related instances whose combination indicates an anomaly even when individual points do not.
- The proposed LSTM RNN predicts current events using current and previous events, then stores recent prediction errors in a circular array.
- Prediction errors exceeding a threshold for a specified duration indicate a collective anomaly.
2 Related Work
Related work includes probabilistic aggregation, LSTM prediction, and LSTM-based denoising for anomaly detection. The paper combines LSTM time-series prediction with standardized errors and group-level scoring for network collective anomalies.
- Existing collective-anomaly work probabilistically aggregates individual anomaly contributions into a group score under unsupervised learning.
- Malhotra et al. trained a stacked LSTM on normal data to predict multiple time steps and assessed anomalies from Gaussian prediction-error likelihoods.
- Marchi et al. combined recurrent denoising autoencoders with LSTM prediction, using reconstruction errors to identify abnormal acoustic signals.
- This paper adapts LSTM prediction to collective anomaly detection by labeling outputs at each time step with standardized error values.
- The proposed application trains on normal traffic behavior and raises alarms for DoS Neptune attacks in the KDD 1999 dataset.
3 Preliminaries
The preliminaries describe LSTM as a recurrent architecture for time-series prediction whose gated cells regulate information flow. Its temporal learning uses evolving prediction errors to adapt weights and represent time-dependent behavior.
- LSTM was introduced as a recurrent neural network technique for time-series prediction.
- Each LSTM cell contains input, forget, and output gates that decide whether information is blocked or passed onward.
- LSTM cells support backpropagation through time, allowing the network to trace evolving prediction error and update weights through gradient descent.
- Training on normal data lets LSTM represent data variation while making predictions based on both sample value and its position in time.
4 Proposed Approach
The approach adapts an LSTM RNN for collective anomaly detection by aggregating recent prediction errors in a circular array and evaluating their density and average magnitude.
- LSTM RNN time-series model: The model uses an LSTM RNN to learn normal time-series behavior and predict coherent outputs using current samples and their temporal context.The paper uses a simple LSTM RNN and describes its ability to remember sample context through prior events.
- Collective anomaly criteria: Danger Coefficient measures the density of anomalous points within the last M_AT time steps, with 0 < DC < 1.N denotes the number of anomalous points in that window.
- Error measures: Prediction errors are measured at each time step as Relative Error, producing a live Relative Error time series.The predicted sample is compared with the real future sample, and the resulting error is used to initialize the Relative Error Threshold from validation data.
- Collective anomaly representation: The circular array stores the M_AT latest error values so the system can evaluate an ensemble of recent points rather than isolated anomalies.This converts point-level error observations into a representation of recent anomaly behavior for collective detection.
- Collective anomaly criteria: A collective anomaly is identified by jointly analyzing Danger Coefficient and Average Relative Error over the recent time-step window.Their values are estimated using a validation set, alongside the model’s error threshold.
- Model operation: The method assumes that training, thresholds, and network parameters are adjusted to obtain satisfactory error decrease and reliable collective-anomaly decisions.The paper states that the method should determine whether a set of points represents a collective anomaly after training and adjustment.
5 Experiments
The experiments convert KDD 1999 network records into time-series data, tune the LSTM RNN and thresholds, and evaluate collective-anomaly detection on normal and attack data.
- Dataset: Experiments use a time-series version of the KDD 1999 dataset, focusing on the Neptune Denial-of-Service attack.The source data was collected in a simulated military-like environment over five weeks.
- Dataset construction: Tcpdump records are time-filtered into pcap files, converted to CSV with tshark, and reduced to relevant fields for classifier processing.The conversion removes irrelevant pcap columns and supports faster computation and system efficiency.
- Dataset construction: The conversion scheme filters and counts attack-relevant packets at each time step, using network-flow characteristics to seek Neptune attacks.The paper describes Neptune and Smurf as DoS attacks characterized by high flows of specific packets.
- Dataset: The dataset is divided into two weeks of normal training data, one validation week with labeled normal and anomalous data, and two testing weeks.The model is trained on n1, thresholds are determined with v1, and performance is evaluated on t1.
- Preliminary experiment: The preliminary experiment selects input history, hidden size, learning rate, and thresholds α and β using prediction error and validation performance.The thresholds are estimated using labeled Neptune attacks and normal data in the validation set.
- Parameter selection: The study notes that increasing inputs may reduce prediction errors but can increase computation time, while hidden-node count can increase information capacity and risk over-fitting.The preliminary analysis examines the trade-off between input history, prediction error, and computation.
- Preliminary experiment: Three inputs produce lower training error than one or two inputs and require less computational time, so this configuration is used in the main experiment.The main experiment trains the selected model on normal n1 data and evaluates it on t1 containing normal and attack data.
6 Results and Discussion
The experiments compare LSTM input configurations and evaluate circular-array collective anomaly detection. Three inputs had lower computational time and training error, while detection performance involved a trade-off between real attacks detected and false alarms.
- The experiments evaluate computation cost, prediction error, and classification accuracy for LSTM models using one, two, and three inputs.
- Three-input models had lower computational time than one- and two-input models and were selected for the main experiment.
- Three-input models achieved lower training error than the other two configurations.
- The main experiment used circular-array detection with M_AT = 12, α = 0.66, and β values of 0.69, 0.66, 0.62, and 0.52.
- 100% collective anomaly detection was possible, but detecting more real attacks resulted in more false alarms.
7 Conclusion and Further work
The paper concludes that its LSTM RNN model can detect collective anomalies on a time-series KDD 1999 dataset. It emphasizes coherent training data and continued work to improve classification accuracy.
- The paper proposes a collective anomaly detection model based on LSTM RNN and investigates inputs, hyperparameters, and thresholds using validation data.
- The model was evaluated on the time-series version of the KDD 1999 dataset and was reported as efficiently capable of detecting collective anomalies.
- Stable system behavior requires training data to be organized coherently.
- Future work focuses on improving the model's classification accuracy and examining how different numbers of LSTM inputs affect outputs.