Source-linked AI summary
A Long Short-Term Memory Recurrent Neural Network Framework for Network Traffic Matrix Prediction
Abdelhadi Azzouni, Guy Pujolle
TL;DR
The paper addresses prediction of future network traffic matrices from historical data, a task relevant to network planning, resource management, and security. It proposes an LSTM recurrent-neural-network framework with traffic-vector preprocessing and evaluates it on real GEANT data. The reported outcome is quick convergence and state-of-the-art prediction performance for relatively small models.
Problem
Traffic-matrix prediction requires modeling relationships among historical traffic data to estimate future matrices for network operation and management.
Method
The paper uses an LSTM-based RNN framework, transforming each traffic matrix into a vector before prediction and assessing accuracy with mean square error.
Results
LSTM RNNs converge quickly and outperform traditional linear methods and feed-forward neural networks by many orders of magnitude on the reported evaluations.
Takeaways & Limitations
The proposed preprocessing and LSTM feeding technique achieves high prediction accuracy with approximately 60 seconds of computation for one hidden layer of 300 nodes.
Abstract
from arXiv · showhide
Network Traffic Matrix (TM) prediction is defined as the problem of estimating future network traffic from the previous and achieved network traffic data. It is widely used in network planning, resource management and network security. Long Short-Term Memory (LSTM) is a specific recurrent neural network (RNN) architecture that is well-suited to learn from experience to classify, process and predict time series with time lags of unknown size. LSTMs have been shown to model temporal sequences and their long-range dependencies more accurately than conventional RNNs. In this paper, we propose a LSTM RNN framework for predicting short and long term Traffic Matrix (TM) in large networks. By validating our framework on real-world data from GEANT network, we show that our LSTM models converge quickly and give state of the art TM prediction performance for relatively small sized models.
I. INTRODUCTION
Network traffic prediction supports timely network management because traffic is difficult to monitor in real time and exhibits complex temporal structure. The paper proposes an LSTM-based framework for large-scale traffic-matrix prediction, reporting quick convergence and state-of-the-art performance with relatively small models.
- Traffic characteristics: Network traffic exhibits self-similarity, multiscalarity, long-range dependence, and strong nonlinearity.
- Motivation: Accurate, timely traffic matrices support scheduling, routing, capacity planning, accounting, and early network-anomaly detection.Early prediction can support faster congestion identification and mitigation than reactive measurement-based methods.
- Existing approaches: Traditional forecasting includes linear ARMA/ARIMA and Holt-Winters models, while nonlinear approaches commonly use neural networks.Reported experiments indicate that neural-network forecasting can outperform traditional linear models that do not meet accuracy requirements.
- Recurrent models: Recurrent neural networks retain activations across time, making them suited to sequence modeling and time-series prediction.
- Proposed framework: LSTM addresses vanishing and exploding gradients in conventional RNNs and is used here for large-scale traffic-matrix prediction.The framework trains and compares LSTM models with varying parameter counts and configurations.
- Scope: The framework does not estimate traffic matrices from raw measurements; it assumes historical TM data is already accurately obtained.
II. TIME SERIES PREDICTION
The paper reviews linear time-series predictors based on autoregressive, differencing, memory-shortening, and exponential-smoothing techniques. These methods model temporal structure through recursive equations and estimated parameters.
- ARMA: ARMA models represent stationary time series using autoregressive and moving-average terms with white-noise innovations.Model identification includes order selection, mean estimation, coefficient estimation, and noise-variance estimation.
- ARIMA: ARIMA extends autoregressive moving-average modeling with differencing to handle nonstationary series.Its formulation uses polynomial terms, a differencing order d, and a backward-shift operator B.
- ARAR: ARAR shortens long memory through transformations before fitting the transformed series as an autoregressive process.The transformed series is processed until it can be treated as short-memory and stationary.
- Holt-Winters: Holt-Winters uses exponential smoothing recursions to forecast series containing a trend.Its level and slope estimates are updated recursively using smoothing parameters α and β.
- Holt-Winters: Holt-Winters initializes its level and trend estimates from the second and first observations, while α and β may be selected randomly or by minimizing one-step errors.
2) Neural Networks for Time Series Prediction:
Neural networks provide adaptive nonlinear models for time-series prediction, learning parameters from observed data rather than a fixed analytical model. Feed-forward networks pass information forward, whereas training uses supervised error-based parameter adjustment.
- Neural-network modeling: Neural networks can learn complex nonlinear traffic patterns through self-learning and self-adaptive capabilities.
- Architecture: A neural network contains weighted neurons organized into input, hidden, and output layers.
- Feed-forward architecture: Feed-forward networks transmit information only from the input layer toward the output layer.
- Training and prediction: Neural-network prediction has training and test phases, with supervised training adjusting parameters until output error decreases.Backpropagation propagates error backward from the output toward the input while changing the weights.
III. LONG SHORT TERM MEMORY NEURAL NETWORKS
RNNs use recurrent connections and internal states to preserve temporal context, but conventional RNN training suffers from vanishing and exploding gradients. LSTMs address these problems and are designed for sequence prediction with longer-range dependencies.
- RNNs store activations from previous time steps in internal states, supporting temporal memory for sequence modeling.
- Conventional RNNs trained with BPTT face vanishing and exploding gradients that limit long-range dependency modeling to 5–10 time steps.
- LSTMs were designed to address conventional RNN training problems and have been applied to sequence prediction and labeling tasks.
A. LSTM Architecture
An LSTM is organized into memory blocks containing cells with self-connections and gates that regulate information flow. Its forget gate adaptively resets memory, while peephole connections can support precise output timing.
- LSTM memory blocks contain memory cells with self-connections that store the network’s temporal state.
- Input, output, and forget gates regulate activation flow into the cell, out of the cell, and through memory updates.
- The forget gate scales the cell’s internal state before recurrently adding it back, enabling adaptive forgetting or memory resetting.
- Peephole connections link internal cells to gates in the same cell to learn precise output timing.
B. LSTM Equations
The LSTM equations specify forward activation and backward gradient calculations for recurrent hidden layers. They define gate functions, state variables, connectivity, and time-recursive BPTT updates for one or multiple memory blocks.
- The equations describe forward activation and backward gradient calculations for an LSTM hidden layer trained with exact-gradient BPTT.
- The notation defines weights, inputs, post-activation values, memory cells, cell states, and the gate, input, and output activation functions.
- The model notation includes I inputs, K outputs, and H hidden-layer cells, while only cell outputs connect to other blocks.
- For an input sequence of length T, the forward pass proceeds from t = 1 while the backward pass proceeds from t = T toward earlier steps.
- Final weight derivatives are obtained by summing derivatives across timesteps, with the objective function denoted by O.
- The forward-pass equations begin with input gates, using sigmoid, g, and h transformations with the stated output ranges.
IV. TRAFFIC MATRIX PREDICTION USING LSTM RNN
The proposed deep LSTM architecture uses deep learning to extract dynamic network-traffic features and predict future Traffic Matrix values. It is designed to capture dependencies among traffic entries across timeslots.
- A deep LSTM architecture extracts dynamic network-traffic features and predicts future Traffic Matrix values.
- The architecture models mutual dependence among traffic entries across different timeslots.
A. Problem Statement
The paper formulates traffic-matrix prediction as estimating a future matrix from historical matrices while modeling relationships among traffic data over time.
- A traffic matrix records traffic volumes between every origin–destination node pair at a given time.
- Prediction uses historical matrices Y_t−1 through Y_t−T to estimate the future matrix Y_t.
- The central challenge is modeling inherent relationships among historical traffic data accurately enough to predict Y_t.
B. Feeding The LSTM RNN
The framework converts each traffic matrix into a traffic vector and feeds complete historical vectors to the LSTM within a fixed sliding learning window.
- Each traffic matrix Y_t is transformed into a traffic vector X_t by concatenating its rows.The vector has size N × N, and entries can be mapped back to matrix positions.
- Predicting individual traffic components separately assumes OD independence, which prior work showed to be incorrect.
- The framework therefore uses previous traffic from all origin–destination pairs to improve traffic-vector prediction.
- A learning window W limits training to a fixed number of previous time slots when continuously predicting the current vector.This window is introduced to control the computational complexity of ongoing prediction.
- The traffic-over-time matrix M contains W historical vectors ordered by time, with T ≥ W and T/W resulting matrices.
C. Performance Metric
The evaluation uses MSE to assess prediction accuracy on real GEANT traffic data, with experiments varying model architecture and comparing prediction methods.
- Mean Square Error measures prediction accuracy by averaging squared differences between forecasted and actual values.
- The experiments evaluate the method using real traffic data from the GEANT backbone network.GEANT is described as a pan-European research network carrying traffic among national research and education networks.
- The dataset contains traffic matrices sampled at 15-minute intervals, and the study compares the proposed method with PCA.
- The short-term evaluation uses 309 traffic matrices, transformed into 529-dimensional vectors and split into training and validation sets.The training and validation matrices contain 263 and 46 consecutive samples, respectively.
- Figures 6 and 7 vary the numbers of hidden units and hidden layers, while Figure 8 compares prediction methods and reports LSTM superiority.The comparison includes linear predictors and feed-forward neural networks.
VI. RELATED WORK
Prior work compares neural and statistical traffic predictors, while this paper reports that LSTM RNNs achieve accurate traffic-matrix prediction with rapid computation.
- Earlier studies found neural-network traffic prediction superior to traditional linear models such as ARMA, ARAR, and Holt–Winters.
- This work reports that LSTM RNN architectures are well suited to traffic-matrix prediction.
- The proposed preprocessing and RNN feeding technique achieves high prediction accuracy in approximately 60 seconds for one hidden layer of 300 nodes.
- The evaluation reports that LSTM RNNs outperform traditional linear methods and feed-forward neural networks by many orders of magnitude.