Source-linked AI summary
Forecasting Economics and Financial Time Series: ARIMA vs. LSTM
Sima Siami-Namini, Akbar Siami Namin
TL;DR
The paper asks how accurate newly introduced forecasting approaches are compared with traditional methods and compares ARIMA with LSTM. It reports that LSTM outperforms ARIMA, reducing average error rates by 84 - 87 percent.
Problem
The paper investigates the accuracy of traditional forecasting methods compared with newly introduced approaches.
Method
The study compares ARIMA and LSTM as representative forecasting techniques.
Results
84 - 87 percent average error-rate reduction was obtained by LSTM compared with ARIMA.
Takeaways & Limitations
LSTM outperforms ARIMA in the reported forecasting comparison.
Takeaways & Limitations
Remembering longer sequences of data is identified as a challenging problem addressed using an introduced “memory line.”
Abstract
from arXiv · showhide
Forecasting time series data is an important subject in economics, business, and finance. Traditionally, there are several techniques to effectively forecast the next lag of time series data such as univariate Autoregressive (AR), univariate Moving Average (MA), Simple Exponential Smoothing (SES), and more notably Autoregressive Integrated Moving Average (ARIMA) with its many variations. In particular, ARIMA model has demonstrated its outperformance in precision and accuracy of predicting the next lags of time series. With the recent advancement in computational power of computers and more importantly developing more advanced machine learning algorithms and approaches such as deep learning, new algorithms are developed to forecast time series data. The research question investigated in this article is that whether and how the newly developed deep learning-based algorithms for forecasting time series data, such as "Long Short-Term Memory (LSTM)", are superior to the traditional algorithms. The empirical studies conducted and reported in this article show that deep learning-based algorithms such as LSTM outperform traditional-based algorithms such as ARIMA model. More specifically, the average reduction in error rates obtained by LSTM is between 84 - 87 percent when compared to ARIMA indicating the superiority of LSTM to ARIMA. Furthermore, it was noticed that the number of training times, known as "epoch" in deep learning, has no effect on the performance of the trained forecast model and it exhibits a truly random behavior.
2. Associate Professor, Department of Computer Science
The paper focuses on forecasting economic and financial time series using deep learning and ARIMA, evaluated with RMSE.
- The paper addresses forecasting economic and financial time series data.
- Deep learning and Long Short-Term Memory (LSTM) are central methodological topics.
- Autoregressive Integrated Moving Average (ARIMA) and Rooted Mean Square Error (RMSE) are also central terms.
1 Introduction
The introduction frames economic and financial forecasting as difficult and compares traditional methods with deep learning approaches. It presents an empirical ARIMA–LSTM comparison focused on forecast error reduction, reporting stronger LSTM performance and no observed epoch effect.
- Economic and financial time-series forecasting is challenging because of unprecedented changes, incomplete information, volatility, and complex trends.
- The introduction asks whether traditional forecasting techniques are as accurate and precise as deep learning-based algorithms for economic and financial time series.
- The paper conducts an empirical comparison of ARIMA and LSTM, representing traditional and deep learning-based forecasting approaches.
- ARIMA is selected for non-stationary data, while LSTM is selected for preserving and training data features over longer periods.
- 84 - 87 percent average error-rate reduction is reported for LSTM compared with ARIMA.
- The study investigates training epochs and reports no effect on forecast-model performance, with epoch behavior described as truly random.
2 Time Series Forecasting: The State-of-the-Art
Time-series forecasting is widely used across applied fields, with ARIMA serving as a longstanding econometric standard. The section contrasts ARIMA's limitations with emerging deep-learning approaches, especially LSTM, and motivates direct performance comparison.
- Time-series analysis studies observed paths to describe data structure and predict future values across economics, business, and finance.
- ARIMA has long been a standard and prevalent method for forecasting economic and financial time series.
- Simple ARIMA models have difficulty modeling nonlinear relationships between variables and assume constant error variance, which may not hold in practice.
- GARCH can relax the constant-error-variance assumption, but optimizing its parameters may be challenging and problematic.
- Deep-learning techniques such as LSTM were developed to address forecasting challenges, and this article compares LSTM with ARIMA for economic and financial time series.
- Prior evidence is mixed: one study reported deep learning underperformed gradient-boosted trees and random forests, while also describing neural-network training as difficult.
3 Mathematical Background
This section introduces ARIMA as a model combining autoregressive, differencing, and moving-average components, alongside neural-network foundations and LSTM's mechanisms for retaining sequential information.
- ARIMA: ARIMA combines autoregressive and moving-average processes, using differencing to convert non-stationary time series into stationary series.Its general form is denoted ARIMA p, d, q.
- ARIMA: The ARIMA notation p, d, q represents autoregressive order, differencing order, and moving-average order, respectively.Seasonal ARIMA additionally specifies P, D, Q, and S for seasonal orders, differencing, and periodic span.
- ARMA components: Autoregressive models use lagged observations, while moving-average models use prior stochastic-error terms to represent temporal dependence.The article presents AR, MA, and combined ARMA formulations as mathematical foundations for ARIMA.
- Artificial neural networks: Neural networks learn by adjusting connection weights through backpropagation over repeated epochs to reduce prediction cost.The network includes input, hidden, and output layers, with activation functions transforming weighted inputs.
- RNN and LSTM: RNNs use sequential observations and internal hidden-layer storage to forecast future steps, but typical RNNs retain only a few earlier steps.LSTM addresses this limitation through a memory line and gates that regulate information flow.
- RNN and LSTM: LSTM is a recurrent network with additional cells and gates that can memorize, filter, add, or dispose of sequential data.Sigmoid layers produce values between zero and one indicating how much information passes through each cell.
5 Algorithms
The algorithms use rolling, one-step-ahead forecasting with expanding histories and RMSE evaluation, comparing a rolling ARIMA baseline with a rolling LSTM model.
- Rolling forecasting: Rolling forecasting predicts the next data point using training sets that expand by one observation at each iteration.The approach is also called walk-forward model validation and is used when prior time steps are dependent.
- Rolling ARIMA: The rolling ARIMA procedure splits each series into 70% training and 30% testing data, then refits ARIMA(5,1,0) before each forecast.The model uses five autoregressive lags, first-order differencing, and no moving-average window.
- Rolling ARIMA: ARIMA(5,1,0) serves as the forecasting baseline, although the authors note it may not be optimal.Each forecasted value is compared with the observed test value, and RMSE is calculated for evaluation.
- Rolling LSTM: The LSTM is trained with a specified number of epochs and neurons, then produces one-step forecasts during walk-forward validation.The reported evaluation metric is RMSE, computed from mean squared error between expected and predicted values.
6 Results
Across the reported financial and economic time series, Rolling LSTM achieves substantially lower average RMSE than Rolling ARIMA, with error reductions between 84% and 87%.
- Financial time series: 87.445% reduction in error rates is achieved by LSTM for financial time series, whose average RMSE falls from 511.481 with ARIMA to 64.213.The comparison uses average RMSE for Rolling ARIMA and Rolling LSTM models.
- Economic data: 84.394% reduction in RMSE is achieved for economic data, where average RMSE declines from 5.999 with Rolling ARIMA to 0.936 with Rolling LSTM.These are the reported average RMSE values for the two rolling models.
- Overall comparison: LSTM-based models outperform ARIMA-based models by a high margin, with reported error reductions between 84% and 87%.The conclusion is based on the RMSE comparisons across the financial and economic datasets.
7 Discussion: The Impact of the Number of Iterations on Fitting Models
The discussion attributes deep-learning performance to iterative optimization intended to minimize errors, while reporting that iteration count has no consistent effect on forecast-model performance.
- Impact of iterations: Iterative optimization repeatedly obtains results and selects the iteration that minimizes errors.The authors describe this process as transforming an under-fitted model into one optimally fitted to the data.
1. Monthly; 2. Weekly
This section examines whether repeatedly training LSTM models on the same economic and financial time-series data improves forecast accuracy. Experiments varying epochs from 1 to 100 found no consistent benefit, with additional training sometimes worsening performance through overfitting.
- Epochs represent the number of times an entire dataset is passed through a model during training.One epoch passes the dataset forward and backward through the network once.
- The study focuses on how the number of training rounds, or epochs, affects forecasting on the same data.
- The experiments varied epoch values from 1 to 100 for each dataset and measured forecast error rates.The upper limit was selected for practical feasibility.
- There is no evidence that training the network with the same dataset more than once improves prediction accuracy.The sensitivity analysis covered both financial and economic time series.
- In some cases, additional training worsens performance, indicating that the trained models are over-fitted.
- Setting epoch = 1 generates a reasonable prediction model, so further training on the same data is unnecessary.
8 Conclusion
The conclusion compares LSTM with ARIMA for forecasting economic and financial time series. The reported results show that LSTM was superior, while changing the number of epochs produced no improvement; the authors propose extending evaluation to other problems and datasets.
- The paper compares ARIMA and LSTM as representative traditional and deep learning techniques for time-series forecasting.
- LSTM was superior to ARIMA in the reported forecasting experiments.The techniques were applied to financial data and economic and financial time series.
- 85% average prediction improvement was reported for LSTM compared with ARIMA.
- Changing the number of epochs produced no improvement in prediction performance.
- The authors advocate applying deep learning-based algorithms and techniques to economic and financial forecasting.
- The authors plan to investigate deep learning on other problems and datasets with various numbers of features.