Source-linked AI summary
Stock Price Prediction Using Machine Learning and LSTM-Based Deep Learning Models
Sidra Mehtab, Jaydip Sen, Abhishek Dutta
TL;DR
The paper investigates whether historical NIFTY 50 data can support accurate stock-price forecasting. It builds machine-learning regressors and LSTM models with walk-forward validation, finding that a univariate LSTM using one-week prior data is the most accurate model.
Problem
The paper addresses the challenge of accurately forecasting highly dynamic and fast-changing stock-price movements from historical data.
Method
The study builds eight machine-learning regression models and four LSTM-based regression models with different architectures and input structures for NIFTY 50 open-value forecasting.
Results
The univariate LSTM model using one-week prior data was the most accurate and optimum in both forecasting accuracy and execution time.
Takeaways & Limitations
LSTM-based deep-learning regression models were found to outperform the machine-learning-based predictive models on the studied NIFTY 50 forecasting task.
Abstract
from arXiv · showhide
Prediction of stock prices has been an important area of research for a long time. While supporters of the efficient market hypothesis believe that it is impossible to predict stock prices accurately, there are formal propositions demonstrating that accurate modeling and designing of appropriate variables may lead to models using which stock prices and stock price movement patterns can be very accurately predicted. In this work, we propose an approach of hybrid modeling for stock price prediction building different machine learning and deep learning-based models. For the purpose of our study, we have used NIFTY 50 index values of the National Stock Exchange (NSE) of India, during the period December 29, 2014 till July 31, 2020. We have built eight regression models using the training data that consisted of NIFTY 50 index records during December 29, 2014 till December 28, 2018. Using these regression models, we predicted the open values of NIFTY 50 for the period December 31, 2018 till July 31, 2020. We, then, augment the predictive power of our forecasting framework by building four deep learning-based regression models using long-and short-term memory (LSTM) networks with a novel approach of walk-forward validation. We exploit the power of LSTM regression models in forecasting the future NIFTY 50 open values using four different models that differ in their architecture and in the structure of their input data. Extensive results are presented on various metrics for the all the regression models. The results clearly indicate that the LSTM-based univariate model that uses one-week prior data as input for predicting the next week open value of the NIFTY 50 time series is the most accurate model.
1 Introduction
The paper situates stock-price prediction amid debate over predictability and reviews indicator- and learning-based approaches. It proposes combining machine-learning and LSTM-based models to forecast NIFTY 50 open values.
- Stock-price prediction remains contested, with the efficient market hypothesis challenging accurate prediction while formal propositions support suitable variables and modeling.
- Technical-analysis research uses indicators such as Bollinger Band, MACD, RSI, moving average, momentum stochastics, and meta sine wave to detect stock-movement patterns.
- The study trains models on NIFTY 50 data from December 29, 2014 to December 28, 2018 and predicts open values from December 31, 2018 to July 31, 2020.
- Four LSTM models vary in architecture and input structure, including three univariate models and one multivariate model.
- The paper combines machine-learning and deep-learning predictive models within a forecasting framework for NIFTY 50 index values.
2 Problem Statement
The paper seeks to learn from historical daily NIFTY 50 movement patterns and forecast future index values over distinct horizons for machine-learning and deep-learning models.
- The study collects NIFTY 50 data over five and a half years to develop a forecasting framework for future index values.
- It hypothesizes that models can learn features of past daily NIFTY 50 movement patterns and exploit them to forecast future index values.
- The chosen forecast horizons are one year for machine-learning models and one week for deep-learning models.
3 Related Work
The related work is organized around variable choice and modeling approach, highlighting limitations in conventional forecasting and motivating learning-based models for dynamic stock movements.
- Existing stock-price forecasting work is categorized into three clusters according to the variables used and the modeling approach.
- Bivariate and multivariate regression models using cross-sectional data are described as simple and dependent on linearity assumptions.
- A major reported shortcoming is difficulty accurately predicting highly dynamic and fast-changing stock-price movement patterns.
- The paper addresses this problem by exploiting machine-learning and deep-learning models, particularly an LSTM network, for future stock-index prediction.
4 Methodology
The study preprocesses daily NIFTY 50 records, trains eight machine-learning regressors, and evaluates them alongside four LSTM models using walk-forward validation. The models forecast open values from historical data using univariate or multivariate inputs and different architectures.
- Data and preprocessing: Daily NIFTY 50 records from December 29, 2014 to July 31, 2020 provide the historical data for model construction and evaluation.The training period ends December 28, 2018, while testing covers December 31, 2018 to July 31, 2020.
- Data and preprocessing: The regression models use open as the response variable and other index variables as predictors after preprocessing and transformation.Derived variables include normalized high, low, close, volume, and range measures.
- Machine-learning models: Eight regression models are evaluated using product-moment correlation and RMSE divided by the mean actual open value.Higher correlation and lower normalized RMSE indicate greater accuracy.
- LSTM models: The deep-learning component uses LSTM networks to address recurrent-network gradient problems and strengthen the forecasting framework.LSTM models are applied to the complex multivariate NIFTY 50 time series.
- Validation and forecasting: Walk-forward validation predicts one week at a time, then feeds the actual week into the model before forecasting the next week.Each forecast covers five working-day open values.
- LSTM models: Four LSTM configurations vary by architecture and input structure, including one- or two-week univariate inputs and two-week multivariate encoder-decoder inputs.The configurations include standard LSTM and encoder-decoder LSTM models.
5 Performance Results
The study evaluates machine-learning and LSTM regression models using correlation, normalized RMSE, execution time, and weekly or daily RMSE measures. Multivariate regression and random forest lead the machine-learning results, while the one-week univariate LSTM is the most accurate and efficient deep-learning model.
- Machine-learning models: Two metrics evaluate machine-learning regressors: correlation between actual and predicted open values and RMSE divided by mean actual open values.
- Machine-learning models: Multivariate regression, MARS, and random forest achieve the strongest test correlation coefficients, while multivariate regression and random forest have the lowest normalized RMSE ratios.
- LSTM models: 18.64s average execution time and a 0.0311 RMSE-to-mean-open ratio characterize LSTM#1, whose mean RMSE increases from Monday through Friday.
- LSTM models: 31.44s average execution time and a 0.0353 RMSE-to-mean-open ratio make LSTM#2 inferior to LSTM#1 on both evaluation measures.
- LSTM models: 14.53s execution time and a 0.0369 RMSE-to-mean-open ratio make LSTM#3 slightly faster but less accurate than LSTM#1.
- LSTM models: LSTM#4 takes almost five times longer than LSTM#3 and has a much higher normalized RMSE, whereas the one-week univariate LSTM is optimal for accuracy and execution time.
6 Conclusion
The study compares eight machine-learning and four LSTM-based regression models for weekly NIFTY 50 forecasting. LSTM models outperform the machine-learning models, while univariate LSTM models are more accurate and faster than multivariate variants.
- Eight machine-learning and four LSTM-based deep-learning regression models were constructed, optimized, and tested using daily NIFTY 50 data.The models target stock-index values and movement patterns on a weekly forecast horizon.
- LSTM-based deep-learning regression models performed far better than the machine-learning-based predictive models.
- The study concludes that deep-learning models have greater capability than corresponding machine-learning models for extracting and learning time-series features.
- Univariate LSTM regression models were more accurate and faster to execute than multivariate LSTM models.