PaperlayerView exact revision on arXiv

Source-linked AI summary

Impact of Data Normalization on Deep Neural Network for Time Series Forecasting

Samit Bhanja, Abhishek Das

arXiv:1812.05519v2cs.LGstat.ML

TL;DR

Nonlinear, highly dynamic time-series data make forecasting challenging, and the paper examines whether normalization affects DRNN prediction. Across BSE and NYSE stock-index data, the authors report lower MSE and MAE with Tanh Estimator normalization than with the other tested methods.

  • Problem

    Stock-market time series are nonlinear and highly dynamic, while normalization is considered important to learning from data with wide-ranging values.

    Source: §6 · p. 3

  • Method

    The paper scales BSE and NYSE data using different normalization techniques and evaluates a DRNN for training, validation, and testing.

    Source: §6 · p. 3, §7 · p. 4

  • Results

    Tanh Estimator normalization produced lower MSE and MAE than the other tested methods for BSE and NYSE stock-index prediction.

    Source: §8 · p. 4

  • Takeaways & Limitations

    The authors conclude that Tanh Estimator is an effective normalization method for time-series prediction with a DRNN.

    Source: §8 · p. 4

Abstract

from arXiv · show

For the last few years it has been observed that the Deep Neural Networks (DNNs) has achieved an excellent success in image classification, speech recognition. But DNNs are suffer great deal of challenges for time series forecasting because most of the time series data are nonlinear in nature and highly dynamic in behaviour. The time series forecasting has a great impact on our socio-economic environment. Hence, to deal with these challenges its need to be redefined the DNN model and keeping this in mind, data pre-processing, network architecture and network parameters are need to be consider before feeding the data into DNN models. Data normalization is the basic data pre-processing technique form which learning is to be done. The effectiveness of time series forecasting is heavily depend on the data normalization technique. In this paper, different normalization methods are used on time series data before feeding the data into the DNN model and we try to find out the impact of each normalization technique on DNN to forecast the time series. Here the Deep Recurrent Neural Network (DRNN) is used to predict the closing index of Bombay Stock Exchange (BSE) and New York Stock Exchange (NYSE) by using BSE and NYSE time series data.

1 INTRODUCTION

Stock-market forecasting matters to investors and regulators, but dynamic markets and nonlinear data make prediction challenging. The paper evaluates data normalization for stock-index forecasting with a deep recurrent neural network.

1 INTRODUCTION

  • Stock-market prediction supports investor guidance and can help regulators take corrective measures.Source: §1 · p. 1
  • Prior stock-market models include fundamental, technical, and analytical approaches [1]–, as well as linear methods such as ARIMA [4].Source: §1 · p. 1
  • Classical training methods effective for shallow networks are inefficient for deep architectures, despite their use in fields including image and speech recognition.The paper identifies deep-network training as difficult.Source: §1 · p. 1
  • The paper seeks an effective data-normalization method for Indian stock-market prediction and applies a DRNN trained on historical market data.The learned knowledge is then used to predict the market.Source: §1 · p. 1

2 PRELIMINARY CONCEPT OF DEEP NEURAL NETWORK

The paper describes a DNN as a network with multiple hidden processing layers, which can represent nonlinear and dynamic data. It also notes that deep networks are difficult to train with classical methods.

2 PRELIMINARY CONCEPT OF DEEP NEURAL NETWORK

  • A DNN commonly has two or more hidden processing-neuron layers, whose outputs feed into successively higher layers.The paper gives multilayer perceptrons and feed-forward networks with many layers as examples [8].Source: §2 · p. 2
  • Multiple hidden layers are presented as useful for processing nonlinear and highly dynamic data, but back-propagation alone is not practically suitable for networks with more hidden layers.Source: §2 · p. 2
  • Figure 1 depicts a simple four-layer DNN.Source: Figure 1 · p. 2

3 BRIEF DESCRIPTION OF RNNS

RNNs model sequential data by repeating a task across sequence elements while making each output depend on previous computation. Their hidden state acts as memory, and the network shares parameters across time steps.

3 BRIEF DESCRIPTION OF RNNS

  • An RNN performs the same task for each sequence element, with outputs depending on prior computations; its hidden state serves as memory.Source: §3 · p. 2
  • Figure 2 depicts an RNN and the unfolding in time of its forward computation.Source: Figure 2 · p. 2
  • The recurrent computation updates hidden state from the current input and previous hidden state, then derives an output from that state.The passage specifies tanh or ReLU as typical nonlinearities and softmax for the output.Source: §3 · p. 2
  • An RNN uses the same parameters at every time step, unlike a traditional DNN with different parameters at each layer, reducing the number of parameters to learn.Source: §3 · p. 2

4 DESCRIPTION OF TIME SERIES DATA

A time series consists of observations collected at equal intervals. Forecasting commonly uses previous values to predict the next value.

4 DESCRIPTION OF TIME SERIES DATA

  • A time series is a sequence of data collected at equal intervals, such as hourly, daily, monthly, or yearly.Source: §4 · p. 2
  • The series is represented as X = {x(1), x(2), …, x(t)}, with x(t) as its most recent value.Source: §4 · p. 2
  • For many time-series problems, the goal is to forecast x(t + 1) from previous feature values related to the predicted value.Source: §4 · p. 2

5 DIFFERENT NORMALIZATION TECHNIQUES

The paper presents normalization as a preprocessing step for scaling time-series values before learning. It describes six techniques, including range scaling, statistical standardization, and nonlinear transforms.

5 DIFFERENT NORMALIZATION TECHNIQUES

  • Normalization is presented as scaling wide-ranging time-series values to support learning, with the aim of speeding the learning process.Source: §5 · p. 2

5.1 Min-Max Normalization

  • Min-Max normalization maps an input value to a chosen range using the attribute's minimum and maximum values.Source: §5.1 · pp. 2–3

5.2 Decimal Scaling Normalization

  • Decimal scaling moves the decimal point according to the number of digits in the attribute's maximum value.Source: §5.2 · p. 3

5.3 Z-Score Normalization

  • Z-Score normalization converts attribute values to a common range based on the attribute's mean and standard deviation.Source: §5.3 · p. 3

5.4 Median Normalization

  • Median normalization divides each input value by the median of all values for that attribute.Source: §5.4 · p. 3

5.5 Sigmoid Normalization

  • Sigmoid normalization applies a sigmoid function to normalize an attribute value.Source: §5.5 · p. 3

5.6 Tanh estimators

  • The Tanh estimator method, introduced by Hample, computes normalized values using the attribute's mean and standard deviation.Source: §5.6 · p. 3

6 DATA SET DESCRIPTION AND PROPOSED FRAME WORK

The study uses BSE and NYSE stock-market time series, which it describes as nonlinear and highly dynamic. Its framework normalizes the data before using a deep recurrent neural network to forecast closing prices.

  • The stock-market series are described as nonlinear and highly dynamic, making successful prediction challenging.Source: §6 · p. 3
  • BSE and NYSE data use opening, high, and low prices as inputs and closing price as the output; both datasets cover 2016–2017 and are divided into training, validation, and test sets.BSE has 493 data points and NYSE has 503; each is split approximately 70%/15%/15%.Source: §6 · p. 3
  • The framework applies normalization before feeding the data to a deep recurrent neural network for training, validation, and testing.Source: §6 · p. 3, Figure 3 · p. 3

7 RESULT AND DISCUSSION

The experiments compare six normalization methods for DRNN forecasts of BSE and NYSE closing prices. Performance is assessed with MSE and MAE, and plots show predicted versus actual values.

  • The DRNN uses one input layer, 20 hidden layers, and an output layer, with Min-Max, Decimal Scaling, Z-Score, Median, Sigmoid, and Tanh Estimator preprocessing.The experiments use Matlab and evaluate performance with Mean Squared Error (MSE) and Mean Absolute Error (MAE).Source: §7 · pp. 3–4
  • The BSE and NYSE plots show predicted versus actual closing prices for the different normalization methods.Source: §7 · p. 4, Figure 4 · p. 4, Figure 5 · p. 4
  • Tables 1 and 2 report MSE and MAE prediction errors for the normalization methods on BSE and NYSE data, respectively.Source: §7 · p. 4, Table 1 · p. 4, Table 2 · p. 4

8 CONCLUSION

The DRNN predicts BSE and NYSE stock indices under different normalization methods, which all yield fair results and small prediction errors. Tanh Estimator normalization produces lower MSE and MAE and is presented as effective for DRNN time-series prediction.

  • Tanh Estimator normalization produces lower MSE and MAE and is concluded to be effective for DRNN time-series prediction.Source: §8 · p. 4
  • All tested normalization techniques produce fair results and very small prediction errors for the BSE and NYSE stock-index forecasts.Source: §8 · p. 4
  • The study applies a DRNN to predict BSE and NYSE stock indices using different normalization methods.Source: §8 · p. 4
Loading 1812.05519v2…