PaperlayerView exact revision on arXiv

Source-linked AI summary

An Introductory Study on Time Series Modeling and Forecasting

Ratnadip Adhikari, R. K. Agrawal

arXiv:1302.6613v1cs.LGstat.ML

TL;DR

Time series forecasting requires models that address varied series behavior and practical modeling issues. This work surveys stochastic, neural-network, and SVM-based models, discusses their strengths and weaknesses, and evaluates them on six real datasets using several accuracy measures and forecast diagrams. The results show successful implementation overall, although some forecasts substantially deviate from the original observations.

  • Problem

    The work addresses the need to understand and select suitable time series forecasting models amid varied model assumptions and practical issues such as stationarity and parsimony.

  • Method

    The study describes stochastic, neural-network, and SVM-based forecasting models and evaluates fitted models on six real datasets using performance measures and forecast diagrams.

  • Results

    The models were successfully implemented and assessed using five performance measures and forecast diagrams for each of the six datasets.

  • Takeaways & Limitations

    The paper provides an introductory overview of major forecasting approaches and their practical evaluation across multiple datasets.

  • Takeaways & Limitations

    Some cases showed significant deviations between the original observations and forecasted values, suggesting that other data preprocessing may be needed.

Abstract

from arXiv · show

Time series modeling and forecasting has fundamental importance to various practical domains. Thus a lot of active research works is going on in this subject during several years. Many important models have been proposed in literature for improving the accuracy and effectiveness of time series forecasting. The aim of this dissertation work is to present a concise description of some popular time series forecasting models used in practice, with their salient features. In this thesis, we have described three important classes of time series models, viz. the stochastic, neural networks and SVM based models, together with their inherent forecasting strengths and weaknesses. We have also discussed about the basic issues related to time series modeling, such as stationarity, parsimony, overfitting, etc. Our discussion about different time series models is supported by giving the experimental forecast results, performed on six real time series datasets. While fitting a model to a dataset, special care is taken to select the most parsimonious one. To evaluate forecast accuracy as well as to compare among different models fitted to a time series, we have used the five performance measures, viz. MSE, MAD, RMSE, MAPE and Theil's U-statistics. For each of the six datasets, we have shown the obtained forecast diagram which graphically depicts the closeness between the original and forecasted observations. To have authenticity as well as clarity in our discussion about time series modeling and forecasting, we have taken the help of various published research works from reputed journals and some standard books.

Declaration

The work introduces major time series forecasting approaches, their assumptions and limitations, and the concepts needed for model development and evaluation. It also covers forecasting experiments, performance measures, and datasets used for comparison.

  • Evaluation: The experiments evaluate forecasts on six real datasets using multiple error measures and forecast diagrams.The listed measures include MSE, MAD, RMSE, MAPE, and Theil’s U-statistics; the contents identify six dataset-specific experimental sections.
  • Modeling principles: Time series modeling requires selecting an appropriate model from past observations while considering stationarity and parsimony.The text emphasizes fitting an adequate model and studying past observations to represent the series structure.
  • Stochastic models: ARIMA and related stochastic models assume a linear time series and a known statistical distribution, while seasonal forecasting uses SARIMA.The book also discusses AR, MA, ARMA, generalized ARFIMA, nonlinear stochastic models, and Box-Jenkins methodology.
  • Neural networks: Artificial neural networks provide data-driven, self-adaptive nonlinear modeling without presuming the observations’ statistical distribution.The covered ANN models include MLPs, TLNNs, and SANN for seasonal forecasting.
  • Support vector machines: SVM methods use support vector kernels and include LS-SVM and DLS-SVM models for time series forecasting.The book introduces SVM fundamentals before discussing these two forecasting models.
  • Scope: The book presents stochastic, neural-network, and SVM approaches to time series forecasting.Its stated objective is a comprehensive discussion of these three widely used approaches.

Basic Concepts of Time Series Modeling

Time series modeling organizes chronologically ordered observations to describe underlying structure and support forecasting. The data may be continuous or discrete, univariate or multivariate, and may contain trend, cyclical, seasonal, and irregular components represented through additive or multiplicative models.

  • Definitions: A time series is a chronologically ordered sequence of observations measured over time.Observations may be represented as random variables indexed by time.
  • Types of time series: A series is univariate when it records one variable and multivariate when it records more than one.
  • Types of time series: Time series may be continuous or discrete, and discrete observations are commonly recorded at equally spaced intervals.Examples include hourly, daily, weekly, monthly, and yearly measurements.
  • Components: The main components of a time series are trend, cyclical, seasonal, and irregular variation.Trend describes long-term movement; cyclical variation typically spans two or more years, while irregular variation is unpredictable.
  • Component models: Additive models sum the four components, whereas multiplicative models multiply them.The multiplicative formulation allows components to affect one another, while the additive formulation assumes independence.
  • Modeling and forecasting: Time series analysis fits a suitable model to past observations so future values can be forecast.Forecasting is useful when the statistical pattern or an explanatory model is limited.

Time Series Forecasting Using Stochastic Models

The stochastic-model discussion covers linear and nonlinear time-series models, emphasizing model selection, stationarity, and autocorrelation analysis for forecasting.

  • Time-series models represent stochastic processes, with linearity determined by whether current values depend linearly or nonlinearly on past observations.
  • AR and MA models form the foundation for ARMA models, which combine autoregressive terms with moving-average error terms.
  • ARIMA generalizes earlier Box-Jenkins models, while SARIMA extends ARIMA for seasonal time-series forecasting.
  • MA-model fitting is more complicated than AR fitting because prior random-error terms are not foreseeable.
  • Stationarity and invertibility impose root conditions on ARMA lag polynomials, while MA processes are stationary regardless of parameter values.
  • ACF and PACF plots summarize dependence across lags and help determine the orders of AR and MA terms for model selection.

Time Series Forecasting Using Artificial Neural Networks

The ANN discussion presents data-driven, nonlinear forecasting architectures that map lagged observations to future values, including feed-forward, time-lagged, and seasonal designs.

  • ANNs are data-driven and self-adaptive, forming models from presented data without requiring a specified model form or distributional assumption.
  • Their nonlinear structure supports modeling complex patterns, and the discussion reports better analysis and forecasting than various linear models.
  • Feed-forward ANN: Feed-forward ANNs use input, hidden, and output layers to map past observations to a future value through network parameters and connection weights.
  • TLNN: TLNNs use selected lagged observations as inputs; for monthly seasonal data, lags 1, 2, and 12 can forecast the value at time t.
  • SANN: SANN requires no raw-data preprocessing and can learn seasonal patterns without removing them; its forecasting ability was empirically verified on practical datasets.
  • SANN: SANN uses observations from one seasonal period as inputs and the next seasonal period as outputs, with input and output sizes determined by the seasonal period.

Time Series Forecasting Using Support Vector Machines

The SVM discussion introduces statistical learning principles for forecasting and regression, emphasizing generalization, structural risk control, and computational trade-offs.

  • Applications: SVM methods extend beyond classification to function approximation, regression estimation, and time-series prediction.
  • SVM foundations: SVM seeks a decision rule with good generalization by selecting support vectors and constructing a separating hyperplane in a higher-dimensional feature space.
  • SVM foundations: SVM training solves a linearly constrained quadratic programming problem, yielding a unique and globally optimal solution.
  • Statistical learning theory: The learning framework assumes training pairs are generated independently from an unknown distribution, with inputs and corresponding outputs observed together.
  • Risk minimization: Expected risk cannot be computed directly because the data distribution is unknown, motivating empirical-risk and structural-risk approaches.
  • Structural risk minimization: A finite training set means minimizing empirical risk does not guarantee minimizing expected risk; SRM bounds their difference using the function class's VC dimension.
  • LS-SVM: LS-SVM transforms traditional quadratic programming into a computationally more efficient procedure for time-series forecasting.

Forecast Performance Measures

Forecast accuracy is assessed by comparing actual and forecasted values on a test dataset using multiple complementary performance measures. The measures differ in how they treat bias, error magnitude, extreme deviations, scale, and interpretability.

  • Forecast models are evaluated on test-set performance, with actual and forecasted values forming the basis of each measure.The text recommends using more than one measure because the metrics capture different aspects of forecast error.
  • MFE measures average signed error and forecast bias, but positive and negative errors can cancel, so zero MFE does not imply error-free forecasts.MFE also depends on measurement scale and data transformations.
  • MAE, also called MAD, measures average absolute deviation without cancellation, but it does not indicate error direction or penalize extreme errors.MAE should be small for a good forecast and is affected by scale and transformations.
  • MAPE expresses average absolute error as a percentage and avoids scale dependence, but it is affected by transformations and does not penalize extreme deviations.Opposite-signed errors do not offset each other in MAPE.
  • MSE averages squared deviations, penalizes extreme errors, and captures overall magnitude without indicating error direction.MSE is sensitive to scale and transformations and is less intuitive than other measures.

Experimental Results

The experiments apply stochastic, neural-network, and SVM-based forecasting methods to six real time series datasets using train/test evaluation and five performance measures. Results vary by dataset: AR(12) performs best for lynx, ANN for sunspots, SVM performs strongly for airline passengers, and SARIMA or SANN lead on other series.

  • Experimental design: Six real time series are evaluated using MAD, MSE, RMSE, MAPE, and Theil’s U-statistics, with forecast diagrams comparing test actual and forecasted observations.The data are divided into training and test sets, with validation sometimes held out from training.
  • Wolf sunspot series: ANN achieves the best forecasting performance for the Wolf sunspot series, whose forecast closely resembles the original series.SVM shows a significant deviation between forecasted and test values in the corresponding forecast diagrams.
  • Airline passenger series: SARIMA and SVM produce quite good forecasts for the airline passenger series, with SVM obtaining the lowest measures except MAD.The airline series has strong multiplicative seasonality, an upward trend, and non-stationarity; the SVM forecast diagram shows excellent performance.
  • Quarterly sales series: The quarterly sales series is best forecast by SARIMA, while SVM performance is not up to expectation and SANN performance is moderate.The supplied results identify SARIMA model (0,1,1) × (0,1,1)_4 as the best-performing model.
  • Beer production and accidental deaths: SANN with four hidden nodes gives the relatively best performance for quarterly U.S. beer production, while SARIMA and SVM also perform quite well.For monthly U.S. accidental deaths, SARIMA obtains the minimum performance measures and shows excellent diagrammatic agreement.

Conclusion

The work presents a state-of-the-art introduction to time series forecasting models, evaluates them across six datasets, and identifies model selection and preprocessing as important practical considerations.

  • The study presents popular forecasting models with their salient features, including stochastic, neural-network, and SVM-based approaches.
  • The covered models include ARIMA for linear series, nonlinear stochastic models such as NMA and ARCH, neural networks, and LS-SVM and DLS-SVM.
  • Successful forecasting depends critically on selecting appropriate ARIMA orders, ANN neuron counts, and SVM hyper-parameters.
  • AIC and BIC support ARIMA model selection, while cross-validation is recommended for ANN neurons and SVM hyper-parameters.
  • The experiments use five performance measures and forecast diagrams across six real time series datasets to assess forecasting accuracy.
  • In some cases, forecasted values significantly deviate from original observations; additional data preprocessing may improve forecast performance.
  • Future work considers finding an efficient combining model that combines different methods to improve forecast accuracy.

1. The Canadian Lynx Dataset

The Canadian lynx dataset is attributed to Elton and Nicholson’s 1942 study of the ten-year cycle in Canadian lynx numbers.

  • The dataset’s original source is Elton and Nicholson’s 1942 study of the ten-year cycle in Canadian lynx numbers.
  • The source was published in the Journal of Animal Ecology, volume 11, pages 215–244.
  • The cited study examines Canadian lynx numbers over a ten-year cycle.

2. The Wolf’s Sunspot Dataset

The Wolf’s Sunspot dataset is attributed to H. Tong’s 1993 book on nonlinear time series and dynamical systems.

  • The dataset’s original source is H. Tong’s 1993 work Non-linear Time Series: A Dynamical System Approach.
  • The source belongs to the Oxford Statistical Science series, volume 6.
  • The cited work was published in July 1993 and spans pages 469–471.

3. The Airline Passenger Dataset

The Airline Passenger dataset is attributed to R. G. Brown’s 1994 book and was also used by Box and Jenkins as Series G.

  • The dataset’s original source is R. G. Brown’s 1994 book Smoothing, Forecasting and Prediction of Discrete Time Series.
  • Brown’s source was published by Prentice-Hall in Englewood Cliffs.
  • Box and Jenkins also used this dataset and named it Series G.

4. The Quarterly Sales Dataset

The supplied passage identifies a published source used in connection with this section.

  • The section cites Makridakis, Wheelwright, and Hyndman’s Forecasting: Methods and Applications, 3rd edition, published by John Wiley & Sons in 1998.

5. The Quarterly U.S. Beer Production Dataset

The supplied passages identify two published sources used in connection with this section.

  • The section cites Wei’s Time Series Analysis: Univariate and Multivariate Methods, 2nd edition, published by Addison Wesley in 2005.
  • The section cites Brockwell and Davis’s Introduction to Time Series and Forecasting, 2nd edition, published by Springer in 2003.
Loading 1302.6613v1…