PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Step-Ahead Time Series Prediction using Multiple-Output Support Vector Regression

Yukun Bao, Tao Xiong, Zhongyi Hu

arXiv:1401.2504v1cs.LGstat.ML

TL;DR

Long-horizon multi-step time-series forecasting requires selecting among prediction strategies for SVR. This paper introduces M-SVR with MIMO and compares it with direct and iterated SVR strategies, finding MIMO-SVR generally the strongest performer.

  • Problem

    Accurate prediction over long future horizons is challenging, motivating comparison of SVR strategies for multi-step-ahead forecasting.

  • Method

    The study applies multiple-output SVR with a multiple-input multiple-output strategy and comparatively evaluates SVR prediction strategies.

  • Results

    MIMO-SVR consistently achieves top-ranked forecast accuracy, while DIR-SVR also performs better than alternatives in most comparisons.

  • Takeaways & Limitations

    MIMO-SVR emerges as the strongest overall strategy, while computational load remains an important practical consideration.

  • Takeaways & Limitations

    The study uses only SVR as the modeling technique, leaving evaluation with other modeling techniques for future research.

Abstract

from arXiv · show

Accurate time series prediction over long future horizons is challenging and of great interest to both practitioners and academics. As a well-known intelligent algorithm, the standard formulation of Support Vector Regression (SVR) could be taken for multi-step-ahead time series prediction, only relying either on iterated strategy or direct strategy. This study proposes a novel multiple-step-ahead time series prediction approach which employs multiple-output support vector regression (M-SVR) with multiple-input multiple-output (MIMO) prediction strategy. In addition, the rank of three leading prediction strategies with SVR is comparatively examined, providing practical implications on the selection of the prediction strategy for multi-step-ahead forecasting while taking SVR as modeling technique. The proposed approach is validated with the simulated and real datasets. The quantitative and comprehensive assessments are performed on the basis of the prediction accuracy and computational cost. The results indicate that: 1) the M-SVR using MIMO strategy achieves the best accurate forecasts with accredited computational load, 2) the standard SVR using direct strategy achieves the second best accurate forecasts, but with the most expensive computational cost, and 3) the standard SVR using iterated strategy is the worst in terms of prediction accuracy, but with the least computational cost.

Note: This preprint copy is only for personal use.

The study compares three SVR-based multi-step prediction strategies. M-SVR with MIMO is reported as best overall, while direct SVR is costliest and iterated SVR cheapest.

  • The study provides empirical evidence comparing three multi-step-ahead prediction strategies.
  • M-SVR using MIMO strategy is identified as the best-performing approach with accredited computational load.
  • Standard SVR using direct strategy has extremely high computational load.
  • Standard SVR using iterated strategy has the lowest computational load.

1. Introduction

Long-horizon time-series forecasting is difficult because uncertainty and error accumulation increase the challenge. The study addresses the limited examination of M-SVR with MIMO for this setting and compares accuracy with computational cost.

  • Multi-step-ahead forecasting becomes more difficult as future-horizon uncertainty grows.
  • Iterated strategy: Iterated forecasting reuses predicted values as inputs, making it susceptible to error accumulation.
  • Direct strategy: Direct forecasting builds separate models for each horizon, but it is time consuming.
  • MIMO strategy: MIMO forecasting predicts a vector of future values while preserving their stochastic dependency.
  • Few studies had examined M-SVR for multi-step-ahead time-series prediction despite prior evidence of its capability in other disciplines.
  • The study proposes MIMO-based M-SVR and compares three SVR strategies using prediction accuracy and computational cost on simulated and real datasets.

2. Methodologies

The methodology formulates multi-step forecasting as a mapping from current and past observations to a future-value vector. M-SVR jointly estimates output regressors, with an IRWLS procedure used for optimization.

  • M-SVR formulation: M-SVR generalizes standard SVR by finding a regressor for every output dimension.
  • Multi-step forecasting maps current and past observations to a vector of future observations over horizon H.
  • M-SVR formulation: The M-SVR cost function uses an epsilon-insensitive loss and balances regularization against error reduction through C.
  • M-SVR formulation: When ε is nonzero, all outputs contribute to constructing each regressor, yielding a shared support-vector set across dimensions.
  • M-SVR optimization: An IRWLS procedure with a quasi-Newton approach iteratively approximates and optimizes the M-SVR objective until convergence.
  • Prediction strategies: The study evaluates iterated, direct, and MIMO strategies using their basic versions.

1) Iterated strategy

The iterated strategy learns a one-step-ahead model and repeatedly feeds each prediction back as the next input until the desired horizon is reached.

  • The iterated strategy minimizes in-sample one-step-ahead residuals to construct a single prediction model.
  • Each predicted value becomes an input for forecasting the subsequent point, continuing until the horizon is reached.
  • The strategy embeds the original series into lagged observations before learning the one-step-ahead model.
  • After learning, the strategy returns estimates for the next H values.

2) Direct strategy

The direct strategy constructs separate forecasting models for each future horizon, using past observations to predict each horizon-specific output.

  • 2) Direct strategy: Each direct model uses only past observations and minimizes associated squared multi-step-ahead errors.
  • 2) Direct strategy: Direct forecasting estimates H different models, one for each horizon-specific output.Each model maps inputs to its corresponding future value.
  • 2) Direct strategy: The strategy first embeds the original series into H horizon-specific datasets.The resulting datasets are denoted D_h for h ∈ {1, …, H}.
  • 2) Direct strategy: After learning, the models return estimates for the next H values across the forecast horizons.

3) MIMO strategy

The MIMO strategy learns one multiple-output model that predicts a vector of future values, preserving temporal stochastic dependency across the forecast horizon.

  • 3) MIMO strategy: MIMO predicts a vector of future time-series values rather than a scalar output.
  • 3) MIMO strategy: Compared with direct forecasting, MIMO uses one multiple-output model instead of H separate models.
  • 3) MIMO strategy: The single model preserves temporal stochastic dependency hidden in the predicted time series.
  • 3) MIMO strategy: MIMO embeds the original series into datasets pairing lagged inputs with multiple future outputs.
  • 3) MIMO strategy: After learning, the model returns estimates of the H next values from the embedded input sequence.

3. Research design

The research design evaluates forecasting strategies on simulated chaotic series and real business time series using hold-out forecasts, multiple accuracy measures, and repeated model selection.

  • 3. Research design: Each series is split into estimation and hold-out samples, with the last 18 observations reserved for out-of-sample evaluation.
  • 3. Research design: Filter methods select inputs, using PMI for iterated and direct strategies and an extended Delta test for MIMO.The extended Delta test is used because PMI handles single outputs but not multiple outputs.
  • 3. Research design: PSO and fivefold cross-validation jointly determine SVR and MSVR hyperparameters after input selection.The study treats input selection and parameter tuning as independent tasks.
  • 3. Research design: The modeling process for each series is repeated 60 times before hold-out performance is summarized.

4. Results and discussion

Across the examined datasets, MIMO-SVR generally provides the strongest forecasting accuracy, while DIR-SVR is substantially more computationally expensive and ITER-SVR is fastest but often less accurate.

  • Forecast accuracy: MIMO-SVR ranks first or near-first across datasets and accuracy measures, with DIR-SVR and ITER-SVR varying by dataset and metric.For NN3, DIR-SVR leads under MAPE, whereas MIMO-SVR leads under SMAPE and MASE; MIMO-SVR leads Hénon and Mackey-Glass rankings more consistently.
  • Forecast accuracy: Across short, medium, and long horizons, MIMO-SVR and DIR-SVR generally forecast more accurately than ITER-SVR, with a few exceptions.The reported explanation is that iterated forecasting accumulates errors, deteriorating prediction accuracy.
  • Forecast accuracy: MIMO-SVR generally outperforms DIR-SVR, possibly because MIMO preserves stochastic dependency among predicted time-series values.This comparison has a few exceptions, but the overall result favors MIMO-SVR.
  • Statistical comparison: For Hénon and Mackey-Glass, MIMO-SVR significantly outperforms competitors for most prediction horizons, while DIR-SVR generally beats ITER-SVR on Hénon.For Mackey-Glass, the DIR-SVR versus ITER-SVR difference is not significant at the 0.05 level in the reported comparisons.
  • Computational cost: DIR-SVR is computationally much more expensive than ITER-SVR, while ITER-SVR and MIMO-SVR are tens of times faster than DIR-SVR across three datasets.ITER-SVR is the least expensive model, but its computational-cost difference from MIMO-SVR is negligible, particularly for small samples.
  • Computational cost: ITER-SVR and MIMO-SVR elapsed times increase slightly with forecast horizon, whereas DIR-SVR cost increases drastically with series sample size.Elapsed times were examined for ITER-SVR, DIR-SVR, and MIMO-SVR across Hénon, Mackey-Glass, and NN3 datasets.

5. Conclusions

The study proposes M-SVR with a MIMO strategy for multi-step-ahead prediction and compares it with direct and iterated SVR strategies. Across simulated and real datasets, MIMO-SVR provides the strongest accuracy–computational-load combination, while direct SVR is costliest and iterated SVR is least costly.

  • Conclusions: M-SVR using the MIMO strategy was assessed against three SVR prediction strategies for multi-step-ahead forecasting.The assessment used simulated and real datasets and considered prediction accuracy and computational cost.
  • Prediction accuracy: MIMO-SVR consistently outperformed DIR-SVR and ITER-SVR in MAPE, SMAPE, and MASE across three datasets, with a few exceptions.
  • Prediction accuracy: The prediction-performance difference between DIR-SVR and ITER-SVR was not significant at the 0.05 level in most cases.
  • Computational cost: DIR-SVR had extremely higher computational load than ITER-SVR and MIMO-SVR.The computational-load difference between ITER-SVR and MIMO-SVR was negligible.
  • Limitations: The study is limited by using only SVR and examining three commonly used prediction strategies.Future work could examine other modeling techniques and richer prediction strategies.

End

The end materials present the experiment procedure, representative actual-versus-predicted hold-out examples, and elapsed-time comparisons across datasets.

  • Fig. 1 presents the experiment procedure for multi-step-ahead prediction.
  • Fig. 2 compares actual values with predicted values on hold-out samples for representative NN3, Hénon, and Mackey-Glass series.
  • Figs. 3–5 present elapsed-time results for three models across individual series in the NN3, Hénon, and Mackey-Glass datasets.

Tables

The tables document the simulated-data setup, PSO parameter selection, prediction-accuracy measures, and ranked model comparisons across NN3, Hénon, and Mackey-Glass datasets.

  • Experimental setup: Tables 1 and 2 report the initialization and sample sizes of simulated time series and the selected PSO parameters.The simulated series include Hénon and Mackey-Glass data; Table 2 concerns PSO parameter selection.
  • Ranked comparisons: Tables 6–8 provide ranked-model multiple comparisons for hold-out samples across NN3, Hénon, and Mackey-Glass prediction horizons.The comparisons are organized by measure and prediction horizon, with adjacent-method differences assessed at the 0.05 significance level.
  • Ranked comparisons: For the reported horizons, the ranking varies: Naive and S-Naive precede MIMO-SVR, DIR-SVR, and ITER-SVR, with some adjacent differences significant.The displayed rankings include rows for horizons 14, 15, and 17; significance is marked for selected adjacent comparisons.
Loading 1401.2504v1…