Source-linked AI summary
Conditional Time Series Forecasting with Convolutional Neural Networks
Anastasia Borovykh, Sander Bohte, Cornelis W. Oosterlee
TL;DR
The paper asks how to forecast noisy, short financial time series while using dependencies across related series. It adapts WaveNet into a conditional convolutional model and compares it with autoregressive and LSTM models, finding that convolutional networks can match or exceed recurrent accuracy while being simpler and more efficient to train.
Problem
Noisy financial series of limited duration make forecasting difficult, while related series may contain exploitable dependencies.
Method
The paper adapts WaveNet into a multivariate conditional CNN using dilated convolutions, ReLU activation, and parametrized skip connections.
Results
The convolutional model is reported to achieve at least as good or better accuracy than linear and recurrent models on nonlinear, noisy forecasting tasks.
Takeaways & Limitations
Convolutional networks provide a simpler, easier-to-train alternative to recurrent networks for forecasting dependencies in and between financial series.
Takeaways & Limitations
The authors identify overfitting as a concern because increasing layers enlarges the receptive field and the number of parameters, and note room to improve nonlinear-dependency learning.
Abstract
from arXiv · showhide
We present a method for conditional time series forecasting based on an adaptation of the recent deep convolutional WaveNet architecture. The proposed network contains stacks of dilated convolutions that allow it to access a broad range of history when forecasting, a ReLU activation function and conditioning is performed by applying multiple convolutional filters in parallel to separate time series which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. We test and analyze the performance of the convolutional network both unconditionally as well as conditionally for financial time series forecasting using the S&P500, the volatility index, the CBOE interest rate and several exchange rates and extensively compare it to the performance of the well-known autoregressive model and a long-short term memory network. We show that a convolutional network is well-suited for regression-type problems and is able to effectively learn dependencies in and between the series without the need for long historical time series, is a time-efficient and easy to implement alternative to recurrent-type networks and tends to outperform linear and recurrent models.
1 Introduction
The paper addresses noisy, short multivariate financial series by developing a conditional convolutional WaveNet model that uses related series alongside each series’ own history. It evaluates this approach against autoregressive and recurrent models, reporting efficient training and competitive or better forecasting accuracy.
- Motivation: Financial forecasting is difficult because the data are noisy, nonlinear, and often available only for limited durations.Long stock-price histories can also be problematic because the financial environment changes.
- Problem and approach: The paper conditions forecasts on multiple related financial series to exploit cross-series correlations and reduce noise in short series.The model uses each target’s history together with the histories of other series.
- Architecture: The proposed WaveNet-inspired CNN uses dilated convolutions to access broad history while retaining a small number of trainable weights for efficient training.Its architecture is simplified for multivariate forecasting and replaces WaveNet’s gated activation with ReLU.
- Evaluation: The study compares the convolutional model with autoregressive and LSTM benchmarks on artificial and financial series including the S&P500, VIX, interest rates, and exchange rates.The evaluation examines both unconditional and conditional forecasting.
- Architecture: Parametrized skip connections condition the network on both target and related series, supporting efficient learning of long- and short-term interdependencies.The paper presents this conditioning design as a novel optimization of the WaveNet-inspired architecture.
- Findings: The WaveNet model is reported as a time-efficient, easy-to-implement alternative that tends to outperform linear and recurrent models on nonlinear, noisy forecasting tasks.Conditioning extracts temporal relationships between series while limiting the need for long historical price series.
2 The model
The model develops convolutional architectures for forecasting, progressing from feedforward and standard convolutional networks to dilated WaveNet-style layers and conditional convolutions. Shared weights reduce parameters, while dilation and parallel conditioning broaden temporal access and capture dependencies within and between series.
- 2.1.1 Feedforward neural networks: Feedforward networks map historical inputs through successive weighted, nonlinear layers to produce the next-step forecast.The first layer forms linear combinations of input variables; later layers repeatedly combine and transform intermediate outputs.
- 2.1.3 Convolutional neural networks: CNNs use local connectivity and shared filters, reducing trainable parameters while detecting the same patterns across positions.Each filter slides over the input to create feature maps, and shared weights make feature detection time-invariant.
- 2.2 Structure: Dilated convolutions apply filters to every dth input element, efficiently learning connections between far-apart data points.The architecture stacks layers with dilations increasing by powers of two, allowing outputs to depend on more input nodes.
- 2.4 Conditioning: Conditional forecasting incorporates a related series through a 1 × k convolution, enabling nonlinear relations to be learned within and between time series.The conditioning design applies parallel convolutional processing to the input and condition and uses the resulting activations in the forecast network.
3 Experiments
The experiments evaluate WaveNet variants on chaotic Lorenz dynamics and financial time series against autoregressive and recurrent baselines. Conditioning generally helps exploit cross-series dependencies, while WaveNet remains robust when such dependencies weaken or data contain extremes.
- 3.1 An artificial example: Replacing WaveNet’s gated activation with ReLU and using parallel filters supports learning linear and nonlinear dependencies within and between series.The conditional Lorenz forecast of each coordinate uses histories from the other two coordinates.
- 3.1 An artificial example: Conditioning improves Lorenz out-of-sample forecasts and produces a more precise X-coordinate forecast, although it does not improve Z one-step RMSE.The authors attribute the Z result to an already accurate unconditional forecast.
- 3.1 An artificial example: Using one filter of width one fails to learn nonlinear dependencies, whereas increasing either filter width or filter count enables them to be learned.This numerically verifies the stated architectural requirement.
- 3.2.3 Results: WaveNet performs similarly to LSTM in HITS during period A, when dependencies remain strong, but conditional gains shrink in later periods as those dependencies weaken.In periods B and C, conditional WaveNet HITS is similar to naive and autoregressive forecasts, while MASE remains consistently lower than competing models.
- 3.2.3 Results: For financial series, WaveNet consistently outperforms vector autoregression and LSTM in MASE, while conditioning can improve directional hit rates through correlated inputs.The exchange-rate results also show stronger correlations among rates containing the same currencies.
- 3.2.3 Results: WaveNet generalizes well under high kurtosis but struggles with outliers during periods of high standard deviation and absolute skewness.In those settings, its performance becomes similar to a linear autoregressive model.
4 Discussion and conclusion
The paper presents WaveNet-based conditional forecasting for multivariate time series and compares it with LSTM and autoregressive models. It finds WaveNet efficient and competitive, while highlighting trade-offs involving nonlinear learning, receptive-field size, and overfitting.
- Method: WaveNet uses dilated convolutional layers and multiple conditions to learn trends and relationships within and between time series.The architecture is adapted for conditional forecasting and applies convolutional processing to various time series.
- Evaluation: The study compares WaveNet with an LSTM and a linear autoregressive model across various time series.The evaluation covers both unconditional and conditional forecasting settings.
- Conclusion: WaveNet is presented as a simple, efficient, and easily interpretable network that can serve as a strong forecasting baseline.The conclusion qualifies this claim by noting that no single model fits every forecasting problem.
- Limitations: Learning nonlinearities requires many layers and filters, creating a trade-off with overfitting as the receptive field and parameter count increase.The paper identifies this imbalance as a limitation of the architecture.
- Future work: The paper identifies possible improvements through combining autoregressive and convolutional models or using parametrized skip connections with an adaptive filter.These alternatives are proposed as future directions for addressing architectural limitations.
- Results: The WaveNet model was a strong competitor to LSTM models, particularly when training time is considered.The discussion distinguishes training-time advantages from prediction-time considerations for longer series.