Source-linked AI summary
Probabilistic Forecasting with Temporal Convolutional Neural Network
Yitian Chen, Yanfei Kang, Yixiong Chen, Zizhuo Wang
TL;DR
The paper tackles probabilistic forecasting for large collections of related time series when histories may be sparse or unavailable. It proposes a CNN-based framework with flexible density estimation and reports superior performance to state-of-the-art methods in point and probabilistic forecasting.
Problem
Forecasting large collections of related series is difficult because millions of series can require prohibitive resources, while some series have sparse or unavailable historical data.
Method
The paper proposes a CNN-based probabilistic forecasting framework supporting both parametric and non-parametric probability density estimation.
Results
The framework yields superior performance compared to other state-of-the-art methods in both point and probabilistic forecasting tasks.
Takeaways & Limitations
The framework provides a scalable and flexible approach for probabilistic forecasting across multiple related time series, including cold-start and sparse-data situations.
Takeaways & Limitations
The motivating real-world scope includes situations where historical data is sparse or unavailable.
Abstract
from arXiv · showhide
We present a probabilistic forecasting framework based on convolutional neural network for multiple related time series forecasting. The framework can be applied to estimate probability density under both parametric and non-parametric settings. More specifically, stacked residual blocks based on dilated causal convolutional nets are constructed to capture the temporal dependencies of the series. Combined with representation learning, our approach is able to learn complex patterns such as seasonality, holiday effects within and across series, and to leverage those patterns for more accurate forecasts, especially when historical data is sparse or unavailable. Extensive empirical studies are performed on several real-world datasets, including datasets from JD.com, China's largest online retailer. The results show that our framework outperforms other state-of-the-art methods in both accuracy and efficiency.
1. Introduction
The paper addresses scalable probabilistic forecasting for large collections of related time series, including settings with sparse histories, cold starts, and influential covariates. It proposes DeepTCN, a non-autoregressive convolutional framework that supports flexible density estimation and compares favorably with state-of-the-art methods.
- Probabilistic forecasting: Probabilistic forecasting estimates distributions of future observations and is important for applications such as product supply, demand, procurement, inventory, and logistics planning.The introduction motivates probabilistic forecasts for retail demand and package delivery volumes.
- Forecasting challenges: Real-world applications may require forecasting thousands or millions of related series, including new products without historical shopping-festival data.These settings also involve exogenous influences such as promotion plans and weather forecasts.
- Forecasting challenges: Classical forecasting models are well-suited to understood structures and sufficient history but require prohibitive labor and computing resources for millions of series.They are also not applicable when historical data is sparse or unavailable.
- Modeling approach: RNN training with backpropagation through time can hamper efficient computation and be difficult, whereas dilated causal convolutions capture long-term correlations with parallel training.Stacked dilated layers increase receptive fields without violating temporal order.
- Proposed framework: DeepTCN is a non-autoregressive probabilistic forecasting framework for large collections of related series, offering parametric and non-parametric density estimation.The framework uses a CNN-based architecture and supports exogenous covariates such as promotion plans and weather forecasts.
- Empirical evaluation: Extensive empirical studies show the framework compares favorably with state-of-the-art methods in both point forecasting and probabilistic forecasting tasks.The framework is designed to learn latent correlations and handle data sparsity and cold starts.
2. Related Work
Related work spans statistical, recurrent, convolutional, and probabilistic approaches to forecasting multiple related time series. The paper positions its method as combining direct, non-autoregressive forecasting with dilated convolutions, residual networks, cross-series information, exogenous variables, and flexible density estimation.
- Traditional univariate methods fit separate models and cannot learn across similar time series.
- Statistical approaches can suffer from overfitting and non-stationarity, motivating joint forecasting methods for multiple series.
- RNNs and CNNs model complex nonlinear feature interactions, with strong performance especially when many related series are available.
- Probabilistic deep-learning methods estimate uncertainty using approaches including autoregressive RNNs, deep state-space models, Gaussian processes, and quantile functions.
- Most probabilistic forecasting frameworks are autoregressive, recursively generating multi-step forecasts and potentially accumulating errors.
- DeepTCN differs by using stacked dilated causal convolutions, residual blocks, a residual decoder incorporating covariates, and flexible probability-density estimation.
3. Method
The method directly models future distributions for multiple related series using historical observations and covariates. DeepTCN combines a dilated-convolution encoder with a residual decoder and supports probabilistic outputs under flexible density-estimation choices.
- The forecasting objective is the conditional distribution of future observations given historical observations, across N series and horizon Ω.
- DeepTCN forecasts the joint future distribution directly rather than factorizing it through recursive autoregressive generation.
- The framework incorporates historical observations and covariates, including future covariates, into direct probabilistic forecasting.
- 3.1.1. Encoder: Dilated causal convolutions: The encoder uses stacked dilated causal convolutions and residual blocks to model historical stochastic processes and capture long-term temporal dependencies.
- 3.1.2. Decoder: Residual neural network: The decoder uses resnet-v to combine encoder outputs with exogenous variables, then applies a dense layer to produce probabilistic forecasts.
- 3.2. Probabilistic forecasting framework: The framework supports multiple probability-density estimation approaches for probabilistic forecasts.
4. Experiments
Experiments evaluate DeepTCN’s quantile and Gaussian variants on JD.com and public datasets using probabilistic and point-forecasting metrics. DeepTCN-Quantile generally achieves the strongest reported accuracy, including sparse-history settings and highly correlated series.
- JD.com datasets: DeepTCN-Quantile achieves the best point-forecasting accuracy across NRMSE, SMAPE, and MASE for all-data, long-series, and short-series groups.Short-series lack historical shopping-festival data, whereas long-series contain more than two years of history.
- JD.com datasets: Without historical shopping-festival data, SARIMA and lightGBM worsen substantially, while DeepTCN-Quantile maintains comparable performance.This pattern is reported for the short-series subgroup.
- JD.com datasets: DeepTCN-Quantile produces more accurate uncertainty estimates than SARIMA, learning uncertainty from data rather than imposing increasing uncertainty over time.SARIMA’s uncertainty becomes especially large during the shopping-festival period because of promotions and intense activity.
- JD.com datasets: DeepTCN-Quantile captures promotion-day spikes and new-warehouse scale and shape patterns when SARIMA and lightGBM lack sufficient historical festival data.The model learns from other warehouses with similar patterns, supporting forecasting for sparse-history series.
- Public datasets: On traffic and parts, both DeepTCN variants outperform other state-of-the-art probabilistic forecasters, while shared nonlinear series correlations help explain the advantage.For point forecasting, DeepTCN-Quantile is best on NRMSE and MASE for traffic and on MASE for electricity.
5. Conclusion
The framework provides convolutional probabilistic forecasting for multiple related time series, supporting both parametric and non-parametric distribution modeling. Across industrial and public datasets, it shows superior performance for point and probabilistic forecasting and supports large-scale applications involving cold starts and sparse data.
- The framework applies convolutional neural networks to probabilistic forecasting for multiple related time series.
- It supports both non-parametric and parametric approaches to modeling probabilistic distributions with neural networks.
- The solution is intended for practical large-scale forecasting applications involving cold starts and data sparsity.
- Results on industrial and public datasets show superior performance compared with other state-of-the-art methods in point and probabilistic forecasting.
A. Dataset
The evaluation uses two JD.com datasets, two public high-frequency datasets, and a monthly automotive-parts dataset. These collections vary substantially in series count, covariates, sampling frequency, duration, and filtering procedures.
- JD.com datasets: JD-demand contains 50,000 regional-demand series covering around 6,000 communication, computer, and consumer-electronics products across seven Chinese regions.
- JD.com datasets: JD-demand includes historical demand plus product-specific information such as region, category, brand, price, and promotions.
- JD.com datasets: JD-shipment includes about 1,450 series from 2014-10-01 to 2018-12-01, including warehouses emerging with business development.
- Public datasets: Electricity contains 370 customer-consumption series recorded every 15 minutes from 2011 to 2014 and aggregated here to hourly data of size N × T = 370 × 26304.
- Public datasets: Traffic contains occupancy rates for 963 San Francisco Bay Area freeway lanes, sampled every 10 minutes and rearranged into calendar order before hourly aggregation.
- Parts dataset: The parts dataset starts with 2,674 monthly-sales series over 51 months; filtering leaves 1,046 series after removing sparse or edge-zero-demand histories.
B. Baselines
The baselines combine conventional statistical forecasting with machine-learning methods. SARIMA is used for selected seasonal series, while lightGBM is applied to both JD.com datasets with tuned features and parameters.
- The baseline set includes univariate forecasting models and machine-learning-based methods.
- SARIMA: SARIMA extends ARIMA with a seasonal term to model seasonal behavior.
- SARIMA: SARIMA is applied to JD-shipment and JD-demand series with more than 14 historical observations.
- SARIMA: SARIMA parameters are automatically selected by minimizing AIC, and its 10% and 90% confidence levels provide probabilistic forecasts.
- lightGBM: lightGBM is a gradient boosting tree method applied to both JD-demand and JD-shipment datasets.
- lightGBM: For JD-shipment, lightGBM uses listed features and grid search to tune learning rate and tree depth using recent and year-ago data.
C. Experiment details
Experiments train and evaluate DeepTCN across JD.com, parts, electricity, and traffic settings using dataset-specific horizons, windows, and histories. Architecture and training hyperparameters are adjusted by task, with evaluation performed after each epoch.
- Implementation: The model is implemented with Mxnet and Gluon and trained on a GPU server with one Tesla P40 and 16 CPUs.
- Implementation: Multiple GPUs can be used to improve training efficiency in real industrial applications, and public-dataset code is available online.
- Dataset-specific settings: JD.com experiments use 31-day training ranges and prediction horizons, with models trained before October and November 2018 for the corresponding months.
- Dataset-specific settings: The parts experiment uses the first 39 months for training and the last 12 months for evaluation.
- Model selection and evaluation: Convolutional hyperparameters are selected by task and dataset to provide a sufficiently large receptive field, while training hyperparameters are manually tuned for metrics and running time.