Source-linked AI summary
DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks
David Salinas, Valentin Flunkert, Jan Gasthaus
TL;DR
Forecasting thousands or millions of related time series requires models that can leverage shared historical behavior for probabilistic predictions. DeepAR addresses this with an autoregressive recurrent network trained globally across series, and substantially improves forecast accuracy over state-of-the-art methods across diverse datasets.
Problem
Forecasting thousands or millions of related time series requires methods that leverage shared historical behavior across series.
Method
DeepAR trains a global autoregressive recurrent network on historical data from all time series to produce probabilistic forecasts.
Results
DeepAR drastically improves forecast accuracy over state-of-the-art forecasting methods across a wide variety of datasets.
Takeaways & Limitations
DeepAR learns complex seasonal and uncertainty patterns and remains applicable to medium-size datasets with little or no hyperparameter tuning.
Takeaways & Limitations
The hyperparameter-selection procedure may overfit the training set, suggesting evaluation across separate time windows and non-overlapping intervals.
Abstract
from arXiv · showhide
Probabilistic forecasting, i.e. estimating the probability distribution of a time series' future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this paper we propose DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an auto regressive recurrent network model on a large number of related time series. We demonstrate how by applying deep learning techniques to forecasting, one can overcome many of the challenges faced by widely-used classical approaches to the problem. We show through extensive empirical evaluation on several real-world forecasting data sets accuracy improvements of around 15% compared to state-of-the-art methods.
1 Introduction
DeepAR addresses probabilistic forecasting for large collections of related time series with a global autoregressive recurrent network. It targets challenges including widely varying series magnitudes while providing accurate forecast distributions with reduced manual feature engineering.
- Forecasting supports operational optimization and data-driven decisions, including retail inventory management, staff scheduling, and supply-chain planning.
- Modern applications increasingly require forecasting thousands or millions of related series, such as household energy use, data-center load, and retailer-wide product demand.
- DeepAR learns a global model from historical data across all series using autoregressive recurrent networks and an LSTM-based architecture.
- A central challenge is that real-world series magnitudes differ widely and follow strongly skewed, approximately power-law distributions.
- The method combines an RNN architecture, a negative Binomial likelihood for count data, and special treatment for varying magnitudes to produce accurate probabilistic forecasts.
- DeepAR learns seasonal behavior and covariate dependencies across series, requires minimal manual feature engineering, and produces Monte Carlo samples for consistent quantile estimates.
2 Related Work
Related forecasting work spans classical models, specialized likelihoods for irregular demand data, information-sharing methods across related series, and neural networks. The paper positions its approach against limitations in classical assumptions and the difficulty of modeling heterogeneous series jointly.
- Classical individual-series forecasting includes ARIMA and exponential smoothing, alongside related techniques reviewed by Hyndman et al.
- Demand data are often erratic, intermittent, or bursty, violating assumptions such as Gaussian errors, stationarity, and homoscedasticity.
- Specialized likelihoods for difficult demand data include zero-inflated Poisson, negative binomial, combined, and tailored multi-stage formulations.
- Matrix factorization and Bayesian hierarchical priors have been proposed to share information across multiple related time series despite data heterogeneity.
- Neural networks have long been studied for forecasting, but recent intermittent-data applications produced mixed results and typically fitted separate models to individual series.
3 Model
DeepAR models the conditional distribution of each series’ prediction range given its conditioning range and known covariates using an autoregressive LSTM network. It supports direct probabilistic sampling and likelihood-based training, with distribution choices and sampling adjustments for different data characteristics.
- Model formulation: The model predicts the prediction range [t0, T] conditioned on [1, t0 −1] and covariates known across all time points.The conditioning range contains observed values, while the prediction range contains values treated as unknown at prediction time.
- Model architecture: DeepAR uses an autoregressive recurrent architecture whose LSTM state consumes the previous observation and recurrent output to parameterize the next-step likelihood.The likelihood parameters are produced from the network output through a parameter function.
- Probabilistic prediction: Joint future samples are generated by ancestral sampling, recursively feeding sampled observations into the network to estimate quantities such as future aggregate quantiles.The recurrent state is initialized from the conditioning range before sampling begins.
- Likelihood models: The network directly predicts likelihood parameters, using Gaussian distributions for real-valued data and negative-binomial distributions for positive count data.The negative-binomial parameterization uses a positive mean and shape parameter, with Var[z] = µ + µ2α.
- Training: Training optimizes the observed-data log-likelihood directly with stochastic gradient descent because recurrent states are deterministic functions of inputs.The method avoids latent-state inference, while scheduled sampling did not significantly improve accuracy and slowed convergence.
- Scale adjustment: Non-uniform sampling selects windows with probability proportional to scale νi to counter underfitting of the few high-scale series.This addresses imbalance when uniformly sampled training instances rarely visit high-scale time series.
4 Applications and Experiments
DeepAR is evaluated across multiple real-world datasets using probabilistic and point-forecast metrics against classical and recurrent baselines. Experiments show improved point accuracy, learned nonlinear uncertainty growth, and better forecast calibration than comparison methods.
- Implementation: A full training and prediction run on the 500K-series ec dataset takes less than 10 hours on one p2.xlarge AWS instance with 4 CPUs and 1 GPU.Prediction is already fast and can be parallelized if necessary.
- Datasets: Evaluations use five datasets, including parts, electricity, and traffic, covering aligned monthly sales, hourly customer consumption, and freeway-lane occupancy.The parts dataset contains 1,046 time series; electricity contains 370 customer series; traffic contains 963 lane series.
- Baselines: DeepAR is compared with Croston, ETS, Snyder, ISSM, and two RNN baselines designed to isolate likelihood, scaling, and sampling effects.The RNN baselines are rnn-gaussian and rnn-negbin.
- Accuracy: DeepAR outperforms MatFact on both electricity and traffic for Normalized Deviation and Normalized RMSE.Probabilistic accuracy is also assessed with 0.5-risk and 0.9-risk across lead times and forecast spans.
- Uncertainty: DeepAR learns nonlinear uncertainty growth over time, including plausibly higher uncertainty around Q4, rather than imposing ISSM’s linear-growth assumption.The aggregate uncertainty analysis is performed over the entire ec dataset.
- Calibration: Forecast calibration improves overall relative to ISSM, while shuffling prediction samples destroys temporal correlation and reduces calibration for longer forecast spans.Shuffling has no impact for a one-time-point span because that span contains only the marginal distribution.
5 Conclusion
DeepAR improves forecast accuracy across diverse datasets by learning a global model from related time series and producing calibrated probabilistic forecasts. It requires little or no hyperparameter tuning and applies to medium-sized datasets with only a few hundred time series.
- DeepAR drastically improves forecast accuracy over state-of-the-art methods across a wide variety of datasets.
- DeepAR learns a global model from related time series, handles widely varying scales, and captures complex patterns including seasonality and uncertainty.It uses rescaling and velocity-based sampling to handle scale differences.
- DeepAR generates calibrated probabilistic forecasts with high accuracy.
- DeepAR works with little or no hyperparameter tuning across a wide variety of datasets.
- DeepAR applies to medium-size datasets containing only a few hundred time series.
6 Supplementary materials
The supplementary materials define aggregated quantile forecasts and evaluation metrics, then describe the implementation, dataset splits, hyperparameter selection, and missing-observation setting. They also note a potential risk of hyperparameter overfitting.
- Quantile forecasting: Aggregated quantile forecasts sum each sample path over the specified span and extract the ρ-quantile from the resulting empirical distribution.The aggregated target is denoted Zi(L, S), with predicted quantile ˆZρ_i(L, S).
- Evaluation metrics: ND and RMSE use predicted median values, with sums taken across all items and prediction-period time points.The supplementary materials also define normalized quantile-loss summaries called ρ-risk.
- Implementation: The implementation uses MxNet, ADAM, early stopping, LSTM cells with forget bias 1.0, and 200 decoder samples for prediction generation.Experiments run on laptops or a single AWS p2.xlarge instance, and end-to-end prediction takes hours even on one machine.
- Experimental setup: For parts, 42 months are used for training and the remaining 8 for evaluation; electricity and traffic use rolling windows without retraining.Other datasets subsample possible training instances according to Table 3.
- Hyperparameter selection: Hyperparameters are selected by fitting on 90% of the training data and minimizing negative log-likelihood on the remaining 10%, after which test metrics are evaluated.The authors note that this procedure could over-fit hyperparameters to the training set.
- Missing observations: Missing target values can arise when observations are unobserved, such as demand during an item’s out-of-stock period.Failing to model these missing observations can lead to systematic forecast underbias in the best case.