PaperlayerView exact revision on arXiv

Source-linked AI summary

N-BEATS: Neural basis expansion analysis for interpretable time series forecasting

Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, Yoshua Bengio

arXiv:1905.10437v4cs.LGstat.ML

TL;DR

The paper asks whether deep learning can solve univariate time-series forecasting without relying on time-series-specific components or classical statistical support. It proposes N-BEATS, a doubly residual deep architecture with hierarchical partial forecasts, and evaluates generic and interpretable configurations. Across diverse M3, M4, and TOURISM datasets, the models achieve state-of-the-art performance while retaining feasible interpretable outputs.

  • Problem

    Pure machine learning had struggled to outperform classical statistical forecasting approaches, motivating investigation of deep learning without time-series-specific components.

  • Method

    N-BEATS uses hierarchical doubly residual stacking, where blocks sequentially analyze residual inputs and aggregate partial forecasts.

  • Results

    N-BEATS achieves state-of-the-art performance on M3, M4, and TOURISM, including 11% improvement over the M4 statistical benchmark and 3% over the M4 competition winner.

  • Takeaways & Limitations

    Pure deep learning can perform exceptionally well on heterogeneous univariate forecasting without time-series domain knowledge, while constrained outputs can remain human-interpretable.

  • Takeaways & Limitations

    Generic N-BEATS outputs are arbitrary and non-interpretable because learned waveforms lack inherent structure.

Abstract

from arXiv · show

We focus on solving the univariate times series point forecasting problem using deep learning. We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers. The architecture has a number of desirable properties, being interpretable, applicable without modification to a wide array of target domains, and fast to train. We test the proposed architecture on several well-known datasets, including M3, M4 and TOURISM competition datasets containing time series from diverse domains. We demonstrate state-of-the-art performance for two configurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year's winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models. The first configuration of our model does not employ any time-series-specific components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufficient to solve a wide range of forecasting problems. Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.

1 INTRODUCTION

Time-series forecasting is financially important, yet pure machine learning has struggled to surpass classical statistical methods. N-BEATS addresses this gap with pure deep learning and interpretable outputs, outperforming statistical approaches across diverse benchmarks.

  • Time-series forecasting supports critical business activities and can have financial impact of millions of dollars per accuracy point.
  • Pure machine-learning methods ranked poorly in M4, while leading entries mostly combined or ensembled classical statistical techniques.
  • N-BEATS empirically demonstrates that pure deep learning without time-series-specific components outperforms statistical approaches on M3, M4, and TOURISM.
  • On M4, N-BEATS improves accuracy by 11% over the statistical benchmark, 7% over the best statistical entry, and 3% over the competition winner.
  • The architecture can produce interpretable outputs usable similarly to traditional seasonality-trend-level decomposition techniques.

2 PROBLEM STATEMENT

The paper formulates univariate point forecasting as predicting a future vector from an observed series history. It evaluates forecasts with scale-free metrics that account for forecast symmetry, naïve seasonal performance, or competition-specific ranking.

  • Given an observed history of length T and forecast horizon H, the task is to predict the vector of future values.
  • A lookback window of length t ending at the latest observation serves as the model input.
  • MAPE, sMAPE, and MASE are standard scale-free forecasting metrics with different error-scaling conventions.
  • MASE scales errors by the average error of a naïve predictor copying the observation m periods earlier, thereby accounting for seasonality.
  • OWA is an M4-specific metric used to rank forecasting performance.

3 N-BEATS

N-BEATS is a generic, deep fully connected forecasting architecture built from doubly residual blocks that produce interpretable or unconstrained basis-expansion forecasts. Its interpretable variant imposes trend and seasonality structure while retaining residual decomposition and ensemble training.

  • N-BEATS is designed as a simple, expressive architecture without time-series-specific feature engineering or input scaling.
  • Basic block: Each block maps a lookback window to a length-H forecast and a backcast estimate of its input.The backcast is intended to remove forecastable components before downstream blocks process the residual.
  • Basic block: The block’s fully connected network predicts forward and backward expansion coefficients, which basis functions transform into forecast and backcast outputs.
  • Doubly residual stacking: Doubly residual stacking combines sequential backcast residuals with hierarchical aggregation of partial forecasts into the final prediction.The backcast branch supports sequential signal analysis, while partial forecasts are aggregated at stack and network levels.
  • Interpretability: Generic N-BEATS learns unconstrained basis waveforms, whereas the interpretable configuration adds inductive biases for structured outputs.The generic basis has no inherent waveform structure; the interpretable design constrains stacks to represent trend and seasonality.
  • Interpretability: The interpretable architecture uses a trend stack followed by a seasonality stack, exposing separate partial forecasts after removing the trend from the input.Trend bases are polynomial, while seasonality bases are Fourier-based and periodic.
  • Ensembling: The reported ensemble uses 180 models diversified by training metric, input-window length, and random initialization, with median aggregation.

4 RELATED WORK

Time-series forecasting methods include established statistical approaches such as exponential smoothing and competition methods such as Theta, alongside other modeling categories.

  • Statistical forecasting approaches based on exponential smoothing are well established and often treated as an industry default.
  • The Theta method, which won the M3 competition, decomposes forecasts into several theta-lines.

5 EXPERIMENTAL RESULTS

N-BEATS is evaluated on diverse M4, M3, and TOURISM forecasting datasets against leading statistical, machine-learning, and hybrid methods. Both generic and interpretable configurations show strong performance, while generic N-BEATS uses no time-series-specific components.

  • Experimental setup: N-BEATS compares generic, interpretable, and ensemble configurations against leading models across M4, M3, and TOURISM.Metrics are OWA and sMAPE for M4, sMAPE for M3, and MAPE for TOURISM.
  • Aggregate results: N-BEATS achieves state-of-the-art performance on three non-overlapping datasets spanning diverse domains, frequencies, and seasonalities.The datasets are M4, M3, and TOURISM.
  • Aggregate results: 0.026 is the M4 OWA gap between N-BEATS and the competition winner, exceeding the winner’s 0.017 gap over the second entry.The cited values are 0.821−0.795 = 0.026 and 0.838−0.821 = 0.017.
  • Generic configuration: Generic N-BEATS uses no feature engineering, scaling, or internal architectural components considered specific to time-series forecasting.Its results are reported across datasets with varied domains, sampling frequencies, and seasonalities.
  • Datasets: The M4 dataset contains 100k business, financial, and economic series with sampling frequencies ranging from hourly to yearly.M3 is smaller at 3003 series, while TOURISM includes monthly, quarterly, and yearly series from governmental and academic sources.
  • Experimental setup: Training uses separate train, validation, and standard test subsets, with validation and training boundaries determined by each series’ final horizon.Hyperparameters are tuned using the train and validation subsets.
  • Interpretability results: Figure 2 shows generic stack outputs as arbitrary and non-interpretable, whereas interpretable outputs separate into slowly moving trend and regular cyclical seasonality.The generic forecast sums two generic stacks; the interpretable forecast sums trend and seasonality stacks.

6 DISCUSSION: CONNECTIONS TO META-LEARNING

The paper frames N-BEATS as a meta-learning system with network parameters acting as outer learning and basis-expansion updates acting as inner learning. Its blocks progressively transform each time-series history into forward forecasts while residual backcasts condition later blocks.

  • Meta-learning framework: Meta-learning separates an inner procedure from an outer procedure, with outer parameters influencing the inner learning process.The paper uses this two-level distinction to interpret N-BEATS.
  • Interpretability comparison: Figure 2 compares actual values and forecasts with generic-stack outputs and interpretable trend- and seasonality-stack outputs across six M4 frequencies.Rows represent yearly, quarterly, monthly, weekly, daily, and hourly examples.
  • N-BEATS as meta-learning: N-BEATS maps whole-network parameters learned by gradient descent to outer learning and block-level basis-coefficient updates to inner learning.Each block modifies expansion coefficients that feed into its basis function.
  • N-BEATS as meta-learning: Each block treats a time-series history segment as a training set, producing forward expansion coefficients that map inputs to predictions.The inner procedure proceeds through sequential stages corresponding to blocks within a stack.
  • N-BEATS as meta-learning: Backward expansion coefficients modify the next block’s input, while forward coefficients generate partial forecasts that are added into the final prediction.This links residual conditioning with forecast construction across blocks.

7 CONCLUSIONS

The paper concludes that N-BEATS is a general, flexible architecture for univariate forecasting, achieving state-of-the-art results in generic and interpretable configurations. It also reports successful multi-task learning and suggests meta-learning as a possible contributor to performance.

  • Conclusions: N-BEATS is empirically validated as a general and flexible architecture for univariate time-series forecasting.The evaluation covers M4, M3, and TOURISM competition datasets.
  • Conclusions: Both generic and interpretable N-BEATS configurations achieve state-of-the-art performance across the three evaluated datasets.The configurations support validation of both accuracy and interpretability hypotheses.
  • Conclusions: The generic configuration performs exceptionally well on heterogeneous forecasting problems without time-series domain knowledge.This is presented as an empirically validated hypothesis rather than a universal guarantee.
  • Conclusions: The interpretable configuration decomposes forecasts into distinct human-interpretable outputs while retaining strong performance.The paper frames this as a viable additional constraint on a deep-learning model.
  • Conclusions: The models train across multiple time series in a multi-task fashion, sharing and transferring individual learnings.The authors speculate that this behavior may partly reflect meta-learning and identify deeper investigation as future work.

A.3 TOURISM DATASET DETAILS

The appendix describes dataset composition and validation ablations for generic and interpretable N-BEATS architectures. Increasing stack depth improves performance before gains saturate, and combining distinct trend and seasonality basis functions provides complementary benefits.

  • TOURISM dataset: The TOURISM dataset contains monthly, quarterly, and yearly series from governmental tourism organizations and academic sources.Its series have positive observed values at every time step and a higher fraction of erratic series than M4 and M3.
  • Layer stacking: The generic-architecture ablation measures validation sMAPE while varying the number of stacks, each containing one residual block.The study asks whether stacking helps and whether different basis functions improve performance.
  • Basis synergy: The interpretable ablation uses trend and seasonality stacks with variable residual-block counts and shared weights within each stack.The largest tested interpretable network is 60 layers deep.
  • Layer stacking: Increasing the number of generic stacks decreases error until the performance gain saturates.Thirty stacks of depth five form a 150-layer network.
  • Basis synergy: Stacks based on different basis functions outperform stacks based on the same basis function, indicating complementary function classes.The paper describes this as a synergistic effect because trend and seasonality function classes have limited overlap.
  • Ensembling: An ensemble of 18 models still achieves state-of-the-art performance, showing that computational efficiency can be traded for ensemble size.The reported result indicates no over-reliance on a large ensemble.

B.3 DOUBLY RESIDUAL STACKING

The ablation study evaluates doubly residual stacking against architectures that disable backward residual or partial forward forecast links. The full topology provides a clear overall advantage, while smaller ensembles retain nearly the same OWA performance.

  • Doubly residual stacking: The doubly residual stacking topology combines backward residual connections with aggregated partial block-level forecasts.This topology is the architectural basis evaluated in the ablation study.
  • Ensemble size: Less than 0.5% of OWA performance is lost when the ensemble is reduced by a factor of 10.The ensemble-size analysis is based on N-BEATS-G M4 test performance.
  • Ablation configurations: The study compares the default DRESS configuration with parallel, no-residual, last-forward, and no-residual-last-forward alternatives.The alternatives selectively disable backward residual or partial forward forecast connections.
  • Ablation results: The doubly residual stacking topology provides a clear overall advantage over alternatives disabling either backcast residual or partial forward forecast links.The comparison uses quantitative ablation results on the M4 dataset.

C.1 DETAILED RESULTS: M4 DATASET

On M4, N-BEATS achieves state-of-the-art performance across studied subsets and shows consistent advantages over the competition winner across many non-overlapping dataset cross-sections.

  • M4 benchmark: N-BEATS achieves state-of-the-art performance on the challenging M4 benchmark using generic, interpretable, and combined configurations.The comparison includes four representative M4 competition entries: pure ML, statistical, ML/TS combination, and DL/TS hybrid.
  • M4 benchmark: 0.026 is the average OWA gap between generic N-BEATS and the M4 winner, exceeding the winner’s 0.017 gap over the second entry.The reported gaps are calculated as 0.821−0.795 and 0.838−0.821, respectively.

C.3 DETAILED RESULTS: TOURISM DATASET

N-BEATS achieves state-of-the-art performance across all TOURISM subsets and gains against the best-known approach and auto-ARIMA on the reported average MAPE.

  • TOURISM results: N-BEATS achieves state-of-the-art performance on all subsets of the TOURISM dataset.The dataset covers separate Yearly and Quarterly/Monthly competition components.
  • TOURISM results: 4.2% is N-BEATS’s average gain over LeeCBaker, while its gain over auto-ARIMA is 11.5%.These gains are reported for average MAPE across the TOURISM dataset.
  • Evaluation metric: The reported average MAPE is calculated across all time series and forecast horizons using the Yearly, Quarterly, and Monthly splits.The aggregation follows the M4 competition methodology.
  • Additional datasets: The experiment compares N-BEATS with MatFact, DeepAR, Deep State, and Deep Factors on ELECTRICITY and TRAFFIC datasets.Both datasets are aggregated to hourly data using sum for ELECTRICITY and mean for TRAFFIC.

D.1 COMMON PARAMETERS

The common experimental parameters define sampling, ensemble construction, input windows, model dimensions, and signal-trace interpretation across datasets and configurations.

  • Training samples: Training samples use a history coefficient LH to select recent anchor points before the final training observation.For M4 Yearly with horizon 6 and LH=1.5, the recent 9 points supply possible anchors.
  • Ensembling: The ensemble mixes models trained on sMAPE, MASE, and MAPE for M4 and M3, while TOURISM uses MAPE alone after validation.The authors report that metric mixing improved M4 and M3 validation performance.
  • Parameter sharing: Interpretable models work best with weights shared across stacks, whereas generic models work best without weight sharing.This is an empirical observation from the respective validation sets.
  • Input and batching: Lookback periods from 2H through 7H are mixed in one ensemble, with batch size fixed at 1024.The lookback period scales with forecast horizon H, and larger batches beyond 1024 provided little additional gain.
  • Signal traces: Numeric traces verify that interpretable stack outputs sum to the interpretable forecast, while generic stack outputs sum to the generic forecast.The traces correspond to the examples reproduced from Figure 5 and Tables 19–24.
Loading 1905.10437v4…