PaperlayerView exact revision on arXiv

Source-linked AI summary

N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting

Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler-Canseco, Artur Dubrawski

arXiv:2201.12886v6cs.LGcs.AI

TL;DR

Long-horizon forecasting suffers from prediction volatility and high computational complexity. N-HiTS combines multi-rate input sampling with hierarchical interpolation to specialize forecasts across scales, reporting stronger accuracy and substantially lower computation than Transformer-based methods. Its hierarchical interpolation also has approximation guarantees for arbitrarily long horizons under smoothness assumptions.

  • Problem

    Long-horizon neural forecasting remains difficult because forecast volatility and computational complexity become especially pronounced as the horizon grows.

  • Method

    N-HiTS combines multi-rate input sampling and hierarchical interpolation to decompose inputs and synthesize forecasts across different frequencies and scales.

  • Results

    N-HiTS significantly outperforms previous Transformer-based multivariate models while using an order of magnitude less computation across six benchmark datasets.

  • Takeaways & Limitations

    N-HiTS provides an interpretable nonlinear decomposition alongside improved long-horizon forecasting accuracy and computational efficiency.

  • Takeaways & Limitations

    The study operates in the univariate regime and motivates further research on effectively using information from multiple variables.

Abstract

from arXiv · show

Recent progress in neural forecasting accelerated improvements in the performance of large-scale forecasting systems. Yet, long-horizon forecasting remains a very difficult task. Two common challenges afflicting the task are the volatility of the predictions and their computational complexity. We introduce N-HiTS, a model which addresses both challenges by incorporating novel hierarchical interpolation and multi-rate data sampling techniques. These techniques enable the proposed method to assemble its predictions sequentially, emphasizing components with different frequencies and scales while decomposing the input signal and synthesizing the forecast. We prove that the hierarchical interpolation technique can efficiently approximate arbitrarily long horizons in the presence of smoothness. Additionally, we conduct extensive large-scale dataset experiments from the long-horizon forecasting literature, demonstrating the advantages of our method over the state-of-the-art methods, where N-HiTS provides an average accuracy improvement of almost 20% over the latest Transformer architectures while reducing the computation time by an order of magnitude (50 times). Our code is available at bit.ly/3VA5DoT

1 Introduction

Long-horizon forecasting is important but becomes increasingly difficult as neural models produce more volatile predictions with rising computational costs. N-HiTS addresses these issues through multi-rate sampling and hierarchical interpolation, with reported accuracy and efficiency gains over Transformer-based methods.

  • Long-horizon forecasting supports planning, maintenance, early warning, and healthcare monitoring applications.
  • Forecasting errors and computation costs inflate dramatically as the forecast horizon grows, with attention and fully connected layers scaling quadratically in horizon length.
  • N-HiTS combines multi-rate data sampling with multi-scale interpolation to specialize predictions across signal frequencies and reduce long-horizon computation.
  • 16% accuracy improvements over existing state-of-the-art Transformer-based techniques were reported across a large array of multivariate forecasting datasets.

2. Hierarchical Interpolation:

N-HiTS uses multi-scale hierarchical interpolation to smooth multi-step predictions and assign different blocks to different signal frequency bands. The approach is evaluated on six large-scale forecasting datasets.

  • Hierarchical Interpolation: Multi-scale hierarchical interpolation enforces smoother multi-step predictions by reducing prediction dimensionality and matching prediction time scale to the final output.
  • Hierarchical Interpolation: N-HiTS synchronizes input sampling rates with output interpolation scales so blocks specialize in forecasting distinct frequency bands.
  • The evaluation covers six datasets spanning electricity transformer temperature, exchange rate, electricity consumption, highway traffic, weather, and influenza-like illness.

2 Related Work

Prior neural forecasting work includes scalable attention-based and fully connected architectures, while interpolation has mainly served data completion, filtering, and quantile regression. N-HiTS applies temporal interpolation to induce multi-scale hierarchical forecasts.

  • Neural forecasting: Deep forecasting methods have been used in industrial resource allocation, transportation, retail, financial trading, and forecasting competitions.
  • Interpolation: Interpolation has been applied to complete unevenly sampled data, filter noise, and support fine-grained quantile regression with recurrent networks.
  • N-HiTS: N-HiTS architecture diagrams show MLP blocks whose multi-rate pooling, hierarchical interpolation, and residual connections specialize additive predictions in different signal bands.
  • Novelty: Temporal interpolation had not previously been used, to the authors’ knowledge, to induce multi-scale hierarchical time-series forecasts.

3 N-HiTS Methodology

N-HiTS extends N-BEATS with multi-rate input sampling and hierarchical forecast synthesis. Its blocks process different signal scales, interpolate reduced-dimensional predictions, and combine residual forecasts, with theoretical approximation guarantees under smoothness.

  • Architecture: N-HiTS extends N-BEATS using multi-rate input sampling and multi-scale forecast synthesis to construct forecasts hierarchically with lower computational requirements.
  • Architecture: Each block uses an MLP to predict backcast and forecast basis coefficients; backcasts clean later inputs while forecasts are summed into the final prediction.
  • Multi-Rate Signal Sampling: MaxPool layers with block-specific kernel sizes expose different effective sampling rates, with larger kernels emphasizing large-scale, low-frequency content.
  • Multi-Rate Signal Sampling: Multi-rate processing narrows most MLP inputs, reducing memory, computation, and parameters while maintaining the original receptive field.
  • Hierarchical Interpolation: Temporal interpolation sets coefficient dimensionality through expressiveness ratio r_l and reconstructs all H horizon points from reduced-dimensional predictions.
  • Hierarchical Interpolation: Blocks near the input use smaller expressiveness ratios and larger pooling kernels, producing low-granularity forecasts before later blocks address remaining signal bands.
  • Theoretical Guarantee: Hierarchical interpolation can approximate arbitrarily long horizons when its basis functions provide multi-resolution approximation and forecast relationships are smooth.
  • Theoretical Guarantee: Piece-wise constants, piece-wise linear functions, and splines are examples of multi-resolution functions with arbitrary approximation capabilities.

4 Experimental Results

Experiments evaluate N-HiTS on established long-horizon forecasting datasets using MAE and MSE, comparing it with multivariate and univariate baselines. N-HiTS achieves stronger forecasting accuracy, lower computational cost, and interpretable scale-specialized decompositions when both proposed enhancements are used.

  • Evaluation Setup: The evaluation uses publicly available long-horizon datasets, MAE and MSE metrics, and multivariate Transformer-based and univariate baselines.Metrics are averaged across dataset features for multivariate datasets, while N-HiTS predicts each variable from its own history.
  • Forecasting Accuracy: N-HiTS outperforms the best baseline, with average relative error decreases of 14% in MAE and 16% in MSE across datasets and horizons.At the longest measured horizon, multivariate MAE decreases by 11% and MSE by 17%.
  • Computational Efficiency: N-HiTS is 45× faster than Autoformer and uses less than 26% of the parameters of the second-best alternative.Compared with N-BEATS, it is 1.26× faster and requires only 54% of the parameters.
  • Ablation Studies: Combining hierarchical interpolation and multi-rate sampling produces the best performance among the evaluated N-HiTS variants.The ablation compares the full model with variants containing only one enhancement or neither enhancement.
  • Ablation Studies: N-HiTS produces interpretable forecast decompositions in which different stacks specialize in trends and seasonality, unlike the control model.The qualitative comparison uses ETTm2 forecasts 720 steps ahead and examines forecast components and residuals.
  • Ablation Studies: The preferred hierarchical interpolation order synthesizes and removes low-frequency, large-scale components before modeling higher-frequency intermittent signals.Linear interpolation also outperforms nearest-neighbor interpolation consistently in the reported ablations.

5 Discussion of Findings

The findings support multi-rate sampling and hierarchical interpolation as complementary inductive biases that improve forecasting while enabling interpretable decomposition. However, the results also leave open whether multivariate baselines fail because of marginal overfitting or weak cross-variable integration.

  • Multi-rate sampling and hierarchical interpolation provide a useful inductive bias compared with both free-form N-HiTS4 and parametric N-BEATSi.The comparison identifies these components as complementary and effective for long-horizon forecasting.
  • N-HiTS combines strong baseline performance with an interpretable non-linear decomposition of forecasts.Synthetic and real-data examples show specialization and reconstruction of latent harmonic signals.
  • Existing long-horizon multivariate approaches are substantially outperformed by N-HiTS despite its univariate inputs.The authors frame this result as a question about marginal overfitting, model parsimony, or inadequate multivariate information integration.

6 Conclusions

N-HiTS combines multi-rate input sampling with hierarchical interpolation to produce improved, interpretable, and computationally efficient long-horizon forecasts. Operating univariately, it outperforms prior Transformer-based multivariate models while using substantially less computation.

  • N-HiTS combines multi-rate input sampling and hierarchical interpolation for long-horizon forecasting.The resulting predictions are described as improved, interpretable, and computationally efficient.
  • N-HiTS uses only each predicted time series’ history yet significantly outperforms previous Transformer-based multivariate models.The paper presents this as a new baseline for multivariate long-horizon forecasting on six popular datasets.
  • N-HiTS uses an order of magnitude less computation than previous Transformer-based multivariate models.The conclusion motivates further research into methods that effectively use information from multiple variables.

A Neural Basis Approximation Theorem

The theorem establishes that hierarchical interpolation can approximate arbitrarily long continuous forecast horizons when multi-resolution functions are sufficiently expressive and forecast relationships are smooth. The appendix builds this result from piecewise-constant approximation and neural approximation of the resulting coefficients.

  • Theorem: Hierarchical interpolation can arbitrarily approximate infinitely long continuous forecast horizons under smoothness conditions.The result assumes interpolating functions are projections onto informed multi-resolution functions and forecast relationships satisfy smoothness conditions.
  • Lemma 1: Any square-integrable forecast function can be approximated by linear combinations of piecewise-constant multi-resolution basis functions.The basis functions are indexed by frequency or interval length and time location.
  • Lemma 1: The multi-resolution basis uses Haar scaling functions whose frequency and active interval are controlled by w and whose location is controlled by h.The projection onto these functions yields an epsilon-approximation for sufficiently fine resolution.
  • Lemma 2: If the projected coefficients vary smoothly with the input history, a three-layer ReLU network can approximate those coefficients.A K-Lipschitz coefficient relationship is given as an example of the required smoothness condition.
  • Proof: The theorem combines arbitrary basis-function approximation with neural approximation of finitely many multi-resolution coefficients.The proof invokes Haar approximation, neural approximation, the triangular inequality, Fubini’s theorem, and bounds on indicator functions.
  • Architecture: N-HiTS reduces prediction dimensionality and synchronizes input sampling with output interpolation so blocks specialize in different frequency bands.Expressivity ratios and multi-rate sampling distribute different time-scale roles across the architecture.

E Main results standard deviations

The reported improvements are robust across repeated forecasting-pipeline runs, while concurrent comparisons preserve an advantage except for a short-horizon Exchange setting.

  • 2.9% MSE and 1.75% MAE standard deviations across eight runs support the robustness of N-HiTS accuracy improvements.Exchange exhibits the greatest variance between runs.
  • 11% MAE and 9% MSE improvements over the second-best alternative hold across benchmark datasets and horizons.The only reported exception is short-horizon Exchange, where ETSformer improves MSE by 9% and MAE by 4%.

F Univariate Forecasting

The univariate experiments compare N-HiTS with Transformer-based and established forecasting benchmarks on ETTm2 and Exchange datasets. N-HiTS significantly improves accuracy across datasets and forecast horizons.

  • Benchmark setup: The appendix positions these experiments as a close comparison with methods specialized in long-horizon forecasting.The comparison focuses on the univariate setting for ETTm2 and Exchange.
  • Benchmark setup: N-HiTS significantly improves over alternative univariate forecasters across datasets and horizons.The experiments compare against Autoformer, Informer, LogTrans, Reformer, N-BEATS, DeepAR, and Prophet.
  • Results: 17% lower MAE and 25% lower MSE are reported for N-HiTS relative to the best alternative across datasets and horizons.These results are summarized in Table A5.
  • Evaluation protocol: The evaluation progressively prolongs the forecast horizon to assess long multi-horizon univariate forecasting performance.Table A5 reports MAE and MSE averaged over eight runs.

G Ablation Studies

Ablation studies examine pooling, interpolation, hierarchical order, and cross-architecture effects. The results favor max pooling, linear or cubic interpolation over nearest neighbors, top-down ordering, and hierarchical interpolation as the main transferable component.

  • Pooling configurations: MaxPool consistently outperforms AveragePool, with MAE improvements up to 15% and MSE improvements up to 8% at the most extended horizon.The average accuracy across datasets and horizons also favors MaxPool.
  • Interpolation configurations: Linear and cubic interpolation consistently outperform nearest-neighbor interpolation, with improvements increasing monotonically along the forecasting horizon.The comparison evaluates nearest-neighbor, linear, and cubic alternatives.
  • Interpolation configurations: 15.8% is the maximum improvement from linear interpolation over nearest neighbors, versus 7.0% for cubic interpolation.Linear and cubic results are otherwise inconclusive relative to each other; linear is favored on average for accuracy and computational performance.
  • Order of hierarchical representations: Top-down predictions outperform bottom-up predictions, with relative improvements of 4.6% in MAE and 7.5% in MSE across horizons and datasets.The top-down hierarchy prioritizes low frequencies before complementing them with higher-frequency details.
  • Enhancements beyond N-HiTS: In DilRNN, hierarchical interpolation drives the main improvements, while adding multi-rate sampling sometimes produces the best performance with marginal differences.Multi-rate sampling does not substantially improve DilRNN, unlike the complementarity observed in the MLP-based N-HiTS setting.
  • Enhancements beyond N-HiTS: The authors did not pursue more complex architectures because their interpretability and signal-decomposition capabilities were not worth losing.Hierarchical interpolation nevertheless exhibits significant benefits in other architectures.

I Hyperparameter Optimization Resources

The section evaluates how computational efficiency affects hyperparameter exploration and training economics. N-HiTS achieves substantial accuracy gains over Autoformer while requiring much less training time and cost.

  • Motivation: Computational efficiency matters because forecasting systems are repeatedly retrained for distributional shifts, linking training speed to deployment cost.The section examines accuracy gains and economic costs associated with hyperparameter optimization and training.
  • Hyperparameter optimization: N-HiTS shows monotonic relative performance gains over the best reported Autoformer as hyperparameter configurations are explored.The experiment uses Bayesian optimization with HYPEROPT and reports results across ablation datasets.
  • Training economic costs: 1.5 hours is the measured training time for N-HiTS1, compared with 92.6 hours for Autoformer and 62.1 hours for Informer.N-HiTS1 uses one randomly sampled HYPEROPT iteration, while Transformer models use repository-reported optimal hyperparameters.
  • Training economic costs: 17% MSE average performance gain over Autoformer is achieved by N-HiTS1 at 1.6% of a single-run cost.N-HiTS20 achieves almost 25% gain at 33% of a single-run cost.
  • Training economic costs: Nearly USD 70.0 is estimated for Autoformer, USD 46.5 for Informer, under USD 1.5 for N-HiTS1, and USD 22.8 for N-HiTS20.The estimates use USD 0.75 per hour for a g4dn.2xlarge instance.
Loading 2201.12886v6…