PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Forecast on Multiple Spatiotemporal Hierarchies

Valentina Moretti, Cesare Alippi, Andrea Cini

arXiv:2610.21414v1cs.LG

TL;DR

Existing deep hierarchical forecasting methods generally train on a single, often fixed, structure, even though different hierarchies provide distinct views of the same forecasting problem. This paper trains one model across sampled hierarchies and introduces a recursive operator for coherent forecasts. Experiments report consistent improvements over single-hierarchy training and competitive performance against forecasting baselines.

  • Problem

    Existing deep hierarchical forecasting methods generally train on a single, often fixed, hierarchy, although different hierarchies induce related tasks with distinct aggregates and information paths.

    Source: §1 · p. 2

  • Method

    The framework trains a shared forecasting model across sampled, structurally diverse hierarchies and uses a recursive operator to enforce coherence with linear time and space complexity.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    Experiments show consistent improvements over single-hierarchy training and competitive performance against forecasting baselines.

    Source: §1 · p. 3, §6 · p. 9

  • Takeaways & Limitations

    Sharing parameters across structurally different views favors representations useful across hierarchies rather than solutions specialized to one structure.

    Source: §3.1 · p. 4

    Projection-based reconciliation requires dense-matrix inversion that becomes increasingly expensive as hierarchies grow, limiting its application to large hierarchies.

    Source: §2 · p. 3

Abstract

from arXiv · show

Hierarchical forecasting exploits aggregation structures that expose multiple views of the same underlying spatiotemporal forecasting problem, while ensuring coherence among forecasts. Individual series provide fine-grained information, while progressively coarser aggregates reveal temporal and spatial patterns at different scales. In the associated deep learning architectures, the hierarchy determines which aggregate series are available and how they are processed together with individual series (e.g., through reconciliation or message-passing operators). Different hierarchies thus provide distinct learning signals and inductive biases for forecasting the same collection of time series. Nonetheless, existing architectures are typically trained on a single, often fixed, structure. In this work, we propose a graph deep learning framework that leverages variations across hierarchies as multi-task regularization. We show that jointly learning across these structures can reduce over-specialization to any single hierarchy and improve forecasting performance. Indeed, training across hierarchies provides an inductive bias toward representations that are useful across different structures, and as such, generalize better. Making forecasts coherent with the aggregation constraints for each sampled hierarchy is computationally expensive: to complement the framework, we propose a novel operator-based on a top-down approach-that produces coherent forecasts with linear time and space complexity w.r.t. the hierarchy size. Experiments show that our approach improves forecasting accuracy over training on a single structure and is competitive with current state-of-the-art methods.

1 Introduction

Hierarchies provide different aggregates, information paths, and forecasting constraints, but existing deep hierarchical methods generally train on one fixed structure. This work trains a shared model across diverse hierarchies and reports improved accuracy over single-hierarchy training.

1 Introduction

  • At inference, the model processes and reconciles each input under multiple hierarchies, then averages the resulting bottom-level forecasts.The implementation also combines graph neural networks, graph pooling, and hierarchical information propagation.Source: Figure 1 · p. 2, §1 · pp. 2–3
  • Hierarchies expose aggregates that provide additional training targets and coherence constraints, while changing the paths through which information propagates across scales.Different structures therefore induce related learning tasks that share the same bottom-level forecasting objective.Source: §1 · pp. 1–2
  • The framework samples a different hierarchy at each training step, sharing model parameters across structures to reduce overfitting to any single hierarchy.The same bottom-level forecasting objective is shared across related tasks induced by different hierarchies.Source: §1 · p. 2
  • A recursive operator combines forecasts across adjacent hierarchy levels and enforces coherence with linear time and space complexity in hierarchy size.This addresses the scaling cost of dense matrix inversions for each sampled hierarchy.Source: §1 · p. 2
  • Joint training across hierarchies improves accuracy over individual-hierarchy training across forecasting benchmarks, aggregation structures, and inductive settings.The experiments support using structural variation as regularization for the forecasting task.Source: §1 · p. 3

2 Preliminaries

Spatiotemporal graph networks combine temporal encoding with message passing among related series. Hierarchical forecasting adds progressively coarser aggregates and coherence constraints, while reconciliation adjusts forecasts to satisfy those constraints.

2 Preliminaries

  • The spatiotemporal forecasting setup predicts future observations from recent series histories, covariates, and a graph; each generic layer combines temporal encoding with neighbor aggregation and updates.The aggregation function is permutation-invariant over a node's neighbors.Source: §2 · p. 3
  • A hierarchy groups bottom-level series into progressively coarser levels, with the aggregation matrix specifying which bottom-level series contribute to each aggregate.All levels are stacked into a single series matrix.Source: §2 · p. 3
  • A coherent forecast satisfies the hierarchy's aggregation constraints, so each aggregate forecast equals the mean of its corresponding bottom-level forecasts.The aggregation matrix is formed from group membership and group sizes, alongside the bottom-level identity matrix.Source: §2 · p. 3
  • Reconciliation maps unreconciled forecasts to coherent forecasts; bottom-up and top-down methods use only one level, unlike weighted projections that combine all levels.Forecasts from other levels can provide complementary information [Bates & Granger, 1969; Hollyman et al., 2021; Di Fonzo & Girolimetto, 2024].Source: §2 · p. 3
  • Dense-matrix inversion in projection-based reconciliation becomes increasingly expensive as hierarchies grow, limiting its use on large hierarchies.The formulation is used in deep hierarchical forecasting models.Source: §2 · p. 3

3 Stochastic Hierarchy Regularization for Spatiotemporal Forecasting

The framework trains one forecasting model across sampled hierarchies, using their structural variation as regularization. It combines hierarchy-aware message passing with a recursive operator for coherent forecasts.

3 Stochastic Hierarchy Regularization for Spatiotemporal Forecasting

  • The framework samples one hierarchy per training step while keeping model parameters shared, changing the aggregates processed and the paths for information exchange.It combines shared bottom-level forecasting with hierarchy-dependent aggregate targets and computational graphs.Source: §3 · p. 4, §3.1 · p. 4

3.1 Learning over a distribution of hierarchies

  • Shared parameters are trained against bottom-level targets and hierarchy-specific aggregate targets, providing multi-task regularization across structurally different views of the same forecasting problem.At inference, forecasts from R independently sampled hierarchies are averaged; this estimates the expected bottom-level prediction and makes it less dependent on any single structure.Source: §3.1 · p. 4

3.2 Designing the hierarchy distribution

  • The hierarchy distribution can vary freely or preserve task-relevant relationships; the implementation samples k and node rankings for three-level k-MIS hierarchies.k-MIS forms intermediate clusters, which connect to a single top-level aggregate.Source: §3.2 · pp. 4–5

3.3 Pyramidal forecaster

  • The pyramidal forecaster encodes node histories, applies bottom-level GNNs and bidirectional message passing between adjacent levels, then maps final representations to forecasts.Assignment matrices aggregate representations upward and propagate them downward; the sampled hierarchy changes the aggregate nodes and paths used.Source: §3.3 · p. 5

3.4 Obtaining coherent forecasts efficiently

  • The recursive top-down operator produces coherent forecasts at every level using local child-to-parent reductions and parent-to-child broadcasts, with O(M) time and memory complexity.Descendant-weighted corrections sum to zero within sibling groups, preserving coherence between adjacent levels.Source: §3.4 · pp. 5–6

4 Related work

Prior work uses hierarchical graphs for multiresolution processing and coherent forecasting, while approaches involving multiple aggregation structures have generally applied hierarchy variation after forecasts are produced.

4 Related work

  • Hierarchical GNNs pool nodes into coarser graphs for multiresolution processing and long-range message passing, and pyramidal designs have been adopted in spatiotemporal models.The passage cites examples across graph learning and spatiotemporal forecasting, including Ying et al. (2018), Gao & Ji (2019), and Yu et al. (2021).Source: §4 · p. 6
  • Deep hierarchical forecasting methods have incorporated predefined aggregation constraints or jointly learned a hierarchy, but both train using a single hierarchy.Rangapuram et al. (2021) use predefined constraints; Cini et al. (2024) jointly learn the hierarchy and forecaster.Source: §4 · p. 6
  • Prior multi-structure approaches reconcile or combine forecasts after production, rather than using hierarchy variation as training-time regularization.The passage notes that Zhang et al. (2025) fit and reconcile separately under each hierarchy, while Pang et al. (2022) and Mattera et al. (2024) jointly use multiple groupings during reconciliation.Source: §4 · p. 6

5 Experiments

Experiments evaluate SHR on sensor-network forecasting benchmarks, comparing it with established models and single-hierarchy variants, and testing hierarchy diversity, reconciliation scalability, and inductive generalization.

Experimental settings

  • Experiments use four real-world sensor-network datasets and evaluate inductive generalization on GLA, a large-scale traffic dataset.The four datasets cover traffic, air quality, and weather; reported performance is mean and standard deviation of MAE across independent runs.Source: §5 · p. 6

Forecasting performance and hierarchy variants

  • SHR is competitive with several established baselines across datasets and outperforms both single-hierarchy variants in bottom-level MAE.Table 1 reports bottom-level performance across datasets; Table 2 compares single- and multiple-hierarchy training and reconciliation procedures.Source: §5 · pp. 6–7, Table 2 · p. 7, Table 1 · p. 7

Sensitivity analysis

  • Increasing training-hierarchy diversity generally improves bottom-level accuracy, and averaging more test-hierarchy forecasts consistently improves accuracy for fixed and varying N1.The analysis also reports similar gains at higher aggregation levels under three test hierarchies; these trends persist without test-time averaging.Source: §5 · pp. 7–8, Figure 3 · p. 8, Figure 4 · p. 8
  • Randomized k-MIS rankings perform best on most datasets, particularly AQI and EngRAD, across the hierarchy distributions compared in Table 3.The evaluated distributions include fixed and randomized rankings for k ∈{2, 3}, as well as randomized rankings with sampled k.Source: Table 3 · p. 7, §5 · p. 8

Computational scalability

  • The recursive coherence operator uses substantially less inference time and GPU memory than projection-based reconciliation, with the gap widening as graph and hierarchy sizes increase.Its time and memory complexities grow approximately linearly, while projection-based reconciliation grows faster; the reported comparison also finds no loss of predictive performance.Source: §5 · pp. 8–9, Figure 5 · p. 9

Inductive setting

  • In the GLA inductive setting, both SHR variants outperform the single-hierarchy model, supporting use when the graph and its aggregation structure change.HiGP is excluded because its node-embedding-based hierarchy construction cannot directly transfer to unseen graphs.Source: §5 · p. 9

6 Conclusion

The framework trains a shared model across structurally diverse hierarchies and includes a scalable procedure for coherent forecasts. Experiments show consistent gains over single-hierarchy training and competitive performance against forecasting baselines.

6 Conclusion

  • Experiments show consistent improvements over single-hierarchy training and competitive performance against forecasting baselines.Table 4 reports performance on the GLA inductive setup and marks the best and second-best MAE.Source: §6 · p. 9, Table 4 · p. 9
  • A single model learns across multiple aggregation structures, with hierarchy variation used as regularization for spatiotemporal forecasting.Future work could explore other hierarchy distributions and forecasting architectures, as well as jointly varying aggregation levels across time and space.Source: §6 · p. 9
  • A scalable procedure produces coherent forecasts under each hierarchy.Source: §6 · p. 9

A Data

The experiments use traffic, air-quality, and weather datasets, with a separate inductive GLA setup. Baselines span recurrent, graph, hierarchical, and transformer-based forecasting approaches.

A Data

  • Traffic, air-quality, and weather datasets support the evaluation, alongside an inductive GLA benchmark built from disjoint graph partitions.GLA uses 3,834 sensors and 98,703 directed, weighted edges; ten balanced partitions are split into training, validation, and test groups, and the figure maps the node split and partitions.Source: A Data · pp. 13–14, Figure 6 · p. 14, Table 6 · p. 15

B Baselines

  • Baselines include recurrent and graph models, a transformer, and methods using hierarchical processing or learned graph dynamics.SHR adopts the TTS pipeline used by HiGP, GConv-TTS, and GUNet-TTS; node embeddings are disabled for all models in the inductive setting.Source: B Baselines · p. 15

C Experimental Details and Additional Results

Models are trained with MAE loss, and reported results average at least three runs with different random seeds. Supplementary results report performance at selected forecasting horizons.

C Experimental Details and Additional Results

C.1 Forecasting performance and hierarchy variants

The hierarchy variants differ in how structures are chosen and averaged during evaluation. Forecasting errors at selected horizons are reported as broadly consistent with the main results.

C.1 Forecasting performance and hierarchy variants

  • Forecasting errors at selected horizons are broadly consistent with the results in Table 1.Table 7 reports performance across datasets at selected horizons and marks the best and second-best MAE.Source: §C.1 · p. 16
  • SHR averages test forecasts over 20 independently sampled k-MIS hierarchies, whereas Fixed hier. uses one spectral-clustering hierarchy throughout the forward pass.The fixed hierarchy governs input aggregation, hierarchical message passing, and forecast reconciliation.Source: §C.1 · p. 16
  • SHR (rand.) samples balanced assignments with N1 from 10 to 30 at each training batch and averages forecasts over three validation and 20 test hierarchies.For validation, N1 is 10, 20, or 30; at test time it is sampled uniformly from 10 to 30.Source: §C.1 · p. 16

C.2 Sensitivity analysis

The sensitivity analysis tests how varying node assignments and hierarchy structure affects forecasting, including evaluation on hierarchies not seen during training. The scalability experiments find that the recursive procedure retains its advantage when processing multiple hierarchies, with the reconciliation procedure largely determining the observed differences.

C.2 Sensitivity analysis

  • Which variation helps more—hierarchy structure or node assignments—differs across datasets.The analysis constructs intermediate aggregates with spectral clustering, except for AQI, where K-means addresses unbalanced spectral partitions.Source: §C.2 · p. 16
  • Training across node assignments improves L0 predictions on fixed hierarchies with N1=10 and 30, cardinalities absent during training.These retests omit inference-time averaging, isolating the effect of training with different node assignments.Source: §C.2 · p. 16
  • Retests on common fixed hierarchies without inference-time averaging show decreasing trends consistent with those in Sec. 5.The evaluation varies both node assignments and the number of intermediate nodes.Source: §C.2 · p. 17

C.3 Computational scalability

  • Training scalability follows trends similar to those observed for inference, and the recursive procedure retains its advantage when processing ten hierarchies.Projection-based reconciliation processes one hierarchy in the comparison; the recursive procedure processes each of ten hierarchies and averages the coherent forecasts.Source: §C.3 · p. 17
  • When hierarchy-building and sampling costs are included, the scalability trends remain consistent with the other experiments.The comparison profiles HiGP with projection-based reconciliation and SHR with the recursive procedure; the procedure for obtaining coherent forecasts largely determines the observed differences.Source: §C.3 · pp. 17–18

C.4 Inductive setting

On held-out GLA graphs, both SHR variants outperform the fixed-hierarchy baseline at horizons 3, 6, and 12; random hierarchy sampling yields lower mean errors than k-MIS. The hierarchy distributions differ between the two SHR variants, which matters when interpreting their performance.

C.4 Inductive setting

  • Both SHR variants outperform the fixed-hierarchy baseline at horizons 3, 6, and 12 on held-out GLA graphs.Table 8 reports MAE performance at these selected horizons.Source: §C.4 · p. 18, Table 8 · p. 18
  • Random hierarchy sampling yields lower mean errors than k-MIS, but their hierarchy distributions differ.Random sampling constrains aggregate counts to {10, . . . , 30} and produces approximately equal-sized groups, while k-MIS can yield different aggregate counts depending on graph topology.Source: §C.4 · p. 18

D Implementation Details

This section lists the implementation software and describes bottom-level performance and scalability evaluations across hierarchy configurations.

D Implementation Details

  • The code is implemented in Python (Van Rossum & Drake, 2009) and uses PyTorch (Paszke et al., 2019), alongside other listed libraries.The additional libraries are PyTorch Geometric (Fey & Lenssen, 2019), PyTorch Lightning (Falcon & The PyTorch Lightning team, 2019), Torch Spatiotemporal (Cini & Marisca, 2022), Hydra, and NumPy (Harris et al., 2020).Source: D Implementation Details · p. 18
  • Bottom-level performance is evaluated across four datasets while varying training-hierarchy count, assignment diversity, intermediate aggregates, and test-hierarchy averaging.Figure 8 evaluates Metr-LA, PeMS-Bay, AQI, and EngRAD without inference-time hierarchy averaging; Figure 9 covers PeMS-Bay, AQI, and EngRAD and varies test-forecast averaging.Source: Figure 8 · p. 19, Figure 9 · p. 20
  • Training scalability compares projection-based reconciliation with the recursive procedure using training time and peak GPU memory.Source: Figure 10 · p. 20
  • Inference scalability compares projection-based reconciliation using one hierarchy with the recursive procedure using ten, reporting inference time and peak GPU memory.Source: Figure 11 · p. 21
  • Inference-time scaling of HiGP and SHR is evaluated as the number of graph nodes increases, tracking inference time and peak GPU memory.Source: Figure 12 · p. 21

Related papers

Loading 2610.21414v1…