PaperlayerView exact revision on arXiv

Source-linked AI summary

HypLTSF: A Hyperbolic Geometric View of Multi-Scale Hierarchies for Long-Term Time Series Forecasting

Namwoo Kim, Hyungryul Baik, Yoonjin Yoon

arXiv:2609.08286v1cs.LG

TL;DR

Existing multi-scale forecasting methods exchange information across temporal resolutions without explicitly representing the hierarchy between fine and coarse scales. HypLTSF embeds scale-wise representations in the Poincaré ball, uses radial and angular hierarchy losses, and achieves state-of-the-art performance across eight benchmarks.

  • Problem

    Existing multi-scale methods exchange information across scales but generally do not explicitly encode their hierarchical ordering in representation geometry.

  • Method

    HypLTSF constructs multi-scale representations, embeds them in the Poincaré ball, and applies radial ordering and angular coherence losses.

  • Results

    HypLTSF achieves state-of-the-art performance across eight benchmarks, ranking first in 30 of 32 MAE settings and 20 of 32 MSE settings.

  • Takeaways & Limitations

    Explicitly modeling the multi-scale hierarchy as a geometric structure provides an effective approach for long-term time series forecasting.

Abstract

from arXiv · show

Multi-scale modeling has become an effective approach for long-term time series forecasting, capturing temporal patterns that range from fine-grained local dynamics to coarse global trends. Representations across these temporal scales are inherently hierarchical, with coarser scales abstracting and aggregating information from finer ones. While existing approaches readily exchange information across these scales, the hierarchy itself is typically left as an emergent byproduct of such interactions rather than captured as a geometric structure in its own right. In this paper, we introduce HypLTSF, a framework that endows the multi-scale hierarchy with a concrete geometric form by embedding scale-wise representations into the Poincaré ball, whose exponentially expanding volume naturally accommodates hierarchical structures. To align this geometry with the temporal hierarchy, HypLTSF imposes two constraints: (1) a radial constraint that orders embeddings by their level of abstraction, and (2) an angular constraint that groups fine-scale patterns sharing a common coarser-scale ancestor. Extensive experiments on long-term time series forecasting benchmarks show that HypLTSF achieves state-of-the-art performance, suggesting that explicitly modeling the multi-scale hierarchy as a geometric structure is effective for forecasting.

I. INTRODUCTION

Multi-scale forecasting captures temporal patterns at different resolutions, but existing methods generally exchange information across scales without explicitly encoding their parent–child hierarchy. HypLTSF models this hierarchy geometrically in hyperbolic space using complementary radial and angular constraints.

  • Multi-scale modeling represents time series at multiple temporal resolutions to capture complex and heterogeneous temporal dynamics.
  • Progressive downsampling or convolution aggregates fine-scale observations into coarser representations, forming parent–child relationships across temporal scales.
  • Existing multi-scale methods exchange information across scales but generally do not structurally enforce hierarchical relationships within the representation space.
  • HypLTSF embeds scale-wise representations in the Poincaré ball and uses radial ordering and angular coherence losses to shape a tree-like hierarchy.
  • HypLTSF achieves state-of-the-art performance across eight benchmark datasets, with results indicating benefits from explicit hierarchical modeling.

B. Hyperbolic Embedding

Hyperbolic embeddings are suited to hierarchical structures because distances expand rapidly with radius, yet their use for long-term time series forecasting remains limited. HypLTSF addresses this gap by imposing geometric ordering on multi-scale temporal representations.

  • Hyperbolic embeddings are widely used for hierarchical or tree-like data because distances grow exponentially with radius.
  • Hyperbolic representations have been applied to hierarchical text and graph structures, including semantic taxonomies and latent graph hierarchies.
  • Hyperbolic geometry has not been widely studied for continuous multivariate long-term time series forecasting, where hierarchy is induced by temporal downsampling.
  • Multi-scale forecasting captures trends, seasonal patterns, and short-term fluctuations that are difficult to model at a single temporal scale.
  • HypLTSF directly models scale hierarchy in hyperbolic space, placing fine-grained child representations deeper than coarse parent representations.
  • The Poincaré ball provides a hyperbolic representation space in which distance from the origin serves as a measure of hierarchical depth.

B. Operations in hyperbolic space

The framework reviews hyperbolic operations used to manipulate representations in the Poincaré ball. These operations include Möbius arithmetic, exponential mapping, and the Einstein midpoint.

  • Möbius addition and scalar multiplication provide hyperbolic analogues of basic vector operations in the Poincaré ball.
  • The exponential map transfers representations between a tangent space and the hyperbolic manifold.
  • The Einstein midpoint serves as a hyperbolic analogue of the Euclidean centroid for aggregating points.
  • The Lorentz factor gives greater weight to points nearer the boundary to compensate for rapid hyperbolic distance growth there.

C. Problem Statement

Multivariate time series forecasting maps historical observations of multiple variables to future values over a specified prediction horizon. HypLTSF adds a structural constraint that explicitly enforces hierarchy among its multi-scale representations.

  • Multivariate forecasting predicts future values of multiple variables from their historical observations.
  • The forecasting function maps an input X_t in R^(C×L) to predictions Ŷ_t in R^(C×H) over horizon H.
  • HypLTSF explicitly enforces hierarchical ordering among scales in a multi-scale forecasting framework.

A. Multi-scale Decomposition

HypLTSF constructs a multi-scale pyramid through progressive average pooling, then independently normalizes and decomposes each scale into trend and seasonal components. Dedicated encoders process these components with scale-specific temporal weights and shared feature weights.

  • Scale construction: Progressive average pooling constructs a pyramid of S+1 scales with progressively shorter temporal lengths.The downsampling window w determines each scale's temporal resolution.
  • Normalization: Each scale is independently normalized with learnable affine parameters to handle distribution shifts across resolutions.Normalization is inverted after prediction to recover the original scale.
  • Trend–seasonal decomposition: Each scale's channel-independent representation is decomposed into trend and seasonal components.The decomposition follows classical time series analysis.
  • Component encoding: Dedicated encoders process trend and seasonal components to capture temporal and cross-feature dependencies in successive stages.The temporal stage uses scale-specific two-layer MLPs because scales have different temporal lengths.
  • Component encoding: The second encoder stage is shared across scales, enabling cross-feature interaction in a consistent representation space.Separate encoders retain independent temporal weights while sharing feature weights across resolutions.

C. Hyperbolic Encoding

HypLTSF fuses trend and seasonal encodings before mapping scale-wise representations into the Poincaré ball. Hyperbolic projections and hierarchy losses then impose a depth ordering in which finer scales lie deeper than coarser scales.

  • Component fusion: Trend and seasonal encodings are fused in Euclidean space using learnable per-scale weights before hyperbolic projection.The weights adapt the relative importance of each component at each scale.
  • Hyperbolic projection: The fused representation is mapped to the Poincaré ball through an exponential map at the origin and a scale-specific hyperbolic linear layer.The layer uses Möbius matrix-vector multiplication and bias addition.
  • Hyperbolic projection: The hyperbolic linear layer transforms d_model-dimensional features into d_h-dimensional representations using learnable weights and bias.The bias is mapped onto the ball via the exponential map before Möbius addition.
  • Hierarchy shaping: Norm clipping after each Möbius operation stabilizes computation, after which hierarchy losses impose finer-than-coarser embedding depth.The resulting embeddings are organized across scales by the hierarchy losses.
  • Pipeline overview: The overview combines pyramid construction, trend–seasonal fusion, hyperbolic encoding, hierarchy losses, and tangent-space prediction.These stages form the complete HypLTSF pipeline.

D. Hierarchy Losses

HypLTSF preserves the temporal hierarchy through complementary radial and angular constraints in the Poincaré ball. Radial ordering encodes parent–child depth, while angular coherence organizes siblings into local branches.

  • Hierarchy structure: Progressive downsampling creates parent–child relationships in which each parent corresponds to w consecutive children at the finer scale.This relationship provides the structural basis for the hierarchy losses.
  • Loss inputs: Channel aggregation produces one representation per time step and scale before hierarchy-loss computation.The Einstein midpoint yields a channel-aggregated representation on which subsequent losses operate.
  • Radial ordering: Radial ordering places each child deeper than its parent, using distance from the origin to encode hierarchical depth.The squared hinge penalizes violations, while a learnable margin determines radial separation.
  • Angular coherence: Angular coherence compares child directions in the tangent space at their parent and pulls them toward their normalized mean branch direction.This local comparison respects the Poincaré ball's geometry.
  • Joint effect: The radial loss organizes depth, whereas the angular loss clusters siblings into coherent subtrees.Together, the losses shape complementary radial and angular axes of the embedding space.

E. Prediction

HypLTSF generates predictions by returning hyperbolic embeddings to the tangent space at the origin and applying scale-specific projections. It then combines forecasts from all scales through a learnable weighted aggregation.

  • Tangent-space predictor: Hyperbolic embeddings are mapped to the tangent space at the origin before standard linear prediction operations.The tangent space is locally Euclidean and supports the predictor's linear projections.
  • Tangent-space predictor: Scale-specific projections produce forecasts by operating along the temporal dimension and reducing the hyperbolic feature dimension to univariate channel predictions.W_time projects temporally, while W_dim reduces the representation to one value per channel.
  • Multi-scale aggregation: The final forecast is a learnable weighted combination of predictions from all scales.The aggregation weights are parameterized by learnable δ(s) values.

F. Training Objective

HypLTSF combines prediction with hierarchy constraints and is evaluated across diverse long-term forecasting benchmarks. It achieves especially strong MAE performance across datasets and horizons, with larger gains on some larger datasets.

  • Training Objective: The complete objective combines prediction loss with radial and angular hierarchy regularizers, guiding temporal resolution to hyperbolic depth and local context to angular position.The prediction term uses L1 loss, while λr and λa control regularizer strengths.
  • Experimental Setup: HypLTSF is evaluated on eight benchmarks, including ETT, Weather, Electricity, Traffic, and Solar, against broad and recent multi-scale baselines.Evaluation uses horizons 96, 192, 336, and 720 with MSE and MAE, while varying look-back windows are considered.
  • Main Results: 30 of 32 horizon settings rank HypLTSF first for MAE, while it ranks first in 20 for MSE across the eight datasets.At dataset-average level, HypLTSF has the lowest MAE on all eight datasets and the lowest MSE on six.
  • Main Results: 14.7% and 7.9% MAE reductions versus TimeMixer occur on Solar and Traffic, respectively, despite their shared MLP-based backbone.The reported Solar values are 0.252→0.215, and the Traffic values are 0.279→0.257.

C. Ablation Studies

Ablations show that the radial and angular hierarchy losses complement one another, shape the intended depth ordering, and generally help more at longer horizons and larger scales. Controlled comparisons further support hyperbolic geometry and show that the proposed module transfers across multi-scale architectures.

  • Loss Complementarity: The radial loss generally contributes more individually, but the relative contribution varies across datasets, with the angular loss stronger in some Traffic settings.On ETTh2 at H=720, radial loss alone nearly recovers the full-model improvement, changing MSE from 0.410 to 0.399.
  • Loss Complementarity: Combining both losses reduces ETTh1 H=720 MSE from 0.443 to 0.433, whereas either loss alone produces no meaningful improvement there.The losses organize radial depth and angular relationships jointly; the full model reduces Solar MSE by 9.1% at H=720.
  • Effect of Horizon and Dataset Scale: 9.1% MSE reduction on Solar at H=720 shows larger hierarchy-loss benefits at longer horizons and on larger-scale datasets.Traffic still gains 1.4% at H=720 despite its 862 channels.
  • Emergence of Hierarchical Structure: Scale 1 > Scale 2 > Scale 3 emerges during training, placing finer resolutions deeper near the Poincaré-ball boundary and coarser resolutions shallower near the origin.After training, the per-scale depth distributions occupy distinct radial bands with minimal overlap.
  • Advantage of Hyperbolic Space: 9.5% Solar MSE improvement at H=720 favors hyperbolic over Euclidean representations, compared with 1.0% at H=192.The H=720 values are 0.220→0.199, and the controlled variants differ only in representation geometry.
  • Plug-and-Play Experiment: The hyperbolic embedding and hierarchy losses improve most settings when inserted into TimeMixer and MICN, though gains vary by base model and horizon.On ETTm1 at H=96, MSE decreases from 0.293 to 0.288 for TimeMixer and from 0.303 to 0.289 for MICN; one longer-horizon exception increases MSE while reducing MAE.

G. Model Efficiency Analysis

HypLTSF is evaluated for both forecasting performance and computational efficiency on Traffic and Solar. It achieves strong accuracy while maintaining competitive computational costs across these large-scale datasets.

  • Table V compares efficiency and forecasting performance on the Traffic and Solar datasets.The evaluation considers memory consumption, training time, inference time, and forecasting performance.
  • HypLTSF achieves the best forecasting accuracy on Traffic with only moderate training and inference-time increases compared with TimeMixer.Traffic uses 862 variables, a look-back window of 512, and a forecasting horizon of 192.
  • HypLTSF achieves the best forecasting accuracy on Solar while also providing the fastest training and inference.Solar uses 137 variables, a look-back window of 512, and a forecasting horizon of 336.
  • TimeKAN has the lowest memory consumption but higher computational latency on both datasets.
  • The results indicate that explicit hierarchical modeling improves forecasting accuracy while maintaining competitive computational efficiency.The framework encodes scale depth and angular proximity as hierarchical structure in the Poincaré ball.
Loading 2609.08286v1…