PaperlayerView exact revision on arXiv

Source-linked AI summary

Channel-Dependent State Space Model for Multivariate Time Series Forecasting

Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu, Duane S. Boning

arXiv:2609.36453v1cs.LGcs.AI

TL;DR

MTSF models must accommodate varying dependencies among variables while remaining generalizable and scalable. Chameleon adds a Kalman-inspired cross-variable update to a GatedDeltaNet-based temporal backbone, with reported wins on strongly dependent datasets and benchmark settings.

  • Problem

    MTSF architectures must handle widely varying cross-variable dependencies in noisy, less abundant time series while avoiding the overfitting, computational cost, and scaling difficulty of unconstrained joint modeling.

    Source: §1 · p. 1

  • Method

    Chameleon combines a GatedDeltaNet temporal backbone with a Kalman-inspired learnable measurement update for data-dependent cross-variable modeling, and uses stochastic RevIN perturbation during training.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Chameleon achieves the best MSE and MAE on strongly dependent ODE and PEMS datasets, and ranks first in 25 of 28 benchmark settings for MSE and 20 of 28 for MAE.

    Source: §5 · p. 9

  • Takeaways & Limitations

    The results suggest Chameleon exploits strong cross-variable dependencies without sacrificing temporal modeling capacity or scalability.

    Source: §5 · p. 9

Abstract

from arXiv · show

Multivariate time series forecasting (MTSF) is critical across many real-world domains. Existing deep learning approaches fall into two paradigms with distinct limitations: channel-independent (CI) methods unconditionally ignore cross-variable dependencies and model only temporal dynamics, while channel-dependent (CD) methods consider both but typically rely on architectural compromises to mitigate overfitting and computational overhead. We therefore propose Chameleon, a specialized CD state space model (SSM) that enables data-dependent, fine-grained interactions across variables while scaling linearly with their number. By connecting selective SSMs with the Kalman filter, we leverage the missing measurement update in the former for cross-variable modeling while preserving the SSM backbone for robust temporal modeling. We further identify favorable inductive biases of GatedDeltaNet for time series, adapt it as our backbone, and improve generalization through additional techniques, including a previously unexplored stochastic perturbation of reversible instance normalization. On strongly dependent ODE and PEMS datasets, Chameleon achieves the best MSE and MAE across all settings, while its CI ablation and prior CD methods incur 61-178% higher MSE on average. Across 28 standard benchmark settings, Chameleon also achieves better MSE and MAE than each baseline in at least 27 and 22 cases, respectively. Training-time and peak-memory analyses on Traffic and ETT further demonstrate competitive efficiency and favorable memory scalability across different variable counts.

1 INTRODUCTION

MTSF must handle widely varying cross-variable dependencies while avoiding the overfitting, computational cost, and scaling difficulty of unconstrained joint modeling. Chameleon combines a temporal SSM backbone with a learnable cross-variable update and reports strong performance across standard benchmarks and strongly dependent systems.

1 INTRODUCTION

  • Unconstrained joint modeling is difficult to generalize and scale because time series are noisy and less abundant, while temporal and variable dimensions can each span hundreds of elements.Cross-variable dependencies also vary widely in strength and structure.Source: §1 · p. 1
  • CI methods omit cross-variable interactions, while CD methods often compromise modeling granularity or temporal capacity to reduce overfitting and computational cost.More expressive CD designs can cost more without consistently improving performance, leaving a trade-off among granularity, temporal capacity, generalization, efficiency, and scalability.Source: §1 · pp. 1–2
  • The paper asks whether fine-grained cross-variable interactions can be added to a strong temporal backbone efficiently, without overfitting across systems with weak and strong dependencies.The questions are motivated by the unresolved trade-offs in existing CI and CD approaches.Source: §1 · p. 2
  • GatedDeltaNet’s rapid forgetting and targeted memory rewrites are identified as favorable time-series inductive biases, and the backbone is adapted for MTSF.The paper also introduces stochastic perturbation of reversible instance normalization during training to improve generalization.Source: §1 · p. 2
  • Chameleon uses a Kalman-filter-inspired measurement update to model cross-variable interactions while retaining linear complexity in the number of variables.The design connects selective SSMs with the Kalman filter.Source: §1 · p. 2
  • Chameleon is reported to achieve state-of-the-art performance across standard long-horizon benchmarks and strongly dependent systems.The introduction states this result without providing quantitative comparisons.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries frame MTSF as forecasting future values from multivariate histories without assuming stationarity. They connect selective SSM updates to the Kalman filter’s time update and motivate a learnable measurement update for cross-variable modeling.

Problem Formulation

  • MTSF forecasts the next F steps from N variables observed over T regularly spaced steps, without explicitly using exact timestamps or assuming stationarity.Trends and seasonality are noted as prevalent in real-world time series.Source: §2 · p. 2

Selective SSMs and GatedDeltaNet

  • Selective SSMs provide efficient long-context modeling and data-dependent state updates, while GatedDeltaNet combines global decay for rapid forgetting with delta-rule targeted memory rewrites.Its input-dependent adaptive memory management is described as a favorable bias for filtering irrelevant information in noisy time series.Source: §2 · pp. 2–3

The Kalman Filter

  • The Kalman filter’s time update parallels recurrent SSM state updates, and its measurement update motivates a learnable state correction for cross-variable modeling.The Kalman gain weights the observation innovation by the relative confidence of prediction and measurement.Source: §2 · p. 3

3 METHOD

Chameleon processes patch-level variable representations through recurrent GatedDeltaNet for temporal evolution and a Kalman-inspired measurement update for cross-variable state calibration, then decodes forecasts per variable.

3.1 INPUT PROCESSING

  • Optional normalization and trend-seasonal decomposition precede patching; separate component projections are summed before encoding.The framework independently transforms each variable into patch-level representations before the encoder.Source: §3 · p. 3, §3.1 · pp. 3–4
  • Perturbing reversible-instance-normalization statistics during training is introduced to improve generalization under distribution shifts.The perturbed mean and standard deviation are used in both normalization and denormalization.Source: §3.1 · p. 3

3.2 CHAMELEON BLOCKS AND ENCODER

  • GatedDeltaNet evolves each variable’s state recurrently, then a measurement update corrects states across variables once after the time recurrence.This factorized ordering preserves chunk-parallel acceleration along the time dimension.Source: §3.2 · pp. 4–5
  • Data-dependent covariance terms and a low-rank gain computation avoid quadratic storage and yield linear scaling in variable count for fixed Dη.The correlation strength can be attenuated when variable dependencies are weak.Source: §3.2 · p. 5
  • A learned scalar gate fuses prior and posterior states, with small initialization keeping the block close to the channel-independent backbone.The gate can reduce cross-variable influence when dependencies are weak.Source: §3.2 · p. 6

3.3 DECODER

  • The decoder flattens each variable’s encoded patch representations, projects them to the forecast horizon, and reverses RevIN when enabled.Source: §3.3 · p. 6

4 EXPERIMENTS

Experiments compare Chameleon with CI and CD baselines on strongly dependent ODE and PEMS datasets and seven long-horizon benchmarks. Results, ablations, and efficiency analyses examine accuracy, design contributions, and scalability.

4.1 DATASETS

  • The seven long-horizon benchmarks are ETTm1, ETTm2, ETTh1, ETTh2, Electricity, Solar Energy, and Traffic; Weather, Illness, and Exchange Rate are excluded.The stated reasons for exclusion include known issues such as low predictability or insufficient samples.Source: §4.1 · p. 6

4.3 RESULTS

  • Chameleon achieves the best MSE and MAE across ODE and PEMS settings, and ranks first in MSE on all seven benchmarks and MAE on six.On ODE and PEMS, results are statistically significant over all baselines in seven of eight settings; on benchmarks, baselines incur 3.2–11.3% higher MSE and 1.9–9.8% higher MAE.Source: Table 1 · p. 7, §4.3 · p. 7, Table 2 · p. 8

4.2 SETUP

  • Benchmarks use forecast horizons 96, 192, 336, and 720, while ODE and PEMS datasets use two horizons each.For each dataset-horizon-model combination, the study evaluates every look-back size over five runs and reports results from the best look-back size.Source: §4.2 · p. 7
  • The six baselines span CD SSM and other CD methods, plus CI SFNN and FITS references for benchmarks.Crossformer is included for strongly dependent datasets; shared hyperparameters are aligned with Chameleon when public configurations are unavailable, mostly for ODE datasets.Source: §4.2 · p. 7

4.3 RESULTS

  • Removing the measurement update raises MSE by 28.5% on PEMS08 and by larger margins on ODE datasets, while benchmark increases are around 2–4% on Traffic and Solar.On weakly dependent ETT datasets, the update provides incremental improvements without degrading performance, with a 4.3% average parameter-count increase.Source: §4.3 · p. 8, Table 3 · p. 8
  • RevIN perturbation and decomposition reduce cross-run standard deviation by 64% and 72% on average, respectively, despite sometimes incremental mean-MSE effects.Removing either design, or replacing GatedDeltaNet with Mamba2, increases MSE by around 1–4% on Traffic and Solar.Source: §4.3 · pp. 8–9, Table 3 · p. 8
  • On Traffic, Chameleon has the best test MSE while keeping training time and peak memory within the same order of magnitude as TimeFilter and S-Mamba.Its peak memory scales comparably to S-Mamba as variable counts change, while TimeFilter and DUET grow considerably faster.Source: Figure 3 · p. 9, §4.3 · p. 9

5 CONCLUSION

Chameleon combines a Kalman-filter-inspired cross-variable update with a GatedDeltaNet temporal backbone and linear complexity in variable count. It achieves strong results on dependent datasets and standard benchmarks, with competitive efficiency and memory scalability.

5 CONCLUSION

  • Chameleon achieves the best MSE and MAE on strongly dependent ODE and PEMS datasets and ranks first in 25 of 28 benchmark settings for MSE and 20 for MAE.Traffic and ETT analyses also show competitive training efficiency and favorable memory scalability across variable counts.Source: §5 · p. 9
  • Chameleon integrates a Kalman-filter-inspired cross-variable update into a GatedDeltaNet temporal backbone, enabling data-dependent fine-grained interactions with linear complexity in variable count.Source: §5 · p. 9
  • The authors report using generative AI for partial code revision, visualization, and writing polish, and state that they reviewed the assisted work.Source: AI USE STATEMENT · p. 10

A DATASET DETAILS

The dataset appendix identifies the benchmark and ODE look-back grids and explains an omission from the latter.

A DATASET DETAILS

B PREPROCESSING GUIDANCE FOR REVERSIBLE INSTANCE NORMALIZATION AND DECOMPOSITION

The section presents training- and validation-split diagnostics to guide RevIN and decomposition, and checks those choices empirically.

Diagnostics

  • Table 6 reports each diagnostic's 95th percentile across variables, with values exceeding the stated selection thresholds highlighted.The diagnostics are computed per variable before aggregation.Source: Table 6 · p. 13, B PREPROCESSING GUIDANCE FOR… · p. 14
  • The diagnostics use training and validation splits; Trend fits an ordinary least squares line over their combined segment to assess fitted net change relative to variable scale.Mean Shift measures level change, while KS compares the training and validation marginal distributions as a broader distribution-shift measure.Source: B PREPROCESSING GUIDANCE FOR… · p. 14

Preprocessing guidance

  • Trend, Mean Shift, and KS above 0.2, or Std. Shift above 1.2, trigger RevIN; decomposition is applied when Trend exceeds 0.2.These thresholds are practical selection criteria, not definitive rules; direct validation comparisons can be more reliable, especially for borderline cases.Source: B PREPROCESSING GUIDANCE FOR… · p. 14
  • At F = 96, RevIN improves both MSE and MAE on ETTh2, ETTm2, and Solar, while disabling it improves both metrics on Double Pendulum.Double Pendulum and Lorenz Coupled fall below all four diagnostic thresholds, so both preprocessing steps are removed for those datasets and retained for the others.Source: B PREPROCESSING GUIDANCE FOR… · p. 14, Table 7 · p. 14

C PERIODICITY ANALYSES

The periodicity analyses distinguish dominant cycles in Traffic and PEMS08 from the lack of clear dominant periods in the two chaotic ODE datasets. The section uses this contrast to motivate different evaluation horizons.

Traffic and PEMS08

ODE datasets

  • Double Pendulum and Lorenz Coupled have no clear dominant period, and Lorenz Coupled shows even weaker periodicity.The analyses include Fourier transforms, autocorrelation, and lag-copy tests; the ODE datasets retain shorter evaluation horizons because their forecast errors grow faster with horizon.Source: C PERIODICITY ANALYSES · p. 14, Figure 6 · p. 16, Figure 7 · p. 16

Evaluation context

  • The contrast between periodic benchmarks and chaotic ODE datasets, together with the ODE datasets' strong and distinct variable dependencies, makes them complementary evaluation settings.The seven benchmarks are periodic, whereas the two ODE datasets lack clear dominant periods.Source: C PERIODICITY ANALYSES · p. 14

D FULL EXPERIMENTAL RESULTS AND ADDITIONAL ANALYSES

The detailed results report Chameleon's benchmark performance and run-to-run variation, alongside efficiency and scalability analyses on ETT and Traffic. Together, these analyses cover datasets with different variable counts and model configurations.

Detailed results

  • Chameleon achieves the best mean MSE and MAE in most cases and generally has lower or comparable cross-run standard deviations, except against FITS.Tables 8 and 9 provide complete means, standard deviations, and statistical significance for the benchmark and strongly dependent dataset settings.Source: D FULL EXPERIMENTAL RESULTS AND… · p. 17, Table 8 · p. 17, Table 9 · p. 18

ETT efficiency

  • On the four ETT datasets at F = 96, Chameleon achieves the best MSE and requires the least training time and peak memory among evaluated CD methods.Only the CI reference SFNN requires less training time and peak memory; the ETT datasets contain seven variables.Source: D FULL EXPERIMENTAL RESULTS AND… · pp. 17–18, Figure 8 · p. 18

Scalability

Related papers

Loading 2609.36453v1…