PaperlayerView exact revision on arXiv

Source-linked AI summary

AsyTO: Asymmetric Temporal Operator for Parameter-Efficient Multivariate Time Series Forecasting

Xiachong Lin, Du Yin, Hao Xue, Wen Hu, Imran Razzak, Arian Prabowo, Matthew Amos, Flora D. Salim

arXiv:2608.16098v1cs.LGcs.AI

TL;DR

Multivariate forecasting must balance parameter efficiency against heterogeneous temporal dynamics. AsyTO preserves distinct history-reading and future-writing roles through compact operator factorization, achieving the lowest lightweight MSE in 30 of 44 settings.

  • Problem

    The open question is whether compact forecasters can preserve variable-specific history–future asymmetry instead of tying evidence extraction to trajectory generation.

  • Method

    AsyTO factorizes variable-specific operators into separate temporal factors, a low-rank phase-aligned periodic prototype, and cycle-separable temporal components.

  • Results

    Lowest MSE among lightweight forecasters occurs in 30 of 44 benchmark cells, with the advantage holding across forecast horizons.

  • Takeaways & Limitations

    The results support sharing temporal factors across variables while specializing their effects through variable-specific responses.

  • Takeaways & Limitations

    AsyTO does not dominate every efficiency axis: at matched compute, another model uses fewer parameters, although it trails in error.

Abstract

from arXiv · show

Multivariate time-series forecasting faces a structural dilemma: sharing one temporal predictor across variables is parameter-efficient but forces heterogeneous variables through an identical history-to-future map, whereas learning an independent predictor per variable restores flexibility at a cost that grows with the product of variable count, context length, and horizon. We argue that this dilemma dissolves once the object being compressed is the forecasting operator rather than the observed series. Auditing per-variable linear history-to-future maps across standard benchmarks, we find that a phase-locked seasonal component paired with a compact residual operator outperforms a dense phase-blind reference in most audited settings. The residual transport is also directional: lag-invariant alternatives consistently underperform asymmetric history-to-future maps. Guided by this structure, we propose AsyTO, an Asymmetric Temporal Operator that factorizes the tensor of per-variable operators into shared but distinct history-reading and future-writing temporal modes with per-variable mode-wise gains, complemented by a low-rank periodic prototype and a cycle-separable factorization of the temporal modes. Each forecast reads only its own variable's history, so parameters and compute grow linearly in the number of variables. Across eleven benchmarks and multiple forecast horizons, AsyTO attains the best lightweight error in 30 of 44 dataset-horizon settings, locating at the accuracy-compute Pareto frontier.

Introduction

The introduction frames parameter-efficient multivariate forecasting as structured compression of per-variable history-to-future operators, arguing that history reading and future writing require distinct temporal roles. AsyTO preserves this asymmetry through periodic structure, factorized temporal operators, and variable-specific gains, improving accuracy across most datasets with fewer parameters.

  • Motivation: Lightweight linear and frequency-domain predictors match or outperform Transformer forecasters on standard long-horizon multivariate forecasting benchmarks.The introduction positions modest parameter counts as a central advantage of recent forecasting models.
  • History–future asymmetry: History–future asymmetry separates selective evidence extraction from coordinating the entire forecast horizon, rather than tying both roles to one shared object.Common temporal compression constructions reuse the same basis, kernel, or pattern set for reading history and writing futures.
  • AsyTO: AsyTO factorizes variable-specific forecasting operators into separate shared history-reading and future-writing factors, with prototype-conditioned gains and cycle-separable temporal compression.It combines a low-rank, phase-aligned periodic prototype with shared temporal factors to encode heterogeneous responses in a low-dimensional space.
  • Empirical validation: AsyTO improves forecasting accuracy across most evaluated datasets while requiring substantially fewer parameters, and its modules further improve popular lightweight backbones in most cases.The evaluation includes structural controls and benchmark tests across several real-world time-series datasets.

Related Works

Related work spans lightweight temporal models using linear, sparse, or spectral designs, alongside approaches that differ in how they model variable interactions. Explicit periodic methods remove or exploit cycles, while AsyTO distinguishes itself by reusing the periodic prototype to parameterize residual forecasting.

  • Compact Forecasting and Temporal Compression: Lightweight temporal forecasting includes linear predictors such as DLinear, MixLinear, and LightTS, plus sparse or spectral designs including FITS, SparseTSF, FreTS, and FilterNet.The literature also distinguishes models by their treatment of variable interaction, including iTransformer and query-based models.
  • Explicit Periodic Modeling: Explicit periodic methods include CycleNet’s learned-cycle removal and approaches exploiting periodicity through downsampling, phase tokenization, or cycle-aligned spectra.The cited examples are SparseTSF, PhaseFormer, and FreqCycle.
  • Explicit Periodic Modeling: AsyTO shares CycleNet’s decomposition step, makes no novelty claim for periodic removal, and instead reuses the periodic prototype to parameterize residual forecasting.This reuse of the periodic prototype is identified as AsyTO’s distinction from prior explicit periodic modeling.

Methodology · Problem Formulation

The problem formulation maps historical multivariate observations to future trajectories, defining forecasting separately for each variable. The forecaster is an operator that transports information from history to future.

  • Problem Formulation: The input is a batch of historical observations X ∈ R^B×L×C.B, L, and C denote batch size, context length, and variable count.
  • Problem Formulation: The target is a batch of subsequent observations Y ∈ R^B×H×C.H denotes the forecast horizon.
  • Problem Formulation: For sample b and variable c, the historical context is X_b,:,c ∈ R^L.Each variable contributes an L-length history for forecasting.
  • Problem Formulation: For sample b and variable c, the future trajectory is Y_b,:,c ∈ R^H.The prediction spans H future observations for that variable.
  • Problem Formulation: Forecasting is defined per variable as Y_b,:,c = f_θ(X_b,:,c; c), for c ∈ {1, …, C}.The formulation applies the forecaster to each variable’s historical context.
  • Problem Formulation: The forecaster f_θ ∈ R^L×H is an operator bridging information transport from history to future.Its dimensions correspond to the context length L and forecast horizon H.

Low-Rank Periodic Prototype

AsyTO replaces a dense phase–variable periodic table with a low-rank prototype that shares within-cycle patterns while placing variables in a compact subspace. Phase-aligned history and future components are read from this prototype, leaving the operator to transport only the periodicity-exclusive residual.

  • Prototype factorization: The factorization reduces parameter cost from PC to Rp(P+C), replacing a dense phase–variable table in R^{P × C}.Here, P is the dataset period, C is the variable count, and r is the prototype rank.
  • Phase alignment: History and future components are read from the same prototype using each sample’s phase-aligned positions.The phase of position τ is determined by the first historical observation’s phase and the dataset period.
  • Prototype factorization: The periodic template is factorized into shared within-cycle patterns and variable-specific coordinates in a low-rank subspace.The assumption concerns low rank of the periodic template, not of the observations themselves.
  • Residual transport: Subtracting the phase-derived periodic component leaves a residual that the operator transports, while the phase-aligned future component is added back.This separates periodic structure from the residual forecasting operation.
  • Initialization: Initialization sets U orthogonally and V to zero, producing a neutral template while preserving gradient flow into V.The zero initialization applies to V, which still receives gradient during training.

Asymmetric Temporal Operator

AsyTO factorizes the per-variable history-to-future operator into shared history-reading and future-writing modes with variable-specific gains, avoiding materialization while preserving target-only forecasting. Separately parameterized history and future modes make the transport directional rather than symmetric.

  • Operator factorization: The CP factorization uses shared history-side factors Ψ, shared future-side factors Φ, and variable–mode responses G.For variable c, ψ_m reads historical residual patterns, ϕ_m writes future trajectories, and G_c,m controls the response strength.
  • Efficient implementation: R_m(L + H + C) parameters and O(BC(L + H)R_m) arithmetic suffice without materializing the operator.The factorization performs contractions in mode space.
  • Target-only forecasting: Each forecast for variable c reads only its own historical residual and never another variable’s observations.This makes the target-only property explicit.
  • Asymmetric transport: Separately parameterized Ψ and Φ preserve directional transport, whereas tying Ψ = Φ would force a symmetric temporal map.Reading evidence from a past position and generating a future one remain distinct operations even when L = H.

Cycle-Separable Temporal Factorization

AsyTO compresses shared history- and future-side temporal factors with a cycle-separable representation that distinguishes cycle distance from within-cycle phase. It supplements this periodic structure with a short recent-history correction and activates future-side factorization only for horizons spanning a full cycle.

  • Cycle-Separable Temporal Factorization: CeST represents each temporal factor as a rank-Rk Kronecker product of cycle-index and within-cycle phase factors.This separates how far back evidence lies from where in the cycle it occurs.
  • Cycle-Separable Temporal Factorization: The history-side temporal cost falls from LRm to RkRm(K + P), with a short dense correction adding MRm parameters.The correction is initialized to zero, so the model begins with the purely periodic basis and departs from it only when data require it.
  • Cycle-Separable Temporal Factorization: Future-side CeST uses K′ = ⌈H/P⌉ output cycles and remains dense when H ≤P.Thus, the future-side term is inactive on short-horizon PEMS settings because the horizon does not span a full cycle.
  • Cycle-Separable Temporal Factorization: The response reuses prototype variable coordinates, replacing CRm parameters with RpRm while each additional variable costs only Rp prototype entries.The factorization is never more expensive because Rp ≤C by construction.
  • Cycle-Separable Temporal Factorization: For future step h, a phase-conditioned, cycle-decayed copy of the matching observation from one cycle earlier is added as a local cue.The cue is gated to zero when s(h) < 1 and therefore vanishes when the window is shorter than one cycle.

Experimental Results · Main Results · Efficiency Analysis

Across 11 benchmarks, AsyTO delivers strong lightweight accuracy while remaining efficient, reaching the lightweight MSE lead in 30 of 44 cells and the accuracy–compute Pareto frontier. Its target-only design has a clear limitation on Traffic, whereas shared temporal factors make the cost of adding variables scale linearly.

  • Experimental Results: AsyTO is evaluated on 11 benchmarks spanning ETT, Electricity, Traffic, Weather, and four PEMS datasets.For seven non-PEMS datasets, the context length is L = 720 and horizons are H ∈{96, 192, 336, 720}.
  • Experimental Results: The comparison covers 12 published forecasters, assessed using mean squared error (MSE) and mean absolute error (MAE).Baselines include lightweight and conventional groups such as SparseTSF, PhaseFormer, CycleNet, DLinear, PatchTST, iTransformer, and FreTS.
  • Main Results: 30 of 44 cells have the lowest MSE for AsyTO within the lightweight group, while its lowest-MAE count is 21 and mean rank is 1.96 on standard benchmarks and 1.06 on PEMS.Its advantage is concentrated on datasets with the assumed periodic structure and persists across forecast horizons.
  • Main Results: Traffic is the sole benchmark where AsyTO loses at every horizon, trailing PhaseFormer by 4.5%, 4.5%, 2.9% and 1.3% as the horizon grows.The target-only design cannot exploit largely shared predictive information across Traffic’s 862 sensors, though the gap narrows at longer horizons.
  • Main Results: 13 of 44 cells have the lowest MSE across all thirteen models for AsyTO, while using two to three orders of magnitude fewer parameters than the conventional group.This trade-off is made explicit by the accuracy–compute comparison.
  • Efficiency Analysis: AsyTO sits on the accuracy–compute Pareto frontier at L=720 →H=96, using 47.5×, 62.2×, and 590.4× fewer MACs than TQNet, iTransformer, and P… on Electricity.Its parameter gap on Electricity is 92.5× against TQNet.
  • Efficiency Analysis: PhaseFormer matches AsyTO’s Electricity compute at 1.0× MACs with a tenth of the parameters, while CycleNet is marginally cheaper at 0.9× MACs; both trail in error by 1.5% and 1.0%.Thus, at this compute scale AsyTO is the most accurate, not the smallest.
  • Efficiency Analysis: Adding one variable increases the parameter count by exactly Rp: 2 on Weather, 4 on ETTm1, 8 on Traffic, and 16 on Electricity.Shared temporal factors produce this linear scaling, whereas a dense per-variable temporal map would add HL coefficients per variable.

Validating the Structural Priors · Cost of Factorization

Ablations show that AsyTO’s gains depend on residual processing, asymmetric history-to-future structure, and variable-specific responses. Factorization then preserves nearly all accuracy while substantially reducing the costs of variable-side and temporal storage.

  • Validating the Structural Priors: AsyTO’s module ablations separately test residual processing, operator factorization, and variable-specific responses against simpler alternatives.w/o Q applies the operator to raw series, w/o ATO uses an independent dense H × L map per variable, and w/o G shares one response across variables.
  • Validating the Structural Priors: The directionality control compares asymmetric factors with tied, parameter-aligned symmetric, and shift-invariant Toeplitz alternatives at L=H.Sym ties the history and future factors, Sym-PA matches Asym’s parameter count, and Toeplitz depends only on relative lag.
  • Validating the Structural Priors: ρasy is 1.37 on ETTm1 and 1.39 on Weather, while learned history modes localize in time and future modes spread across the horizon.These observations indicate specialized history-reading and future-writing temporal bases rather than convergent bases.
  • Cost of Factorization: Independent prototype storage increases parameters by 1.06×, 1.45×, 2.41×, and 7.32× on ETTm1, Weather, ECL, and Traffic, respectively, without improving reported MSE.Independent responses scale by 1.01–2.19× and match or underperform AsyTO except on Weather.
  • Cost of Factorization: On Weather, independent responses reduce MSE from .311 to .307 for a 4.5% increase in parameters, whereas full sharing helps Weather but degrades ETTm1 and ECL.The coordinate-generated form provides an intermediate between channel-homogeneous sharing and costly per-variable storage.
  • Cost of Factorization: Dense temporal factors raise total model size by 1.63–2.90× while changing MSE by at most .003, with no consistent direction across datasets.Dense factors slightly improve ECL and Traffic but match or underperform AsyTO on ETTm1 and Weather.
  • Cost of Factorization: Cycle-separable temporal factors remove 39–65% of the parameters required by dense counterparts while retaining nearly all accuracy.Variable-side and temporal factorizations address complementary scaling terms.

Transferability Analysis

Transferability tests attach the periodic prototype and CeST to frozen PhaseFormer and MixLinear backbones. Jointly training both modules reduces error broadly, with especially large gains on PEMS, where the look-back is shorter than one cycle.

  • Transferability setup: The transferability test freezes independently trained PhaseFormer and MixLinear backbones while comparing T0, T1, and T2 module-training configurations.T1 trains only the periodic prototype Q, whereas T2 trains Q together with CeST.
  • Transferability results: 39 of 44 PhaseFormer cells and 40 of 44 MixLinear cells show lower MSE with T2.Across seven standard benchmarks, mean reductions are 1.3% for PhaseFormer and 4.6% for MixLinear.
  • Transferability results: On PEMS, T2 reduces mean error by 30.5% for PhaseFormer and 68.4% for MixLinear.PEMS has a shorter look-back than one cycle: L = 96 < P = 288.
  • Component contribution: CeST improves T2 over T1 on 21 of 22 displayed backbone–dataset pairs, including PEMS gains from 15.2% to 30.5% and from 37.5% to 68.4%.The only material regression relative to T0 is MixLinear on ETTh1 (+2.7%), where the input already spans many cycles.

Discussion and Conclusion

AsyTO is a compact forecasting operator that separately parameterizes history-side evidence extraction and future-side trajectory generation. It combines low-rank periodic removal, asymmetric residual mapping, cycle-separable temporal factorization, and prototype-derived variable-specific responses.

  • Discussion and Conclusion: AsyTO assigns distinct parameterizations to extracting history-side evidence and generating future-side trajectories.This is the core asymmetric operator design.
  • Discussion and Conclusion: After estimating and removing a low-rank periodic component, AsyTO maps the residual through separate history and future factors.The residual pathway separates history processing from future generation.
  • Discussion and Conclusion: Cycle-separable factorization controls temporal cost, while prototype-derived coordinates generate variable-specific responses.Matched-budget controls substantiate the asymmetric design.
Loading 2608.16098v1…