PaperlayerView exact revision on arXiv

Source-linked AI summary

XLinear: A Lightweight and Accurate MLP-Based Model for Long-Term Time Series Forecasting with Exogenous Inputs

Xinyang Chen, Huidong Jin, Yu Huang, Zaiwen Feng

arXiv:2601.09237v1cs.LG

TL;DR

Long-term forecasting with exogenous inputs must balance efficiency and accuracy while accounting for temporal patterns and relevant cross-variable dependencies. XLinear uses sigmoid-gated MLPs and endogenous global tokens to model these signals, and reports superior accuracy and efficiency across benchmark and real-world forecasting tasks.

  • Problem

    Long-term forecasting with exogenous inputs needs to balance computational efficiency and accuracy while modeling temporal patterns and relevant drivers of target dynamics.

    Source: Introduction · pp. 1–2

  • Method

    XLinear uses sigmoid-gated MLPs, endogenous global tokens, and a cross-channel prediction head to combine temporal and variate-wise signals.

    Source: Introduction · p. 2, Conclusion · p. 9

  • Results

    Across seven public benchmarks and five real-world datasets, XLinear reports superior accuracy and efficiency for multivariate forecasts and univariate forecasts influenced by exogenous variables.

    Source: Conclusion · p. 9

  • Takeaways & Limitations

    The reported evaluation supports XLinear for both multivariate forecasting and univariate forecasting influenced by exogenous inputs.

    Source: Conclusion · p. 9

    When a variable is both a prediction target and an exogenous driver, VGM input dimensionality expands, posing a challenge for high-dimensional datasets.

    Source: Main Results · p. 6

Abstract

from arXiv · show

Despite the prevalent assumption of uniform variable importance in long-term time series forecasting models, real world applications often exhibit asymmetric causal relationships and varying data acquisition costs. Specifically, cost-effective exogenous data (e.g., local weather) can unilaterally influence dynamics of endogenous variables, such as lake surface temperature. Exploiting these links enables more effective forecasts when exogenous inputs are readily available. Transformer-based models capture long-range dependencies but incur high computation and suffer from permutation invariance. Patch-based variants improve efficiency yet can miss local temporal patterns. To efficiently exploit informative signals across both the temporal dimension and relevant exogenous variables, this study proposes XLinear, a lightweight time series forecasting model built upon MultiLayer Perceptrons (MLPs). XLinear uses a global token derived from an endogenous variable as a pivotal hub for interacting with exogenous variables, and employs MLPs with sigmoid activation to extract both temporal patterns and variate-wise dependencies. Its prediction head then integrates these signals to forecast the endogenous series. We evaluate XLinear on seven standard benchmarks and five real-world datasets with exogenous inputs. Compared with state-of-the-art models, XLinear delivers superior accuracy and efficiency for both multivariate forecasts and univariate forecasts influenced by exogenous inputs.

Introduction

Long-term forecasting with exogenous inputs must balance computational efficiency and accuracy while modeling temporal patterns and relevant cross-variable information. XLinear uses sigmoid-gated MLPs and endogenous global tokens to model these dependencies, with strong reported accuracy and efficiency.

Introduction

  • Forecasting with exogenous inputs must account for relevant drivers that can influence target dynamics, alongside persistent temporal patterns and cross-variate information.The introduction describes asymmetric relationships, such as local weather driving lake surface or crop canopy temperature.Source: Introduction · p. 1
  • Transformer models capture long-range dependencies, but prior approaches have limitations in cross-variable or temporal modeling, and patch-based attention can lose temporal information.The introduction notes that early Transformers paid little attention to cross-variable dependencies, while patch-based models may be unsuitable for some forecasting tasks.Source: Introduction · pp. 1–2
  • Lightweight models capture temporal dependencies, but the introduction argues that real-world forecasting also needs information from exogenous drivers.It identifies DLinear, TimeMixer, and xPatch as examples of models that process seasonal-trend components.Source: Introduction · p. 2
  • On Electricity, XLinear delivers the highest forecast accuracy among 10 comparative models while matching DLinear and RLinear’s training speed and using less memory.Figure 1 covers univariate forecasting with 320 exogenous variables; TimesNet and TiDE are excluded for clarity because of slow training.Source: Figure 1 · p. 1, Introduction · p. 2
  • XLinear uses sigmoid-activated MLP gating and endogenous global tokens to capture temporal patterns and cross-variable dependencies from exogenous signals.Its prediction head integrates the extracted signals; the model is designed to balance efficiency with forecasting accuracy.Source: Introduction · p. 2

Method

XLinear uses learnable endogenous global tokens as hubs for exogenous interactions, with MLP gating to capture temporal patterns and cross-variable dependencies. It embeds both input types, applies time-wise and variate-wise gating, then combines the resulting signals to predict future endogenous values.

Method

  • Learnable global tokens for endogenous variables serve as hubs for exogenous interactions, while MLP gating captures temporal and cross-variable dependencies.The gated cross-variable information is combined with each temporally enhanced endogenous sequence before the prediction head.Source: Method · p. 2, Figure 2 · p. 3

Forward Process

  • Historical endogenous and exogenous sequences are mapped to predictions of the endogenous variables for the next S time steps.The model learns the mapping from input sequences to S-step-ahead predictions.Source: Forward Process · p. 2

Embedding

  • A joint parameterized embedding processes the RevIN-processed endogenous and exogenous sequences, then concatenates a learnable global context token with the endogenous embedding along time.The resulting enhanced representation carries the token used to extract relevant exogenous information.Source: Embedding · p. 3

Time-wise Gating Module (TGM)

  • The Time-wise Gating Module uses an MLP to capture temporal dependencies, transfer sequence information to the global token, and sigmoid-scale weights for selective feature filtering.Its stated purpose is to extract salient historical temporal patterns.Source: Time-wise Gating Module (TGM) · p. 3

Variate-wise Gating Module (VGM)

  • The Variate-wise Gating Module links endogenous global tokens with exogenous sequences, mapping relevant exogenous information into the tokens to optimize endogenous predictions.The formulation concatenates along the channel dimension.Source: Variate-wise Gating Module (VGM) · p. 3

Prediction Head

XLinear’s prediction head maps integrated endogenous and exogenous information to future target values, while the loss aggregates prediction error across endogenous channels. The section also specifies multivariate handling and the datasets, training, forecasting, and evaluation settings.

Prediction Head

  • The fully connected prediction head fuses endogenous temporal characteristics with information extracted from exogenous variables to map integrated features to the target sequence.It maps concatenated endogenous sequences of shape (batch size, dx, 2*dmodel) to S future steps in a channel-independent manner.Source: Prediction Head · p. 3, Experimental Settings · p. 4

Loss Function

  • The loss averages MSE across the M endogenous-variable sequences after aggregating losses from individual channels.The error is evaluated over the forecast steps T+1 to T+S.Source: Loss Function · p. 4

Multivariate Forecasting

  • In multivariate forecasting, each variable is both a prediction target and an exogenous source for predicting other variables.The TGM is shared across variables, and the VGM across features.Source: Multivariate Forecasting · p. 4

Experimental Results and Comparisons Experimental Settings

  • The evaluation uses seven established datasets and five exogenous-input datasets, with MSE and MAE for public benchmarks and additional environmental metrics.The environmental and Crop datasets also use NSE, KGE, and MAPE; Table 1 summarizes dataset characteristics, and Table 2 covers univariate forecasts with exogenous inputs.Source: Experimental Settings · pp. 4–5, Table 1 · p. 5, Table 2 · p. 5
  • Experiments use fixed seed 2025, Adam optimization, dataset-selected learning rates, and typically 30 epochs with early stopping.Batch sizes are selected from 4 to 256; inputs are generally length 96, with horizons 96, 192, 336, or 720, while Crop uses 12, 24, 36, or 48.Source: Experimental Settings · p. 4

Main Results

XLinear performs strongly across benchmark and real-world forecasting tasks, combining frequent accuracy wins with efficient training and memory use. Its results also show a weaker comparative position on high-dimensional Traffic.

Main Results

  • XLinear has the lowest MSE in 89.3% (25/28) of benchmark configurations and the lowest MAE in 75.0%.It also achieves at least 39.3% faster training, 13.9% lower memory usage, and 2.8% lower MSE than other baselines, except DLinear and RLinear.Source: Main Results · pp. 5–6
  • On five real-world applications, XLinear has the lowest MSE and MAE in over 75% of cases and ranks first on NSE in 75%.It reaches NSE > 0.65 in 65% of cases; Table 3 reports single-endogenous-variable forecasts using environmental datasets.Source: Main Results · p. 6, Table 3 · p. 6
  • For multivariate forecasting, XLinear is best on both MSE and MAE in over 91.7% of cases on the first six benchmark datasets, ranking second in the rest.Table 4 presents multivariate results, with TimeMixer results reported in Wang et al. (2024a) and other baselines in Wang et al. (2024b).Source: Main Results · p. 6, Table 4 · p. 7
  • On Traffic with 862 variables, iTransformer outperforms XLinear; across forecast horizons, XLinear ranks second in MSE and third in MAE on Traffic.Each variable is both a target and an exogenous driver in this setting, substantially expanding VGM input dimensionality.Source: Main Results · p. 6
  • On the last Electricity variable with 96-step input and prediction horizons, XLinear’s multivariate forecasts closely align with ground truth.Figure 3 compares XLinear with five SOTA baselines.Source: Figure 3 · p. 6, Main Results · p. 7
  • On Weather multivariate forecasting, XLinear trains at least 39.3% faster, uses 43.7% less memory, and has 5.1% lower MSE than other models except DLinear and RLinear.The exception models have MSEs 31.5% and 28.9% higher than XLinear, respectively.Source: Main Results · p. 7, Figure 4 · p. 7

Model Analysis

XLinear’s analysis highlights fine-grained forecasting, interpretable temporal and variable weighting, and efficient use of longer input windows. Ablations indicate that endogenous global tokens and temporal sequences contribute complementary information.

Model Analysis

  • On volatile PEMS datasets, XLinear forecasts short-term fluctuations more precisely than patch-based models across most scenarios.The authors attribute this to direct modeling of individual dimensions after embedding, while patching may hinder learning high-resolution temporal details.Source: Model Analysis · p. 8
  • XLinear’s temporal and variable weights highlight important time steps and variables; on DE data, higher weights for Wind power and Electricity price align with observed correlations.The analysis also describes temporal weights mapping input features to global tokens.Source: Model Analysis · p. 8, Figure 6 · p. 8
  • Forecast MSE drops rapidly as lookback grows to 96, then decreases more gradually with further increases.The authors report that XLinear benefits from longer input sequences.Source: Model Analysis · p. 8
  • On Weather, XLinear’s training time and GPU memory grow approximately linearly with lookback, while TimeMixer and PatchTST resource demands grow at least quadratically.XLinear outperforms baseline models on training time and memory, except that it is comparable to DLinear and RLinear.Source: Model Analysis · p. 8, Figure 8 · p. 9
  • With a lookback of 512, XLinear outperforms TSMixer and PatchTST at horizons 96–336, and performs comparably to PatchTST at horizon 720.Table 6 reports performance comparisons at lookback length 512.Source: Model Analysis · p. 9, Table 6 · p. 9

Ablation Study

  • Using both endogenous sequences and global tokens gives the best ablation performance; global tokens alone generally outperform the pure temporal sequence.The authors state that global tokens capture target temporal features and exogenous information, making the two sources complementary.Source: Ablation Study · p. 9, Table 7 · p. 9

Conclusion

XLinear uses gated MLPs and learnable global tokens to model temporal patterns and exogenous-to-endogenous dependencies. Experiments show strong accuracy and efficiency, while scalability to series with hundreds of variables remains future work.

Conclusion

  • XLinear uses sigmoid-gated MLPs and learnable global tokens to capture temporal patterns and variate-wise dependencies from exogenous to endogenous variables.Time-wise and variate-wise gating outputs are fused with input feature maps to amplify salient signals and suppress noise.Source: Conclusion · p. 9
  • Across seven public benchmarks and five real-world datasets, XLinear achieves superior accuracy and efficiency in multivariate and exogenous-input univariate forecasting.It matches other lightweight models’ speed and is at least 30% faster than the most efficient Transformer-based alternatives.Source: Conclusion · p. 9
  • Future work targets scalability to time series with hundreds of variables, alongside lookback optimization and forecast reliability for operational deployment.Source: Conclusion · p. 9

Appendix

The supplementary materials outline related work, dataset descriptions, forecast-accuracy comparisons, and an activation-function ablation.

Appendix

  • The supplement is organized to cover related work, datasets, model comparisons, and a targeted gating activation ablation.Source: Appendix · p. 11
  • Subsection A is designated for a brief review of related work, and Subsection B for dataset descriptions.Source: Appendix · p. 11
  • Subsection C compares forecast accuracy with advanced time-series and GNN-based models; Subsection D examines the gating module’s activation function.Source: Appendix · p. 11

A. Related Work

Prior work addresses temporal and cross-variable forecasting through Transformers and MLPs, while exogenous-input models increasingly connect drivers to endogenous targets. XLinear builds on this line with MLP-based gating.

A. Related Work

  • Transformer variants target efficiency and temporal or variable interactions, but patch dependence and permutation invariance can limit their suitability or preserve temporal information poorly.PatchTST adds local semantic information; Crossformer, iTransformer, and TimeXer model interactions across variables in different ways.Source: A. Related Work · pp. 11–12
  • MLP forecasters use decomposition, normalization, mixing, or encoder-decoder designs, while xPatch processes variables independently and cannot incorporate exogenous inputs.Examples include DLinear, RLinear [Li et al. 2023], TSMixer, TiDE, TimeMixer, and xPatch.Source: A. Related Work · p. 12
  • Exogenous-input tasks require models to capture endogenous history and use exogenous information; TSMixer fuses heterogeneous features through time- and feature-mixing operations.The passage gives irrigation scheduling, population flows, yield forecasting, and environmental science as examples.Source: A. Related Work · p. 12
  • TiDE encodes historical endogenous and exogenous values and uses future exogenous values during decoding, while TimeXer uses global tokens and cross-attention for exogenous interaction.XLinear extends this research direction with MLP-based gating modules.Source: A. Related Work · p. 12
  • Foundation time-series models are mostly Transformer-based, while cited convolutional and graph approaches lack explicit exogenous-input mechanisms and have reported multivariate performance limitations.The comparison in the passage names TimeXer and XLinear as stronger-performing models.Source: A. Related Work · p. 12

B. Datasets

The evaluation primarily uses 12 datasets, spanning public forecasting benchmarks and real-world datasets with environmental, operational, or agricultural measurements. The authors identify high-dimensional VGM inputs as a scalability challenge.

B. Datasets

  • The evaluation primarily incorporates 12 datasets.The supplied descriptions include ETT, Weather, Electricity, Traffic, Dissolved Oxygen, water temperature, and Crop Yield Prediction datasets.Source: B. Datasets · pp. 12–13
  • ETT includes four subsets with oil temperature targets and six power-load variables; Weather, Electricity, and Traffic provide meteorological, consumption, and road-occupancy measurements.The datasets span sampling intervals from 10 minutes to one hour, as described in the passage.Source: B. Datasets · p. 12
  • Dissolved Oxygen, water-temperature, and Crop Yield Prediction datasets pair target measurements with environmental factors.The passage describes two river monitoring stations, two dam monitoring sites, and agricultural-field data, respectively.Source: B. Datasets · p. 13
  • High-dimensional VGM inputs challenge signal extraction, and future work proposes reducing their dimensionality while preserving essential inter-variable dependencies.Source: B. Datasets · p. 13

C. Comparison with GNN-based Models

Across the evaluated comparisons, XLinear performs strongly against both state-of-the-art models and GNN-based baselines. Its exception is the high-dimensional Traffic dataset, where it ranks below first.

C. Comparison with GNN-based Models

D. Ablation Study on Activation Functions

The activation-function ablation reports that replacing sigmoid with Swish or Tanh causes slight declines in some cases, while Softmax substantially worsens performance. The described sigmoid gate maps weights to a stable range for feature modulation.

D. Ablation Study on Activation Functions

Related papers

Loading 2601.09237v1…