PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspective

Xingjian Wu, Xiangfei Qiu, Hanyin Cheng, Zhengyu Li, Jilin Hu, Chenjuan Guo, Bin Yang

arXiv:2510.14510v5cs.LG

TL;DR

Fixed-stride adjacent patching creates fixed representations that may retain unhelpful information when time-series patterns shift or contain anomalies. SRS adaptively selects and reassembles patches; experiments report state-of-the-art SRSNet performance and improvements to patch-based models.

  • Problem

    Fixed-stride adjacent patching produces fixed representation spaces that may include harmful information when periods change, shift, or contain anomalies.

    Source: §1 · p. 2

  • Method

    SRS combines Selective Patching and Dynamic Reassembly to adaptively select and sequence patches, and SRSNet pairs SRS with an MLP head.

    Source: §1 · p. 2

  • Results

    SRSNet achieves state-of-the-art performance on real-world datasets, and SRS improves the performance of patch-based models.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    SRS is presented as a plug-and-play way to enhance patch-based forecasting models.

    Source: §1 · p. 2, §7 · p. 10

    SRS may not suit non-patch models; compulsory use with patch size 1 loses patch semantics and adds computational overhead.

    Source: §6 · p. 10

Abstract

from arXiv · show

Time Series Forecasting has made significant progress with the help of Patching technique, which partitions time series into multiple patches to effectively retain contextual semantic information into a representation space beneficial for modeling long-term dependencies. However, conventional patching partitions a time series into adjacent patches, which causes a fixed representation space, thus resulting in insufficiently expressful representations. In this paper, we pioneer the exploration of constructing a selective representation space to flexibly include the most informative patches for forecasting. Specifically, we propose the Selective Representation Space (SRS) module, which utilizes the learnable Selective Patching and Dynamic Reassembly techniques to adaptively select and shuffle the patches from the contextual time series, aiming at fully exploiting the information of contextual time series to enhance the forecasting performance of patch-based models. To demonstrate the effectiveness of SRS module, we propose a simple yet effective SRSNet consisting of SRS and an MLP head, which achieves state-of-the-art performance on real-world datasets from multiple domains. Furthermore, as a novel plug-and-play module, SRS can also enhance the performance of existing patch-based models. The resources are available at https://github.com/decisionintelligence/SRSNet.

1 Introduction

Patching builds contextual time-series representations by grouping adjacent points, but fixed strides can retain harmful information when periods shift, change, or contain anomalies. The paper introduces SRS to adaptively select and arrange patches, and reports strong performance for SRSNet and improvements to patch-based models.

1 Introduction

  • Patching groups adjacent time points whose semantic information tends to be similar, helping capture local features and patterns while reducing computational complexity.The paper frames patching as a way to construct contextual representations for forecasting.Source: §1 · p. 1
  • Fixed-stride adjacent patching creates the same representation positions across contextual series, which can retain anomalies or shifting processes and disrupt period semantics.The paper motivates adapting patch selection to include useful forecasting information.Source: §1 · p. 2
  • SRS uses Selective Patching to choose patches and Dynamic Reassembly to determine their sequence, forming a flexible representation space.Both techniques are designed for gradient-based optimization on forecasting tasks.Source: §1 · p. 2
  • SRSNet achieves state-of-the-art performance across multiple real-world datasets, while SRS can improve existing patch-based models.SRS combines Selective Patching and Dynamic Reassembly; the paper describes the module as plug-and-play and gradient-friendly.Source: §1 · p. 2

2 Related works

The related-work section traces forecasting from statistical and machine-learning approaches to deep neural models, then reviews patch-based methods and their representation-space limitations.

2.1 Development of Time Series Forecasting

  • Forecasting methods progress from classical statistical approaches to machine learning and deep neural networks, including sequence, Transformer, and MLP-based models.The section notes that machine-learning methods require manual feature engineering and model design, while deep methods leverage representation learning.Source: §2.1 · pp. 2–3

2.2 Progress in Patch-based Time Series Forecastng Methods

  • Patch-based forecasting segments series into subseries tokens, with later methods extending patch modeling across variables, time dimensions, architectures, and scales.The passage cites Triformer, PatchTST, Crossformer, xPatch, Pathformer, and PatchMLP as examples.Source: §2.2 · p. 3
  • Existing patch methods retain fixed strides, which can lose information or include unhelpful content; SRS adapts patch selection and sequence to use contextual information.Source: §2.2 · p. 3

3 Methodology

SRS replaces fixed adjacent-patch representations with flexible spaces built by selecting and reordering patches from each contextual series. Its learned patch operations are combined with conventional patch embeddings, and the resulting representations can feed patch-based backbones.

3.1 Structure Overview

  • SRS is a plug-and-play module that replaces fixed-index patching with adaptive patch selection and ordering for each contextual time series.The module is optimized through a gradient-based strategy and can be integrated into patch-based backbones.Source: §3 · p. 3, §3.1 · p. 4

3.2 Selective Patching

  • Stride-1 scanning exposes K candidate patches; an MLP assigns n scores per patch, selects the highest-scoring patch at each sampling, and permits selection with replacement.The selected patches are retrieved using their maximum-score indices.Source: §3.2 · pp. 4–5

3.3 Dynamic Reassembly

  • An MLP scores each selected patch once, then sorts patches by score to determine their order while retaining gradient propagation.Dynamic Reassembly adaptively orders the selected patches to construct the representation space.Source: §3.3 · pp. 5–6

3.4 Embedding

  • SRS and conventional patch embeddings are adaptively fused, then position embeddings are added before the representations enter patch-based backbones.The fusion balances the two embedding sets with α ∈ [0, 1]^(n×d).Source: §3.4 · p. 6

4 SRSNet

SRSNet combines the SRS module with a small Linear/MLP head for forecasting. The paper reports strong performance without a complex backbone.

4 SRSNet

  • SRSNet consists of the SRS module and a Linear/MLP head of at most two layers.Source: §4 · p. 6
  • Flattened embeddings feed an MLP that predicts the future horizon, with forecasting trained using Mean Squared Error.Source: §4 · p. 6
  • SRSNet’s reported strong performance demonstrates the SRS module’s use of contextual time series without a complex backbone.Source: §4 · p. 6

5 Experiments

Across multivariate forecasting benchmarks, SRSNet earns most first rankings. The experiments also examine plug-in gains, component contributions, efficiency costs, and parameter sensitivity.

5.1 Experimental Details

  • Experiments cover eight forecasting benchmarks spanning ETT, Weather, Electricity, Solar, and Traffic, using mse and mae across four horizons and three look-back sizes.The look-back sizes are 96, 336, and 512; each method’s best results are reported for fair comparison.Source: §5.1 · p. 7, Table 1 · p. 7

5.2 Main Results

  • SRSNet achieves most first rankings across datasets and forecasting horizons, with strong results against models of different structures and fixed-patching baselines.Table 2 reports multivariate forecasting averages for horizons F ∈ {96, 192, 336, 720}.Source: Table 2 · p. 7, §5.2 · p. 7

5.3 Ablation Study and Analysis

  • Pairing SRS with a naive MLP improves prediction accuracy by approximately 5% to 15%.The authors attribute SRSNet’s gains to adaptively constructing a selective representation space with Selective Patching and Dynamic Reassembly.Source: Table 3 · p. 8, §5.3 · p. 8
  • Adding SRS improves PatchTST, Crossformer, xPatch, and PatchMLP by approximately 5% on average, and can be integrated into PatchMLP at each patch scale.The authors describe SRS as orthogonal to multi-scale techniques.Source: Table 3 · p. 8, §5.3 · p. 8
  • Selective Patching has the greatest impact; Dynamic Reassembly reassembles selected patches, while Adaptive Fusion combines conventional-patching and SRS embeddings.The authors report that all three components contribute positively when used together.Source: Table 4 · p. 8, §5.3 · pp. 8–9

5.4 Efficiency Analysis

  • SRSNet balances efficiency and performance, with reported efficiency advantages over Transformer-based models, TimeMixer, and TimeKAN.DLinear and Amplifier are more efficient, but the authors report poor performance on large datasets such as Solar and Traffic.Source: Table 5 · p. 9, §5.4 · p. 9
  • Adding SRS to PatchTST and Crossformer introduces about 10% increases in Max GPU Memory, Inference Time, and Training Time, and under 5% in MACs.The efficiency analysis reports these overheads for plugin integration into the two patch-based models.Source: Table 6 · p. 9, §5.4 · p. 9

5.5 Parameter Sensitivity

  • SRSNet is reported to remain robust across patch sizes, with 16 or 24 often chosen as common configurations.The authors link this robustness to SRS adaptively selecting useful patches.Source: Figure 4 · p. 9, §5.5 · p. 9

6 Discussion

The authors identify limitations in SRS's applicability, scaling evidence, interpretability, and initialization. They also describe potential directions for addressing these concerns.

Potential limitations

  • SRS may be impractical for non-patch models because patch size 1 loses patch semantics and adds computational overhead.The authors note that current expert and foundation models mainly use patch-based modeling.Source: §6 · p. 10
  • SRS's effectiveness and generalization with billion-scale time-series pretraining, and whether it follows scaling laws, remain unverified.The authors report stable behavior with fixed hidden size and layer count in end-to-end scenarios.Source: §6 · p. 10
  • Gradient-based optimization coupled to downstream models means selected patches are useful for forecasting but not necessarily interpretable.Future work proposes environment-aware mechanisms to perceive patch-wise distributions and patterns more explicitly.Source: §6 · p. 10
  • Initializing α using dataset prior knowledge may improve SRS convergence and stability.The authors suggest increasing α for periodic, stationary data and decreasing it for non-stationary, shifting data; more efficient α updates remain future work.Source: §6 · p. 10

Future work

  • Future work proposes MoE routing to distinguish heterogeneous patterns and pattern-supervised α updates to address initialization and interpretability.The proposed environment-aware mechanism also aims to make representation spaces more adaptive and explainable.Source: §6 · p. 10

7 Conclusion

The paper presents SRS as a gradient-based, plug-and-play module for patch-based forecasting models and introduces SRSNet, which combines SRS with an MLP head. The authors report state-of-the-art performance for SRSNet on real-world datasets and performance improvements when SRS is added to patch-based models.

7 Conclusion

  • SRSNet achieves state-of-the-art performance on real-world datasets, while SRS effectively improves patch-based models.SRS adaptively selects and reassembles contextual-series patches; the module is gradient-based and plug-and-play, and SRSNet combines it with an MLP head.Source: §7 · p. 10

NeurIPS Paper Checklist

The checklist reports that the paper's claims reflect its contributions and scope, and that experimental details, code, and data are available for reproduction. It also records disclosures on limitations, ethics, societal impact, assets, human subjects, and LLM use.

Claims and scope

  • The authors report that the abstract and introduction accurately reflect the paper's contributions and scope.They state that the claims match the contributions and scope of the work.Source: NeurIPS Paper Checklist · p. 17

Limitations

  • The authors state that they discuss SRS limitations in Section 6.The checklist marks the paper as discussing its limitations.Source: NeurIPS Paper Checklist · p. 17

Reproducibility

  • The authors report providing reproducible code and datasets, with experimental instructions in the repository README and full details in Appendix A.They also report that the paper specifies training and test details.Source: NeurIPS Paper Checklist · pp. 18–19

Experiments

  • The authors report standard deviations for SRSNet settings in Table 8 and computational resource requirements in Table 5.The checklist marks both experiment statistical significance and compute-resource reporting as addressed.Source: NeurIPS Paper Checklist · pp. 19–20

Ethics and societal impact

  • The authors report adherence to the NeurIPS Code of Ethics and mark societal-impact discussion as not applicable.They state that they believe the work's societal implications are not significant enough to warrant specific mention.Source: NeurIPS Paper Checklist · p. 20

Assets and release

  • The authors state that the paper poses no high-risk release concerns and that its public code and datasets are properly credited.They also report plans to provide documentation alongside the code upon acceptance.Source: NeurIPS Paper Checklist · pp. 21–22

Other disclosures

  • The authors mark human-subjects research and theoretical results as not applicable, and state that the method has no LLM component.The checklist reports that the work involves neither crowdsourcing nor human subjects.Source: NeurIPS Paper Checklist · pp. 17, 18, 22, 23

A Experimental Details

The appendix identifies a dataset-statistics table and reports the shared experimental implementation and training setup.

A Experimental Details

  • Table 7 presents dataset statistics.The supplied caption identifies the table's scope but does not provide its values.Source: Table 7 · p. 24
  • Experiments use the TFB repository for unified evaluation, with baseline results also derived from TFB and the “Drop Last” trick disabled.The authors follow the TFB settings of Qiu et al. [2024] for that comparison.Source: A Experimental Details · p. 24
  • SRSNet experiments use PyTorch in Python 3.8 on an NVIDIA Tesla-A800 GPU, with MSE loss and the ADAM optimizer.Batch size starts at 64 and can be halved to a minimum of 8 in response to an out-of-memory issue.Source: A Experimental Details · p. 24

A.1 Full results

The full-results materials include multivariate forecasting results, ablation studies, and plug-in studies across datasets and forecasting horizons. The captions specify what each table covers but do not provide its cell values.

A.1 Full results

  • Table 8 reports multivariate forecasting results across datasets and horizons F ∈{96, 192, 336, 720}, with SRSNet standard deviations calculated over five random seeds.The caption marks best results in red and second-best in blue.Source: Table 8 · p. 24
  • The caption identifies Table 9 as ablation studies for the datasets at forecasting horizons F ∈ {96, 192, 336, 720}, and says red marks the best result.Source: Table 9 · p. 24
  • Table 10 is also labeled an ablation study across datasets and horizons F ∈{96, 192, 336, 720}, with best results marked in red.No ablation values are included in the supplied caption.Source: Table 10 · p. 25
  • Tables 12 and 11 are labeled plug-in studies across datasets and horizons F ∈{96, 192, 336, 720}, with improvements marked in black.The supplied captions do not identify which models or metrics improved.Source: Table 12 · p. 25, Table 11 · p. 25

A.2 Showcases

Figures 5–7 visualize input-96-predict-96 results for SRSNet on ETTh1, ETTm2, and Traffic, describing its handling of special cases with SRS. Grey rectangles indicate selected patches of size 24.

A.2 Showcases

  • On ETTh1, Figure 5 visualizes SRSNet processing special cases with SRS; grey rectangles mark selected patches of size 24.The visualization uses input-96-predict-96 results.Source: Figure 5 · p. 26
  • On ETTm2, Figure 6 visualizes SRSNet processing special cases with SRS; grey rectangles mark selected patches of size 24.The visualization uses input-96-predict-96 results.Source: Figure 6 · p. 26
  • On Traffic, Figure 7 visualizes SRSNet processing special cases with SRS; grey rectangles mark selected patches of size 24.The visualization uses input-96-predict-96 results.Source: Figure 7 · p. 26

B Related Works

The related-work discussion places time-series analysis across several application domains and describes forecasting methods ranging from vector-sequence and Transformer models to MLP-based approaches. It frames forecasting as a widely used real-world task.

B Related Works

  • Time-series analysis spans economy, transportation, health, weather, and energy, and includes forecasting, anomaly detection, classification, and imputation.The discussion identifies forecasting as the most widely used task in real-world applications.Source: B Related Works · pp. 26–27
  • Vector-sequence methods such as TimesNet [Wu et al., 2023a] and SegRNN [Lin et al., 2023] use CNNs or RNNs to capture temporal dependencies.The passage also notes that learned features may outperform human-engineered features.Source: B Related Works · p. 27
  • Transformer approaches, including PatchTST [Nie et al., 2023], are described as capturing relationships between time points and enhancing forecasting performance.Source: B Related Works · p. 27
  • MLP-based methods such as DUET [Qiu et al., 2025a] use simpler, smaller architectures while attaining competitive forecasting accuracy.Source: B Related Works · p. 27

Related papers

Loading 2510.14510v5…