Source-linked AI summary
CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns
Shengsheng Lin, Weiwei Lin, Xinyi Hu, Wentai Wu, Ruichao Mo, Haocheng Zhong
TL;DR
Long-horizon forecasting needs stable periodicity, but existing methods often pursue it through long-range dependency modeling rather than directly modeling the patterns. The paper introduces RCF, which learns recurrent cycles and forecasts residuals, and combines it with Linear or shallow MLP backbones in CycleNet. CycleNet achieves state-of-the-art results across multiple domains, while RCF also improves several existing models.
Problem
Long-horizon forecasting requires capturing stable periodic patterns, motivating explicit periodicity modeling beyond indirect long-range dependency extraction.
Method
RCF learns recurrent cycles to model periodic components and forecasts residuals, while CycleNet combines RCF with a Linear layer or shallow MLP.
Results
CycleNet achieves consistent state-of-the-art performance across multiple domains, and RCF improves existing models by approximately 5% to 10% for PatchTST and iTransformer and approximately 20% for DLinear.
Takeaways & Limitations
Explicit periodicity modeling enables simple Linear and MLP backbones to achieve strong forecasting performance with RCF.
Takeaways & Limitations
RCF does not explicitly model inter-channel relationships, and applying it directly to iTransformer does not significantly improve MSE in some spatiotemporal scenarios.
Abstract
from arXiv · showhide
The stable periodic patterns present in time series data serve as the foundation for conducting long-horizon forecasts. In this paper, we pioneer the exploration of explicitly modeling this periodicity to enhance the performance of models in long-term time series forecasting (LTSF) tasks. Specifically, we introduce the Residual Cycle Forecasting (RCF) technique, which utilizes learnable recurrent cycles to model the inherent periodic patterns within sequences, and then performs predictions on the residual components of the modeled cycles. Combining RCF with a Linear layer or a shallow MLP forms the simple yet powerful method proposed in this paper, called CycleNet. CycleNet achieves state-of-the-art prediction accuracy in multiple domains including electricity, weather, and energy, while offering significant efficiency advantages by reducing over 90% of the required parameter quantity. Furthermore, as a novel plug-and-play technique, the RCF can also significantly improve the prediction accuracy of existing models, including PatchTST and iTransformer. The source code is available at: https://github.com/ACAT-SCUT/CycleNet.
1 Introduction
Long-horizon forecasting depends on capturing stable periodicity, yet existing approaches primarily seek it indirectly through long-range dependency modeling. CycleNet instead explicitly models periodic patterns with RCF and simple backbones.
- Motivation: Stable periodicity provides a practical foundation for accurate long-horizon time series forecasting.Long horizons cannot rely solely on recent means, trends, or other short-term information.
- Motivation: Electricity data exhibits clear shared daily periodic patterns that can be represented by repeating a globally shared daily segment.The repeated segment represents cyclic components across multiple days.
- Method: RCF explicitly models inherent periodic patterns with learnable recurrent cycles, then forecasts the residual components.This reframes long-term prediction around modeled cycles rather than only feature extraction from long sequences.
- Contribution: Combining RCF with a single-layer Linear model or dual-layer MLP forms the proposed CycleNet method.The resulting method is described as simple yet powerful.
- Contribution: RCF is presented as a technique that enhances both basic and existing forecasting models.The paper positions it as a flexible contribution for improving LTSF performance.
2 Related work
CycleNet extends seasonal-trend decomposition by explicitly learning globally shared periodic patterns within independent sequences. It combines this decomposition with a simple Linear or MLP backbone.
- Seasonal-trend decomposition: Classical seasonal-trend decomposition separates time series into seasonal and trend components, often using a moving-average kernel.Autoformer, FEDformer, and DLinear use this approach to model the components independently.
- RCF: RCF can be viewed as a seasonal-trend decomposition method whose key difference is explicit global periodic-pattern modeling with learnable recurrent cycles.The cycles are modeled within independent sequences.
- RCF: RCF is described as conceptually simple, computationally efficient, and capable of improving prediction accuracy.These properties distinguish it from the reviewed decomposition techniques in the paper’s positioning.
- CycleNet: CycleNet combines RCF with a Linear- or MLP-based backbone for simple and efficient time series forecasting.The paper reviews Transformer-, RNN-, and other forecasting categories to position the method.
3 CycleNet
CycleNet transforms long-horizon forecasting into residual modeling after explicitly representing periodic components with learnable recurrent cycles. A Linear or shallow MLP backbone forecasts the residuals.
- Framework: For a D-channel series, forecasting maps past L observations to H future steps, where periodicity is especially important over horizons of 96–720 steps.These horizons correspond to several days or months.
- Residual Cycle Forecasting: RCF first models periodic patterns through learnable recurrent cycles within independent channels, then predicts residual components.These are the two principal stages of the technique.
- Periodic patterns modeling: Learnable recurrent cycles Q have length W, are initialized to zero, globally shared within channels, and trained jointly with the prediction backbone.Cyclic replication of Q produces sequence-length cyclic components.
- Periodic patterns modeling: The cycle length W should be set to the dataset’s maximum stable cycle and can be examined using dataset characteristics or autocorrelation.The paper cites daily and weekly cycles in electricity and traffic as examples.
- Residual forecasting: Residual forecasting subtracts modeled cyclic components, predicts the residual sequence with a backbone, and adds predicted residuals to future cyclic components.This converts the original task into cyclic residual component modeling.
- Cycle alignment: Equivalent input and forecast cycle subsequences are obtained by shifting Q by t mod W and repeating it to match the required lengths.The construction uses floor-based full repetitions plus remainder segments.
- Backbone and normalization: CycleNet uses a single-layer Linear or dual-layer MLP backbone, while instance normalization uses the input window’s mean and standard deviation.The default loss is Mean Squared Error for consistency with mainstream methods.
4 Experiments
Experiments evaluate CycleNet on standard multivariate LTSF benchmarks, comparing accuracy, efficiency, and the contribution of RCF. Results show strong overall performance and lightweight operation, while revealing limitations on traffic and under extreme points.
- Main results: CycleNet achieves state-of-the-art multivariate LTSF performance overall, with CycleNet/MLP ranking first and CycleNet/Linear second except on Traffic.Results use look-back length L = 96 and average forecast horizons H ∈{96, 192, 336, 720}.
- Limitations: CycleNet remains below iTransformer on Traffic because independently modeling channels does not capture the dataset’s necessary inter-channel relationships.Traffic contains spatiotemporal and temporal-lag dependencies between neighboring detection points.
- Ablation study: RCF improves Linear and MLP prediction accuracy by approximately 10% to 20%, indicating that gains largely come from periodic-cycle modeling.The ablation combines both backbones with the same default instance normalization strategy used by CycleNet.
- Ablation study: RCF improves existing models by approximately 5% to 10%, including PatchTST and iTransformer, and improves DLinear by approximately 20%.These results support RCF as a flexible plug-and-play technique across different backbones.
- Limitations: On Traffic, RCF can reduce MAE while increasing MSE because extreme points and locally exaggerated average cycles create a few large residual errors.MSE amplifies the effect of these local errors more strongly than MAE.
- Periodic-pattern analysis: CycleNet’s learned periodic patterns differ across channels despite shared cycle lengths, motivating separate pattern modeling for each channel.One electricity channel shows intermittent weekday periodicity, while other channels show relatively uniform weekday patterns.
5 Discussion
CycleNet is most effective when periodic patterns are prominent, stable, and compatible with its fixed-cycle, channel-independent design. Its limitations include varying cycle lengths, outliers, long-range cycles, and unmodeled inter-channel relationships.
- CycleNet may not suit datasets whose cycle length varies over time because RCF learns only a fixed-length cycle.
- A shared cycle length can be problematic when different channels exhibit different periodicities under CycleNet’s channel-independent strategy.The paper suggests splitting data by cycle length or modeling channels separately as possible remedies.
- Significant outliers can distort RCF’s learned historical average cycles, causing inaccurate periodic and residual component estimates.
- RCF is effective for mid-range stable cycles such as daily or weekly patterns, but yearly cycles may require decades of history for training.The paper identifies long-range cycle modeling as a target for future techniques.
- RCF does not explicitly model inter-channel relationships, which are important in spatio-temporal scenarios containing spatial and temporal dependencies.
6 Conclusion
The paper argues that explicitly modeling periodicity can improve long-term forecasting. It introduces RCF and combines it with lightweight Linear or MLP backbones in CycleNet, while identifying inter-channel modeling as a promising direction.
- 6 Conclusion: The paper identifies inherent periodic patterns as important for accurate long-horizon forecasting and advocates explicitly modeling them.
- 6 Conclusion: CycleNet combines recurrent-cycle periodic modeling with single-layer Linear or dual-layer MLP backbones.
- 6 Conclusion: RCF models shared periodic patterns through recurrent cycles and predicts residual components through a backbone.
- 6 Conclusion: Future work includes integrating CycleNet with methods that effectively model inter-channel relationships.
B.2 Utilizing ACF analysis to determine cycle length
ACF analysis helps select CycleNet’s cycle-length hyperparameter W by locating periodic peaks. The observed maximum cycles align with manually inferred dataset cycle lengths, supporting those settings.
- W must match the length of the dataset’s maximum stable periodic cycle for RCF to function as intended.
- ACF measures correlation between a time series and lagged values, helping identify periodicity through lag-dependent peaks.
- The largest ACF peak corresponds to the lag aligned with the dataset’s maximum cycle length.
- The datasets show evident periodicity through prominent ACF peaks and troughs in the training-set visualizations.
- The maximum cycles in Figure 6 align with pre-inferred cycle lengths, indicating that W should be set strictly to those values.
B.3 Experimental details
The experiments use standard LTSF benchmark datasets and fixed train-validation-test splits, with implementation choices adjusted for dataset size. RevIN is omitted on Solar because it substantially degrades performance there.
- The experiments use ETT, Electricity, Solar-Energy, Traffic, and Weather benchmark datasets.
- ETT datasets use 6:2:2 train-validation-test splits, while the other datasets use 7:1:2 splits.
- CycleNet training used PyTorch on one NVIDIA RTX 4090 GPU for up to 30 epochs with early stopping.
- Batch size was 256 for ETTs and Weather and 64 for the remaining datasets because those datasets have more channels.
- RevIN was disabled for Solar because it caused a significant performance drop associated with zero-valued nighttime segments and non-day-aligned look-back windows.
C More experimental results
RCF learns periodic patterns under varying forecasting configurations, with horizon changes leaving patterns nearly unchanged and cycle length determining the captured periodic structure.
- The learned periodic pattern remains almost unchanged as the forecast horizon varies.
- Changing the look-back window preserves the overall pattern but makes it smoother when the window is longer.A longer look-back supplies richer periodic information and reduces reliance on the learned pattern component.
- The backbone affects learned patterns: DLinear smooths them, iTransformer changes them through multichannel interactions, and PatchTST resembles Linear.
- With W = 168 on Electricity, the recurrent cycle learns the complete weekly and daily periodic pattern.
C.2 Full results with different look-back lengths
CycleNet maintains state-of-the-art performance across forecast horizons and look-back lengths, with mostly very low variability, though Traffic remains a challenging exception.
- CycleNet achieves state-of-the-art results in most settings across forecast horizons, with standard deviations mostly below 0.001.Table 7 reports CycleNet results averaged over five random seeds.
- The longer-look-back experiments use L ∈ {336, 720} and reproduce baseline results after fixing discarded test batches.
- Even with longer look-back lengths, CycleNet generally maintains a significant advantage and state-of-the-art performance in most scenarios.
- PatchTST and SegRNN outperform CycleNet on Traffic despite their channel-independent designs.The Traffic dataset contains more outliers and up to 862 channels, favoring stronger nonlinear modeling capacity.
C.3 Full results with different STD techniques
Against other Seasonal-Trend Decomposition techniques, RCF consistently performs best overall, especially when the data exhibit strong periodicity, but not on noisy weather data.
- The comparison uses a pure Linear backbone, look-back length 336, and no additional instance normalization strategies.Under this setup, CLinear denotes CycleNet/Linear without RevIN.
- RCF consistently outperforms the other Seasonal-Trend Decomposition techniques.
- On the relatively noisy weather dataset, RCF shows no significant advantage, while several decomposition techniques perform similarly to pure Linear.
C.4 Ablation study of RevIN
RevIN usually improves CycleNet, but comparisons against versions without RCF identify RCF—not RevIN—as the key accuracy-enhancing component.
- RevIN leads to better performance in most cases.Its effects are especially examined on the ETTh2 and Weather datasets.
- CycleNet significantly outperforms RLinear and RMLP, which represent CycleNet without the RCF technique.
- The ablation concludes that RCF is the key factor significantly enhancing CycleNet’s prediction accuracy.
C.5 Further Analysis in Traffic Scenarios
CycleNet performs strongly across most evaluated domains but is less effective in traffic scenarios with severe outliers and important spatiotemporal relationships. The authors identify these conditions as limitations of the current RCF design and propose robustness and multi-channel modeling as future directions.
- Performance in traffic scenarios: CycleNet achieves state-of-the-art performance overall except on the Traffic dataset, where it still remains top-tier on PEMS.The PEMS evaluation uses four public subsets with forecast horizons of 12, 24, 48, and 96 steps.
- Effectiveness of RCF: RCF substantially contributes to CycleNet’s performance despite using only a single-layer Linear or two-layer MLP backbone.Comparisons with RLinear and DLinear indicate that RCF narrows the gap between a simple Linear model and state-of-the-art models.
- Traffic-specific limitation: Traffic contains numerous and large outliers that can skew RCF’s learned average cycles and produce inaccurate residual components.Subtracting an exaggerated local average cycle can distort point predictions, with large errors affecting MSE more strongly than MAE.
- Traffic-specific limitation: Models that explicitly capture inter-channel relationships, such as iTransformer and GNN, are better suited to traffic scenarios involving extreme points and temporal lag characteristics.The authors illustrate this with sudden traffic surges propagating between junctions, which current CycleNet does not model adequately.
- Performance in traffic scenarios: On PEMS, the MSE gap relative to the state-of-the-art decreases from approximately 10% on Traffic to about 5%, consistent with less severe extreme points.The authors report that CycleNet improves on PEMS despite both datasets representing traffic.
- Future directions: The authors identify two improvement directions: making RCF more robust to outliers and developing more suitable multi-channel modeling within the RCF framework.These directions target the limitations observed in traffic scenarios.