PaperlayerView exact revision on arXiv

Source-linked AI summary

TimeES: Probabilistic and Deterministic Time Series Forecasting via Evolutionary Spectra

Weiwei Ye, Renhe Jiang, Hangchen Liu, Dongyuan Li, Yoshihide Sekimoto

arXiv:2609.32384v2cs.LGcs.AI

TL;DR

Non-stationary time series forecasting must account for evolving spectral patterns and uncertainty, which current methods do not explicitly model together. TimeES learns parameterized evolutionary spectra for deterministic and probabilistic forecasting, and reports state-of-the-art performance across both task types, with reduced spectral-estimation complexity.

  • Problem

    Current deep learning approaches do not explicitly model both evolution and randomness in Fourier spectra, limiting accurate forecasts of trajectories and uncertainty in non-stationary time series.

    Source: §1 · p. 1

  • Method

    TimeES parameterizes evolutionary spectra as a learned amplitude representation modulated by random variables, supporting deterministic decomposition and probabilistic generation.

    Source: §1 · p. 2, §4.1 · p. 4

  • Results

    TimeES achieves state-of-the-art performance across deterministic and probabilistic forecasting, with a 17.23% average reduction in CRPS across datasets.

    Source: §1 · p. 2, §5.1 · p. 7

  • Takeaways & Limitations

    Exploiting Hermitian symmetry and spectral energy sparsity reduces estimation complexity from O(NM) to O(NK), where K ≪ M/2.

    Source: §1 · p. 2, §6 · p. 9

    Frequency selection may be ineffective under high noise when spectral energy is not concentrated in a few frequencies, and learning evolutionary spectra requires sufficient data.

    Source: §6 · p. 9

Abstract

from arXiv · show

Real-world time series are inherently non-stationary, with trends, periodic patterns, and uncertainty evolving over time. While the Fourier domain offers a natural lens to model time series, current deep learning approaches do not explicitly model evolution and randomness in the Fourier spectra, which limits their ability to accurately predict both the expected trajectory and its uncertainty in non-stationary time series. Motivated by Evolutionary Spectra (ES) theory, we propose TimeES, a general framework that enables probabilistic and deterministic forecasting via the evolutionary spectra theory. Specifically, we derive a parameterizable evolutionary spectra formulation, recasting non-stationary random process modeling as learning an evolving representation modulated by random variables. Furthermore, we reduce the complexity of the estimated spectra from O(NM) to O(NK), where K << M/2, by exploiting Hermitian symmetry and spectral energy sparsity for frequency selection. Based on a simple linear backbone, our proposed TimeES achieves consistent state-of-the-art performance across both deterministic and probabilistic forecasting tasks, with high efficiency and interpretability. Code is available at: https://github.com/wwy155/TimeES.

1 Introduction

Non-stationary time series combine evolving spectral patterns with uncertainty, but existing frequency-domain and probabilistic methods do not adequately represent both. TimeES estimates evolutionary spectra for deterministic and probabilistic forecasting, and the paper reports state-of-the-art performance across both task types.

1 Introduction

  • Analytically derived Fourier features rely on locally invariant spectra and therefore do not capture the spectral evolution characteristic of non-stationary series.The introduction identifies FFT and STFT among these frequency-domain approaches.Source: §1 · p. 1
  • Probabilistic methods may conflate predictable periodic dynamics with random fluctuations, while STFT can blur spectral structure through leakage and limited frequency resolution.The ETTm1 case study describes time-varying frequency components and stochastic fluctuations; the cited passage reports blurred spectrograms and spikes between trend and daily spectra.Source: §1 · pp. 1–2
  • 17.23% average improvement over recent baselines on the most challenging probabilistic forecasting benchmarks accompanies state-of-the-art performance across deterministic and probabilistic tasks.The ETTm1 case study illustrates TimeES's point-wise spectral estimates and period-aware uncertainty intervals.Source: §1 · p. 2, Figure 1 · p. 2
  • TimeES parameterizes evolutionary spectra as a generative model, enabling direct estimation, interpretable decomposition, and probabilistic or deterministic forecasting.The framework exploits Hermitian symmetry and spectral energy sparsity to reduce estimation complexity from O(NM) to O(NK), where K ≪ M/2.Source: §1 · p. 2

2 Related work

Prior work offers tools for frequency-aware deterministic forecasting, non-stationary generative modeling, and evolutionary spectra estimation, but each leaves limitations that TimeES addresses.

2.1 Evolutionary Spectra for Non-stationary Process

  • Existing evolutionary-spectrum estimators rely on limiting assumptions or perturbation-based optimization requiring strict initialization, whereas TimeES estimates spectra directly from data through representation learning.The cited passage describes evolutionary spectra as non-identifiable, with infinitely many valid solutions.Source: §2.1 · p. 2

2.2 Deterministic Time Series Forecasting

  • Deterministic forecasting spans RNNs, Transformers, frequency-aware models, distribution-shift methods, and Koopman or DMD approaches, whose spectra concern lifted operators rather than the observed process's time-varying power spectra.The passage distinguishes evolutionary spectra as modeling the observed non-stationary random process itself.Source: §2.2 · pp. 2–3

2.3 Probabilistic Time Series Forecasting

  • Many probabilistic forecasting methods adapt computer-vision generative models and assume instance-independent or stationary data; NsDiff allows a varying endpoint distribution to better capture non-stationarity.The passage contrasts these approaches with TimeES's premise that time series arise from a non-stationary random process.Source: §2.3 · p. 3

3 Preliminary

The preliminary section introduces evolutionary spectra as a representation of non-stationary processes whose spectral properties vary over time. Estimating them is challenging because the true spectra are unknown and admit infinitely many solutions.

3 Preliminary

  • A stationary process may lack an absolutely integrable realization, so its classical Fourier integral may not exist; the spectrum representation theorem extends the representation using a random spectral measure.The passage gives white noise as an example of a stationary process.Source: §3 · p. 3
  • Evolutionary spectra extend spectral representation to non-stationary processes through a time-varying complex amplitude A(t, ω) and random measure dZ(ω).The energy spectral density is defined as dS(t, ω) = |A(t, ω)|^2.Source: §3 · p. 3
  • Evolutionary spectra reflect non-stationary random structure across frequencies, but their unknown values and infinitely many valid solutions make identification challenging.The paper proposes a tractable formulation so evolutionary spectra can be estimated through representation learning.Source: §3 · p. 4

4 Methodology

TimeES estimates an evolving amplitude representation and combines it with physical frequency phases and random variables to synthesize forecasts. Its design supports deterministic and probabilistic prediction while reducing the number of estimated frequency components.

4.1 Discrete Evolutionary Spectra

  • DES parameterizes evolutionary spectra as a point-wise generative model, with A(n, ωk) specifying evolving amplitudes and complex Gaussian Wk variables providing stochastic realizations.Sampling Wk multiple times generates multiple signal realizations.Source: §4.1 · p. 4
  • The matrix formulation separates the learned amplitude A, physical frequency basis F, and probabilistic vector w to represent a sample path of a non-stationary process.TimeES estimates A with a neural network before DES synthesizes the series using F and w.Source: Figure 2 · p. 4, §4.1 · p. 5

4.2 Probabilistic and Deterministic Forecasting via Evolutionary Spectra

  • Hermitian symmetry and energy-based frequency selection reduce spectral estimation from O(NM) to O(NK), with K ≪ M/2.The method selects the smallest frequency subset whose cumulative periodogram energy reaches a target ratio r; for M = 96, K is typically 2 to 5 across datasets.Source: §4.2 · p. 5
  • Estimating only K selected frequencies reduces the network output dimension to 2NK rather than 2NM, with unselected frequencies assigned zero amplitude.The selected frequency columns also restrict the phase matrix used in synthesis.Source: §4.2 · pp. 5–6
  • Fixing Wk = 1 yields deterministic time-frequency decomposition, whereas treating Wk as random yields a generative model for non-stationary stochastic processes.Probabilistic forecasting incurs extra sampling and storage costs; deterministic decomposition avoids that overhead and provides a physically interpretable amplitude representation.Source: §4.2 · p. 6

4.3 Connections with DFT and STFT

  • With time-invariant amplitudes DES reduces to inverse DFT, and with blockwise constant amplitudes it recovers inverse STFT with a rectangular window.Under local stationarity and a normalized window, continuous-time STFT approximates the evolutionary spectrum.Source: §4.3 · p. 6

5 Experiments

Experiments evaluate TimeES on probabilistic and deterministic forecasting, then test its components and inspect frequency-band reconstructions.

5 Experiments

  • The evaluation covers multivariate probabilistic forecasting and univariate and multivariate deterministic forecasting across real-world benchmarks.The probabilistic study uses nine benchmarks; the deterministic study evaluates eight benchmarks and includes an additional single-variate task.Source: §5 · p. 6, §5.1 · p. 7, §5.2 · p. 8

5.1 Probabilistic Time Series Forecasting

  • 17.23% lower CRPS and 18.66% lower MSE on average than recent baselines, with TimeES achieving state-of-the-art results on both measures.On Traffic, reductions were 35.15% for CRPS and 28.34% for MSE; gains were marginal on ILI, where limited samples hinder reliable ES estimation.Source: Table 3 · p. 7, §5.1 · p. 7

5.2 Deterministic Time Series Forecasting

  • TimeES achieves state-of-the-art MSE on 8 of 9 datasets in single-variate forecasting, improving over FreTS by 19.38% and 16.00% over single-variate and multivariate settings, respectively.It also surpasses FEDformer by 9.98% and 12.28% in those respective settings.Source: Table 4 · p. 8, §5.2 · p. 8

5.3 Ablation Study

  • TimeES performs best across all three ablation tasks; omitting evolutionary spectra yields worse performance than omitting DES on the reported long-term forecasting comparisons.Without DES, relative MSE improvements over w/o E are 24.02% for single-variate and 8.27% for multivariate long-term forecasting.Source: Table 5 · p. 8, §5.3 · pp. 8–9

5.4 Interpretability Analysis

  • The learned spectra separate frequency-band information: low frequencies capture long-term trends, while high frequencies reflect abrupt changes and fine-grained dynamics.The ETTh1 reconstruction is shown in Figure 4.Source: Figure 4 · p. 9, §5.4 · p. 9

6 Conclusion

The conclusion presents TimeES as a deep-learning framework for forecasting through evolutionary spectra, while the accompanying background reviews non-stationary spectral modeling and its theoretical representation.

6 Conclusion

  • TimeES combines evolutionary spectra theory with deep learning for deterministic and probabilistic forecasting, reducing spectral-estimation complexity from O(NM) to O(NK), where K ≪ M/2.The reduction exploits Hermitian symmetry and spectral energy sparsity.Source: §6 · p. 9

A.1 Evolutionary Spectra for Non-stationary Time Series

  • Evolutionary spectra provide a point-wise spectrum for harmonizable stochastic processes, but are non-identifiable; existing estimation techniques rely on restrictive conditions or strict initialization.TimeES estimates evolutionary spectra directly from data without strong parametric assumptions or constraints.Source: §A.1 · p. 15

A.2 Deep Learning for Deterministic Time Series Forecasting

  • Deep-learning methods for deterministic forecasting, including frequency-aware models, do not explicitly model spectral evolution and therefore cannot effectively capture frequency variations.The passage contrasts this limitation with TimeES’s focus on non-stationary stochastic processes.Source: §A.2 · p. 15

A.3 Deep Learning for Probabilistic Time Series Forecasting

  • Most existing probabilistic time-series generators use computer-vision architectures and implicitly rely on i.i.d. data; NsDiff varies its endpoint distribution to better capture non-stationarity.TimeES instead assumes a non-stationary, stochastic data-generating process.Source: §A.3 · p. 15

B.1 Theorem 3.1

  • The theorem’s spectral representation models the process through a stochastic integral with a random function whose increments are orthogonal and whose squared-magnitude expectation is governed by h(ω).The construction uses truncated Fourier integrals and normalized expected power because sample paths typically are not absolutely integrable over the infinite domain.Source: §B.1 · p. 16

B.2 Theorem 4.1

The appendix develops a discrete stochastic representation of evolutionary spectra, examines its relation to STFT and frequency selection, and reports spectral energy concentration across benchmark datasets. It also describes the benchmark and evaluation metrics used in the experiments.

B.2 Theorem 4.1

  • A discrete stochastic representation uses an evolving amplitude to modulate complex Gaussian noise, with frequency discretization and a Riemann-sum approximation.The frequency increments are represented using i.i.d. standard complex normal variables.Source: §B.2 · p. 17

B.3 Theorem 4.3

  • A real-valued signal is equivalent to imposing Hermitian symmetry on its frequency coefficients, including real-valued coefficients at the zero and Nyquist frequencies.The Nyquist-frequency condition applies when M is even.Source: §B.3 · p. 18

B.4 Continuous STFT Approximates ES under Local Stationarity

  • The STFT approximates the local spectral amplitude through window-based frequency smoothing, while narrowing the window improves temporal localization but increases frequency smearing.Approximation accuracy depends on the signal’s time–frequency structure and the chosen window.Source: §B.4 · p. 19

B.5 Relationships between SNR, r and K

  • Selecting the smallest frequency set that reaches energy fraction r makes the retained count K adapt to signal-to-noise ratio without prior frequency-structure or noise-level knowledge.Higher-SNR signals need fewer frequencies, whereas noise-dominated signals need more.Source: §B.5 · p. 20

C Experimental Details

  • The experimental details identify benchmark datasets and include a supplementary classification evaluation using learned representations.The benchmark table is summarized in Table 7.Source: C Experimental Details · p. 21, Table 7 · p. 21

C.1 Datasets Details

  • The datasets cover electricity transformer and client consumption, exchange rates, traffic, weather, and time-series classification benchmarks.ETT includes hourly and 15-minute series; the ECL dataset records hourly consumption for 321 clients.Source: §C.1 · p. 21

C.2 Metrics

  • Evaluation uses MAE and MSE for forecasting, QICE and CRPS for probabilistic forecasts, and accuracy for classification.CRPS is approximated using 100 generated samples.Source: §C.2 · p. 22

C.3 Spectral Energy Sparsity

  • ETTm1 retains 90% of its energy in six frequencies; ETTm2 and ETTh1 need only two to five, while ExchangeRate places over 80% in one.Across datasets, even 99% energy capture rarely requires more than 10–20 frequencies, supporting the use of r = 0.9.Source: §C.3 · pp. 22–23, Figure 5 · p. 23

C.4 Ablation Experiments

The ablations show that evolutionary-spectra modeling is central to forecasting performance, while DES supplies generative forecasting and uncertainty estimates. Additional experiments report TimeES results on synthetic spectral extrapolation and time-series classification.

C.4 Ablation Experiments

  • Removing evolutionary-spectra modeling raises ETTm1 single-variate MSE from 0.893 to 1.306 and probabilistic CRPS from 0.310 to 0.417.Removing DES changes deterministic results only slightly, but makes the model non-generative and unable to produce predictive distributions or uncertainty estimates.Source: §C.4 · pp. 23–24, Table 8 · p. 24

C.5 Additional Baseline Experiments

  • The additional-baseline experiments compare TimeES with six models spanning Koopman methods, bridge-based temporal modeling, mixing, and Transformer architectures.The supplied passage describes the comparison setup but gives no outcome values.Source: §C.5 · p. 24

C.6 Synthetic Experiments

  • 0.290 MSE is TimeES’s lowest result in the chirp-signal extrapolation comparison, where other models struggle to track increasing frequency.The synthetic signal’s periodicity changes from 64 to 128 and magnitude from 1 to 3; TimeES (NES) captures the evolving frequency and amplitude.Source: §C.6 · pp. 24–25

C.7 Time Series Classification

  • 62.96% average accuracy, 2.85% above TimesNet, makes TimeES the top model on five UEA classification tasks.It also reaches 92.83% on SelfRegulationSCP1 and 36.81% on EthanolConcentration, while scoring 86.50% on UWaveGestureLibrary versus iTransformer’s 87.50%.Source: §C.7 · pp. 25–26, Figure 7 · p. 26, Table 10 · p. 26

D Efficiency Analysis

On ETTh1, TimeES is reported as the most efficient among compared methods for probabilistic and 720-step multivariate forecasting, using one forward pass for probabilistic samples.

D Efficiency Analysis

  • TimeES achieves the highest efficiency among compared methods in probabilistic and deterministic forecasting on ETTh1.It generates probabilistic samples in one forward pass, unlike diffusion approaches that typically require hundreds or thousands of denoising steps; it also runs faster than optimized frequency-based models in deterministic forecasting.Source: D Efficiency Analysis · p. 26, Figure 8 · p. 27

E Hyperparameter Sensitivity

The frequency-retention parameter r controls how many components are kept according to their share of total energy. The reported r = 0.9 is stable across forecasting tasks, while frequency selection remains dataset-sensitive.

E Hyperparameter Sensitivity

  • r = 0.9 yields stable performance across the evaluated forecasting tasks, while aggressive pruning degrades Traffic results.On several ETT multivariate forecasting datasets, appropriate frequency selection also improves accuracy; the authors contrast energy-based selection with rigid top-k selection that may miss critical components.Source: E Hyperparameter Sensitivity · pp. 27–28, Figure 9 · p. 28

F Additional Experimental Results

The appendix provides full experimental results for classification, probabilistic and deterministic forecasting, and decomposition-framework evaluations. It also shows additional predictive-uncertainty examples across datasets.

F Additional Experimental Results

G Additional Showcases

  • Figure 10 presents predictive uncertainty examples across datasets, plotting ground truth, predicted means, and 96% prediction intervals.The plots show the last 500 predicted time steps for the first four variables in each dataset.Source: Figure 10 · p. 33, G Additional Showcases · p. 33

H Additional Interpretability Analysis

The analysis attributes predictive variance to specific frequencies, revealing channel-level differences. The comparison with Koopman models distinguishes what the spectrum represents and how its contributions combine.

H Additional Interpretability Analysis

I Relation to Koopman Operators and Dynamic Mode Decomposition

  • The comparison with KooNPro considers how the models treat spectral randomness, decomposition, non-stationarity, forecasting, and their relation to DMD.KooNPro is used as a representative probabilistic Koopman model.Source: I Relation to Koopman Operators and… · p. 35

I.1 Randomness in the Spectrum

  • KooNPro models uncertainty over frequencies, whereas evolutionary spectra resolve observation variance by frequency, which adds across independent retained frequencies.On one ETTh1 batch, analytic variance summed to 1.310 versus an empirical 1.307 from 4,000 sampled paths; mean relative error was 1.8%.Source: §I.1 · p. 36

I.2 Spectral Decomposition

  • Evolutionary spectra decompose the observation additively, so any frequency sub-band yields a time-domain component without a decoder.The paper connects this additivity to band-wise reconstruction and per-channel uncertainty attribution.Source: §I.2 · p. 36

I.3 Parameterization of Non-stationarity

TimeES fixes frequencies on a DFT grid while allowing amplitudes to evolve, and predicts the forecast horizon in closed form. The paper compares this design with DMD and KooNPro, and examines frequency support, uncertainty, and input-length sensitivity.

I.3 Parameterization of Non-stationarity

  • On ETTh1 and an off-grid synthetic series, learnable frequencies leave accuracy unchanged and move at most 0.065 of a bin, versus 0.40 and 0.27 needed to reach the true components.DES uses amplitude modulation on fixed grid frequencies, the form prescribed by Priestley; a time-varying amplitude can represent shifted frequencies.Source: §I.3 · pp. 36–37, Table 17 · p. 36

I.4 Recursive vs. Closed-form Forecasting

  • DES predicts the whole amplitude field at once, producing the horizon as a single matrix product rather than recursive applications of an operator.Unlike recursion, lengthening the horizon adds terms to a sum rather than links to a chain, and one draw of {Wk} keeps a sampled path spectrally coherent.Source: §I.4 · p. 37

I.5 Relation to Dynamic Mode Decomposition

  • In DES terms, DMD uses deterministic weights Wk = 1 and amplitudes restricted to exponential envelopes, while DES predicts arbitrary amplitudes from the lookback and uses random weights.DMD reconstructs from an operator’s eigen-decomposition; DES generates the horizon at once rather than iterating a one-step operator.Source: §I.5 · p. 37

I.6 Empirical Comparison with KooNPro

  • TimeES outperforms KooNPro in all tested settings; at H = 24, it reduces CRPS by 27%–54% with L = 168 and 33%–42% with L = 10, except ETTm1’s 5% margin.At H = 192, KooNPro’s CRPS rises roughly an order of magnitude, and 83%–88% of test points fall below its 10th predictive percentile.Source: §I.6 · p. 37, Table 18 · p. 37

J.1 Frequency Support and Evolving Spectra

  • Frequencies selected on the training split capture 92.3% of test spectral energy on average, close to the target ratio r = 0.9.The selected support is estimated from the training split; evolving spectral content within that support is modeled by A(n, ωk).Source: §J.1 · p. 38, Table 19 · p. 38

J.2 The Spectral Random Variables

  • TimeES achieves best QICE on 5 of 9 datasets and second-best on 3, but Gaussian spectral weights limit flexibility in per-step distribution tails.A shared draw of {Wk} keeps sampled trajectories spectrally coherent, and the predictive covariance between forecast steps is available in closed form; non-Gaussian or mixture spectral measures are proposed as future work.Source: §J.2 · p. 38

J.3 Sensitivity to Input Length

  • On ETTh1, a 24-step input stays within 2% of default; on ETTm1 it raises CRPS 28%, while inputs from 72 steps onward stay within 4%.The 24-step ETTm1 input covers only six hours, not a daily cycle; 96- and 336-step inputs improve on the default.Source: §J.3 · pp. 38–39, Table 20 · p. 39

J.4 Operating Range under Limited Data

TimeES performs well across most benchmark datasets but is less stable when training data are limited. The checklist reports reproducibility details and statistical-significance information, alongside disclosures about resources and research practices.

J.4 Operating Range under Limited Data

  • Best CRPS on 8 of 9 datasets and best QICE on 5 of 9; TimeES estimates evolutionary spectra directly from data rather than assuming an analytic form.The passage attributes these gains to direct estimation and refers to Table 3 for CRPS and Table 11 for QICE.Source: §J.4 · p. 39
  • On ILI, 473 training windows accompany a CRPS standard deviation of 0.172 over 3 seeds; ETTh1 has 17,420 time steps and CRPS standard deviation below 0.01.The passage identifies ILI as the benchmark’s extreme limited-data case and reports the ETTh1 comparison.Source: §J.4 · p. 39
  • NsDiff is less affected in the limited-data regime using trailing-window variance estimates, and has better decile-level QICE on ETTh1, ETTh2, and ETTm1.The passage links NsDiff’s limited-data behavior to variance estimated from a trailing sliding window and its flexibility with asymmetric residuals.Source: §J.4 · p. 39

NeurIPS Paper Checklist

  • The checklist says Section 5 provides information needed to reproduce the main experiments, and Appendix F reports statistical significance below 3% for probabilistic and 1% for deterministic results.The checklist also marks experimental settings and details as specified in Section 5.Source: NeurIPS Paper Checklist · pp. 41–42
  • The checklist reports that experiments used an NVIDIA A6000 GTX GPU, computational costs appear in Section D, and limitations are discussed in Section 6.These are checklist justifications, not additional resource or limitation measurements.Source: NeurIPS Paper Checklist · pp. 40, 43
  • The checklist marks ethics, societal-impact discussion, and asset attribution as addressed; it marks human-subject review and LLM-use declarations as not applicable.The paper states that it involves neither crowdsourcing nor human subjects and that core method development does not involve LLMs.Source: NeurIPS Paper Checklist · pp. 43–46

Related papers

Loading 2609.32384v2…