PaperlayerView exact revision on arXiv

Source-linked AI summary

FreKoo++: Learning Continuous Spectral Dynamics for Temporal Domain Generalization

En Yu, Xiaoyu Yang, Wei Duan, Guangquan Zhang, Jie Lu

arXiv:2608.22224v1cs.LGcs.AI

TL;DR

TDG methods face difficulty handling multi-scale drift, local uncertainty, and irregular observations when generalizing from historical to future domains. FreKoo++ uses continuous Koopman modal dynamics and adaptive spectral weighting with regularization to model and disentangle these dynamics. The paper reports consistent improvements on discrete and continuous TDG benchmarks, alongside theoretical approximation and generalization bounds.

  • Problem

    Existing TDG methods struggle with multi-scale or recurring drift, domain-specific uncertainties, and irregularly sampled observations.

  • Method

    FreKoo++ models source-domain parameter trajectories with continuous Koopman modes whose complex eigenvalues encode frequency and growth or decay, then adaptively weights modes using spectral and stability regularization.

  • Results

    FreKoo++ consistently improves temporal generalization on discrete and continuous TDG benchmarks and is supported by modal approximation and continuous-time generalization bounds.

  • Takeaways & Limitations

    Continuous spectral-dynamical modeling provides a principled direction for robust learning in evolving environments with irregular observations and complex drift.

Abstract

from arXiv · show

Temporal Domain Generalization (TDG) aims to learn from historical domains and generalize to unseen future distributions under concept drift. Nevertheless, prevailing TDG methods struggle with complex real-world streaming scenarios involving both multi-scale drift patterns (e.g., long-term periodicity intertwined with short-term incremental changes) and local uncertainties, especially in continuous settings where observations arrive irregularly. To address this limitation, we propose FreKoo++, a novel continuous spectral-dynamical framework that pioneers the unification of continuous Koopman modal dynamics with adaptive spectral disentanglement. Specifically, FreKoo++ maps source-domain parameters into a compact latent space, modeling their evolution as a superposition of learnable continuous modes where complex eigenvalues jointly encode oscillatory frequency and temporal growth or decay. This formulation naturally accommodates irregular timestamps and supports arbitrary horizon extrapolation without rigid discrete stepping. Furthermore, we propose a new adaptive soft spectral weighting mechanism backed by stability and spectral regularization, which automatically isolates persistent dominant dynamics from transient noise without relying on manual frequency thresholds. We derive modal approximation and generalization bounds that characterize how amplitude and eigenvalue estimation errors propagate with the prediction horizon. Extensive experiments on both discrete and continuous TDG benchmarks demonstrate that FreKoo++ achieves state-of-the-art performance under complex multi-scale drifts and irregular sampling.

I. INTRODUCTION

Temporal Domain Generalization addresses future-distribution generalization under concept drift, but existing methods struggle with multi-scale recurrence, local uncertainty, and irregular sampling. FreKoo++ responds with continuous Koopman spectral dynamics, adaptive disentanglement, theoretical guarantees, and benchmark validation.

  • Temporal Domain Generalization learns from chronologically ordered source domains to generalize to unseen future distributions under evolving temporal shifts.
  • Existing discrete methods assume uniformly spaced temporal grids, limiting their ability to model asynchronously observed streams.Their step-wise transitions are bound to fixed temporal intervals.
  • Current approaches struggle with multi-scale drift because long-term periodicity and recurring patterns extend beyond locally incremental changes.Examples include seasonality, weekly or daily user patterns, and economic cycles.
  • Segmenting streams into temporal domains can violate within-domain IID assumptions, exposing methods to localized non-IID structures and domain-specific artifacts.These uncertainties can complicate optimization and undermine stable cross-temporal generalization.
  • FreKoo++ models parameter evolution through continuous Koopman-inspired modes whose complex eigenvalues encode oscillatory frequency and temporal growth or decay.The formulation supports irregular observations and prediction at arbitrary future timestamps without discrete time stepping.
  • An adaptive soft spectral weighting mechanism with spectral and stability regularization separates persistent dynamics from transient fluctuations without manual frequency selection.The paper also provides approximation and generalization bounds and reports extensive validation across discrete and continuous TDG benchmarks.

B. Challenges

Continuous temporal domain generalization must handle noisy parameter trajectories, multi-scale drift, irregular timestamps, and arbitrary-time extrapolation. FreKoo++ addresses these constraints by combining continuous Koopman modal dynamics with adaptive spectral disentanglement.

  • Parameter-space formulation: Empirical parameter estimates are perturbed by finite-sample effects, intra-domain non-stationarity, and domain-specific variations.These perturbations are represented by ϵ(t_i) around the underlying optimal parameter trajectory.
  • Multi-scale drift patterns: Incremental or locally smooth models struggle to capture periodic and multi-scale parameter dynamics that recur after temporal lags.The challenge concerns long-range recurring behavior rather than only gradual adjacent-domain shifts.
  • Parameter-space uncertainties: Treating noisy empirical parameters as clean signals can overfit transient fluctuations instead of tracking persistent dynamics, degrading future-domain generalization.The perturbation term captures finite-sample effects and domain-specific variations that contaminate the observed sequence.
  • Irregular timestamps: Discrete transitions designed for uniform grids do not scale to varying temporal gaps, limiting prediction at arbitrary future timestamps.Continuous temporal domain generalization must handle non-uniform intervals between historical observations.
  • FreKoo baseline: FreKoo decomposes trajectories into low- and high-frequency components, extrapolates low-frequency Koopman dynamics, and regularizes high-frequency variations.Its Fourier-based decomposition identifies dominant trends and long-term periodic patterns in the low-frequency component, while high-frequency context is not forward-extrapolated.
  • FreKoo++ response: FreKoo++ unifies spectral analysis and continuous modal dynamics, adaptively separates persistent dynamics from fluctuations, and analytically predicts parameters at arbitrary future horizons.The framework operates directly in continuous time, accommodating irregular observations without step-wise numerical integration.

A. Latent Trajectory Construction

FreKoo++ encodes source-domain parameter trajectories in a compact latent space and models them with continuous Koopman-inspired modes. Learnable spectral gates and regularizers separate persistent dynamics from transient uncertainty during training.

  • Latent trajectory encoding: Source-domain parameters are encoded from R^D into a compact latent space R^m, producing a continuous trajectory across observed timestamps.The encoder maps each θ(t_i) to z(t_i), with m ≪ D.
  • Continuous modal dynamics: Continuous Koopman modes represent latent evolution through eigenvalues whose imaginary parts encode frequency and real parts encode growth or decay.The modal coefficients and mode vectors are combined into unified complex amplitudes.
  • Continuous spectral representation: The model replaces rigid DFT frequency bins with data-driven eigenvalue-indexed continuous modes.This establishes a continuous spectral coordinate system for the observed latent trajectory.
  • Adaptive spectral disentanglement: Learnable soft gates assign high dominance to low-frequency, persistent modes and route high-frequency or rapidly attenuating modes to the transient component.The frequency and decay scores use learnable thresholds and sigmoid gating functions.
  • Joint optimization: Training jointly fits source-domain prediction, latent modal reconstruction, parameter auto-encoding, and spectral regularization objectives.The stability constraint penalizes positive growth in dominant modes, while the spectral constraint suppresses transient modal energy and avoids gate collapse.

E. Future Time Inference

At an arbitrary future timestamp, FreKoo++ analytically extrapolates persistent dominant modes while retaining transient variation from the last observed time. This avoids recursive rollouts and supports constant-time forecasting.

  • Asymmetric extrapolation: Persistent dominant modes are extrapolated directly to the future timestamp, while transient modes are frozen at the last observation.The asymmetric protocol treats volatile local fluctuations as stationary residual context during future prediction.
  • Analytical inference: Closed-form exponential modal evaluation bypasses step-wise rollouts and numerical ODE solvers.The method evaluates the dominant branch at t_s and the transient branch at t_T.
  • Framework advances: FreKoo++ uses continuous Koopman eigenvalues, learnable spectral gates, and analytical modal extrapolation to support irregular sampling and arbitrary-horizon forecasting.These changes distinguish FreKoo++ from FreKoo’s DFT grids, manual cutoffs, and discrete transitions.

VI. THEORETICAL ANALYSIS

The theoretical analysis establishes a finite-mode approximation bound and explains how amplitude, eigenvalue, and truncation errors depend on the prediction horizon. Stability and spectral regularization target the corresponding error sources.

  • Mode approximation: Under finite modal dynamics with a non-growing transient remainder, the learned latent trajectory admits a mode approximation bound.The bound applies to any horizon and decomposes error into amplitude, eigenvalue, and truncation terms.
  • Error decomposition: Amplitude, eigenvalue, and truncation errors are separately identified in the continuous modal approximation analysis.The proof bounds exponential differences through amplitude and eigenvalue discrepancies and controls the remainder using non-growing modes.
  • Stability control: Positive learned growth rates can exponentially amplify eigenvalue errors over the prediction horizon.The stability regularizer encourages non-positive dominant growth so error growth scales at most linearly with the future time.
  • Spectral control: The spectral regularizer suppresses amplitudes of transient-weighted modes and prevents degenerate gate collapse.It acts as an empirical surrogate for controlling truncation error from unmodeled dynamics.

B. Future Time Generalization Bound

The continuous-time generalization analysis bounds future-domain excess risk using modal mismatch, transient holding error, and horizon-dependent spectral growth. Stable dominant modes and frozen transients limit error amplification.

  • Risk bound: Theorem 2 bounds future-domain excess risk under Lipschitz task, backbone, and decoder assumptions and a realizable latent parameter trajectory.The bound follows the inference protocol that extrapolates dominant dynamics and freezes transients.
  • Dominant-branch error: Dominant extrapolation error is governed by anchor mismatch and spectral estimation error, with horizon dependence through exponential growth factors.The stability regularizer drives the positive-growth factor toward zero, suppressing exponential divergence.
  • Regularized error control: When all transient-weighted modes are non-growing, the transient energy factor remains bounded by the mode count.R_stab penalizes growing dominant modes, while R_spec suppresses amplitudes of modes routed to the transient branch.
  • Transient-branch error: Freezing transient modes at t_T makes transient holding error independent of the future horizon.This structurally insulates predictions from horizon-dependent amplification of volatile noise.
  • Stable-spectrum corollary: Under stable true and learned dynamics with exact spectral recovery, future error is bounded by anchor mismatch regardless of the horizon.This is the stated elimination of exponential error compounding.

VII. EXPERIMENTS

FreKoo++ is evaluated across continuous and discrete temporal domain-generalization benchmarks, with comparisons spanning time-agnostic, regularized, adaptation, and temporal-modeling baselines. On CTDG, it consistently outperforms existing methods for irregular timestamps and arbitrary-horizon forecasting.

  • Benchmark settings: FreKoo++ is tested on six irregularly sampled CTDG datasets and seven uniformly sampled DTDG datasets.The CTDG suite includes continuous variants of 2-Moons and Rot-MNIST, Twitter, Yearbook, Cyclone, and House; the DTDG suite includes 2-Moons, Rot-MNIST, ONP, Shuttle, Elec2, HousePrices, and ApplianceEnergy.
  • Benchmark settings: The CTDG comparison includes time-agnostic, domain-generalization, continuous-adaptation, and temporal-modeling baselines.Baselines include Offline, LastDomain, IncFinetune, IRM, V-REx, CIDA, TKNets, DRAIN, DRAIN-∆t, DeepODE, NeuralLio, and Koodos.
  • CTDG results: FreKoo++ consistently outperforms existing baselines across CTDG classification and regression tasks with irregular timestamps and arbitrary-horizon targets.The reported comparison attributes the gains to modeling parameter trajectories in a continuous spectral-dynamical space rather than omitting temporal ordering or enforcing discrete transitions.
  • Metrics: Table I reports classification error rates or Twitter AUC and regression MAE, while Table II reports classification error rates and regression MAE.A dash indicates that a method does not support the corresponding task.
  • CTDG results: Continuous spectral extrapolation analytically extends stable dominant dynamics while insulating predictions from volatile, non-generalizable noise.The formulation uses eigenvalue components whose real and imaginary parts represent temporal persistence and oscillatory frequency.

2) Discrete Temporal Domain Generalization:

Under uniformly sampled DTDG benchmarks, FreKoo++ remains competitive with FreKoo, while ablations show that continuous modal dynamics, adaptive gating, and regularization are important for robust extrapolation. Qualitative analyses visualize spectral separation and stable long-horizon decision-boundary behavior.

  • Discrete temporal domain generalization: FreKoo++ improves upon FreKoo on five of seven DTDG datasets, while FreKoo remains stronger on Elec2 and House-D.The improved datasets are 2-Moons-D, Rot-MNIST-D, ONP, Shuttle, and Appliance.
  • Ablation studies: Fixed-spectrum modes cause classification error to rise from 1.3% to 9.7% on 2-Moons-C, Twitter AUC to fall to 0.64, and Cyclone MAE to worsen to 17.2.The fixed-spectrum variant removes temporal growth and decay by enforcing σk ≡ 0.
  • Ablation studies: Hard gating worsens Cyclone MAE to 18.7, while Frozen gating consistently degrades performance relative to adaptive soft gating.These variants impose rigid or fixed spectral boundaries rather than adapting them to data-specific temporal profiles.
  • Qualitative spectral analysis: Figure 4 separates parameter evolution into smooth dominant dynamics and short-lived residuals, then extrapolates the dominant component while holding the transient component fixed.Learned modes are organized in a frequency–decay plane, where soft gates separate persistent and transient modes without manual frequency thresholds.
  • Ablation studies: Removing transient spectral regularization raises 2-Moons-C error to 6.7%, and disabling stability regularization changes the error from 1.3% to 2.5%.The reported effects support suppressing transient amplitudes and controlling positive growth rates in extrapolation.
  • Qualitative decision-boundary analysis: Across six unseen future domains, FreKoo++ maintains coherent decision boundaries, whereas Koodos reaches 5.3% classification error at step 44.The comparison attributes Koodos’s degradation to unconstrained continuous trajectory modeling that overfits local noise and amplifies integration errors.

A. Proof of Theorem 1

The proof decomposes modal prediction error into amplitude estimation, eigenvalue estimation, and modal truncation components, then extends the analysis to dominant extrapolation and transient holding errors. The resulting bounds track growth through estimated and true eigenvalue real parts over the prediction horizon.

  • Assumptions: The supplementary conventions require omitted transient modes to be non-growing and bound learned transient growth over the historical window by finite Gtrans(tT ).The real-part operator is treated as 1-Lipschitz when transferring complex-valued bounds to the real parameterization.
  • Error decomposition: The mode approximation proof bounds total latent error by summing amplitude estimation, eigenvalue estimation, and modal truncation errors.The triangle inequality gives ∥z(t) − ˆz(t)∥2 ≤ ∥Eamp(t)∥2 + ∥Eest(t)∥2 + ∥Etrunc(t)∥2.
  • Eigenvalue estimation error: Eigenvalue estimation error grows with horizon through a factor proportional to |λk − ˆλk| t e^(ϱk t), where ϱk is the largest nonnegative relevant real part.Because t e^(ϱk t) is nondecreasing for ϱk ≥ 0, the supremum on [0, ts] occurs at ts.
  • Modal truncation error: The truncation component is bounded because omitted modes satisfy Re(λk) ≤ 0 and therefore have non-growing exponential magnitude.For k > K, |e^(λk t)| ≤ 1 for all t ≥ 0.
  • Future-domain risk: Future-domain excess-risk analysis decomposes error into dominant extrapolation and transient holding components under Assumptions 1–2.The proof uses the continuous inference rule ˆz(ts) = ˆzdom(ts) + ˆztrans(tT ).
  • Future-domain risk: Decoder Lipschitz continuity converts latent prediction error into parameter error through ∥ˆθ(ts) − θ⋆(ts)∥2 ≤ Ldec∥ˆz(ts) − z⋆(ts)∥2.The proof also uses Lipschitz continuity of the loss and predictor to relate parameter or latent errors to excess risk.

C. Proof of Proposition 2

The proposition bounds the cumulative temporal energy of a decaying spectral mode. For negative real eigenvalue components, the resulting exponential-energy term is finite and controlled by the mode amplitude and decay rate.

  • Energy bound: For λk = σk + jωk with σk < 0, the proof bounds the mode’s cumulative temporal energy over [0, Tmax].The real-valued trajectory energy is upper-bounded using ∥Re(u)∥2 ≤ ∥u∥2.
  • Energy bound: Because σk < 0, the exponential factor satisfies 0 < e^(2σkTmax) < 1, making 1 − e^(2σkTmax) ≤ 1.This monotonic decay condition yields the finite bound stated in Eq. (22).

A. Datasets

FreKoo++ is evaluated on continuous and discrete temporal domain generalization benchmarks spanning synthetic and real-world classification and regression tasks. Continuous benchmarks use irregular temporal sampling and chronological source-to-future target splits, while discrete benchmarks test backward compatibility under standard sequential protocols.

  • Continuous benchmarks: Six continuous benchmarks use asynchronous arrivals and irregularly sampled temporal gaps, with the earliest 70% of domains for training and latest 30% for future extrapolation.Timestamps are normalized to [0, 1] while preserving relative temporal intervals; the '-C' suffix denotes continuous irregular variants.
  • Continuous benchmarks: The continuous suite covers rotational synthetic classification, influenza-risk prediction from tweets, yearbook gender classification, cyclone wind-intensity regression, and housing-price regression.Rotated 2-Moons-C and Rotated MNIST-C model continuous rotational drift; Twitter, Yearbook, Cyclone, and House-C represent real-world streams with temporal shifts.
  • Continuous benchmarks: House-C uses 40 non-overlapping monthly domains from 2013–2019, training on the first 28 and testing on the remaining 12 under non-uniform sampling intervals.Its regression target is housing price, with drift attributed to evolving local real-estate market conditions.
  • Discrete benchmarks: Seven discrete benchmarks follow chronological source-domain training and reserve the immediately succeeding unseen future domain as the target.The suite includes synthetic rotational drift, article popularity, flight-status classification, seasonal electricity demand, housing prices, and appliance energy consumption.
  • Discrete benchmarks: The discrete suite includes real-world concept drift from popularity patterns, flight-status changes, seasonal demand, economic shifts, and energy-usage changes.These datasets use sequential or chronological domain partitions, with the latest domain held out for testing.
  • Baselines: The experimental setup compares FreKoo++ with twelve representative continuous temporal domain generalization baselines grouped into four technical paradigms.One paradigm contains time-agnostic methods such as Offline, LastDomain, and IncFinetune.

SECTION III IMPLEMENTATION DETAILS

FreKoo++ combines task prediction, latent autoencoding, and continuous Koopman spectral dynamics, trained end-to-end with jointly weighted objectives. Experiments examine representation stability, loss-weight robustness, and the effect of Koopman modal dimension across continuous benchmarks.

  • Model components: The implementation integrates a dataset-specific task predictor, a symmetric four-layer latent autoencoder, and a continuous Koopman spectral-dynamical module.The Koopman module operates in R^m, while task and autoencoder architectures are summarized across datasets.
  • Optimization: End-to-end Adam optimization uses separate learning rates for the task model, autoencoder, and spectral module, with four weights regulating reconstruction, prediction, spectral, and stability objectives.The supplied implementation passages identify α, β, γ, and δ as objective weights, including spectral and stability regularization.
  • Optimization: Dataset-specific training schedules range from 200 to 800 epochs, with learning rates and objective weights adjusted across 2-Moons-C, Rot-MNIST-C, Twitter, Yearbook, Cyclone, and House.Examples include 300 epochs for 2-Moons-C, 400 for Rot-MNIST-C, 200 for Twitter, 800 for Yearbook, and 500 for Cyclone.
  • Representation analysis: Across 15 Rot-MNIST-C and 12 Yearbook future domains, t-SNE features retain tight class-separated clusters during continuous parameter extrapolation.The visualization is intended to inspect geometric evolution of penultimate features across unseen target domains.
  • Representation analysis: Classification errors remain bounded at 1.4%–4.8% on Rot-MNIST-C and 2.6%–6.7% on Yearbook without representation collapse or distant-horizon dispersion.The authors present this as qualitative evidence supporting their stability theorem and corollary.
  • Sensitivity analysis: Performance is generally robust across loss-weight sweeps, although excessively large reconstruction weight increases Rot-MNIST-C error by fitting transient noise.Moderate stability and spectral regularization constrain positive modal growth and filter transient noise without sacrificing model capacity.
  • Sensitivity analysis: Modal dimensions K ≤ 4 underfit, K = 16–32 generally performs best, and K = 32 achieves peak Twitter AUC and lowest House-C and 2-Moons-C MAE.Increasing to K = 64 slightly degrades performance on noisy real-world datasets such as House-C and Twitter; Fig. 9 marks K = 32 as adopted.
Loading 2608.22224v1…