PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Directly Learning Periodic Trajectories Can Fail

Kaixin Zheng, Anita Layton

arXiv:2609.32254v1cs.AIcs.NAmath.CAmath.NA

TL;DR

Periodic-orbit learning must choose a starting phase and time parameterization, so regular orbits can still yield difficult trajectory targets. The paper studies phase alignment and separate waveform-period learning, and reports substantially lower fixed-window OOD errors with period normalization. Its analysis connects these failures to phase accumulation over long timescales.

  • Problem

    Periodic orbits do not specify a unique trajectory target because starting phase and period vary, making the parameter-to-trajectory map depend on representation choices.

    Source: §1 · p. 1

  • Method

    The paper analyzes phase accumulation and learns an aligned, normalized waveform separately from its period, with continuity results under regular attracting-limit-cycle assumptions.

    Source: §1 · p. 2

  • Results

    OOD validation MSE falls from 0.228 to 0.0104 on Goodwin and from 0.200 to 0.0163 on GOLD95 with period normalization.

    Source: §2.1 · p. 3

  • Takeaways & Limitations

    Data-generation choices such as long transient removal and wide observation windows merit scrutiny alongside the model, since reported periodic-benchmark errors may partly reflect them.

    Source: §8 · p. 11

    Decoupling reduces but does not eliminate OOD error, and waveform errors remain larger than their IID counterparts.

    Source: §7 · p. 10

Abstract

from arXiv · show

Operator learning of periodic solutions requires deciding how simulation data should be recorded and represented. A natural choice is to integrate long enough for transients to decay and record a window wide enough to contain at least one full period of all trajectories. We find that these conservative choices can make the resulting trajectories difficult to learn, even when the underlying periodic orbits vary regularly with system parameters. Unaligned trajectories generalize poorly even within the training distribution. Phase alignment substantially improves in-distribution generalization, but models trained on a fixed physical-time window still have large errors on trajectories with periods outside the training range. We explain both failures through a common mechanism: frequency differences accumulate over time, so the target phase varies rapidly with the parameters. Predictors that cannot track this variation incur a population MSE floor in both settings; for fixed window prediction, we also derive a per-sample lower bound. We then study one of the simplest representations that escape these floors: learning an aligned, normalized waveform and its period separately. We establish regularity of the decoupled targets under ODE assumptions and show experimentally that this approach avoids both failures in ODE systems and a PDE case study.

1 Introduction

Learning periodic solutions directly requires choosing a phase and time parameterization, choices that can make trajectory targets difficult to predict despite regular periodic orbits. The paper identifies two resulting failures, explains them through phase accumulation, and studies a decoupled representation that avoids this mechanism.

1 Introduction

  • The paper frames periodic direct learning as the long-time counterpart to learning steady states from system parameters.It focuses on periodic rather than stationary long-time solutions.Source: §1 · p. 1
  • A periodic orbit does not specify a unique trajectory target: its starting phase and parameter-dependent period shape the parameter-to-trajectory map.These recording choices leave the orbit unchanged but can make accurate prediction difficult.Source: §1 · p. 1
  • Prior oscillatory-data work modifies representations or learns reduced phase dynamics, while this paper studies parameterized systems generated by a common equation family.That setting supplies mathematical structure for analyzing how orbit and period vary with parameters.Source: §1 · pp. 1–2
  • Phase misalignment obstructs IID generalization, while fixed physical-time windows can remain accurate IID yet fail on unseen periods.These are two distinct representation-induced failures identified through ablation studies.Source: §1 · p. 2
  • Both failures arise when parameter-dependent frequency differences accumulate over large timescales, producing phase variation that exceeds predictor sensitivity.For fixed-window representations, the paper also gives a per-sample MSE lower bound near the training period range.Source: §1 · p. 2
  • Separately learning an aligned, normalized waveform and its period removes uncontrolled observation phase and yields continuous, neural-network-approximable targets under regular attracting-limit-cycle assumptions.One full-state snapshot can recover physical-time predictions from these targets.Source: §1 · p. 2

2 When Conservative Data Preparation Hurts

The section examines two conservative recording choices: waiting for transients to decay and using a wide physical-time window. Its ablations show that phase alignment addresses IID error, while period normalization is important for generalization to longer, unseen periods.

2 When Conservative Data Preparation Hurts

  • Waiting until a common late time and recording a wide physical-time window are natural ways to capture periodic behavior, but each can independently cause generalization failure.The target is the cycle as it varies with the parameter, rather than the initial approach from x0.Source: §2 · p. 2

2.1 Two choices, two failure modes

  • The experiments use the Goodwin oscillator and GOLD95, comparing aligned and unaligned waveforms and period normalization under IID and period-based OOD splits.Both systems have stable periodic orbits in the selected parameter domains; waveform MSE is computed after transforming and standardizing outputs.Source: §2.1 · p. 3
  • Phase-aligned fixed-window targets remain accurate IID without period normalization, with mean validation MSE of 5.48 × 10−5 on Goodwin and 8.28 × 10−4 on GOLD95.On Goodwin, this error is lower than with period normalization.Source: §2.1 · p. 3
  • Unaligned IID validation MSE reaches 0.997 without period normalization and 0.970 with it, despite training error below 1.1 × 10−3.A common long integration after aligning converged GOLD95 orbits also produces this failure, so transient contamination is not required.Source: §2.1 · p. 3
  • OOD validation MSE falls from 0.228 to 0.0104 on Goodwin and from 0.200 to 0.0163 on GOLD95 with period normalization.The fixed-window errors are more than an order of magnitude higher on both systems.Source: §2.1 · p. 3
  • Across five seeds, FM1 validation MSE is near or above 1 for unaligned targets but below 10−3 when phase-aligned; FM2 gaps exceed 10-fold on both systems.Ranges for the compared settings do not overlap, and their standard deviations are small relative to the gaps.Source: §2.1 · p. 3

3 Error Floors for Parameterized Periodic Targets

The section connects late-time and fixed-window trajectory failures to phase variation amplified by long time scales. It derives a population error floor for slowly varying predictors and a per-sample lower bound for fixed-window OOD prediction near the training boundary.

3 Error Floors for Parameterized Periodic Targets

  • Long time scales amplify parameter-dependent frequency differences into rapid phase variation, providing a common mechanism for both trajectory-learning failures.The section represents both settings in a common form where phase variation grows with the time scale.Source: §3 · pp. 3–4

3.1 Population error

  • Under regularity assumptions, a predictor whose parameter Lipschitz constant grows more slowly than the time scale has a population MSE floor equal to the phase variance.The theorem’s interpretation is conditional: training does not enforce the predictor Lipschitz bound.Source: §3.1 · p. 5

3.2 Per-sample OOD error

  • For fixed-window OOD prediction near a training-frequency boundary, the normalized squared error is bounded below asymptotically by min{z(p)^2, 1}, where z(p) measures accumulated phase drift.The bound applies under the stated regularity and predictor-sensitivity conditions, uniformly over nearby OOD points with bounded positive z.Source: §3.2 · pp. 5–6
  • A nearby OOD trajectory can be predicted poorly in a wide window even when boundary error is small, because frequency mismatch accumulates into phase drift.The proof compares the OOD prediction with its value at a nearby training-boundary point and freezes the waveform at the test parameter.Source: §3.2 · pp. 5–6

4 Numerical validation

The numerical tests examine whether long burn-in shifts and wide recording windows create the predicted learning floors. Across both ODE systems, the results support population floors for large shifts and windows, and a per-sample lower-bound trend for fixed-window OOD prediction.

4.1 FM1: shift sweep

  • At large burn-in shifts, validation MSE reaches or exceeds σ2_phase on both systems, and the best validation result closely matches the floor.The best validation error occurs within four epochs; continued training raises final validation MSE to 1.69–1.82 times the floor.Source: §4.1 · p. 6

4.2 FM2: window sweep

  • For large windows, best and final population validation MSE remain above σ2_phase on both systems, consistent with a floor for limited-sensitivity predictors.The best validation error stays above the floor, suggesting the result is not simply due to insufficient training.Source: §4.2 · p. 7, Figure 2 · p. 7
  • At large tested windows, the per-sample OOD error-bin minima lie above min{z2, 1}, matching the predicted lower-bound trend near the training-frequency boundary.The bin means lie above the curve at every tested window; minima do so starting at W = 302.1 h for Goodwin and W = 195.4 h for GOLD95.Source: §4.2 · p. 7, Figure 3 · p. 8

5 Escaping the Error Floor: A Decoupled Target

The section proposes learning an anchored, normalized waveform and its period separately, then using a snapshot to recover physical-time phase. Under regular attracting-limit-cycle assumptions, the decoupled targets are continuous and network-approximable; experiments report accurate snapshot-based reconstructions on two ODE systems.

5 Escaping the Error Floor: A Decoupled Target

  • The decoupled target is one way to escape the floor by removing or separately representing accumulating phase.The regularity conditions for this result are independent of the population assumptions used to establish the error floors.Source: §5 · p. 8, §5.2 · p. 8

5.1 Decoupled representation

  • The decoupled representation separately predicts the anchored waveform and period, removing the large phase term from the waveform target.After transient removal, phase is anchored at the unique maximum of a scalar observable and one cycle is sampled at fixed phases.Source: §5.1 · p. 8, §5 · p. 8

5.2 ODE regularity and approximability

  • Under the stated regular periodic setting, the period and anchored waveform are continuous and uniformly approximable by feedforward networks with suitable activations.The assumptions include a unique attracting hyperbolic limit cycle for each parameter, with no bifurcation, loss of hyperbolicity, or branch switching.Source: §5.2 · p. 9

5.3 How can we use decoupled targets?

  • With a predicted waveform and period, a single full-state snapshot supplies phase for reconstructing the physical-time trajectory.The phase is estimated by matching the predicted waveform to the observed state, then the waveform is evaluated at the shifted, period-scaled time.Source: §5.3 · p. 9
  • A single full-state snapshot reconstructs trajectories on both ODE systems with mean period error below 0.5% across five training seeds.The reported waveform MSEs are 1.3×10−4 on Goodwin and 7.6×10−4 on GOLD95, measured in state coordinates scaled to unit training-set variance.Source: §5.3 · p. 9, Table 2 · p. 9

6 Beyond ODEs: a FitzHugh–Nagumo PDE Case Study

A FitzHugh–Nagumo reaction–diffusion case study tests periodic traveling-wave fields using phase-aligned one-period records. The qualitative results match the ODE ablations: both failure modes persist, while the decoupled method learns the periodic solution accurately.

6 Beyond ODEs: a FitzHugh–Nagumo PDE Case Study

  • The case study uses a one-dimensional FitzHugh–Nagumo reaction–diffusion system and phase-aligns each one-period field before comparing representations.The ablations change the representation while keeping the underlying periodic solution fixed.Source: §6 · p. 10
  • In the FitzHugh–Nagumo PDE case study, both failure modes persist for spatial fields, while the decoupled method accurately learns the periodic solution.The tested branch is a winding-one traveling wave with periodic boundary conditions, aligned using the phase of the first spatial Fourier coefficient of u.Source: §6 · p. 10
  • Extending the decoupled-representation continuity theory to PDEs requires separate treatment of traveling-wave branches and translation symmetry.The paper leaves that extension for future work.Source: §6 · p. 10

7 Discussion and Limitations

The regularity results are scoped to ODEs and can fail at bifurcations; analogous PDE regularity remains future work. Decoupling reduces, but does not eliminate, OOD error, and its scalar-period assumption does not cover more complex periodic structure.

7 Discussion and Limitations

  • ODE target continuity can fail at bifurcations, and analogous regularity results for PDE systems remain future work.The stated regularity conditions concern the decoupled targets for regular attracting limit cycles.Source: §7 · p. 10
  • Decoupling reduces OOD error in the experiments but does not eliminate it; waveform errors remain larger than their IID counterparts.Parameter extrapolation remains a general difficulty in operator learning.Source: §7 · p. 10
  • The method assumes period depends only on the parameter, so region-varying periods require targets designed for that structure.Atrial fibrillation is given as an example in which period length varies by heart region.Source: §7 · p. 10
  • Operator-learning targets beyond trajectories, including steady states, Green’s functions, and Newton solvers, remain an open design space.The paper identifies target design for a wider range of physical quantities as future work.Source: §7 · p. 10

8 Conclusion

The conclusion argues that learning targets can dominate error when long-time data generation makes phase sensitivity grow faster than a predictor can track. The appendices support this account with phase-averaging arguments, a population error decomposition, and a boundary-based per-sample bound.

8 Conclusion

  • Errors can depend mainly on the learning target: long transient removal or wide observation windows create large timescales that demand comparably growing parameter sensitivity.When predictor sensitivity grows more slowly, an error floor remains, even if the periodic orbit depends regularly on parameters.Source: §8 · p. 11
  • Rather than learn a full trajectory, the paper proposes targets reflecting dynamics: period and waveform, separately, for a periodic orbit; a general target-selection principle remains open.Initial phase is described as a choice of observation start, not a parameter-determined property of the orbit.Source: §8 · p. 11

A.1 Setup and lemmas

  • The proof setup uses local frequency coordinates and phase averaging to show oscillatory terms vanish against sufficiently slow predictors.The argument partitions local frequency intervals and applies periodic-averaging estimates before extending the result over the parameter domain.Source: §A.1 · pp. 14–15

A.2 Proof of Theorem 1

  • Theorem 1’s proof combines convergence to phase variance with asymptotic orthogonality for predictors whose parameter Lipschitz constants grow more slowly than the time scale.The decomposition is applied along any bounded-error sequence to establish the lower bound.Source: §A.2 · p. 19

B.1 Setup and lemmas

  • The fixed-window appendix compares phase separation over a long window with error at the training-period boundary.Its lemmas provide the phase-separation estimate and a local comparison to the training boundary used in the per-sample argument.Source: §B.1 · pp. 19–20

B.2 Derivation of per-sample bound

  • The per-sample argument considers nearby OOD points with bounded window-scaled distance from the training period range and derives an error lower bound.The displayed intermediate bound controls the prediction error by phase separation minus a term that vanishes under the stated growth conditions.Source: §B.2 · p. 21

C Proof of Theorem 3

Under smoothness, hyperbolicity, unique-cycle, and unique-anchor conditions, the proof establishes continuity of the period and anchored waveform. It also describes the ODE datasets’ sampling and acceptance procedure.

C Proof of Theorem 3

  • A transverse section and smooth flow yield local continuation of the cycle and return time; the unique-cycle assumption then makes the period continuous across parameters.The first return is the minimal period after shrinking the neighborhood if needed.Source: C Proof of Theorem 3 · p. 22
  • A unique nondegenerate maximum of the anchor observable continues locally with parameters, yielding a jointly continuous anchored waveform and uniformly approximable period and sampled waveform targets.The approximation conclusion uses compactness of the parameter domain and coordinatewise universal approximation.Source: C Proof of Theorem 3 · p. 22

D.1 Goodwin oscillator

  • The ODE experiments use peak anchoring for the Goodwin X coordinate and the GOLD95 M coordinate.The GOLD95 model has five state variables representing mRNA and cytosolic and nuclear PER forms.Source: §D.1 · p. 22, §D.2 · p. 23

D.2 Goldbeter 1995 circadian model

  • GOLD95 sampling fixes four phosphorylation Michaelis constants equal and sets the Hill coefficient to 4, with 14 free parameters.The parameter vector is listed in the passage.Source: §D.2 · p. 23

D.3 Sampling, integration, and acceptance

  • The datasets integrate candidate ODEs until transients decay, estimate periods from successive peaks, and resample one anchored cycle; accepted samples number 5,000 for Goodwin and 10,000 for GOLD95.Rejection rates were 1.2% for Goodwin and 28.4% for GOLD95.Source: §D.3 · p. 23

E.1 Main decoupled training setup

The main decoupled setup trains separate period and waveform predictors. The appendices specify alternative alignment and window constructions and a shift-sweep evaluation configuration.

E.1 Main decoupled training setup

  • The sampling box uses intervals ±30% of baseline values, except the Goodwin Hill exponent, sampled from 10 to 14.The configuration table is identified as the default for the period and waveform models.Source: Table 5 · p. 23, Table 4 · p. 24
  • The main setup trains a period MLP and waveform FNO separately; across five seeds, period validation RMSE averages 0.0077 h for Goodwin and 0.165 h for GOLD95.The training split contains 4,000/1,000 Goodwin and 8,000/2,000 GOLD95 samples.Source: §E.1 · pp. 23–24

E.2 Ablation implementation

  • The ablations vary phase alignment and period normalization using aligned or unaligned cycles and fixed physical-time windows, including period-based OOD splits.The fixed-window OOD validation trajectories are clipped to the training window, while training trajectories are tiled to fill it.Source: §E.2 · p. 25

F.1 FM1 shift sweep: configuration

  • The FM1 shift sweep changes only each orbit’s target start time across specified hourly shifts, sharing the same train-validation split across shifts and seeds.The FNO uses parameter and normalized-time inputs, with final and best validation MSE tracked over training.Source: §F.1 · pp. 25–26

F.2 FM1 shift sweep: detailed results

The FM1 shift sweep shows early best-validation checkpoints followed by rising validation error despite falling training loss. The appendices also specify FM2 window-sweep, per-sample, theorem-applicability, and snapshot-conditioned evaluation procedures.

F.2 FM1 shift sweep: detailed results

  • For all five seeds marked †, the best validation checkpoint occurs in epochs 0–3; afterward, training loss falls while validation MSE rises.Best val MSE is measured at that checkpoint, whereas Final val MSE is measured at the last epoch.Source: §F.2 · p. 26

F.3 FM2 window sweep: configuration

  • The FM2 window sweep uses a 1D FNO and varies window length across six multiples of each system’s cutoff while holding the time step fixed.The sweep uses five seeds per system; Table 8 lists shared hyperparameters.Source: §F.3 · pp. 26–27, Table 8 · p. 26
  • Validation samples have periods at or above the cutoff and are treated as OOD; training samples have periods below it.Goodwin has 2,020 training and 2,980 validation samples; GOLD95 has 4,822 training and 5,178 validation samples.Source: §F.3 · p. 27

F.4 FM2 window sweep: detailed results

  • The FM2 sweep reports final and best-validation MSE as mean ± one standard deviation across five seeds for Goodwin and GOLD95.The corresponding results are presented in Tables 9 and 10.Source: §F.4 · p. 27, Table 9 · p. 27, Table 10 · p. 27

F.5 Per-sample analysis for FM2

  • Per-sample analysis bins validation examples by z = Wdω(p), filters low-variance waveforms, and reports binwise threshold counts and seed variability.Table 11 summarizes the per-sample results; the variance filter retains parameters with V(p) ≥ 0.1 V(p∂).Source: §F.5 · p. 28, Table 11 · p. 28

F.6 Applicability of Theorem 2

  • The theorem-applicability sweeps vary one parameter while fixing the others at baseline; period decreases with δ1 in Goodwin and increases with vd in GOLD95.Table 12 lists the single-parameter sweeps used to check Assumption 2.Source: §F.6 · p. 28, Table 12 · p. 29

G Snapshot-Conditioned Prediction Details

  • Snapshot-conditioned evaluation predicts a period and waveform from one post-transient full-state snapshot per orbit, then matches the snapshot by periodic interpolation.It uses 100 candidates at each of three search levels; Table 13 reports per-seed validation mean, median, and 95th-percentile results.Source: G Snapshot-Conditioned Prediction… · pp. 28–29, Table 13 · p. 29
  • The reported per-sample waveform error is defined in the training output scaler’s encoded state space and compared against the ODE solution initialized at the snapshot.The returned prediction is evaluated over the requested physical-time horizon.Source: G Snapshot-Conditioned Prediction… · pp. 28–29

H FitzHugh–Nagumo PDE implementation details

The PDE case study constructs phase-aligned, one-period FitzHugh–Nagumo traveling-wave fields and compares target representations using shared periodic records. A separate period predictor has reported validation NMSEs on iid and period-OOD splits.

PDE and data generation

  • The PDE dataset retains 5,000 parameter samples whose post-transient solutions lie on the winding-one traveling-wave branch, generated with a scrambled Sobol sequence.The remaining 2,500 samples are unused; the spatial domain is periodic.Source: H FitzHugh–Nagumo PDE implementation… · p. 30

Numerical simulation and initial condition

  • Simulations use a centered second-order finite-difference Laplacian and explicit fourth-order Runge–Kutta integration after initializing a traveling wave from phase-shifted homogeneous limit cycles.The PDE is integrated for Tburn,PDE = 600 before measuring the post-transient wave.Source: H FitzHugh–Nagumo PDE implementation… · p. 30

Period estimation and phase alignment

  • For winding-one rigid traveling waves, period is estimated from the first spatial Fourier coefficient’s unwrapped phase velocity, and a canonical phase fixes the alignment.The recorded field is advanced to the next occurrence of the canonical phase, then one period is resampled on a 128 × 128 time-space grid.Source: H FitzHugh–Nagumo PDE implementation… · pp. 30–31

Construction of learning targets

  • All target representations use the same phase-aligned one-period field, so the ablations vary representation while holding the periodic solution fixed.The canonical field has shape 128 × 128 × 2, with channels for u and v.Source: H FitzHugh–Nagumo PDE implementation… · p. 31

PDE experiments and data splits

  • The study compares normalized-field prediction with fixed physical-time-window prediction, using the same phase for both target constructions and the same samples for Experiments 4 and 5.The iid experiments use 4,000 training and 1,000 validation samples; the fixed window spans approximately 1.79–2.70 periods.Source: H FitzHugh–Nagumo PDE implementation… · p. 31

Model and training configuration

  • The normalized-field model uses a 2D FNO and a separate MLP predicts period; the period predictor achieves validation NMSE 2.72 × 10^-9 iid and 2.16 × 10^-3 OOD.The waveform FNO takes normalized time, space, and parameters for normalized targets, or physical time for fixed-window targets; Table 16 summarizes configurations.Source: H FitzHugh–Nagumo PDE implementation… · p. 32, Table 16 · p. 32
Loading 2609.32254v1…