DAW: Dynamics-Aware Weighting for Deep Learning Forecasts of Chaotic Systems

Zhou Fang; Gianmarco Mengaldo

Source-linked AI summary

DAW: Dynamics-Aware Weighting for Deep Learning Forecasts of Chaotic Systems

Zhou Fang, Gianmarco Mengaldo

arXiv:2608.22277v1Aug 23, 2026cs.LGphysics.comp-ph

TL;DR

Long-term autoregressive forecasts of chaotic systems accumulate errors because uniform training underrepresents rare, dynamically complex states. DAW weights training by local dynamical dimension, outperforming uniform, statistical-density, and shuffled-weight baselines on chaotic KS forecasts.

Problem
Deep-learning surrogates for chaotic systems remain prone to long-term autoregressive error accumulation, while uniform objectives emphasize frequently visited states over rare regimes.
Method
Dynamics-Aware Weighting reshapes the loss using each sample’s local dimension to emphasize rare, dynamically complex states with larger forecast errors.
Results
DAW attains the lowest long-term autoregressive error and slowest spatial-correlation loss among uniform, statistical-density, and randomly shuffled weighting baselines.
Takeaways & Limitations
DAW’s advantage is associated with aligning weights to local dynamical dimension, suppressing error during high-dimension events such as wave mergers.
Takeaways & Limitations
Experiments cover only the KS and Lorenz-63 systems, and robustly estimating local dimension becomes more costly as attractor effective dimension increases.

Abstract

from arXiv · show

Deep learning surrogates for forecasting chaotic dynamical systems suffer from catastrophic error accumulation over long-term autoregressive rollouts. This behavior is partly tied to the underlying systems: chaotic spatiotemporal systems, such as the Kuramoto-Sivashinsky (KS) equation, visit phase space unevenly - dominated by recurrent, low-dimensional quiescent states (e.g., near-laminar flows) and punctuated by rare, dynamically complex topological transitions (e.g., wave-merging events). Under a sample-wise uniform objective, standard neural surrogates allocate their finite capacity to the statistically numerous quiescent states, under-representing the transient regimes that trigger disproportionate, localized errors. Existing imbalanced-regression methods reweight samples by target-space density. However, statistical target-space rarity need not coincide with the intrinsic dynamical rarity - the recurrence geometry of the attractor that is the source of the imbalance. To address this, we introduce Dynamics-Aware Weighting (DAW), a data-centric objective reweighting framework. Using the local dimension $d$ from dynamical systems theory as an a priori measure of a state's active degrees of freedom, DAW reshapes the loss landscape to allocate representational capacity toward the sparse, high-$d$ regimes where forecast errors are systematically large. On the chaotic KS equation, DAW consistently outperforms uniform training, purely statistical density weighting, and its randomly permuted ablation, reducing long-term autoregressive error relative to all baselines. Event-level analysis shows that DAW achieves this by suppressing the localized error amplifications incurred during sharp jumps in $d$, which accompany complex physical processes such as wave-merging in the KS system.

1 Introduction

Deep-learning surrogates can achieve strong accuracy at low computational cost but remain vulnerable to long-term autoregressive error accumulation. DAW addresses the resulting dynamical imbalance by weighting rare, high-dimensional regimes using the local dimension d, where forecast errors are disproportionately concentrated.

Motivation: DL surrogates model complex dynamical systems effectively and can match or exceed traditional approaches at lower computational cost, yet accumulate errors during long-term autoregressive rollouts.The paper frames long-horizon error accumulation as a central limitation despite strong short-term modeling performance.
Motivation: Chaotic attractors are visited unevenly, so uniform objectives prioritize frequently recurring states and under-represent rare states.This sampling imbalance causes approximation capacity to follow recurrence frequency rather than the informativeness of dynamically rare regimes.
Motivation: This imbalance is an instance of imbalanced regression, where minimizing average error favors densely sampled majority regions and performs poorly on sparse target-space regions.The mismatch has been observed in simulated and real-world dynamical systems but has received limited attention.
Research gap: The paper identifies a gap in dynamically meaningful indicators and principled data-centric schemes for exploiting the intrinsic imbalance of chaotic systems.Existing reweighting strategies are introduced as responses to imbalance, but the stated gap concerns indicators tied to the underlying dynamics.
Contribution: DAW uses the local dimension d to assign higher weights to rare, dynamically complex regimes where forecast errors tend to be larger.Dynamical-systems indicators characterize regime imbalance, and prior work reports disproportionately concentrated errors in higher-d and higher-θ states, with the association stronger for d.

2 Methodology

The methodology uses local dimension d to quantify dynamical complexity and attractor sparsity, then constructs DAW by reshaping density-based weights toward rare high-d states. The resulting normalized weighted objective reallocates training emphasis while preserving global gradient scale.

2.1 Local dimension: Local dimension d characterizes how invariant-measure probability mass scales near a phase-space state, providing a measure of local dynamical geometry.Smaller d indicates geometrically denser regions, while higher d identifies sparser attractor regions at the scales resolved by the data.
2.1 Local dimension: Finite-data estimates of d combine Poincaré recurrence with Extreme Value Theory, using a high threshold quantile q = 0.99 and a Generalized Pareto Distribution.The estimate is state dependent at finite resolution because it reflects local attractor geometry at the scales probed by the data.
2.2 DAW construction: For each training pair (x_i, y_i), DAW computes a localized indicator d_i in the concatenated input-output phase space and estimates its empirical density P(d) with KDE.The density is min-max normalized to P′(d) ∈ [0, 1], exposing imbalance in local-dimension values across the training set.
2.3 Theoretical motivation: The input-output pairing preserves pointwise local dimension asymptotically under the stated Lipschitz conditions, although paired and state-only indicators may differ at finite estimation scales.This establishes the theoretical basis for computing d on concatenated training pairs while acknowledging finite-resolution effects.
2.2 DAW construction: DAW first applies a DenseWeight-style base weight to under-represented d regions, then multiplies it by normalized local dimension to suppress quiescent low-d states and emphasize the high-d tail.This reshaping targets rare critical transitions such as wave-merging events while avoiding indiscriminate upweighting of infrequent low-d states.
2.2 DAW construction: The reshaped weights are globally normalized to expectation 1.0, and the neural surrogate is trained with the resulting sample-wise weighted MSE.Global normalization preserves the overall gradient scale and optimizer stability.

3 Experimental Setup

Experiments use the chaotic Kuramoto–Sivashinsky equation at domain size L = 22 and compare weighting methods under a shared MLP training setup. Models are evaluated through closed-loop autoregressive rollouts using accuracy, spatial-consistency, and high-dimensional-event diagnostics.

Benchmark system: The benchmark is the one-dimensional Kuramoto–Sivashinsky equation, which exhibits quiescent regimes, traveling waves, and transient wave-merging events.The domain size is fixed at L = 22, the smallest domain sustaining spatiotemporal chaos in this configuration.
Model and training: All methods use an identical six-hidden-layer MLP with 128 neurons per layer, Adam optimization, batch size 128, learning rate 5 × 10−4, weight decay 1 × 10−9, and 1000 epochs.The shared backbone, optimizer, and training budget isolate differences caused by sample weighting.
Objective: Experiments minimize a weighted mean-squared error, with Standard recovered by wi = 1 ∀i and every weighting scheme normalized to Ei[wi] = 1.0.This normalization preserves the global gradient scale across weighting schemes.
Evaluation: Evaluation reports Mean Absolute Error, spatial Pearson Correlation, Cumulative Error, and Overall Win Rate during high-d event windows defined by the 75th percentile of local dimension.Closed-loop testing feeds each prediction back as the next input; event windows use contiguous intervals above the threshold and compare cumulative error across methods.
Baselines: DAW is compared with Standard, DenseWeight, and RandomWeight, which respectively use equal weighting, statistical target rarity, and a random permutation of DAW weights.RandomWeight preserves DAW’s marginal weight distribution while destroying its correspondence with high-d regimes.

4 Results

Results show that one-step errors concentrate in rare, high-local-dimension states, while DAW’s advantage emerges during long autoregressive rollouts. Event-level analysis attributes this improvement to reduced error amplification around high-d transitions, with ablations supporting robustness across factor-space choices and chaotic-system geometries.

One-step error by local dimension: Across five equal-mass local-dimension bins, mean one-step MAE increases monotonically for all objectives, concentrating the largest errors in rare, geometrically complex states.Each bin contains 20% of test states.
Aggregate rollout performance: At Step 1, DAW and DenseWeight perform comparably and both improve on Standard and RandomWeight, while all methods retain near-unity spatial correlation.One-step accuracy is governed primarily by overall learned-map quality rather than differential rare-state coverage.
Aggregate rollout performance: By 0.25 LT, DAW leads on both metrics; by 0.5 LT, it preserves high spatial correlation while Standard reaches the 0.5 predictability threshold.The separation between methods grows as closed-loop rollout length increases.
Aggregate rollout performance: Standard accumulates error fastest and crosses the spatial-correlation threshold near 0.5 LT, whereas DAW maintains the lowest MAE throughout the rollout.RandomWeight provides only a marginal delay to the loss of spatial coherence.
Event-level error analysis: Across 21,146 high-d event windows from 14,778 initial conditions, most Standard-versus-DAW events lie above the equality diagonal, indicating lower DAW event-level cumulative error.The comparison uses per-event Cumulative Error over windows centered on high-d events.
Ablations and generalization: DAW’s factor-space ablation finds input-only d(x) and paired d(x, y) perform similarly and slightly outperform target-only d(y), consistent with a finite-scale disadvantage.The three indicators estimate the same local-dimension field asymptotically.

5 Discussion · Appendix A Convergence and Consistency of Local Dimension d

The discussion attributes DAW’s long-horizon gains primarily to aligning non-uniform weights with local dynamical dimension, while emphasizing its lightweight implementation and scaling limitations. Appendix A evaluates how trajectory length affects the reliability and consistency of local-dimension estimates.

5 Discussion: DAW’s improvement stems primarily from placing weight according to local dimension rather than from non-uniform weighting alone.RandomWeight improves over uniform training but remains behind DAW at every autoregressive horizon and event-level evaluation.
5 Discussion: RandomWeight improves uniform training over autoregressive rollouts and at the event level, but remains well behind DAW at every horizon.The control experiments show that weighting itself contributes, while dynamical alignment supplies the larger share of the gain.
5 Discussion: DAW is plug-and-play because local dimension d is computed offline, requiring no architectural change or inference overhead.The trained surrogate remains as lightweight as its unweighted counterpart while curbing rollout error growth and preserving spatial-pattern consistency.
5 Discussion: The study restricts experiments to the KS and Lorenz-63 systems, although DAW is agnostic to state dimension and network architecture.This point concerns the reported experimental scope and stated framework generality.
5 Discussion: Robustly estimating the a priori signal d becomes more costly as the attractor’s effective dimension increases.A reliable GPD fit requires trajectory length to grow with effective dimension, so scaling to higher-dimensional systems requires care.
Appendix A Convergence and Consistency of Local Dimension d: Appendix A validates the trajectory length required for reliable local-dimension estimation by examining convergence and consistency across varying observation lengths.Figure 5 presents this convergence and consistency analysis.
Appendix A Convergence and Consistency of Local Dimension d: Short sequences of 50 to 200 LT produce significant fluctuations in the normalized local-dimension distributions.Panel (a) shows probability density functions of dL under Z-score normalization, (dL −µdL)/σdL.

Appendix B Sensitivity of α

DAW remains robust across a broad range of weighting intensities, with its advantage emerging at long forecast horizons and optimal performance around α = 1.0. Increasing α forms a stable performance basin, while α →0 reduces DAW to a less effective pure ˜d-tilt.

Parameter range: The sensitivity analysis evaluates α from approximately 0.0 to 2.0, spanning pure linear ˜d-tilt to aggressive penalization of high-d regimes.α controls the magnitude of the loss penalty applied to dynamically complex regimes.
Forecast-horizon sensitivity: At Step 1, predictive errors are universally low and vary negligibly with α, whereas differences become important at 0.5 LT and 1.0 LT.MAE is evaluated at Step 1, Step 20 (0.25 LT), Step 40 (0.5 LT), and Step 80 (1.0 LT).
Performance basin: As α increases, MAE drops significantly and forms a robust performance basin, while α →0 yields pure ˜d-tilt that accumulates error faster than full weighting.Pure ˜d-tilt improves on uniform training but is less stable over long rollouts.
Robustness: From α = 0.5 to α = 2.0, DAW consistently outperforms the Standard and DenseWeight baselines at long forecast horizons, with optimal performance around α = 1.0.The Standard baseline is unweighted, while DenseWeight uses α = 1.0.
Robustness: The broad stable basin indicates that DAW’s superior long-term stability is rooted in phase-space topological alignment rather than brittle hyperparameter fine-tuning.This interpretation is stated as the basis of DAW’s robustness across the tested α range.

Appendix C Comparison of d quantile

DAW remains robust across the tested local-dimension quantiles q = 0.98, 0.99, and 0.995, without requiring a finely tuned threshold. At q = 0.98 and q = 0.995, it achieves the best autoregressive performance among the compared methods.

Quantile robustness: DAW is evaluated at q = 0.98, 0.99, and 0.995 to test whether its performance depends on a finely tuned local-dimension threshold.The main-text setting is q = 0.99.
Quantile robustness: Standard and DenseWeight are independent of q, so only DAW and RandomWeight are retrained at q = 0.98 and q = 0.995 with other settings fixed.The local dimension d is recomputed for each alternate quantile.
Autoregressive performance: Across q = 0.98 and q = 0.995, DAW attains the lowest MAE and the slowest spatial-correlation decay.The comparisons include Standard, DenseWeight, and RandomWeight.
Autoregressive performance: Across both tested quantiles, DAW delays crossing the 0.5 predictability threshold the longest.This supports robustness of the d estimate over the tested q range and preserves the relative ranking of dynamically complex states.

Appendix D Comparison of using d(x, y), d(x) and d(y) · Appendix E Statistical significance of the win rate with bootstrapping

Appendix D finds that d(x) and d(x, y) provide statistically indistinguishable forecast-diagnostic performance across the rollout, while d(y) is not substantively reported in the supplied passages. The results support using the input-only d(x) as an equally valid a priori DAW indicator.

Appendix D Comparison of using d(x, y), d(x) and d(y): The paired indicator d(x, y) concatenates each input–output pair into a higher-dimensional phase space.This is the one-step-map construction used in the main text.
Appendix D Comparison of using d(x, y), d(x) and d(y): Proposition 1 states that d(x, y) and d(x) coincide asymptotically in the vanishing-radius limit.At the finite quantile used in practice, the paired and state-only estimates need not agree.
Appendix D Comparison of using d(x, y), d(x) and d(y): d(x) and d(x, y) are statistically indistinguishable in MAE and spatial correlation from Step 1 through 1.0 LT.Their agreement remains within run-to-run variability at every reported horizon.
Appendix D Comparison of using d(x, y), d(x) and d(y): The target component adds no separable information at the resolved scales once the input state's recurrence geometry is retained.This interpretation is given as consistent with Proposition 1 and the observed agreement between d(x) and d(x, y).
Appendix D Comparison of using d(x, y), d(x) and d(y): d(x) recovers d(x, y)'s performance while requiring only the input state.It can be obtained from the historical trajectory without reference to the one-step target.
Appendix E Statistical significance of the win rate with bootstrapping: The supplied passages for Appendix E contain no substantive description of bootstrap-based statistical-significance results.

E.1 Method

The study estimates win-rate uncertainty with a trajectory-level cluster bootstrap over high-d event windows, preserving within-rollout dependence. It additionally tests DAW against each baseline using pairwise bootstrap contrasts rather than relying on separate per-method intervals.

Cluster bootstrap: Win-rate uncertainty is estimated with a cluster bootstrap because windows from the same rollout share initial conditions and temporal proximity.The analysis covers 21,146 high-d event windows, which are not mutually independent.
Cluster bootstrap: The bootstrap samples the T = 13,111 test trajectories that contribute at least one high-d window, excluding trajectories without events.Per trajectory and method, the procedure stores won-window counts and total event-window counts.
Cluster bootstrap: The procedure uses B = 10,000 bootstrap replicates, resampling complete trajectories so within-trajectory correlation remains intact.Each replicate aggregates wins and events after trajectory-level resampling, then derives percentile confidence intervals.
Pairwise comparisons: Because the four win rates sum to one and are negatively correlated, the analysis directly tests DAW–baseline pairwise contrasts.A contrast interval entirely above zero indicates a higher DAW win rate across essentially every resample.

E.2 Results · Appendix F Additional results on Lorenz

DAW achieves a stable, statistically tested advantage in event-level forecasting performance, outperforming RandomWeight, DenseWeight, and Standard. The study also extends evaluation to Lorenz-63 to test whether DAW’s advantage generalizes beyond spatiotemporal chaos.

E.2 Results: Each per-method confidence interval spans less than 1.4 percentage points, indicating stable point estimates rather than test-set artifacts.The intervals do not overlap across methods.
E.2 Results: The 95% ordering by event-level win rate is DAW > RandomWeight > DenseWeight > Standard.Non-overlapping intervals make the ordering unambiguous at the 95% level.
E.2 Results: 42.4% is DAW’s event-level win rate, with a 95% CI of [41.7, 43.1].This converts the reported point estimate into a tested result rather than an isolated measurement.
E.2 Results: 18.0 points is DAW’s contrast over DenseWeight, compared with 14.3 points over RandomWeight and 37.3 points over Standard.The corresponding 95% confidence intervals are [16.8, 19.2], [13.0, 15.5], and [36.5, 38.2], respectively; all exclude zero.
E.2 Results: Table 3 evaluates Overall Win Rate over 21,146 high-d event windows using cluster bootstrap resampling of 13,111 event-bearing test trajectories.The analysis uses B = 10,000 bootstrap resamples and reports 95% percentile confidence intervals.
E.2 Results: DAW more reliably suppresses per-event error accumulation in demanding high-d regimes than uniform training, statistical density weighting, or shuffled-weight ablation.The result is presented as corroborating the central finding across the tested baselines.
Appendix F Additional results on Lorenz: The study repeats the evaluation on Lorenz-63 to test whether DAW’s advantage is specific to spatiotemporal chaos.Lorenz-63 is described as a low-dimensional chaotic attractor with geometry largely distinct from the KS field.

F.1 Experimental setup

The Lorenz-63 experiment uses canonical chaotic parameters, long Runge-Kutta-integrated trajectories, and standardized train/validation/test splits. It estimates local dimension to identify rare lobe-switching transitions and evaluates four weighting schemes with a smaller MLP.

Experimental setup: The Lorenz-63 system uses canonical parameters (σ, ρ, β) = (10, 28, 8/3), with leading Lyapunov exponent λmax ≈0.906 and Lyapunov time LT = 1/λmax ≈1.10.
Experimental setup: Trajectories are integrated using fourth-order Runge-Kutta at dt = 0.01 over approximately 5000 LT, with a 70%/15%/15% training/validating/testing split and Z-score standardization.
Experimental setup: Local dimension d is estimated on the three-dimensional Lorenz state, where rare high-d excursions correspond to lobe-switching transitions between the attractor’s two wings.These transitions are described as the low-dimensional analogue of KS wave-merging events.
Experimental setup: The Lorenz experiment uses an MLP with 4 hidden layers and 64 neurons per layer, comparing Standard, DAW, DenseWeight, and RandomWeight weighting schemes.

F.2 Results

On Lorenz closed-loop rollouts, DAW preserves the method ordering established on KS, achieving the lowest MAE and slowest correlation decay. DAW also extends the 0.5 predictability horizon to approximately 4.1 LT, roughly 80% beyond the uniform objective.

Figure 9 evaluates mean absolute error and Pearson correlation across the three Lorenz state components at each autoregressive step.Both metrics are averaged over the full test set.
DAW achieves the lowest MAE and slowest Pearson-correlation decay throughout Lorenz autoregressive rollouts.The ordering is DAW, DenseWeight, RandomWeight, then Standard across both metrics.
≈4.1 LT is DAW’s predictability horizon at the 0.5 correlation threshold, exceeding ≈3.7 LT for DenseWeight, ≈3.4 LT for RandomWeight, and ≈2.3 LT for Standard.
Roughly 80% is DAW’s increase in predictability horizon over the uniform objective.The uniform objective corresponds to Standard in the reported comparison.