Source-linked AI summary
Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model
Julien Brajard, Alberto Carassi, Marc Bocquet, Laurent Bertino
TL;DR
The paper tackles emulation and prediction of hidden dynamics from sparse, noisy observations. It alternates data assimilation with neural-network training to build a data-driven surrogate model. In the 40-variable Lorenz 96 case, the surrogate reproduces forecast and long-term dynamical properties, while performance depends on observation noise, coverage, and tuning.
Problem
Existing machine-learning dynamics reconstruction commonly assumes noise-free, complete observations, unlike realistic sparse and noisy settings.
Method
The method iteratively uses data assimilation to estimate complete states and machine learning to learn a surrogate model from those analyses.
Results
The Lorenz 96 surrogate reproduces forecast skill and long-term properties, including energetic frequencies and positive Lyapunov exponents.
Takeaways & Limitations
The hybrid approach supports dynamics emulation and prediction from partial, noisy observations within the tested Lorenz 96 setting.
Takeaways & Limitations
The method has substantial computational cost and its convergence and skill are sensitive to training-set length, observation noise, coverage, and hyperparameters.
Abstract
from arXiv · showhide
A novel method, based on the combination of data assimilation and machine learning is introduced. The new hybrid approach is designed for a two-fold scope: (i) emulating hidden, possibly chaotic, dynamics and (ii) predicting their future states. The method consists in applying iteratively a data assimilation step, here an ensemble Kalman filter, and a neural network. Data assimilation is used to optimally combine a surrogate model with sparse noisy data. The output analysis is spatially complete and is used as a training set by the neural network to update the surrogate model. The two steps are then repeated iteratively. Numerical experiments have been carried out using the chaotic 40-variables Lorenz 96 model, proving both convergence and statistical skill of the proposed hybrid approach. The surrogate model shows short-term forecast skill up to two Lyapunov times, the retrieval of positive Lyapunov exponents as well as the more energetic frequencies of the power density spectrum. The sensitivity of the method to critical setup parameters is also presented: the forecast skill decreases smoothly with increased observational noise but drops abruptly if less than half of the model domain is observed. The successful synergy between data assimilation and machine learning, proven here with a low-dimensional system, encourages further investigation of such hybrids with more sophisticated dynamics.
1. Introduction
The paper addresses dynamics emulation from sparse, noisy observations, where existing machine-learning approaches often assume complete, noise-free system states. It proposes combining data assimilation, which estimates states, with machine learning, which emulates a surrogate model.
- Motivation: Machine learning has reconstructed low-order chaotic dynamics and produced forecasts, but commonly relies on noise-free, complete observations.Partial and noisy observations are closer to practical geoscience settings.
- Research gap: Most existing machine-learning algorithms are not suited to observations that are sparse, noisy, and variable in location and number.Here, sparse means the system is not densely observed, with observation locations and counts varying in space and time.
- Motivation: Data assimilation is designed to estimate system states from noisy, unevenly distributed observations together with a dynamical model.Its optimization can be related to machine learning when model error is present.
- Contribution: The proposed hybrid algorithm uses data assimilation to estimate the state and machine learning to emulate the surrogate model.The approach targets cases where both the system state and its dynamical model must be inferred.
- Contribution: The paper evaluates the resulting data-driven surrogate model using forecast skill and long-term properties.The paper also discusses sensitivity to observation number, noise statistics, and other control parameters.
2. Methodology
The method alternates data assimilation and machine learning: assimilation estimates complete states from observations, while a convolutional neural network learns the surrogate resolvent and updates its parameters. The cycle uses assimilation uncertainty in training and requires careful initialization and hyperparameter choices.
- Problem formulation: The objective is to predict x_k+1 from x_k by learning a surrogate model G of the dynamical resolvent.The resolvent maps the state between consecutive observation times.
- Neural-network surrogate: The surrogate model G_W uses a residual neural-network form, with convolutional layers and weights W.Convolution applies locally around each state variable, reflecting a locality hypothesis.
- Neural-network surrogate: Training minimizes multi-step simulation error over a forecast lead N_f using a state-weighting matrix P_k.P_k acts as a surrogate-model error covariance, assigning lower weight to more uncertain states.
- Iterative hybrid algorithm: Data assimilation provides complete state analyses for training, while the neural network provides the surrogate forecast model used by assimilation.The implementation uses the finite-size ensemble Kalman filter EnKF-N, although other adequate assimilation methods can be substituted.
- Iterative hybrid algorithm: Each cycle applies data assimilation to the observation time series and then trains the neural network, passing P_k from assimilation to machine learning.The loss uses the diagonal of the assimilation covariance matrix for computational efficiency and reduced numerical bias.
- Algorithm properties: The data-assimilation and machine-learning steps are separate, so their algorithms can be chosen independently or combined with an external system.The approach can leverage machine-learning libraries for parallel optimization of high-dimensional parameters.
- Initialization and sensitivity: Convergence is not guaranteed for arbitrary initial weights, so the neural network is initialized by training on an interpolated observation field.Convergence is also sensitive to model-noise level and observational-period length.
3. Numerical experiment setup
The experiment tests the hybrid surrogate-model approach on chaotic 40-variable Lorenz 96 data with randomly varying, noisy partial observations. Evaluation covers forecast accuracy, data-assimilation accuracy, long-term dynamics, climate statistics, and training/setup sensitivity.
- Model setup: The surrogate model is evaluated on the 40-variable chaotic Lorenz 96 system, integrated with fourth-order Runge–Kutta using time step h = 0.05.The model uses periodic boundary conditions, forcing F = 8, and has largest Lyapunov exponent Λ1 ≈ 1.67.
- Observations and data: The reference dataset contains K = 40,000 truth-model time steps, with p = 20 randomly sampled state values observed at each time step.The observations correspond to 50% of the 40-variable field, and their locations vary over space and time.
- Observations and data: The observational-error standard deviation is σobs = 1, approximately 5% of the total state range, and observations share the model integration interval.The noise level was selected so that reducing observational noise has a measurable effect on results.
- Evaluation metrics: Performance scores assess forecast skill, analysis accuracy, long-term dynamical properties, and climate frequency content.The evaluation uses RMSE-f for surrogate forecasts, RMSE-a for analyses, the Lyapunov spectrum, and power spectral density; average values are also computed.
- Surrogate-model training: The ensemble Kalman filter uses an ensemble of size N = 30, while initial weights are trained from cubic interpolation of observations rather than random initialization.The initialization uses interpolated observations as inputs but not as targets where data are missing; the initial training uses Nf = 4 and 40 epochs.
- Surrogate-model training: The neural network uses convolutional layers with a bilinear layer, and its architecture and training settings were selected by cross-validation on complete and interpolated datasets.The input layer uses batch normalization; the bilinear layer multiplies two convolutional outputs before concatenation with a third output.
4. Results and discussion
The hybrid method converges toward both accurate forecasts and consistent long-term dynamics, while reproducing key statistical properties of the Lorenz 96 system. Its performance depends on observational coverage, noise, training-set size, and model-noise tuning.
- Convergence of the algorithm: RMSE-f decreases to 0.21 while the first Lyapunov exponent approaches the true-model value of 1.67.Neither quantity is explicitly optimized within a training cycle.
- Interpolation: A factor-of-3 error reduction over cubic interpolation is achieved when the surrogate model is used in data assimilation.The analysis RMSE is also below the observation standard deviation of 0.8 for σobs = 1.
- Emulating the underlying dynamics: After convergence, the surrogate reproduces the power spectrum up to 5 Hz but adds high-frequency noise beyond that range.Low frequencies are better observed and reproduced, whereas high frequencies are not recovered from observations; long simulations remain stable over 1336 Lyapunov time units.
- Emulating the underlying dynamics: The surrogate’s first twelve Lyapunov exponents are very close to the true model’s, while discrepancies remain in the null and negative parts of the spectrum.The shared positive spectrum indicates similar unstable-subspace properties and average perturbation growth.
- Forecast skill: After 2 Lyapunov time units, the surrogate trajectory diverges significantly from the true trajectory, with error saturation after 4–5 Lyapunov time units.Both simulations use the same initial condition.
- Forecast skill: Forecast skill deteriorates gradually as observational noise increases, but is significantly degraded below 50% observation coverage.More than 50% coverage shows no obvious improvement, and the 50% threshold may be specific to the Lorenz 96 model.
- Sensitivity to hyperparameters: The optimal model-noise level is approximately 1.0 for forecast skill and approximately 0.01 for long-term dynamical reconstruction.Larger model noise improves data assimilation, whereas smaller noise limits interference with Lyapunov-exponent calculations.
5. Conclusions
The proposed iterative hybrid method builds a data-driven surrogate from partial, noisy observations by alternating data assimilation and machine learning. On the 40-variable Lorenz model, it reconstructs key dynamics and forecasts effectively, while performance depends on observation conditions, tuning, and computational cost.
- Method: Each cycle alternates data assimilation, which interpolates and de-noises observations, with machine learning that learns the analyzed dynamics.The data assimilation analysis supplies complete states for neural-network training, while the neural network supplies the surrogate forecast model.
- Results: Accurate forecasts extend to 2 Lyapunov time units, while positive Lyapunov exponents and frequencies below 5 Hz are well represented.The reference experiment observed 50% of the field with noise standard deviation 1; reconstruction error was smaller than observational error.
- Sensitivity: Forecast skill worsens with increasing observation noise, and results deteriorate when less than 50% of the field is observed.When the observed fraction exceeds 50%, there is no significant difference in surrogate-model skill.
- Sensitivity: Tuning the neural-network forecast steps and data-assimilation model-noise standard deviation can favor forecast skill or long-term dynamics reconstruction.The method therefore exposes a trade-off between these two evaluation targets through its control parameters.
- Limitations: The method is computationally costly because each cycle requires complete data assimilation and neural-network training until convergence.These operations are equivalent to 30 and 20 forward model runs, respectively; the authors suggest stopping criteria and parallel or GPU computing as possible improvements.
- Limitations: The approach may rely partly on the autonomous character of the Lorenz 96 model, although extensions to non-autonomous systems are proposed.Suggested extensions include treating system variations as slow or supplying external forcing as a neural-network input.
CRediT authorship contribution statement
The authors contributed across conceptualization, methodology, software, validation, writing, project administration, and funding acquisition.
- Julien Brajard contributed conceptualization, methodology, software, validation, and original-draft writing.
- Alberto Carrassi contributed methodology, validation, review and editing, project administration, and funding acquisition.
- Marc Bocquet contributed methodology, validation, and review and editing.
- Laurent Bertino contributed methodology, validation, review and editing, project administration, and funding acquisition.