Source-linked AI summary
Gaussian process learning with flow map refinement for parameter estimation in dynamical systems
Yue Hao, Dongwei Ye
TL;DR
Derivative matching can be locally efficient but may fail to enforce global dynamical consistency, especially with scarce or noisy data. GPL-FMR combines Gaussian-process derivative matching with Bayesian flow-map refinement and generally improves parameter estimates across tested systems, including with 20 observations and 30% noise.
Problem
Derivative matching is computationally efficient but local in time, so it does not directly impose the dynamical system’s global evolution under scarce or noisy observations.
Method
GPL-FMR first obtains probabilistic parameter estimates by Gaussian-process derivative matching, then refines physical parameters and latent shooting states through Bayesian flow-map optimization.
Results
GPL-FMR consistently yields more accurate parameter estimates across tested cases, including settings with only 20 observations and 30% noise.
Takeaways & Limitations
The framework improves parameter estimation accuracy for the tested dynamical systems, particularly when observations are scarce and noisy.
Takeaways & Limitations
Full Bayesian posterior estimation can be computationally expensive as observation counts and state dimensions grow, motivating MAP estimation with a Laplace approximation.
Abstract
from arXiv · showhide
Parameter estimation is a central task in data-driven learning of dynamical systems. It aims to recover the underlying physical parameters from observed time-series data, thereby providing interpretable insights into the physical mechanisms governing the system. Gradient/derivative matching methods based on Gaussian process provide an efficient way to perform parameter estimation. Those methods avoid repeated numerical integration and enforce local derivative consistency. However, such local matching may result in global inconsistency with the governing flow map, particularly under scarce and noisy observations. To address this limitation, we propose a framework based on Gaussian process learning with flow map refinement (GPL-FMR), a two-stage parameter estimation framework. The first stage is based on Gaussian process learning algorithm and the posterior obtained from which is transferred as an informative prior to the second stage based on flow-map refinement. The second stage further improves the parameter estimation via optimisation based on global dynamical constraints. We demonstrate and analyse its performance on multiple numerical examples, including the Van der Pol oscillator, the Lotka-Volterra model, and the Lorenz-63 system. The results show that the proposed framework consistently improves parameter estimation accuracy, particularly under scarce and noisy observations.
1. Introduction
The introduction motivates parameter estimation from dynamical-system observations and identifies a global-consistency gap in GP-based derivative matching. It presents GPL-FMR as a two-stage method that combines efficient probabilistic derivative matching with flow-map trajectory refinement.
- Data-driven learning infers continuous- or discrete-time dynamics from observations for surrogate modelling and insight into underlying system mechanisms.
- System identification seeks to uncover dynamical mechanisms directly from data without prior system knowledge, with SINDy cited as an example.
- NeuralODE represents continuous dynamics with a neural network whose trained weights and biases typically lack physical meaning unless intentionally constructed.
- Trajectory-fitting approaches estimate unknown parameters by minimizing discrepancies between observed states and numerically integrated model trajectories.
- GP-based derivative matching enforces local agreement at observed points but lacks the ODE flow map’s global constraint, so reconstructed states need not form one inferred trajectory.
- GPL-FMR transfers a GP-derived parameter estimate and uncertainty into multi-shooting trajectory refinement, enforcing global dynamical constraints through the ODE flow map.
2. Preliminary
The preliminary section formulates parameter estimation for noisy observations of an autonomous dynamical system and introduces Gaussian-process regression as the probabilistic modeling basis. It defines the system state, parameterized vector field, observation noise, inference objective, GP kernels, hyperparameter estimation, and predictive distributions.
- Dynamical-system formulation: The system state x(t) evolves under a parameterized vector field f(·; θ), while observations are collected as a time series of noisy states.The state vector is x(t) = [x_1(t), x_2(t), · · · , x_N(t)]^T, with x_0 specifying the initial condition at t = t_0.
- Dynamical-system formulation: The objective is to infer the unknown parameters θ from the observations while quantifying uncertainty.The observations can also be represented component-wise through vectors u_i formed from each observed state component over time.
- Gaussian-process regression: A Gaussian process defines a distribution over functions through a mean function and covariance function whose finite function collections follow joint Gaussian distributions.The covariance function is a positive definite kernel whose choice reflects properties such as sample-path smoothness.
- Gaussian-process regression: In GP regression, observations combine a latent function with zero-mean Gaussian noise, and kernel hyperparameters are estimated by minimizing the negative log marginal likelihood.The covariance matrix includes both kernel evaluations and the noise variance term.
- Gaussian-process regression: Conditioning the joint GP distribution on observed data yields a predictive distribution at a new point with mean k_*K^-1y and variance k̃_ϕ(x_*, x_*) − k_*K^-1k_*^T.The predictive mean and variance are obtained from the covariance matrix and cross-covariance vector between the new point and the observations.
3. Two-stage parameter estimation with uncertainty quantification
GPL-FMR estimates parameters in two stages: Gaussian-process derivative matching first provides an uncertainty-aware probabilistic estimate, then Bayesian flow-map refinement uses global dynamical constraints and inherited GPL information. The refinement jointly considers physical parameters and latent states, with MAP optimization used when full posterior inference is computationally prohibitive.
- Gaussian-process learning: The first stage uses Gaussian process learning with a derivative-matching likelihood to obtain an efficient probabilistic estimate of unknown parameters while accounting for derivative-estimate uncertainty.The posterior precision incorporates derivative uncertainty through the data-fidelity term.
- Flow-map refinement: Derivative matching compares the vector field with GP-induced derivatives locally in time, motivating a second-stage Bayesian flow-map refinement for global dynamical consistency.The refinement uses priors inherited from GPL and also estimates latent states as nuisance variables.
- Flow-map refinement: Multi-shooting partitions the observation window into consecutive intervals to reduce sensitivity and poor conditioning caused by accumulated initial-state, parameter, and integration errors.Flow consistency is imposed between consecutive latent shooting states, with a soft relaxation accommodating continuity and numerical integration error.
- Flow-map refinement: The second-stage posterior combines the flow-map likelihood with informative GPL parameter priors and priors on latent shooting states.The likelihood enforces both observation fidelity and dynamical consistency between consecutive latent states.
- Computational treatment: MAP estimation replaces full posterior approximation when the number of unknowns, Np + NtN, makes Bayesian inference computationally prohibitive.Np is the number of physical parameters, Nt the number of observation times, and N the state-space dimension.
4. Numerical Experiments
The numerical experiments evaluate GPL-FMR against feature regression and GPL on the Van der Pol oscillator, Lotka–Volterra model, and Lorenz-63 system. Across these examples, GPL-FMR generally improves parameter accuracy, reduces posterior uncertainty, and preserves global dynamical consistency under scarce and noisy observations.
- Experimental setup: GPL-FMR is evaluated against feature regression and GPL using parameter relative errors, Laplace-approximated standard deviations, inferred shooting states, and GP-smoothed trajectories.The framework is tested on three numerical examples and compares both physical parameter estimates and dynamical reconstructions.
- Van der Pol oscillator: GPL-FMR outperforms the other methods in nearly all Van der Pol test cases and recovers µ close to the ground truth even with 30% noise and Nt = 20.Feature regression estimates cluster around 1 instead of the ground truth µ = 3, while GPL degrades with increased noise and fewer observations.
- Uncertainty reduction: GPL-FMR standard deviations remain of order 10−3, compared with GPL values of approximately 10−1 to 10−2 in the Van der Pol example and 10−2 to 10−1 for Lotka–Volterra.Increasing observations from Nt = 20 to Nt = 100 further reduces GPL-FMR uncertainty for all Lotka–Volterra parameters.
- Lotka–Volterra model: GPL-FMR provides the most consistently accurate Lotka–Volterra estimates across the four physical parameters, especially for ρ, β, and δ as noise increases.For α, GPL-FMR performs well in most cases, with a few exceptions slightly outperformed by another method.
- Lorenz-63 system: For Lorenz-63, GPL-FMR remains more accurate under 20 observations and 30% noise, reducing parameter discrepancies by approximately one order of magnitude in most cases.Jointly inferred latent states reproduce chaotic dynamics and recover rapid local features that GP-smoothed trajectories over-smooth, even when Nt = 20.
5. Discussion
GPL-FMR combines GP-based derivative matching with ODE flow-map consistency, enabling refinement when scarce and noisy data produce inaccurate smoothed states and derivatives. Its Laplace approximation offers a practical compromise because full Bayesian posterior sampling becomes computationally expensive as the inference problem grows.
- GPL-FMR combines GP-based derivative matching with dynamical consistency from the ODE flow map.The flow-map refinement corrects limitations inherited from the GPL stage.
- Under scarce and noisy observations, GP-smoothed states and derivative estimates can deviate substantially from the underlying dynamics.These deviations motivate global dynamical refinement beyond local derivative matching.
- Full Bayesian posterior sampling can be computationally expensive because inference dimension grows with observations and state variables, while evaluations require numerical integration.The formulation involves physical parameters and latent shooting states across all shooting intervals.
- The Laplace approximation provides a practical compromise for the Bayesian formulation.
6. Conclusion
The work proposes a two-stage framework for parameter estimation from scarce and noisy observations. Gaussian process learning provides probabilistic estimates through local derivative matching, followed by Bayesian flow-map refinement that jointly infers parameters and latent shooting states.
- The framework targets parameter estimation in dynamical systems using scarce and noisy observations.
- Its first stage uses Gaussian process learning for efficient probabilistic parameter estimation through local derivative matching.
- Its second stage applies multi-shooting Bayesian flow-map refinement to jointly infer physical parameters and latent shooting states.
- Gaussian process posteriors are incorporated as informative empirical priors for the refinement stage.