Source-linked AI summary
Identifying parameter couplings and uncertainties of mixed-noise stochastic systems via full-covariance Gaussian mixture network
Xiaolong Wang, Xiangwen Hao, Jing Feng, Yuanyuan Liu, Yong Xu
TL;DR
Mixed-noise stochastic systems are difficult to identify because their likelihoods can be intractable and their parameters may be coupled or non-identifiable. PENN-GMD learns full-covariance Gaussian mixture likelihood distributions from single partially observed trajectories, accurately recovering coupling and multi-modal uncertainty across five challenging examples.
Problem
Mixed-noise stochastic systems can have intractable likelihoods, strongly coupled parameters, pronounced uncertainties, and complete non-identifiability, making point estimates insufficient.
Method
PENN-GMD maps single trajectories to closed-form, full-covariance Gaussian mixture likelihood distributions with hard-encoded constraints.
Results
Across five progressively challenging numerical examples, PENN-GMD accurately estimates parameters while revealing coupling structures and uncertainty landscapes, including multi-modal distributions under complete non-identifiability.
Takeaways & Limitations
PENN-GMD provides uncertainty-aware parameter identification that exposes parameter couplings and non-identifiability in complex mixed-noise systems.
Takeaways & Limitations
Because Gaussian mixtures have exponentially decayed tails, they cannot precisely fit heavy-tailed likelihoods induced by Lévy noise outside the main probability mass.
Abstract
from arXiv · showhide
Parameter identification of stochastic dynamical systems driven by mixed noises is challenging due to intractable likelihood functions. We propose PENN-GMD, a parameter estimation neural network that maps partially observed trajectories to a Gaussian mixture distribution (GMD) over the system parameters. Unlike conventional uncertainty estimates, the GMD employs full covariance matrices to explicitly reveal parameter couplings and multi-modal likelihood structures. The network is trained by minimizing the negative log-likelihood via a surjective parameterization that hard-encodes all GMD constraints, thereby approximating the true likelihood. We validate the method on five numerical examples with increasing complexity, including systems driven by fractional Gaussian and Lévy noises, oscillators with colored noise, coupled neurons under different observability, and an aeroelastic airfoil with unidentifiable stochastic disturbances. Results demonstrate that PENN-GMD accurately recovers likelihood distributions, captures parameter couplings, and naturally diagnoses non-identifiability through variance broadening or mode splitting. These capabilities establish PENN-GMD as a practical tool for uncertainty-aware parameter identification in complex stochastic systems where conventional likelihood-based methods are infeasible.
1 Introduction
Mixed-noise stochastic systems make likelihood-based parameter identification difficult because analytically tractable noise likelihoods are often unavailable. PENN-GMD addresses this challenge by mapping trajectories to full-covariance Gaussian mixture likelihood distributions that expose parameter couplings, multimodality, and non-identifiability.
- Motivation: Mixed disturbances couple nonlinear dynamics and noise, making accurate joint identification of system and driving-noise parameters essential but challenging.
- Motivation: Traditional maximum likelihood estimation requires an explicit, analytically tractable noise likelihood, which is problematic for fractional Gaussian and Lévy noises.
- Proposed method: PENN-GMD maps single trajectories to Gaussian mixture likelihood distributions with full covariance matrices, providing access to parameter couplings and multimodal structures.
- Proposed method: Cholesky decomposition and softmax/softplus transformations hard-encode distributional constraints, yielding a single-term objective without constraint-related hyperparameters.
- Results: Across four mixed-noise examples, PENN-GMD recovers accurate parameter estimates, reveals physically interpretable couplings, and diagnoses non-identifiability through multimodal GMD structure.The method’s likelihood outputs numerically align closely with the theoretical likelihood for a Gaussian-driven Ornstein-Uhlenbeck process.
2 Problem definition
The problem considers nonlinear stochastic systems with parameters in a bounded hyperrectangle, where only partial state observations may be available. PENN learns from simulated trajectories to approximate the parameter likelihood, with distributional covariance and multimodality exposing parameter couplings and uncertainty.
- System and observations: The system is a D-dimensional nonlinear stochastic model with M parameters constrained to a hyperrectangle P covering all configurations of interest.The parameters are denoted Θ* = [θ*1, …, θ*M]ᵀ ∈ P ⊂ R^M.
- System and observations: Because the full state may be unobservable, the method uses measured trajectories Z = {Q(x_i)} obtained through a measurement function Q.The observed trajectory contains partial states or combinations of states sampled at discrete times.
- Training objective: Training data cover P by uniformly sampling parameter vectors, simulating the known system, and recording N_train observed trajectories.Each dataset pair consists of an observed trajectory and its corresponding sampled parameter vector.
- Training objective: Given a test trajectory Z, PENN outputs Φ defining q(Θ; Φ), which approximates p(Z | Θ) and reveals parameter couplings and uncertainties through covariance and multimodality.The inferred distribution is intended to represent the likelihood of the underlying system parameters.
- Training objective: The network is trained by minimizing empirical negative log-likelihood, whose population limit is equivalent to minimizing the Kullback–Leibler divergence.The empirical objective is a Monte Carlo approximation, with approximation error vanishing as N_train → ∞.
3 Parameter estimation neural network with Gaussian mixture distribution
PENN-GMD maps partially observed trajectories to a Gaussian mixture distribution through an encoder-decoder and a surjective parameterization. Full covariance matrices represent parameter couplings and multimodal likelihoods, while hard-encoded constraints allow recovery of any N-component GMD in principle.
- Architecture: PENN-GMD encodes an observed trajectory into a fixed-length embedding, decodes it into an unconstrained vector y, and maps y to feasible GMD parameters Φ.The architecture consists of encoder fENC, decoder fDEC, and surjection fSUR components.
- Gaussian mixture representation: The GMD models multidimensional, multimodal likelihoods while encoding parameter couplings and uncertainties in mixture-component covariance matrices.Its universal approximation and identifiability properties support representing smooth densities up to label switching.
- Gaussian mixture representation: Full covariance matrices efficiently represent strongly correlated likelihoods, and their off-diagonal elements explicitly characterize interdependencies among system parameters and noises.Diagonal-covariance mixtures may require many axis-aligned components to capture tilted ridges, especially in high-dimensional parameter domains.
- Surjective parameterization: The surjection uses Softmax weights, direct means, and Cholesky-factor covariance parameters to hard-encode normalization and positive-definiteness constraints.The network therefore outputs only the free vector y, simplifying construction and ensuring feasible GMD parameters during optimization.
- Surjective parameterization: Theorem 1 establishes that fSUR is surjective, and the hard-encoded construction can theoretically recover any N-component GMD without loss of generality.For any feasible GMD parameter set Φ, a corresponding unconstrained preimage y exists.
- Architecture: An LSTM-based encoder retains historical information while suppressing irrelevant disturbances, supporting variable-length time series with partial observations.The decoder refines the compact temporal representation into y before the surjection produces feasible GMD parameters.
4 Numerical experiments
Across numerical experiments, PENN-GMD recovers parameter likelihoods and uncertainties while exposing couplings and non-identifiability in systems driven by diverse stochastic noises. Its full-covariance GMDs become more concentrated with informative data and split or broaden when parameters cannot be uniquely determined.
- OUP benchmark: In the OUP test, the analytical likelihood confirms positive coupling between lg τ and D because larger τ must be offset by larger D to preserve variability.The analytical likelihood serves as ground truth for validating PENN-GMD estimates.
- Fractional Gaussian and Lévy noises: Despite long-range fGn correlations and Lévy jumps, PENN-GMD generally estimates all five parameters effectively, with lg τ estimated best.The mixed-noise system is non-Markovian and lacks a closed-form likelihood, making it a challenging testbed.
- Fractional Gaussian and Lévy noises: When H = 0.5 or α = 2, fGn and Lévy noise become equivalent Gaussian noise, making D_FGN and D_Lévy non-identifiable and producing large errors.PENN-GMD represents the resulting possibilities along D_Lévy + D_FGN = 1, including negatively correlated intensity combinations.
- Fractional Gaussian and Lévy noises: Away from H = 0.5 and α = 2, marginal GMDs are concentrated and point estimates are accurate, whereas non-identifiable cases yield multiple physically meaningful Gaussian-component groups.The component groups distinguish alternative noise-source explanations and the intensity tradeoff.
- Data informativeness: Longer trajectories with n = 200, 1000, and 10,000 measurements produce visibly smaller GMD variances, showing that additional temporal information reduces estimation uncertainty.The experiments use the same parameters while increasing trajectory length.
5 Discussion
The discussion finds that PENN-GMD trains stably with a hyperparameter-free NLL objective, while its uncertainty estimates improve with sufficient Gaussian components and training data. Full covariance is assessed as practically important for representing strongly coupled parameters.
- Training and computation: PENN-GMD training uses a hyperparameter-free NLL loss and validation curves on 20,000 trajectories across the numerical examples.Training datasets contain 300,000 or 500,000 trajectories, depending on the example.
- Training and computation: GPU-accelerated PyTorch implementations generate 100,000 stochastic airfoil trajectories in approximately 48 seconds using about 3 GiB of GPU memory.Approximately 37 seconds are spent on warm-up and transient removal.
- Ablation studies: Increasing Gaussian components from N = 1 to N = 20 slightly reduces point-estimate mean absolute errors while leaving their error standard deviations almost identical.The near-invariance of point estimates follows because they are weighted means of the Gaussian mixture distribution.
- Ablation studies: Using 10 or 20 Gaussian components correctly captures all coupling patterns, whereas one component is overly uncertain and two components reproduce an incorrect positive coupling near α = 2.The comparison concerns the marginal distributions of the noise parameters α and H.
- Ablation studies: Larger training datasets improve point-estimate accuracy and precision, reduce mean NLL, and better recover the T-shaped likelihood of the stability index and Hurst exponent.Datasets with Ntrain ≤5×10^4 fail to precisely capture that T-shaped distribution.
- Ablation studies: Validation losses initially decrease steadily, but overfitting occurs for N ≤2 or Ntrain ≤ 5×10^4 when training continues for sufficiently many epochs.These results link more components and training data with improved uncertainty-estimation accuracy.
- Covariance ablation: For strongly coupled DVDP parameters D1 and τ1, the full-covariance model with N = 10 produces a smooth, narrowly confined likelihood.The comparison uses additional diagonal-covariance models with N = 10 and N = 30 under otherwise identical settings.
6 Conclusion
PENN-GMD learns closed-form full-covariance Gaussian mixture approximations to intractable mixed-noise SDE likelihoods from single trajectories, exposing parameter couplings and multimodal uncertainties. Across five progressively challenging numerical examples, it demonstrated accurate point-estimation capability, while fixed mixture size and quadratic covariance scaling remain limitations.
- Contribution: PENN-GMD learns full-covariance Gaussian mixture distributions from single trajectories to approximate intractable mixed-noise SDE likelihoods.Hard-encoded constraints enable direct extraction of parameter couplings and multimodal uncertainties.
- Validation: Five progressively challenging numerical examples demonstrated the framework across fractional Gaussian and Lévy noises, colored-noise oscillators, coupled neurons, and aeroelastic airfoil dynamics.The examples also included a standard Ornstein-Uhlenbeck process with known likelihood and different observability scenarios.
- Limitations: The mixture component number N is fixed to 10, and full covariance matrices scale quadratically with parameter dimension, increasing training cost for high-dimensional systems.The authors identify adaptive component selection as an extension and covariance scaling as a practical limitation.