Source-linked AI summary
Machine Learning of Linear Differential Equations using Gaussian Processes
Maziar Raissi, George Em. Karniadakis
TL;DR
The paper addresses learning parametric linear conservation laws from scarce and noisy observations. It adapts Gaussian process priors to the relevant operators and infers their parameters probabilistically. The approach handles diverse linear operators and provides uncertainty estimates for learned parameters and predictions.
Problem
The paper asks how to infer governing parametric linear equations from scarce or noisy observations of black-box functions.
Method
The method modifies Gaussian process priors according to the linear operator, turning operator parameters into kernel hyperparameters.
Results
The algorithm learns operator parameters from as few as 7 noise-free training data and handles highly noisy observations, including fractional equations.
Takeaways & Limitations
The framework provides a unified probabilistic treatment of differential, integro-differential, time-dependent, and fractional linear equations with posterior uncertainty quantification.
Takeaways & Limitations
Training requires inversion of dense covariance matrices, whose computational cost scales cubically with the number of training data.
Abstract
from arXiv · showhide
This work leverages recent advances in probabilistic machine learning to discover conservation laws expressed by parametric linear equations. Such equations involve, but are not limited to, ordinary and partial differential, integro-differential, and fractional order operators. Here, Gaussian process priors are modified according to the particular form of such operators and are employed to infer parameters of the linear equations from scarce and possibly noisy observations. Such observations may come from experiments or "black-box" computer simulations.
1. Introduction
The paper targets a unified framework for learning conservation laws represented by parametric linear operators from noisy observations. It uses Gaussian processes to connect black-box functions while supporting multi-fidelity information sources.
- Research problem: The central problem is learning parameters φ of a linear operator from noisy observations of u(x) and f(x).The learned parameters identify the governing equation that best describes the data.
- Research problem: Gaussian processes provide flexible priors, analytical tractability, probabilistic inference, and posterior variances for uncertainty quantification.These properties motivate their use for inverse problems involving scarce or noisy data.
- Related formulations: The framework includes convolved Gaussian processes for multi-output settings and recursive co-kriging for blending information sources of variable fidelity.In recursive co-kriging, f(x) combines φu(x) with a latent function v(x) capturing unexplained factors.
2. Methodology
The method modifies Gaussian process priors to reflect the relevant linear operator. Operator parameters become kernel hyperparameters, allowing them to be inferred jointly with the functions.
- Prior construction: The algorithm uses Gaussian process priors tailored to the corresponding differential operators.It assumes u(x) is a Gaussian process with covariance kuu(x, x′; θ).
- Prior construction: Because linear transformations of Gaussian processes remain Gaussian processes, operator-induced functions can be represented through transformed covariance functions.This supports differential and integral transformations within the same probabilistic framework.
- Parameter learning: The operator parameters φ are converted into hyperparameters of the kernels kff, kuf, and kfu.This is the mechanism by which the governing linear equation becomes learnable through Gaussian-process inference.
Kernels [3]
The method uses Gaussian-process kernels to model linear transformations, learn operator and kernel parameters, and quantify predictive and parameter uncertainty.
- Kernels [3]: The squared exponential covariance function is used as the default kernel and implies smooth approximations.More complex function classes can be represented by choosing different kernels.
- Kernels [3]: The hyper-parameters and operator parameters are trained by minimizing the negative log marginal likelihood.The marginal likelihood balances data fit against model complexity through an Occam’s-razor effect.
- Kernels [3]: Training requires inversion of dense covariance matrices, whose computational cost scales cubically with the number of training observations.Recent work has addressed this Gaussian-process scaling limitation.
- Kernels [3]: Posterior variances indicate confidence in estimated operator parameters and predictions, providing Bayesian uncertainty quantification.The uncertainty information could support active-learning data acquisition, although that application is not pursued here.
3. Results
The framework is designed to learn linear operators across integro-differential, time-dependent, and fractional equations, with results illustrated using benchmark problems and Figure 1’s one-dimensional case.
- 3. Results: The algorithm treats linear operators agnostically, allowing integro-differential, time-dependent, and fractional equations to be learned within one framework.
- 3. Results: Figure 1 compares exact functions, training observations, predictive means, and two-standard-deviation bands for left- and right-hand sides under noise-free and noisy data.
- 3. Results: The one-dimensional benchmark infers parameters α and β from noise-free and noisy observations of u(x) and f(x).The functions satisfy the integro-differential equation when (α, β) = (2, 5).
Noise-free data
Using seven randomly selected noise-free observations, the algorithm recovers the parameters of the one-dimensional integro-differential operator and provides uncertainty information.
- Noise-free data: (2.012627, 4.977879) are the learned values of (α, β) from seven noise-free training data.
- Noise-free data: The seven observations comprise four data points for u(x) and three for f(x), sampled randomly in [0, 1].
- Noise-free data: Posterior distributions for u(x) and f(x) are shown in Figure 1(A, B), while posterior variances indicate uncertainty in parameters and predictions.
Noisy data
The framework learns linear-equation parameters from scarce, noisy observations and represents uncertainty in reconstructed functions. Demonstrations cover heat, fractional, and gene-expression dynamics, including inferred physical parameters.
- Noisy data: With 14 and 10 noisy observations, the algorithm learned (α, β) = (2.073054, 5.627249) despite deliberately sizable noise.The posterior variances quantify both observation scarcity and data-noise levels.
- Heat equation: For the heat equation, 40 scattered space-time observations yielded α = 0.999943 from noise-free data.The example uses 20 observations for each of u(t, x) and f(t, x).
- Fractional equation: Fractional operators are handled by deriving the operator-specific kernel through an inverse Fourier transform.The approach bypasses numerical discretization and the costly convolution operations associated with non-local fractional dynamics.
- Fractional equation: The fractional-equation example learned α = 1.412104 from only 9 noise-free one-dimensional observations.The data comprised five observations for u(x) and four for f(x).
- Drosophila gap gene dynamics: For Drosophila gap genes, the method learns decay λa and diffusion Da rates while treating production parameters Ra, T ab, and ha as given.The model covers Hb, Kr, Gt, and Kni protein dynamics using expression data and posterior predictions.
4. Discussion
The work introduces a probabilistic technology for learning general parametric linear equations from noisy data and demonstrates its generality across diverse benchmarks and functional genomics. The framework also identifies several feasible extensions based on the preservation of Gaussian-process structure under linear transformations.
- The framework learns general parametric linear equations from noisy data.
- Its generality is demonstrated on benchmark problems with different attributes and an application in functional genomics.
- The methodology can be applied to inverse problems in materials characterization, tomography, electrophysiology, and metamaterial design.
- The framework can be generalized to multiple-fidelity data, variable coefficients, complex geometries, non-Gaussian or input-dependent noise, and systems of linear integro-differential equations.These extensions remain feasible because any linear transformation of a Gaussian process is still a Gaussian process.