Source-linked AI summary
Inferring solutions of differential equations using noisy multi-fidelity data
Maziar Raissi, Paris Perdikaris, George Em. Karniadakis
TL;DR
The paper addresses the problem of inferring differential-equation solutions from scarce, noisy, multi-fidelity forcing and solution data without requiring observations on domain boundaries. It develops Gaussian-process priors tailored to linear operators and uses their predictive posteriors for uncertainty quantification and active refinement. The framework handles spatio-temporal and high-dimensional problems while avoiding temporal discretization and associated timestepping consistency and stability issues.
Problem
The paper asks how to estimate unknown solutions of linear differential equations from limited noisy forcing and solution observations that may be scattered away from boundaries.
Method
The method uses Gaussian-process priors transformed by linear integro-differential operators to model multi-fidelity forcing and infer posterior distributions over solutions.
Results
The framework returns solution fields with quantified uncertainty and applies to spatio-temporal and high-dimensional linear problems using scattered noisy variable-fidelity data.
Takeaways & Limitations
Posterior uncertainty supports adaptive solution refinement, while the framework avoids numerical discretization and timestepping consistency and stability restrictions.
Takeaways & Limitations
The squared-exponential prior limits approximation capability to sufficiently smooth functions.
Abstract
from arXiv · showhide
For more than two centuries, solutions of differential equations have been obtained either analytically or numerically based on typically well-behaved forcing and boundary conditions for well-posed problems. We are changing this paradigm in a fundamental way by establishing an interface between probabilistic machine learning and differential equations. We develop data-driven algorithms for general linear equations using Gaussian process priors tailored to the corresponding integro-differential operators. The only observables are scarce noisy multi-fidelity data for the forcing and solution that are not required to reside on the domain boundary. The resulting predictive posterior distributions quantify uncertainty and naturally lead to adaptive solution refinement via active learning. This general framework circumvents the tyranny of numerical discretization as well as the consistency and stability issues of time-integration, and is scalable to high-dimensions.
1. Introduction
The paper situates probabilistic machine learning within a broader shift toward data-driven, probabilistic approaches to mathematical and scientific modeling.
- Probabilistic methods are increasingly being applied to mathematical and scientific modeling.
- This perspective supports computing with probability distributions rather than relying solely on deterministic methods.
- Recent work has extended probabilistic approaches into numerical analysis and scientific computing.
2. Problem setup
The paper considers linear differential equations with unknown solutions driven by expensive, noisy forcing data available at multiple fidelity levels. Sparse, noisy solution observations called anchor points supplement forcing observations and need not lie on boundaries or initial-value locations.
- The target problem is a general linear integro-differential equation L_xu(x) = f(x), with u the unknown solution and f the external force.
- High-fidelity forcing observations are scarce and noisy, while lower-fidelity data may come from simplified models, inexpensive sensors, or uncalibrated measurements.
- Anchor points are scattered noisy observations of u that may be located in the interior or on domain boundaries.
- The objective is to estimate u from multi-fidelity forcing data and anchor points, together with its uncertainty or error.
3. Solution methodology
The methodology combines Gaussian-process priors with linear differential operators and autoregressive multi-fidelity modeling to infer solution posteriors from forcing data and anchor observations. Posterior variance supports adaptive acquisition, while the approach has computational and modeling limitations.
- Gaussian processes provide probabilistic priors and posterior variance estimates for adaptive refinement and active learning.
- A two-level autoregressive model represents the solution as u(x) = ρu_1(x) + δ_2(x), with independent Gaussian-process components.
- Linearity of L_x transfers the Gaussian-process structure from u to the forcing f, with covariance kernels transformed as k_ℓ = L_xL_x′g_ℓ.
- The trained model returns a predictive Gaussian distribution for u, whose mean predicts the solution and whose variance quantifies uncertainty.
- The training cost scales as O(mn^3), and the fixed cross-correlation parameter ρ may fail to represent more complex input-dependent correlations.
- Active learning selects the next high-fidelity forcing observation where the posterior variance of f is maximized.
4. Results
Across one-dimensional, time-dependent, high-dimensional, and fractional examples, the proposed framework reconstructs linear-equation solutions from sparse forcing and anchor data while avoiding numerical discretization. Its posterior uncertainty supports error estimation and active refinement, and its kernel-based treatment discovers low-dimensional structure.
- 4.1. Integro-differential equation in 1D: The one-dimensional example accurately approximates the solution without discretizing the linear operator or requiring solution data beyond minimal anchor points.Anchors need not be initial or boundary conditions and may be noisy.
- 4.1. Integro-differential equation in 1D: Posterior variance quantifies uncertainty in the predicted solution, identifies less-trusted regions, and supports active-learning data acquisition.The variance provides an a-posteriori error signal for refining knowledge of u.
- 4.2. Active learning and a-posteriori error estimates for the 2D Poisson equation: Selecting new forcing observations where posterior variance is highest yields fast convergence and accurate solution predictions after a handful of iterations.Figure 2 tracks relative errors and posterior standard deviations for both f and u during refinement.
- 4.3.1. Time-dependent linear advection-diffusion-reaction equation: The method reconstructs a time-dependent advection-diffusion-reaction solution from 10 noisy high-fidelity forcing observations while avoiding temporal discretization and timestepping restrictions.Posterior variance agrees with the maximum absolute point-wise solution error in the illustrated case.
- 4.3.2. Poisson equation in 10D: In a 10-dimensional Poisson problem, the method automatically discovers the two active dimensions and attains high-accuracy solution predictions from multi-fidelity data.The active dimensions are x1 and x3, while the low-fidelity model varies across all dimensions.
- 4.3.3. Fractional sub-diffusion equation: For fractional-order linear equations, the framework handles the operator by deriving its covariance kernel through an inverse Fourier transform, demonstrated at α = 0.3.The fractional operator is defined in the Riemann-Liouville sense, and the exact solution is obtained using Fourier analysis.
5. Discussion
The framework learns solutions to general linear integro-differential equations from noisy, variable-fidelity data while handling diverse problem structures without workflow changes. Its scope includes several extensions, but the chosen prior limits approximation to sufficiently smooth functions and nonlinear operators remain unresolved.
- The framework learns solution fields for general linear integro-differential equations from scattered noisy data of variable fidelity, with quantified uncertainty.
- Figure 3 demonstrates unchanged treatment of temporal integration, high-dimensionality, and non-local interactions across diverse benchmark problems.
- The methodology can extend to more than two fidelity levels, variable coefficients, complex geometries, non-Gaussian noise, and broader linear boundary conditions.
- The stationary squared exponential Gaussian-process prior limits approximation capability to sufficiently smooth functions.
- Nonlinear operators remain an open problem because assigning Gaussian-process priors to the solution may not be reasonable.
Appendix A. Computer software
The manuscript provides Matlab code enabling access to and reproduction of all presented data and results.
- Matlab code provides access to and enables reproduction of all data and results presented in the manuscript.
Appendix B. Movie S1
Movie S1 illustrates convergence of the active-learning procedure by tracking posterior variances and errors for the solution and forcing term, alongside the data-acquisition process.
- The animation tracks posterior variance and error against the exact solution for both the solution u and forcing term f.
- A distinct colormap highlights that active learning selects new data using the maximum posterior variance of f.