Source-linked AI summary
A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
Xuhui Meng, George Em Karniadakis
TL;DR
The paper addresses how to learn accurate physical models when high-fidelity data are scarce but low-fidelity data are abundant. It proposes composite multi-fidelity DNNs and MPINNs with adaptive linear/nonlinear correlation learning, and reports high-accuracy identification in function-approximation and inverse-PDE tests using very few high-fidelity data.
Problem
Accurate high-fidelity data are scarce and expensive, while inverse nonlinear PDEs may require large high-fidelity datasets for identifying unknown parameters or functions.
Method
The paper combines a low-fidelity network with linear and nonlinear high-fidelity correlation networks, then extends PINNs with a PDE-induced network to form MPINNs.
Results
The methods learn adaptive correlations, approximate benchmark and high-dimensional functions, and identify unknown parameters or functions in inverse PDE problems with very few high-fidelity data.
Takeaways & Limitations
The proposed networks support multi-fidelity approximation and inverse-PDE identification when inexpensive low-fidelity data complement a small high-fidelity dataset.
Takeaways & Limitations
The reactive-transport study assumes ψ = 0.4, q = 0.5m/y, and D = 10−8m/s2 are known, while reaction parameters are difficult to measure directly.
Abstract
from arXiv · showhide
We propose a new composite neural network (NN) that can be trained based on multi-fidelity data. It is comprised of three NNs, with the first NN trained using the low-fidelity data and coupled to two high-fidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the low-fidelity and the high-fidelity data. We first demonstrate the accuracy of the new multi-fidelity NN for approximating some standard benchmark functions but also a 20-dimensional function. Subsequently, we extend the recently developed physics-informed neural networks (PINNs) to be trained with multi-fidelity data sets (MPINNs). MPINNs contain four fully-connected neural networks, where the first one approximates the low-fidelity data, while the second and third construct the correlation between the low- and high-fidelity data and produce the multi-fidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, in the two high-fidelity NNs a relaxation parameter is introduced, which can be optimized to combine the linear and nonlinear sub-networks. By optimizing this parameter, the present model is capable of learning both the linear and complex nonlinear correlations between the low- and high-fidelity data adaptively. By training the MPINNs, we can:(1) obtain the correlation between the low- and high-fidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of high-fidelity data.
1. Introduction
The paper addresses the cost and scarcity of high-fidelity data by developing multi-fidelity DNNs and MPINNs that combine abundant low-fidelity data with limited accurate measurements. The proposed approach targets adaptive correlation learning, high-dimensional approximation, and inverse nonlinear PDE problems.
- Motivation: High-fidelity data are accurate but expensive and scarce, whereas low-fidelity data are cheaper, more abundant, and can provide useful trends.Multi-fidelity modeling combines these sources to improve prediction accuracy with fewer high-fidelity samples.
- Related work: Existing correlation methods include response surfaces, polynomial chaos, Gaussian processes, artificial neural networks, and moving least squares.Gaussian-process models with linear autoregressive schemes have received substantial attention in applications.
- Research gap: Gaussian-process autoregressive approaches capture linear correlations, while extensions are needed for complex nonlinear correlations and challenging high-dimensional or discontinuous problems.The cited limitations motivate a neural-network-based alternative.
- Contribution: The proposed multi-fidelity DNNs and MPINNs are designed to learn linear and nonlinear correlations adaptively and handle high-dimensional problems.The MPINN extension combines multi-fidelity learning with physics-informed neural networks for nonlinear PDE settings.
- Study scope: The paper evaluates the methods on function approximation and inverse PDE problems, with the study organized around methods, results, and conclusions.Applications include learning unknown quantities in geophysical systems.
2. Multi-fidelity Deep Neural Networks and MPINNs
The method constructs a composite multi-fidelity network that maps low-fidelity predictions to high-fidelity outputs through adaptive linear and nonlinear correlation components. For MPINNs, the resulting approximation is passed to a PDE-induced network that encodes governing equations.
- Correlation modeling: Multi-fidelity modeling begins by discovering and exploiting the relation between low- and high-fidelity data.The conventional autoregressive relation uses multiplicative and additive correlation surrogates.
- Correlation modeling: A generalized autoregressive scheme uses an unknown function F(.) to map low-fidelity data to the high-fidelity level, including linear and nonlinear cases.This extends models restricted to linear correlations.
- Adaptive correlation: F(.) is decomposed into linear and nonlinear parts, whose combination is controlled by a data-determined hyper-parameter α.The value of α determines the degree of nonlinearity in the learned correlation.
- MPINN architecture: MPINNs contain four fully connected networks: one for low-fidelity approximation, two for low–high-fidelity correlation, and one induced by the governing PDEs.The first three networks produce the multi-fidelity approximation used by the PDE-encoding network.
- Training: The composite DNNs dynamically learn linear and nonlinear correlations without prior knowledge of the relationship between fidelity levels.Training uses a loss function with regularization and is optimized with L-BFGS and Xavier initialization.
3. Results and Discussion
The results section tests the multi-fidelity DNN and MPINN on function approximation and two inverse PDE problems involving geophysical applications.
- Evaluation scope: The study evaluates the multi-fidelity DNN on function-approximation problems and MPINNs on two inverse PDE problems.The inverse problems concern hydraulic conductivity and reaction models in porous-media settings, as described elsewhere in the paper.
3.1. Function approximation
The multi-fidelity DNN approximates continuous, discontinuous, high-dimensional, and complex nonlinear-correlated functions by learning relationships between sparse high-fidelity and more abundant low-fidelity data. Across these tests, it adaptively discovers correlations without prior assumptions and can extend to non-functional correlations through embedding theory.
- Function approximation: The multi-fidelity DNN is evaluated on continuous and discontinuous functions with both linear and complicated nonlinear low–high-fidelity correlations.The experiments use different data sets and correlation structures to assess the model’s approximation capability.
- Discontinuous function with linear correlation: With 38 low-fidelity and 5 high-fidelity points, the multi-fidelity DNN accurately captures the discontinuity at x = 0.5 despite no data in 0.4 < x < 0.6.The learned correlation differs only slightly from the exact correlation, whereas the high-fidelity-only prediction is not good.
- Continuous function with nonlinear correlation: With 51 low-fidelity and 14 high-fidelity points, the model accurately predicts high-fidelity values even when low- and high-fidelity trends are opposite over part of the domain.The learned correlation agrees well with the exact one in this adversarial-data case.
- Phase-shifted oscillations: For phase-shifted oscillations, Method II using [x, yL(x), yL(x−τ)] succeeds, while Method I using [x, yL(x)] fails with the available data.The delayed low-fidelity value implicitly approximates the first derivative needed to capture the correlation.
- 20-dimensional function approximation: The framework can handle high-dimensional cases and extend through embedding theory to non-functional correlations caused by phase errors.The section summary reports adaptive correlation learning without prior assumptions and improved capability for complex nonlinear correlations.
3.2. Inverse PDE problems with nonlinearities
The MPINNs are applied to inverse PDE problems in unsaturated flow and reactive transport, using scarce high-fidelity observations together with multi-fidelity data. They estimate hydraulic conductivity and reactive-model parameters, with improved results reported over single-fidelity modeling.
- Unsaturated flows: MPINNs use scarce high-fidelity pressure-head measurements to estimate the hydraulic conductivity field in unsaturated porous-media flow.The setup assumes hydraulic conductivity is inferred from pressure-head observations rather than measured directly.
- Unsaturated flows: For constant-flux unsaturated flow, replacing the differential PDE formulation with an integral mass-conservation formulation greatly improves the hydraulic-conductivity prediction.The integral formulation is used instead of Eq. (22), and the results improve greatly.
- Unsaturated flows: MPINNs significantly improve prediction accuracy for α0 and m compared with estimates based on high-fidelity data only.The comparison is reported using mean parameter values for different initial guesses in Table 2.
- Reactive transport: The reactive-transport problem learns the effective chemical reaction rate and reaction order from partial observations of the solute concentration field.The reaction setup assumes known porosity, Darcy velocity, and dispersion coefficient, while the reaction parameters are difficult to measure directly.
- Reactive transport: MPINN estimates of kf and ar are much better than single-fidelity estimates, with rather small standard deviations.The results compare PINNs trained on high-fidelity data with MPINNs for the considered reactive-transport cases.
4. Conclusion
The paper presents composite multi-fidelity neural networks and MPINNs that combine low-fidelity data with a small high-fidelity set for function approximation and inverse PDE problems. The approach adaptively learns linear and nonlinear correlations and identifies unknown parameters or functions with high accuracy using very few high-fidelity data.
- The composite network uses separate subnetworks for low-fidelity approximation and linear or nonlinear correlations with high-fidelity data.MPINNs extend this design with four fully connected neural networks, including a final network that encodes the governing PDEs.
- Training a relaxation parameter lets MPINNs combine linear and nonlinear correlation subnetworks without assuming the relation between fidelity levels in advance.
- Multi-fidelity neural networks can approximate continuous and discontinuous functions in high dimensions, while MPINNs address inverse problems governed by nonlinear PDEs.
- The method was tested on function approximation, hydraulic-conductivity estimation for unsaturated flows, and reaction-model identification in reactive transport.
- The proposed MPINN identifies unknown parameters or functions with high accuracy using very few high-fidelity data, potentially reducing high-fidelity experimental collection costs.
Appendix A. Data-driven manifold embeddings
The appendix extends multi-fidelity neural networks with data-driven manifold embeddings to represent more complicated nonlinear correlations. Delayed low-fidelity observations enlarge the network inputs, while the time delay is learned from training data.
- Embedding theory provides a route for learning more complicated nonlinear correlations between low- and high-fidelity data.
- Under the smooth-function assumption, high-fidelity data are represented as a function of the input and delayed low-fidelity observations.The formulation uses yL(x), yL(x−τ), through yL(x−(m−1)τ), together with a small number of high-fidelity pairs.
- The neural-network inputs can be extended from [x, yL(x)] to a vector containing multiple time-delayed low-fidelity values.
- The time delay τ is learned from training data because selecting its optimal value is important and existing selection methods are often problem-dependent.