Source-linked AI summary
Transfer learning based multi-fidelity physics informed deep neural network
Souvik Chakraborty
TL;DR
The paper addresses reliability-analysis settings where governing physics are approximate and high-fidelity data are scarce or expensive. It proposes MF-PIDNN, which first trains on approximate physics and then transfers learning from high-fidelity data. Across benchmark reliability problems, the method correctly predicts failure probability and reliability index and outperforms high-fidelity data-driven and low-fidelity physics-driven alternatives.
Problem
Reliability analysis is challenging when governing equations are unavailable or approximate and only few, costly high-fidelity observations can be collected.
Method
MF-PIDNN trains a low-fidelity physics-informed model from approximate governing equations, then updates it with available high-fidelity data through transfer learning.
Results
Across benchmark reliability problems, MF-PIDNN correctly predicts probability of failure and reliability index and outperforms high-fidelity data-driven and low-fidelity physics-driven models.
Takeaways & Limitations
MF-PIDNN combines information from approximate low-fidelity physics and high-fidelity data without requiring low-fidelity data generation.
Takeaways & Limitations
The study uses unregularized mean-squared loss during transfer learning and manually selects transfer-learning parameters, network architecture, and activation functions.
Abstract
from arXiv · showhide
For many systems in science and engineering, the governing differential equation is either not known or known in an approximate sense. Analyses and design of such systems are governed by data collected from the field and/or laboratory experiments. This challenging scenario is further worsened when data-collection is expensive and time-consuming. To address this issue, this paper presents a novel multi-fidelity physics informed deep neural network (MF-PIDNN). The framework proposed is particularly suitable when the physics of the problem is known in an approximate sense (low-fidelity physics) and only a few high-fidelity data are available. MF-PIDNN blends physics informed and data-driven deep learning techniques by using the concept of transfer learning. The approximate governing equation is first used to train a low-fidelity physics informed deep neural network. This is followed by transfer learning where the low-fidelity model is updated by using the available high-fidelity data. MF-PIDNN is able to encode useful information on the physics of the problem from the {\it approximate} governing differential equation and hence, provides accurate prediction even in zones with no data. Additionally, no low-fidelity data is required for training this model. Applicability and utility of MF-PIDNN are illustrated in solving four benchmark reliability analysis problems. Case studies to illustrate interesting features of the proposed approach are also presented.
1 Introduction
The paper identifies limits in existing multi-fidelity methods and proposes MF-PIDNN, which combines approximate physics with scarce high-fidelity data without requiring low-fidelity data generation.
- Motivation: Existing multi-fidelity methods can fail when low- and high-fidelity models have complex, space-dependent, nonlinear correlations.Conventional co-Kriging and MLMC approaches rely on trend capture and strong linear correlation between fidelities.
- Motivation: Existing approaches commonly assume that low-fidelity solvers are computationally efficient enough to generate sufficient low-fidelity data.Large eddy simulation illustrates a low-fidelity solver whose computational cost can remain significant.
- Motivation: Physics-informed multi-fidelity methods may also assume that the exact physics corresponding to high-fidelity data is known.The paper notes that governing equations can be unavailable or only approximate because of simplifying assumptions.
- Proposed framework: MF-PIDNN trains an initial low-fidelity model directly from approximate governing equations, then updates it using transfer learning and available high-fidelity data.The framework is designed to capture physical laws such as invariance and symmetries without generating low-fidelity data.
- Evaluation: The paper evaluates the proposed framework on selected reliability analysis problems and presents numerical results in Section 4.The paper structure places the approach in Section 3 and numerical performance results in Section 4.
2 Problem statement
Reliability analysis is difficult when limit-state equations are unavailable or approximate and laboratory data are scarce. The paper formulates a framework that trains directly from approximate physics and limited high-fidelity observations.
- Reliability formulation: Reliability analysis defines failure through a limit-state function, with J(ξ) < 0 representing the failure domain and J(ξ) ≥ 0 the safe domain.The failure probability is formulated using the distribution of the stochastic input and an indicator of the failure region.
- Reliability formulation: The multivariate integral for failure probability generally lacks a closed-form solution, requiring numerical integration or asymptotic approximations.This makes repeated evaluation of the limit-state function central to reliability calculations.
- Data constraints: When governing equations are unavailable or approximate, design-stage reliability analysis may depend on costly and limited laboratory experiments.Field data are unavailable in the stated design-phase setting, leaving laboratory experiments as the feasible alternative.
- Data constraints: A surrogate model can map stochastic inputs, spatial locations, and times to system responses when enough laboratory data are available.The paper lists Gaussian processes, polynomial chaos, analysis of variance, support vector machines, and hybrid polynomial expansions as examples.
- Proposed approach: MF-PIDNN directly trains on the low-fidelity differential model without generating low-fidelity data, together with the available high-fidelity data.The low-fidelity model is given by an ordinary or partial differential equation, including Eq. (7).
3 Multi-fidelity physics informed deep neural network
MF-PIDNN combines physics-informed and data-driven deep learning to address approximate governing physics and scarce high-fidelity data. It trains one low-fidelity physics-informed model, then updates selected parameters using high-fidelity data through transfer learning.
- Physics-informed foundation: PI-DNN trains a DNN directly from governing differential equations rather than simulation data.Its physics-informed loss uses residuals evaluated at collocation points, with derivatives obtained by automatic differentiation.
- Motivation: Data-driven DNNs are unsuitable when only a few high-fidelity data points are available, while PI-DNNs can be inaccurate when governing equations are approximate.The paper identifies data scarcity and imperfect physics as complementary limitations of the two baseline approaches.
- Proposed approach: MF-PIDNN first solves the low-fidelity model with PI-DNN, then updates that model using available high-fidelity data.The approach therefore combines approximate physics with data-driven refinement instead of relying on a separate low-fidelity surrogate.
- Proposed approach: The proposed framework requires no low-fidelity data and is designed for problems with approximate physics and limited high-fidelity observations.Its high-fidelity update uses mean-squared error, while the initial model is trained from the approximate governing equation.
- Transfer learning: Transfer learning reduces the number of parameters updated during high-fidelity training and retains features learned from the low-fidelity model.The procedure freezes the initial layers while tuning the final one or two layers.
4 Numerical illustration
The numerical illustrations evaluate MF-PIDNN across four reliability-analysis problems using transfer learning from low-fidelity physics and sparse high-fidelity data. Across the examples, MF-PIDNN closely matches benchmark MCS results, supports extrapolation, and outperforms standalone low- or high-fidelity neural networks.
- Numerical illustration: MF-PIDNN is evaluated on four benchmark reliability-analysis problems spanning stochastic variables, linear and nonlinear problems, and ordinary and partial differential equations.The benchmark results use Monte Carlo simulation for comparison.
- MF-PIDNN procedure: The algorithm first trains a physics-informed DNN with the low-fidelity model, then freezes initial layers and updates the network using high-fidelity data.The workflow also modifies the DNN to satisfy initial and boundary conditions and minimizes physics-informed and data-driven loss functions.
- Ordinary differential equation: MF-PIDNN matches MCS results in the first reliability example, whereas low-fidelity PI-DNN and high-fidelity DNN produce erroneous results.The comparison reports both probability of failure and reliability index.
- Nonlinear oscillator: For the nonlinear oscillator, MF-PIDNN closely matches MCS across thresholds and outperforms both HF-DNN and LF-PIDNN at t_t = 3.0 and x_0 = 2.0.These results indicate that the model captures the response over the whole domain.
5 Conclusions
MF-PIDNN combines approximate physics with limited high-fidelity data through transfer learning, achieving accurate reliability predictions while requiring no low-fidelity data. The study also identifies several aspects requiring further automation and regularization.
- MF-PIDNN first trains on approximate low-fidelity physics, then updates the model using high-fidelity data through transfer learning.
- MF-PIDNN requires no low-fidelity data and can provide reasonable predictions in zones without data by retaining information from approximate physical laws.
- Across benchmark reliability problems, MF-PIDNN correctly predicts failure probabilities and reliability indices, outperforming high-fidelity data-driven and low-fidelity physics-driven models.
- Transfer-learning updates may overfit because the study uses an unregularized mean-squared loss and manually selects tunable parameters and network design.