Source-linked AI summary
Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations
Zixue Xiang, Wei Peng, Xiaohu Zheng, Xiaoyu Zhao, Wen Yao
TL;DR
PINNs for incompressible Navier–Stokes equations require balancing competing data, constraint, and PDE-residual losses, but baseline accuracy and manual weight tuning remain limited. The paper introduces lbPINNs, which learn loss weights through Gaussian likelihood and adaptive noise parameters, and reports improved accuracy across Kovasznay, cylinder-wake, and Beltrami flows.
Problem
Baseline PINNs converge at about 10−2 ± 10−3, while balancing multiple competing loss terms and manually tuning their weights is challenging.
Method
lbPINNs model loss terms probabilistically and update noise parameters through maximum likelihood and gradient descent to adaptively assign loss weights during training.
Results
Across two-dimensional steady and unsteady flows and three-dimensional unsteady Beltrami flow, lbPINNs achieve approximately 10−4 ± 10−5 errors and outperform baseline PINNs in reported experiments.
Takeaways & Limitations
Adaptive loss weighting improves the reported accuracy and adaptability of PINNs for simulating incompressible Navier–Stokes flows across the tested cases.
Takeaways & Limitations
The learned noise parameters depend on neural-network optimization algorithms, whose gradient-based methods are not guaranteed to find the exact solution; theoretical robustness and scalability remain open concerns.
Abstract
from arXiv · showhide
There have been several efforts to Physics-informed neural networks (PINNs) in the solution of the incompressible Navier-Stokes fluid. The loss function in PINNs is a weighted sum of multiple terms, including the mismatch in the observed velocity and pressure data, the boundary and initial constraints, as well as the residuals of the Navier-Stokes equations. In this paper, we observe that the weighted combination of competitive multiple loss functions plays a significant role in training PINNs effectively. We establish Gaussian probabilistic models to define the loss terms, where the noise collection describes the weight parameter for each loss term. We propose a self-adaptive loss function method, which automatically assigns the weights of losses by updating the noise parameters in each epoch based on the maximum likelihood estimation. Subsequently, we employ the self-adaptive loss balanced Physics-informed neural networks (lbPINNs) to solve the incompressible Navier-Stokes equations,\hspace{-1pt} including\hspace{-1pt} two-dimensional\hspace{-1pt} steady Kovasznay flow, two-dimensional unsteady cylinder wake, and three-dimensional unsteady Beltrami flow. Our results suggest that the accuracy of PINNs for effectively simulating complex incompressible flows is improved by adaptively appropriate weights in the loss terms. The outstanding adaptability of lbPINNs is not irrelevant to the initialization choice of noise parameters, which illustrates the robustness. The proposed method can also be employed in other problems where PINNs apply besides fluid problems.
1 Introduction
The introduction motivates improving PINN accuracy for incompressible Navier–Stokes flows by adaptively balancing competing loss terms. It presents lbPINNs, which update loss weights through Gaussian probabilistic modeling, and reports improved accuracy across several flow problems.
- PINN training combines data, boundary, initial, and equation-residual losses, making appropriate weighting important for optimization.Fixed weights can be time-consuming to tune and may lead to locally optimal solutions.
- lbPINNs automatically update the weight of each loss term during every iteration using Gaussian probabilistic models and maximum likelihood inference.Dynamic noise parameters represent loss uncertainty, with each loss weight determined by the corresponding noise magnitude.
- The method is evaluated on two-dimensional steady Kovasznay flow, two-dimensional unsteady cylinder wake, and three-dimensional unsteady Beltrami flow.
2 Related Work
Related work addresses the challenge of balancing multiple PINN loss terms, but prior approaches do not fully account for their competitive relationship. The paper responds with uncertainty-based automatic weighting for incompressible Navier–Stokes problems.
- PINNs minimize losses defined by sampled data and physical laws, but convergence and accuracy remain challenging.
- Existing approaches use gradient statistics, NTK principles, annealing, or cold starting to assign or tune loss weights during training.Annealing and cold-starting methods can require choosing an appropriate sigmoid function that influences accuracy.
- Prior studies often examine generalization effects without considering the competitive relationship among physics-objective loss terms.
- The proposed approach uses uncertainty to automatically balance multiple losses while approximating incompressible Navier–Stokes flows.
3 Methods
The method represents incompressible Navier–Stokes solutions with a neural network and trains it using physics, boundary, initial, and data-related losses. It then replaces manually selected loss weights with Gaussian uncertainty-based parameters updated during optimization.
- Physics-informed neural networks: PINNs approximate the Navier–Stokes solution with a feed-forward neural network taking space–time coordinates as input.The network outputs the modeled flow quantities, with weights and biases collected in θ.
- Physics-informed neural networks: Automatic differentiation computes the Navier–Stokes residuals, which are incorporated into the loss alongside boundary, initial, and sample-data constraints.LPDE penalizes equation residuals, LBC fits boundary conditions, LIC enforces initial constraints, and Ldata corresponds to sample data.
- Loss weighting: The baseline loss uses a weighted sum of competing constraint losses, and Kovasznay-flow experiments show that accuracy depends on the selected weights.The tested boundary and physics weights achieved their best reported accuracy at [ωf,ωb] = [0.25,0.75].
- Training procedure: The PINN training procedure initializes θ, constructs residuals and losses, and updates parameters with gradient-based optimization over repeated training steps.The algorithm specifies training, boundary, initial, and residual points before updating θ with Adam and a learning rate.
- Self-adaptive loss balanced method: lbPINNs model each loss term with Gaussian likelihoods and use trainable noise parameters to determine adaptive loss weights through maximum likelihood.The noise collection describes uncertainty for each loss term; decreasing εf increases the weight and penalty of LPDE, while log εf discourages unbounded growth.
4 Results
The experiments evaluate self-adaptive loss-balanced PINNs on Kovasznay flow, cylinder wake, and Beltrami flow, with comparisons against baseline PINNs and varying initial noise settings. Across these cases, lbPINNs achieve accurate solutions while dynamically adjusting loss weights and noise parameters.
- Adaptive weighting: The method updates noise parameters and loss weights during training using a Gaussian probabilistic model and maximum-likelihood-based weighted loss.Decreasing noise increases the corresponding loss penalty, while larger uncertainty reduces a term’s contribution.
- Experimental cases: lbPINNs are evaluated on two-dimensional steady Kovasznay flow, two-dimensional cylinder wake, and three-dimensional unsteady Beltrami flow.The experiments use analytic or exact solutions and compare predicted velocity and pressure fields with reference results.
- Kovasznay flow: 6.411×10^-4 ± 4.320×10^-5 relative L2 error is obtained for Kovasznay flow, versus 4.435 × 10^-3 ± 2.872 × 10^-3 for baseline PINNs.The comparison uses the same training duration and initial noise setting [εf,εb] = [2,2].
- Kovasznay flow: 1.106×10^-3 is the lbPINNs test loss for Kovasznay flow, compared with 1.923 × 10^-3 for PINNs.The reported convergence is associated with dynamically changing weights and noise configurations.
- Cylinder wake: 4.818×10^-6 ± 4.564×10^-6 relative L2 velocity error is reported for the cylinder wake, versus 5.321 × 10^-4 ± 3.753 × 10^-4 for original PINNs.The cylinder-wake experiment uses exact data and collocation points to assess predicted velocity and pressure fields.
- Beltrami flow: Beltrami-flow velocity and pressure predictive errors attain 10^-4 ± 10^-5 over 5000 epochs.The study also reports dynamically evolving noise parameters, with εb decreasing more slowly than the other parameters.
5 Conclusions
The paper identifies competing loss terms and weight selection as central challenges for PINNs and proposes adaptive balancing for incompressible Navier–Stokes reconstruction. Across several flow problems, lbPINNs show faster loss decay, lower relative error, and adaptability to different uncertainty-parameter initializations, while theoretical robustness remains unresolved.
- 5 Conclusions: PINN performance and convergence are susceptible to the selection of weights among competing physics loss terms.The paper frames combining multiple loss functions as a multi-objective optimization challenge.
- 5 Conclusions: lbPINNs adaptively assign weights to competing loss terms by maximizing Gaussian likelihood with scalable uncertainty parameters.The method updates the loss weights during training through learned uncertainty parameters.
- 5 Conclusions: Across Kovasznay flow, cylinder wake, and Beltrami flow, lbPINNs exhibit slightly faster loss decay and lower relative error than original PINNs.The cited flow cases include two-dimensional steady, two-dimensional unsteady, and three-dimensional unsteady Navier–Stokes problems.
- 5 Conclusions: Different initial uncertainty-parameter sets still indicate outstanding adaptability of lbPINNs.The study explicitly investigates how initialization affects accuracy.
- 5 Conclusions: Theoretical analysis is still needed because gradient-based optimization may not guarantee the exact solution for the noise parameters.The authors identify robustness and scalability as areas requiring further improvement.