Source-linked AI summary
Stiff-PINN: Physics-Informed Neural Network for Stiff Chemical Kinetics
Weiqi Ji, Weilun Qiu, Zhiyu Shi, Shaowu Pan, Sili Deng
TL;DR
PINNs face challenges when solving stiff chemical-kinetic ODE systems. This paper reduces stiffness with QSSA before applying PINNs, finding that regular PINNs fail while the resulting Stiff-PINN captures system dynamics.
Problem
The paper investigates why PINNs struggle to solve stiff chemical-kinetic systems governed by stiff ODEs.
Method
The approach applies QSSA to reduce ODE-system stiffness before enforcing the governing equations with a PINN.
Results
Regular PINNs failed to predict system evolution, whereas QSSA-based Stiff-PINNs captured the systems’ dynamic responses.
Takeaways & Limitations
The authors identify physical stiffness as the failure source and present Stiff-PINN as an effective approach for stiff chemical-kinetic systems.
Takeaways & Limitations
The authors note open questions about developing a robust approach and more general stiffness-removal methods.
Abstract
from arXiv · showhide
Recently developed physics-informed neural network (PINN) has achieved success in many science and engineering disciplines by encoding physics laws into the loss functions of the neural network, such that the network not only conforms to the measurements, initial and boundary conditions but also satisfies the governing equations. This work first investigates the performance of PINN in solving stiff chemical kinetic problems with governing equations of stiff ordinary differential equations (ODEs). The results elucidate the challenges of utilizing PINN in stiff ODE systems. Consequently, we employ Quasi-Steady-State-Assumptions (QSSA) to reduce the stiffness of the ODE systems, and the PINN then can be successfully applied to the converted non/mild-stiff systems. Therefore, the results suggest that stiffness could be the major reason for the failure of the regular PINN in the studied stiff chemical kinetic systems. The developed Stiff-PINN approach that utilizes QSSA to enable PINN to solve stiff chemical kinetics shall open the possibility of applying PINN to various reaction-diffusion systems involving stiff dynamics.
1. Introduction
PINNs encode governing differential equations through residual-loss minimization, but stiff chemical kinetics create training challenges associated with widely separated species time scales and other sources of imbalance. The paper therefore evaluates regular PINN and introduces Stiff-PINN, which incorporates QSSA to reduce stiffness before solving the kinetic systems.
- PINN framework: PINN constructs differential-equation solution spaces with neural networks and enforces governing equations by minimizing automatically differentiated residual losses.The framework supports both forward differential-equation problems and inverse parameter inference from observations.
- Challenges for stiff kinetics: Numerical stiffness can cause PINN failure through unbalanced back-propagated gradients, while physical stiffness may impose additional training challenges.The imbalance occurs between initial/boundary-condition losses and differential-equation residual losses.
- Challenges for stiff kinetics: Stiff chemical kinetics challenge PINN because reaction-network species span widely separated characteristic time scales, while state dimensionality, nonlinearity, and concentration ranges add further difficulty.Stiff kinetics is relevant to atmospheric, energy, materials, chemical-engineering, biomedical, and pharmaceutical systems.
- QSSA-based reduction: QSSA reduces stiffness by replacing selected fast-species ODEs with differential-algebraic equations, reducing the largest chemical-Jacobian eigenvalue and eliminating fast species.It can also reduce the number of state variables and transport equations, greatly lowering computational cost.
- Study objective: The study evaluates PINN on two classical stiff dynamics problems and compares regular-PINN with Stiff-PINN, which incorporates QSSA to reduce stiffness.The paper presents the kinetic systems, PINN models, QSSA implementation, and comparative performance investigation in subsequent sections.
2. Methodology
The methodology formulates chemical kinetics as stiff ODE systems, evaluates regular PINN performance on ROBER and POLLU, and incorporates QSSA to reduce stiffness. Stiff-PINN replaces selected fast-species ODEs with algebraic relations before applying the PINN framework.
- Chemical-kinetic ODE formulation: Chemical kinetics are modeled as ODEs for species concentrations over an interval from t_0 to t_final with initial concentrations y_0.The state vector y contains the concentrations of N chemical species.
- Stiffness characterization: Stiffness arises from widely separated species time scales, requiring explicit integrators to use very small time steps.The integration-step ratio S = τ_final/τ_fast indicates stiffness when it is on the order of 1000 or larger.
- Benchmark problems: Regular PINN performance is investigated on ROBER and POLLU, classical stiff chemical-kinetic test problems.ROBER has three species and five reactions, whereas POLLU has 20 species and 25 reactions describing atmospheric air-pollution formation.
- Regular-PINN framework: The regular PINN maps time to species concentrations, evaluates ODE residuals through a second network, and combines residual and initial-condition losses.Time derivatives for residual evaluation are obtained using automatic differentiation, and Adam optimizes the neural network.
3. Results and Discussions
Regular-PINN fails on the stiff ROBER and POLLU kinetic systems, whereas QSSA-based Stiff-PINN removes stiffness sufficiently to predict species profiles accurately. The results attribute regular-PINN’s failure primarily to multiscale stiffness and show that QSSA improves both approximation and loss convergence.
- ROBER: Regular-PINN captures ROBER species y1 and y3 only for t∈[0, 10 s] before substantially deviating from the exact solutions.Adaptive loss weights do not prevent training failure for ROBER.
- Stiffness mechanism: ROBER’s stiffness challenges regular-PINN because y2~O(10^-5), whereas y1 and y3~O(1), creating a large scale separation for simultaneous approximation.In Stiff-PINN, y1 and y3 are of the same order after QSSA eliminates y2.
- ROBER: Stiff-PINN with QSSA accurately captures all three ROBER species profiles, while its loss decreases by a factor of 6 orders of magnitude.QSSA eliminates y2 from the neural-network output and mitigates residual-loss imbalance between fast and slow species.
- POLLU: In POLLU, regular-PINN fails, while QSSA reduces 20 ODEs to 10 ODEs and 10 algebraic equations and enables accurate Stiff-PINN predictions.The reduction uses 10 empirically selected QSS species with maximum concentrations lower than 1e-4.
- POLLU: QSSA has little effect on POLLU non-QSS profiles and preserves stable-species predictions, but it can cause larger errors for some QSS species during induction.The larger errors occur particularly for QSS species y13, y19, and y20, whose concentrations are close to non-QSS species.
- POLLU: In POLLU training, the Stiff-PINN loss becomes four orders of magnitude smaller than the regular-PINN loss.Both losses decrease during training, but Stiff-PINN achieves the lower final magnitude.
4. Conclusion and Outlook
Regular-PINN failed on the ROBER and POLLU stiff chemical kinetic systems, whereas QSSA-based stiffness reduction enabled Stiff-PINN to capture their dynamics. The conclusion identifies open challenges in general stiffness reduction, automatic QSSA construction, and training on residual mild stiffness, while noting broader relevance to data-driven kinetic modeling.
- Main findings: Regular-PINN failed on ROBER and POLLU, while QSSA-based stiffness reduction enabled Stiff-PINN to capture their dynamic responses.The results indicate that physical stiffness caused PINN failure and support Stiff-PINN as an effective approach for stiff chemical kinetics.
- Open challenges: Fast time scales involving linear combinations of species remain difficult, and generalized reduction methods may also reduce QSSA approximation error, including for POLLU species y12.Suggested approaches include computational singular perturbation, intrinsic low dimensional manifolds, and Global Quasi-Linearization.
- Open challenges: Manually deriving QSSA formulas for complex systems is time-consuming, motivating automatic identification of QSS species and derivation of differential-algebraic equations for Stiff-PINN.Automatic reduction tools and general low-order approximation approaches are proposed as possible aids.
- Open challenges: Residual mild stiffness requires improved neural-network optimization, potentially using stiff ODE solvers, explicit or semi-implicit methods, or time-scale-based loss normalization.Stiff-solver optimization may require the loss Hessian with respect to network parameters, while newer explicit and semi-implicit methods may mitigate that requirement.
- Broader relevance: Estimating slow-time-scale species may help other data-driven kinetic models, including Neural ODEs, where excluding QSS species could improve training for stiff systems.The paper connects this strategy to reported Neural ODE training challenges in stiff chemical kinetics.
Supporting Information.
The Supporting Information provides details of the POLLU model, including its full formulation and QSSA reduction, along with regular-PINN training results.
- POLLU model: The POLLU model’s original full formulation and QSSA reduction are documented.These details support the reduction of the model for subsequent analysis.
- Regular-PINN training: Training results for the POLLU model using regular-PINN are provided.The supporting material includes the corresponding regular-PINN training outcomes.