PaperlayerView exact revision on arXiv

Source-linked AI summary

A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations

Revanth Mattey, Susanta Ghosh

arXiv:2106.07606v1math.NAphysics.comp-ph

TL;DR

Standard PINNs can lose accuracy on strongly nonlinear and higher-order PDEs. The paper proposes bc-PINN, which sequentially retrains one neural network over time segments while preserving earlier solutions, and reports improved accuracy with fewer collocation points; phase-space representation further improves accuracy and efficiency.

  • Problem

    PINN accuracy reduces significantly for strongly nonlinear and higher-order partial differential equations.

  • Method

    bc-PINN sequentially retrains a single neural network over successive time segments while satisfying the solutions already obtained for all previous segments.

  • Results

    The method provides accurate solutions for nonlinear and higher-order PDEs, using fewer collocation points, while phase-space representation reduces derivative-computation time for higher-order PDEs.

  • Takeaways & Limitations

    bc-PINN provides continuous solutions across the full spatio-temporal domain with one neural network despite time-domain segmentation.

  • Takeaways & Limitations

    The Allen Cahn formulation uses a double-well potential with specified parameter choices, and the reviewed IBVP assumes periodic boundary conditions and a real-valued initial condition.

Abstract

from arXiv · show

A physics informed neural network (PINN) incorporates the physics of a system by satisfying its boundary value problem through a neural network's loss function. The PINN approach has shown great success in approximating the map between the solution of a partial differential equation (PDE) and its spatio-temporal input. However, for strongly non-linear and higher order partial differential equations PINN's accuracy reduces significantly. To resolve this problem, we propose a novel PINN scheme that solves the PDE sequentially over successive time segments using a single neural network. The key idea is to re-train the same neural network for solving the PDE over successive time segments while satisfying the already obtained solution for all previous time segments. Thus it is named as backward compatible PINN (bc-PINN). To illustrate the advantages of bc-PINN, we have used the Cahn Hilliard and Allen Cahn equations, which are widely used to describe phase separation and reaction diffusion systems. Our results show significant improvement in accuracy over the PINN method while using a smaller number of collocation points. Additionally, we have shown that using the phase space technique for a higher order PDE could further improve the accuracy and efficiency of the bc-PINN scheme.

1. Introduction

PINNs use neural-network loss functions to satisfy PDE boundary value problems, but accuracy can decline for strongly nonlinear and higher-order equations. The proposed bc-PINN addresses this by sequentially retraining one network across time segments while preserving prior solutions.

  • Traditional numerical PDE methods are accurate but computationally expensive for complex nonlinear problems and often require problem-specific techniques.
  • PINNs satisfy an initial boundary value problem through the neural network loss function and learn mappings from spatio-temporal inputs to PDE solutions.
  • The study examines PINN accuracy for strongly nonlinear Allen Cahn and higher-order Cahn Hilliard equations.Allen Cahn has strong nonlinearity, while Cahn Hilliard combines strong nonlinearity with a fourth-order derivative.
  • Backward compatible PINN solves the PDE over successive time segments by retraining the same neural network while satisfying solutions from prior segments.The approach is named backward compatibility because prior time-segment solutions remain enforced during later training.
  • The proposed method is reported to handle higher-order and strongly nonlinear PDEs with fewer iterations and collocation points while achieving higher accuracy than standard PINN.
  • A single neural network represents the entire domain, with continuity across time segments enforced for the predicted solution and its derivatives.

2. A brief review of physics informed neural network (PINN) for partial differential equations

A PINN approximates a PDE solution across a spatio-temporal domain by training a neural network whose loss enforces the governing equation and initial and boundary conditions. Its loss combines mean squared errors for these three components, with residual errors evaluated at collocation points.

  • A PINN predicts the PDE solution at any point in the entire spatial-temporal domain while satisfying the governing PDE through its loss function.
  • The PDE, periodic boundary conditions, and real-valued initial condition form the initial-boundary value problem considered in this study.
  • The PINN loss contains mean squared errors for the initial condition, boundary condition, and PDE residual.
  • Initial and boundary errors are evaluated at their respective sampled points, while PDE residual error is computed at collocation points obtained through Latin hypercube sampling.
  • Prediction accuracy is evaluated at unknown testing points against a highly accurate Chebyshev-polynomial numerical solution treated as exact.
  • Relative total error normalizes prediction error over the entire domain, while relative error normalizes absolute error at each testing point by the true solution.

3. The proposed backward compatible sequential PINN method (BC-PINN)

bc-PINN solves an initial-boundary value problem sequentially across time segments with one neural network, preserving earlier solutions while training on the current segment. The method uses a loss that combines current-segment PDE and boundary constraints with penalties for departure from previously obtained solutions.

  • bc-PINN solves the initial-boundary value problem sequentially in time.
  • The time domain [0, T] is discretized into successive segments ∆T_n = [T_n−1, T_n].
  • A single neural network is retrained on each current segment while satisfying solutions obtained on all previous segments.
  • The loss for subsequent segments penalizes departure from previously obtained solutions while enforcing initial, boundary, and PDE residual errors on the current segment.
  • The network uses spatial and temporal inputs and approximates the PDE solution with four 200-neuron tanh hidden layers.
  • The approach is applied to the Allen Cahn and Cahn Hilliard equations to demonstrate its advantages for nonlinear and higher order PDEs.

4. Allen Cahn Equation

The Allen Cahn equation models phase separation, while standard PINN accuracy is challenged by its strong nonlinearity. The bc-PINN formulation trains sequentially over time segments while enforcing backward compatibility with previously obtained solutions.

  • Allen Cahn equation and parameters: The Allen Cahn equation is a semilinear PDE used for phase separation, with h representing component concentration and c1 representing interfacial thickness.The solution progressively develops interfaces separating different phases.
  • PINN for Allen Cahn equation: Standard PINN training used 20,000 collocation points and 100,000 ADAM iterations, but produced an erroneous solution for the Allen Cahn equation.The individual equation terms were analyzed to understand the failure.
  • bc-PINN for Allen Cahn equation: bc-PINN uses a loss function that combines initial, boundary, PDE-residual, and backward-compatibility mean-squared errors for each time segment.The backward-compatibility term penalizes departure from the neural-network solution obtained on previous time steps.
  • bc-PINN for Allen Cahn equation: bc-PINN accurately predicts the Allen Cahn solution at t = 0.25, whereas the standard PINN fails at that time.Relative total errors for both methods are evaluated over the entire domain against the Chebfun solution.
  • bc-PINN for Allen Cahn equation: bc-PINN maintains high accuracy across the entire domain, although its error increases slowly with time because phase boundaries become sharper and sequential errors accumulate.Decreasing c1 also reduces prediction error because c1 controls the effect of the double derivative ∇2 h.

5. Cahn Hilliard Equation

The Cahn Hilliard equation is a nonlinear fourth-order PDE for phase separation, challenging standard PINNs because automatic differentiation becomes more difficult for higher-order derivatives. The proposed phase-space bc-PINN reformulates it into coupled lower-order equations and trains across successive time segments while preserving prior solutions.

  • 5.1. Cahn Hilliard equation and parameters: The Cahn Hilliard equation models phase separation in two-phase systems through a nonlinear fourth-order PDE.The order parameter represents the rescaled density or concentration of one material component, while α relates to surface tension and κ is mobility.
  • 5.2. Standard PINN: Standard PINN training used 20,000 collocation points and 100,000 ADAM iterations, but produced an inaccurate Cahn Hilliard solution.The setup used Ω = [−1, 1], periodic boundary conditions, and time domain (0, 1].
  • 5.3. bc-PINN for Cahn Hilliard equation: Phase-space representation converts the fourth-order Cahn Hilliard equation into two coupled second-order PDEs.The neural network predicts both h(x,t) and μ(x,t), with losses for initial conditions, boundary conditions, PDE residuals, and backward compatibility.
  • 5.3. bc-PINN for Cahn Hilliard equation: The bc-PINN loss includes a backward-compatibility error that enforces agreement with the known neural-network solution from previous time segments.The total loss for each segment combines initial, boundary, residual, and backward-compatibility errors.
  • 5.3. bc-PINN for Cahn Hilliard equation: 0.036 versus 0.8594: bc-PINN achieved lower relative total error than standard PINN against the Chebfun solution.Only the endpoint error from one time segment is propagated into the next segment, rather than errors from all earlier time steps.
  • 5.3. bc-PINN for Cahn Hilliard equation: The bc-PINN phase-space solution closely matches the exact solution across the entire spatio-temporal domain.Figures compare exact and bc-PINN solutions, errors, and results for different α and κ parameters.

6. Conclusions

The paper proposes bc-PINN for nonlinear and higher-order PDEs by retraining one neural network over successive time segments while preserving solutions from previous segments. It reports accurate solutions with fewer collocation points and highlights phase-space and logarithmic-residual extensions.

  • 6. Conclusions: bc-PINN retrains one neural network over successive time segments while satisfying solutions from all previous segments.
  • 6. Conclusions: bc-PINN targets nonlinear and higher-order PDEs, including the Cahn Hilliard and Allen Cahn equations where standard PINN faces difficulties.
  • 6. Conclusions: The method combines backward compatibility with logarithmic residuals and phase-space representations to improve accuracy.
  • 6. Conclusions: bc-PINN achieves high accuracy with fewer collocation points and uses phase-space reformulation to reduce higher-order derivative computation time.
  • 6. Conclusions: Despite time segmentation, one network produces a continuous solution over the entire spatio-temporal domain.

Appendix A. Hyper-parameter selection for bc-PINN

The appendix studies hyper-parameter choices for bc-PINN using the Cahn Hilliard equation as a canonical example. Accuracy depends on selecting the number of iterations, segment time steps, and collocation points appropriately.

  • Appendix A. Hyper-parameter selection for bc-PINN: The hyper-parameters include ADAM iterations per segment, time steps per segment, and collocation points per segment.
  • Appendix A. Hyper-parameter selection for bc-PINN: The appendix evaluates different hyper-parameter cases and metrics to optimize bc-PINN settings for the Cahn Hilliard equation.
  • Appendix A. Hyper-parameter selection for bc-PINN: The accuracy of bc-PINN solutions depends on proper hyper-parameter selection.

Appendix B. bc-PINN with a logarithmic residual for Allen Cahn Equation

The appendix compares bc-PINN with and without a logarithmic residual against standard PINN for the Allen Cahn equation. The logarithmic residual improves accuracy when applied selectively, but applying it to all loss terms decreases accuracy.

  • Appendix B. Method: The logarithmic-residual bc-PINN replaces the standard PDE residual loss term while retaining the rest of the bc-PINN loss.
  • Appendix B. Comparison: The appendix compares standard PINN, bc-PINN, and bc-PINN with a logarithmic residual.
  • Appendix B. Results: The logarithmic-residual bc-PINN is more accurate than bc-PINN without the logarithmic residual.The authors suggest that the logarithmic function reduces the relative weight of the PDE residual term, improving satisfaction of initial and boundary terms.
  • Appendix B. Evaluation: The appendix reports relative errors over the entire domain with respect to the Chebfun solution for different methods.
  • Appendix B. Results: Applying the logarithmic function to all four loss terms decreases accuracy.

Appendix C. Minimization of the bc-PINN loss function

The bc-PINN loss function is minimized during training on the Cahn–Hilliard time segment [0.45, 0.5] using ADAM and LBFGS optimizers.

  • Training of the bc-PINN on time segment [0.45, 0.5] minimizes the loss function from equation (24).The minimization is presented for the Cahn–Hilliard equation.
  • The training setup uses learning rates and stopping criteria for both the ADAM and LBFGS optimizers.
  • Figure C.15 plots loss versus iterations for training with ADAM and LBFGS optimizers.ADAM results are shown in the top panel and LBFGS results in the bottom panel.
Loading 2106.07606v1…