PaperlayerView exact revision on arXiv

Source-linked AI summary

NeuroSEM: A hybrid framework for simulating multiphysics problems by coupling PINNs and spectral elements

Khemraj Shukla, Zongren Zou, Chi Hin Chan, Additi Pandey, Zhicheng Wang, George Em Karniadakis

arXiv:2407.21217v2cs.LGphysics.flu-dyn

TL;DR

Engineering multiphysics simulations face challenges incorporating experimental data, while PINNs are less accurate than existing high-fidelity solvers. NeuroSEM couples PINNs with a spectral-element solver across subdomains, and the paper demonstrates it in thermal-convection scenarios, including a PIV comparison showing horseshoe-vortex structure.

  • Problem

    In engineering multiphysics problems, integrating available experimental data into numerical solvers is challenging, while PINNs are comparatively less accurate than existing high-fidelity solvers.

    Source: §1 · p. 2

  • Method

    NeuroSEM trains PINNs in data-bearing subdomains and couples their inferred boundary conditions to Nektar++ SEM for the rest of the domain.

    Source: §1 · p. 2, §3.1 · p. 16

  • Results

    NeuroSEM was demonstrated across five thermal-convection scenarios, with a PIV comparison depicting horseshoe-vortex structure near the bottom boundary.

    Source: §4 · pp. 20–21

  • Takeaways & Limitations

    The framework uses domain decomposition and can extend its coupling beyond Dirichlet conditions to interface fluxes and stresses.

    Source: §1 · p. 2

    Convergence rates across arbitrary PDEs remain lacking, and accurate time-dependent use requires sufficient data and PINN training across the full time domain.

    Source: §4 · p. 22

Abstract

from arXiv · show

Multiphysics problems that are characterized by complex interactions among fluid dynamics, heat transfer, structural mechanics, and electromagnetics, are inherently challenging due to their coupled nature. While experimental data on certain state variables may be available, integrating these data with numerical solvers remains a significant challenge. Physics-informed neural networks (PINNs) have shown promising results in various engineering disciplines, particularly in handling noisy data and solving inverse problems in partial differential equations (PDEs). However, their effectiveness in forecasting nonlinear phenomena in multiphysics regimes, particularly involving turbulence, is yet to be fully established. This study introduces NeuroSEM, a hybrid framework integrating PINNs with the high-fidelity Spectral Element Method (SEM) solver, Nektar++. NeuroSEM leverages the strengths of both PINNs and SEM, providing robust solutions for multiphysics problems. PINNs are trained to assimilate data and model physical phenomena in specific subdomains, which are then integrated into the Nektar++ solver. We demonstrate the efficiency and accuracy of NeuroSEM for thermal convection in cavity flow and flow past a cylinder. We applied NeuroSEM to the Rayleigh-B\'enard convection system, including cases with missing thermal boundary conditions and noisy datasets, and to real particle image velocimetry (PIV) data to capture flow patterns characterized by horseshoe vortical structures. The framework's plug-and-play nature facilitates its extension to other multiphysics or multiscale problems. Furthermore, NeuroSEM is optimized for efficient execution on emerging integrated GPU-CPU architectures. This hybrid approach enhances the accuracy and efficiency of simulations, making it a powerful tool for tackling complex engineering challenges in various scientific domains.

1. Introduction

Engineering multiphysics simulations need to incorporate experimental data, but conventional assimilation can be costly and PINNs are less accurate than high-fidelity solvers. NeuroSEM combines PINNs in data-bearing subdomains with a spectral-element solver elsewhere, and is presented for Rayleigh–Bénard convection and flow past a cylinder.

1. Introduction

  • PINNs can integrate scattered measurements or images into PDEs, but are comparatively less accurate than high-fidelity solvers; conventional FEM data assimilation can substantially increase computational cost.The cited FEM techniques include 3D-VAR, 4D-VAR, and Kalman filters.Source: §1 · p. 2
  • NeuroSEM couples PINNs in data-bearing subdomains with a high-fidelity solver such as Nektar++ SEM elsewhere, using domain decomposition to extend to multiphysics or multiscale problems.The framework uses Dirichlet boundary conditions for coupling in the demonstrated cases and can be extended to interface fluxes or stresses.Source: §1 · p. 2, Figure 1 · p. 3
  • Rayleigh–Bénard convection is introduced as a prototype problem for evaluating the framework.Source: §1 · pp. 2–3
  • The study considers steady-state flow on a square domain and unsteady flow past a cylinder, with potentially noisy velocity or temperature data available across the domain or in a limited region.The scenarios include data for velocity, temperature, or both.Source: §1 · p. 3

2. Methodology: NeuroSEM

NeuroSEM combines PINNs, which fit data while encoding PDE constraints, with the SEM solver Nektar++ to integrate learned surrogates into multiphysics simulations. It describes three coupling cases and contrasts its space-time data assimilation with 3D-VAR and 4D-VAR.

2.1. Physics-informed neural networks (PINNs)

  • PINNs represent the PDE solution with neural networks and combine data-fit and PDE losses, but large-domain forward problems can be costly and difficult to optimize.The stated challenges include automatic-differentiation costs, many residual points, and difficulty reaching the loss's global minimum.Source: §2.1 · p. 4

2.2. Integration of PINNs with Nektar++

  • SEM uses high-order polynomial approximations on mesh elements, but its use may be limited by requirements for noise-free initial and boundary conditions and restrictions on sparse or noisy data assimilation.The paper presents PINN integration as a way to combine data assimilation with a conventional numerical solver.Source: §2.2 · pp. 4–5
  • The framework couples PINN surrogates with Nektar++ in three ways: replace temperature, replace velocity, or impose PINN-derived boundary conditions while solving the coupled system.The first two cases decouple the Navier–Stokes and advection equations; the third solves the coupled system and uses PINNs for Dirichlet, Neumann, and Robin boundary conditions.Source: §2.2 · p. 5

2.2.1. Case A: Integration of the PINNs model for T into Nektar++

  • In Case A, Nektar++ queries a trained temperature PINN at each time step and adds its output as the body force when solving the Navier–Stokes equations.The implementation calls a PyTorch-traced model with x, y, and t; the Robin boundary condition spans Neumann at R = 0 and approaches Dirichlet as R approaches infinity.Source: §2.2.1 · pp. 5, 7, Figure 2 · p. 6

2.3. NeuroSEM versus existing data assimilation approach

  • NeuroSEM incorporates spatial and temporal dimensions together, whereas 3D-VAR uses observations at a single time and 4D-VAR relies on an adjoint model.The passage describes adjoint development as complex and computationally demanding, particularly for nonlinear systems or models with physical parameterizations.Source: §2.3 · p. 8

3. Computational experiments

The experiments test NeuroSEM on steady Rayleigh–Bénard convection, unsteady flow past a cylinder, and real PIV data from a horseshoe-vortex flow. Across these cases, PINN-derived fields are integrated with SEM to simulate or reconstruct flow.

3.1. A steady-state flow example

  • For Ra = 10^4, 10^5, and 10^6, Scenario A NeuroSEM streamlines agree with reference SEM, and the approach recovers corner vortices often difficult for PINNs alone, especially at large Ra.Source: Figure 5 · p. 10, §3.1 · p. 11
  • For Ra = 10^4, 10^5, and 10^6, Scenario B reconstructs temperature with high accuracy, although error increases with Rayleigh number; its Nusselt numbers show excellent agreement with SEM and.The u surrogate is supplied as a continuous, differentiable function to the SEM solver to evaluate advection velocity.Source: §3.1 · pp. 12–13
  • With missing thermal Dirichlet data and noisy observations, Scenario C reconstructs velocity at Ra = 10^4 with relative L2 errors of 0.87% (u) and 0.93% (v); T-data noise degrades accuracy more.The tested observations have additive Gaussian noise, and the scenario uses known Neumann but missing Dirichlet temperature boundary conditions.Source: §3.1 · pp. 13–14, Figure 8 · p. 14
  • For sparse data confined to a small cut-out, PINNs provide boundary values for SEM on the surrounding domain, and NeuroSEM agrees with the reference solution.The tested cut-out is Ωc = [0.4, 0.6]²; the PINNs are trained on that subdomain without imposed velocity or temperature boundary conditions.Source: §3.1 · pp. 14, 16, 17, Figure 12 · p. 16

3.2. An unsteady-state flow example

  • For unsteady cylinder flow, NeuroSEM matches SEM in vorticity, lift, drag, and Nusselt number; degree-1 SEM instead diverges later, with lift and drag errors of 2.39% and 10.41%.The temperature surrogate is trained on snapshots over t ∈ [0, 5], while force and Nusselt comparisons extend to t ∈ [0, 20].Source: Figure 13 · p. 17, Figure 14 · p. 17, §3.2 · p. 18
  • With partially missing temperature boundary conditions and noisy interior data in unsteady flow, NeuroSEM’s relative L2 errors are 2.58% for velocity magnitude and 7.11% for pressure.This case uses interior measurements of temperature and velocity over t ∈ [0, 5].Source: §3.2 · pp. 18–19

3.3. NeuroSEM with Particle Imaging Velocimetry (PIV) data

  • NeuroSEM and PIV both show horseshoe vortices near the wall, and their velocity-magnitude profiles agree along P1 and P2 in a separate cut-out test.The PINNs first fit PIV data while satisfying Navier–Stokes; the cut-out experiment imposes PINN-inferred velocity and pressure boundary conditions on the SEM subdomain.Source: §3.3 · pp. 19–20
  • For the unsteady PIV cut-out test, PINN-inferred boundary conditions let SEM simulate the surrounding domain, with NeuroSEM and PIV velocity-magnitude profiles agreeing along P1 and P2.The comparison is shown at t = 4 × 10^-2.Source: §3.3 · p. 20, Figure 16 · p. 20

4. Summary

NeuroSEM couples PINN surrogates with Nektar++ across thermal-convection scenarios and a real PIV flow. The authors also identify convergence and PINN training requirements as limitations.

4. Summary

  • NeuroSEM integrates PINN models with Nektar++ across five thermal-convection scenarios, including missing-boundary-condition cases, and is also applied to real PIV data.The scenarios cover steady and unsteady flow; the framework is described as extensible to other physical problems and three dimensions.Source: §4 · pp. 20, 22
  • High-vorticity regions in the NeuroSEM flow correspond to horseshoe-vortex structures near the bottom boundary in the PIV comparison.The velocity profiles from NeuroSEM are overlaid with PIV profiles along P1 and P2.Source: §4 · p. 21
  • PINN inference for large computational domains can be offloaded to GPUs or other accelerators through PyTorch’s differentiable framework.The authors describe this as an implementation capability of NeuroSEM.Source: §4 · p. 22
  • NeuroSEM convergence analysis is difficult because PINNs lack convergence-rate results across arbitrary PDEs.The stated error is bounded below by O(h^(n+1)) for the SEM mesh size h and polynomial order n, and above by the PINN error.Source: §4 · p. 22
  • Accurate NeuroSEM computations depend on successful PINN training, which often requires sufficient data and, for time-dependent problems, training across the entire time domain.The authors suggest uncertainty-quantification techniques for addressing sparse, noisy data.Source: §4 · p. 22

Appendix A. Session file for the integration of PINNs into Nektar++

The appendix provides Nektar++ session-file configurations for three ways of integrating PINN surrogates into the solver.

Appendix A. Session file for the integration of PINNs into Nektar++

  • Case A parses a PINN surrogate for temperature as a body-forcing function while Nektar++ solves Eqs. (1a) and (1b).The configuration is shown in Listing 1.Source: Appendix A · p. 29
  • Case B uses a PINN surrogate for velocity to supply advection velocities while Nektar++ solves Eq. (1c).The configuration is shown in Listing 2.Source: Appendix A · p. 29
  • Case C infers subdomain boundary values for velocity, pressure, and temperature from a PINN and imposes them as boundary conditions in Nektar++.The configuration is shown in Listing 3.Source: Appendix A · pp. 29–30

Appendix B. Details of PINNs

This appendix section introduces the numerical-experiment details for PINN architectures and training procedures.

Appendix B. Details of PINNs

  • The appendix describes the neural-network architecture and training procedure used in the numerical experiments.Source: Appendix B · p. 32

Appendix B.1. The steady-state flow example

The appendix specifies PINN architectures and training schedules for the steady- and unsteady-flow examples.

Appendix B.1. The steady-state flow example

  • For steady-flow Scenarios A and B, the networks use four 100-neuron tanh hidden layers; Scenario D uses two such networks, trained for 600,000 iterations.Scenarios A and B use 100,000 iterations for Ra = 10^4 and 10^5, and 300,000 for Ra = 10^6; Scenario D uses a three-stage learning-rate schedule.Source: Appendix B.1 · p. 32

Appendix B.2. The unsteady-state flow example

  • For the unsteady cylinder-flow example, a four-layer, 100-neuron tanh network takes x, y, and t as inputs and predicts T, with training over 1,000,000 iterations.The learning rate is scheduled across four training intervals, and the PINN loss weights are set to one.Source: Appendix B.2 · p. 32

Appendix B.3. The real PIV data example

The real PIV example models the time-dependent flow fields with a neural network trained on sampled space-time residual points and data.

Appendix B.3. The real PIV data example

  • The network takes x, y, and t as inputs and outputs u, v, and p.It uses five hidden layers, each with 100 neurons and tanh activation.Source: Appendix B.3 · p. 32
  • Training uses 1,000,000 residual points randomly sampled across the spatial-temporal domain by Latin hypercube sampling.Adam runs for 1,000 epochs at a learning rate of 1 × 10−3, with mini-batches of 50,000.Source: Appendix B.3 · p. 32
  • The data-loss weight is 10, while the PDE-loss weight is one.Source: Appendix B.3 · p. 32

Appendix C. Acceleration of training of PINNs

This section evaluates PINN variants and combinations for accelerating training and improving accuracy at Rayleigh numbers 10^4 and 10^5.

Appendix C. Acceleration of training of PINNs

  • The section compares separable, gradient-enhanced, and self-adaptive PINNs, including combinations, at Ra = 10^4 and Ra = 10^5.Source: Appendix C · p. 33
  • The vanilla PINN baseline has five tanh layers with 32 neurons each and is trained with Adam at learning rate 10^-4 for 600,000 iterations.Source: Appendix C · p. 33
  • For Ra = 10^5, the network parameters are initialized from the Ra = 10^4 case, and speed and accuracy evaluations are reported in Tables C.7 and C.8.Source: Appendix C · p. 33

Appendix C.1. Using separable PINNs

Separable PINNs process each one-dimensional coordinate in a separate subnetwork and aggregate the resulting outputs.

Appendix C.1. Using separable PINNs

  • Separate subnetworks receive individual one-dimensional coordinates, and an aggregation module combines their outputs using an outer product and summation.Source: Appendix C.1 · p. 33
  • Inputs are ensured to have at least two dimensions before entering the subnetworks so dense layers can process them.Source: Appendix C.1 · p. 33
  • Convergence is sensitive to the random seed, while measured training time is sensitive to the relative-error calculation printed during training.The implementation details are given in Table C.4 for Ra = 10^4 and Ra = 10^5.Source: Appendix C.1 · p. 33, Table C.4 · p. 33

Appendix C.2. Using self-adaptive PINNs

The self-adaptive PINN implementation learns residual and boundary loss weights, emphasizing terms with larger residuals.

Appendix C.2. Using self-adaptive PINNs

  • Self-adaptive weighting computes loss coefficients from residual values and penalizes larger residuals more than smaller ones.Source: Appendix C.2 · p. 33
  • The implementation adapts weights for residual and boundary losses, denoted wu and wb, respectively, across Dirichlet and Neumann boundary terms.Source: Appendix C.2 · pp. 33–34
  • The weighting coefficients are trainable and updated by gradient ascent on the loss; training is serialized in three 200,000-iteration stages to reach 600,000 iterations.Training time sums the three stages, while error is measured after all 600,000 iterations.Source: Appendix C.2 · p. 34

Appendix C.3. Using gradient-enhanced PINNs

Gradient-enhanced PINNs add a loss term that drives PDE-residual gradients toward zero at selected points. The appendix describes its Rayleigh-number-specific weights and combinations with separable and self-adaptive PINNs.

Appendix C.3. Using gradient-enhanced PINNs

  • Gradient-enhanced PINNs [69] add a loss term enforcing zero PDE-residual gradients at selected points, with residual gradients computed in both x and y.The appendix combines this term with other PINN variations.Source: Appendix C.3 · pp. 34–35
  • The gradient-loss weight was set to 10^-4 for Ra = 10^4 and 10^-6 for Ra = 10^5.The implementation details are referenced in Table C.6.Source: Appendix C.3 · p. 35, Table C.6 · p. 35
  • For Ra = 10^4, the appendix describes combining gradient enhancement with self-adaptive weighting, separable PINNs, or both.The gradient-enhanced self-adaptive variant calculated residual gradients at 100 collocation points; the gradient-enhanced separable variant used 100 × 100 points.Source: Appendix C.3 · p. 35
  • Tables C.7 and C.8 report average training time and relative L2-error over three runs for Ra = 10^4 and Ra = 10^5, respectively.The captions warn that training times and error values may vary across compilation and GPU-node differences.Source: Table C.7 · p. 36, Table C.8 · p. 36

Related papers

Loading 2407.21217v2…