PaperlayerView exact revision on arXiv

Source-linked AI summary

NEXT: Physics-Informed Neuro-Spectral Exponential Time Differencing Architectures

Márcio Marques, Leonardo Mendonça, Leonardo M. Moreira, Christian Júnior de Oliveira, Vitor Balestro, Tiago Novello, Daniel Yukimura, Pavel Petrov, Lucas Nissenbaum

arXiv:2609.31539v1cs.LGcs.NAmath.NA

TL;DR

PINNs and NeuSA address different limitations in PDE learning, but NeuSA's numerical integration can be unstable for stiff systems. NEXT combines spectral representations with exponential time differencing, and its benchmarks report stronger accuracy than baselines across tested equations alongside inverse-problem results from sparse measurements.

  • Problem

    PINNs can suffer spectral bias and lack of causality, while NeuSA's integration can be vulnerable to stiffness that imposes severe stability constraints and may cause divergence.

    Source: Introduction · p. 1, Neuro-Spectral Architectures · pp. 2–3

  • Method

    NEXT combines NeuSA's spectral representation with high-order exponential time differencing, treating the linear part explicitly and learning the remaining possibly nonlinear dynamics.

    Source: Introduction · p. 1, NEXT architecture · p. 3

  • Results

    NEXT outperformed the strongest baseline on every tested equation and consistently required shorter training times; inverse experiments recovered Burgers parameters and reconstructed transient boundary heat flux from two sensors.

    Source: Results and Discussion · p. 7

  • Takeaways & Limitations

    The experiments support NEXT for parameter identification and unknown boundary-condition reconstruction from very few measurements.

    Source: Results and Discussion · p. 7

    The formulation is presented for autonomous PDEs, although the inverse heat problem demonstrates capability on a non-autonomous PDE.

    Source: Conclusion · p. 7

Abstract

from arXiv · show

Physics-Informed Neural Networks (PINNs) build neural representations of time-dependent PDE solutions, naturally incorporating physics knowledge and observational data, which makes them well suited to both forward and inverse PDE problems. PINNs, however, are known to suffer from spectral bias and lack of causality. Neuro-Spectral Architectures (NeuSA), a recently proposed alternative to PINNs, mitigate both issues, but their numerical integration becomes unstable for stiff differential equations arising in many relevant physical problems. This study proposes Neuro-Spectral Exponential Time Differencing Architectures (NEXT), which combines the spectral representation of the PDE solution in NeuSA with high-order exponential integrators. Within this approach, the linear stiff part of the vector field induced by the PDE is integrated exactly through matrix exponentials, while the possibly nonlinear remainder is modeled by a neural network. The effectiveness of NEXT is verified through benchmark experiments on a set of stiff PDEs, in which NEXT is stable and accurate while NeuSA diverges numerically. It is also shown that NEXT can be applied to inverse problems, where the model has to learn unknown parameters or boundary conditions from sparse data. All code used in this work is publicly available at: https://github.com/marcioh2m/next.git .

Introduction

PINNs can struggle with high-frequency components and causal propagation, while NeuSA mitigates these issues but remains vulnerable to stiffness. NEXT incorporates exponential time differencing into NeuSA to retain its representational strengths while addressing stiff dynamics.

Introduction

  • PINNs face spectral bias in learning high-frequency components and may fail to enforce or propagate PDE initial conditions.These limitations restrict their capabilities in practical problems.Source: Introduction · p. 1
  • NeuSA uses spectral-domain representations and neural ODEs to mitigate PINN limitations, but stiff systems can make its integration diverge unless timesteps are extremely small.Its sequential integration also makes long-horizon training computationally expensive.Source: Introduction · p. 1
  • NEXT incorporates Exponential Time Differencing into NeuSA, combining its representation power with the robustness and computational efficiency of high-order ETD methods.The approach retains high-frequency representation and causality by construction while targeting stiff dynamics and inverse parameter or boundary-condition identification.Source: Introduction · pp. 1–2
  • Figure 1 illustrates NEXT's intended combination of NeuSA's spectral fidelity and causality with exponential integrators' robustness to stiffness for sharp-interface PDEs.The illustration uses a heterogeneous heat equation.Source: Figure 1 · p. 2

Neuro-Spectral Architectures

NeuSA represents PDE solutions in a spectral basis and uses a neural ODE for the resulting coefficient dynamics. This structure addresses spectral bias and causality, but stiffness can make explicit time integration impractical.

Neuro-Spectral Architectures

  • NeuSA represents a PDE solution in a spectral basis, reducing the PDE to coefficient-space ODEs whose vector field is represented by a neural network.The spectral basis can be Fourier modes, and a suitable choice can enforce boundary conditions such as periodicity.Source: Neuro-Spectral Architectures · p. 2
  • NeuSA combines Neural ODE time flow with pseudospectral methods, addressing spectral bias and lack of causality while achieving high accuracy and fast training on benchmark problems.Neural ODEs enforce causality by construction.Source: Neuro-Spectral Architectures · p. 2
  • NeuSA remains dependent on its numerical integrator, and sequential time integration adds substantial computational overhead for large time horizons.For stiff systems, standard explicit integrators can make integration inefficient and training infeasible.Source: Neuro-Spectral Architectures · pp. 2–3
  • Spectral discretizations can produce stiff systems with rapidly decaying modes alongside slow dynamics, imposing severe stability constraints on explicit time stepping.For high-order spatial derivatives, high-frequency modes can be orders of magnitude larger than low-frequency modes, forcing prohibitively small steps.Source: Neuro-Spectral Architectures · pp. 2–3

NEXT architecture

NEXT uses high-order exponential time differencing in a spectral NeuSA architecture, integrating the linear part analytically while learning the nonlinear remainder. This targets stiffness while retaining the framework's causal and spectral properties.

NEXT architecture

  • NEXT integrates fourth-order ETD into NeuSA's temporal evolution, using spectral-space known linear structure and learned nonlinear dynamics.The resulting architecture uses ETDRK4 to integrate the linear term exactly and a neural network to model the nonlinear term; diagonal L makes the required matrix exponentials efficient.Source: NEXT architecture · p. 3
  • NEXT combines high-order ETD with spectral discretization and physical-loss training, improving stability and computational efficiency in stiff PDE settings.Earlier work explored first-order exponential integrators for simple stiff systems and Taylor-approximation extensions.Source: NEXT architecture · p. 3
  • The linear and nonlinear terms are separated in the spectral coefficient system, with the linear part integrated exactly and the nonlinear remainder handled by ETD.The loss may be evaluated in physical or Fourier space; data-driven training or complex boundary conditions require physical-space loss.Source: NEXT architecture · p. 3, Figure 2 · p. 3
  • Handling the often-stiff linear term analytically makes NEXT perform better under stiffness than original NeuSA, with only the nonlinear part integrated numerically.The approach extends the framework to PDEs for which NeuSA's Runge–Kutta solver would require a prohibitive number of time-integration steps.Source: NEXT architecture · p. 4

Experimental Results

Across stability-limited forward PDE benchmarks, NEXT outperforms baseline methods in accuracy and training time; inverse experiments recover unknown Burgers parameters and boundary heat flux from sparse observations.

Experimental Results

Forward problems

  • For viscous Burgers, NEXT beats the strongest baseline by more than an order of magnitude in accuracy at less than half its training time.Source: Forward problems · p. 4

Results and Discussion

Forward problems

  • For the wave equation, NeuSA with RK4 diverges unless its timestep is sufficiently small, whereas NEXT uses larger steps and outperforms baselines in accuracy and training time.Source: Wave equation · p. 5
  • On KdV, NEXT outperforms baselines by roughly an order of magnitude and tracks the solution more closely as the dispersive shock’s high-frequency components emerge after t > 0.5.Source: Wave equation · p. 5
  • On the 1D heat equation, NEXT improves accuracy by over an order of magnitude while training faster, and better captures changes in thermal diffusivity.Source: Forward problems · p. 4, Wave equation · p. 5

Inverse Problems

Conclusion

NEXT combines NeuSA with a fourth-order exponential solver to extend it to stiff PDEs, integrating stiff linear terms exactly while representing nonlinear terms with a neural network. The conclusion also identifies broader equation classes and computational issues for future work.

Conclusion

  • NEXT combines NeuSA with a fourth-order exponential solver, integrating possibly stiff linear terms exactly and representing nonlinear terms with a neural network.The authors describe this as generalizing NeuSA to a broader class of PDEs.Source: Conclusion · p. 7
  • The heat inverse problem shows NEXT can handle non-autonomous PDEs, although the formulation is presented for autonomous PDEs.Navier–Stokes, elasticity, multiphase-flow, and pseudo-differential equations are identified as future applications.Source: Conclusion · p. 7
  • Further improvement may come from other high-order exponential integrators, while artificial domain truncation remains an issue to address.The authors mention perfectly matched layers or transparent boundary conditions as possible approaches to domain truncation.Source: Conclusion · p. 7

Supplementary Material

The supplementary material explains how stiffness limits explicit time stepping and how exponential time differencing treats the linear dynamics analytically. It connects this approach to stiffness arising in spectral PDE discretizations.

A Exponential Time Differencing

  • Runge–Kutta methods balance simplicity, implementation ease, cost, and accuracy, but stiffness can force prohibitively small time steps.ETD addresses stiffness from linear operators by computing that part of the propagator analytically, usually with a matrix exponential.Source: A Exponential Time Differencing · p. 11, §A.2 · p. 11

A.1 Stiffness

  • When components evolve on different timescales, the fastest dynamics constrain stability and can waste computation on slower modes.In spectral heat-equation discretizations, increasing the number of modes makes high-frequency components evolve faster and stiffens the system.Source: §A.1 · p. 11

A.2 Exponential Time Differencing

  • ETD uses an integrating factor to derive a step formula; approximating the nonlinear term as constant gives first-order ETD-Euler with φ1(ΔtL).The passage defines φ1(A) = A^-1(e^A − I).Source: §A.2 · p. 11
  • Higher-order ETD schemes approximate the integral with weighted slopes; ETD-RK4 uses high-order φ-functions and relaxes stability constraints by treating stiff linear dynamics exactly.Nonlinear terms are still evaluated explicitly.Source: §A.2 · p. 11

A.3 Details on the stiffness of the ODEs emerging from PDE discretization

  • For a spatial operator of order m, spectral discretization yields eigenvalue scales λmax = O(K^p) and λmin = O(1), with p = m for parabolic and m/2 for second-order hyperbolic equations.Thus λmax/λmin = O(K^p); high modes decay rapidly in parabolic equations or oscillate rapidly in hyperbolic equations, constraining explicit time steps.Source: §A.3 · p. 12

B Experiment Details

Experiment details report baseline configuration choices, precision and learning-rate settings, and the relative RMSE evaluation metric.

B Experiment Details

  • Baseline hyperparameters follow Bizzi et al. (2025b), while NEXT configurations were chosen to keep a similar parameter count.Other hyperparameters were selected after preliminary tests to optimize each model under the same order of magnitude for training time.Source: B Experiment Details · p. 12
  • NEXT experiments use 64-bit floating-point precision, while MLP baselines use 32-bit because float64 did not improve their results and slowed training and inference.The authors cite the Neural ODE’s reliance on numerical integration as the reason precision matters for NEXT.Source: B Experiment Details · p. 12
  • The exponential learning-rate decay factor is γ = 0.995 for NEXT and γ = 0.999 for baselines.The values were selected after testing for the best performance of each method.Source: B Experiment Details · p. 12

B.1 Evaluation metric

  • All experiments are evaluated using relative root mean squared error, following Wu et al. (2024).Source: §B.1 · p. 12

B.2 Additional results

The appendix reports variability alongside mean training time and rRMSE, then details the spectral formulations and training setups for the one-dimensional heat and viscous Burgers equations.

B.2 Additional results

  • The reported standard deviations for training time and rRMSE complement the means in Table 1, with the variability results listed in Table 4.The supplied passages do not provide the Table 4 values.Source: §B.2 · p. 12

B.3 1D Heat equation

  • For the 1D heat equation, NEXT uses a homogeneous linear field with κ0 = 0.1, diagonal in a sine basis that enforces homogeneous Dirichlet conditions.The reference solution uses a pseudo-spectral method with fourth-order Runge–Kutta; NEXT uses 199 frequencies and 201 time steps.Source: §B.3 · p. 12

B.4 Viscous Burgers’ equation

  • For viscous Burgers, periodic boundaries and odd-order derivatives motivate a Fourier basis, in which the candidate linear field is diagonal.The reference solution uses a pseudo-spectral method with fourth-order Runge–Kutta; NEXT uses 200 frequencies and 401 time steps.Source: §B.4 · pp. 12–13

B.5 1D Wave equation

The wave experiment uses a smoothed three-layer velocity profile and represents its second-order dynamics as a first-order system. The KdV setup uses a Fourier basis for periodic boundaries and expresses its linear field in that basis.

B.5 1D Wave equation

  • The wave experiment uses a three-layer medium with a faster central bump, slower edges, and sigmoid-smoothed interfaces.Its second-order temporal equation is recast with state [u, v], where v is the first derivative of u; NEXT uses a homogeneous wave field with c0 = 1 and a sine basis.Source: §B.5 · p. 13
  • NEXT represents the wave equation’s nonlinear part with an MLP and uses 299 sine frequencies, which enforce Dirichlet boundary conditions.The model integrates for 101 time steps; the MLP baseline uses 201 temporal points and upweights initial and boundary conditions.Source: §B.5 · p. 13

B.6 Korteweg–De Vries equation

  • For KdV, a Fourier basis hard-enforces periodic boundaries and makes the linear field’s exponential a 2 × 2 matrix exponential.The reference solution uses a pseudo-spectral method with fourth-order Runge–Kutta; NEXT uses 100 spatial frequencies and 501 time steps.Source: §B.6 · p. 14

B.7 2D Heat equation

The 2D heat setup uses a sine-basis exponential integrator, while the identification experiments and solver-order comparison describe training choices and reported stability and accuracy trade-offs.

B.7 2D Heat equation

  • For the 2D heat equation, homogeneous Dirichlet conditions motivate a sine basis for spectral decomposition; the second-spatial-derivative operator is diagonal and invertible, enabling ETD integration through elementwise Hadamard products.The NEXT model was trained with 99 frequencies in each spatial dimension and 201 time steps.Source: §B.7 · p. 14

B.9 Boundary heat flux identification

  • For transient boundary heat-flux identification, the spatial domain is doubled while the physics loss is evaluated only on the original domain.The experiment uses a Fourier basis because the boundary constraint should be less strict, and runs 10 seeds.Source: §B.9 · p. 14

B.7 2D Heat equation

  • NEXT correctly predicts the 2D heat solution across snapshots, while baseline models struggle with sharp material interfaces and show errors that grow after missing the initial condition.NEXT’s error profile is more stable, which the figure caption attributes to its causal nature and automatic enforcement of initial conditions.Source: Figure 11 · p. 15, Figure 12 · p. 15

C.1 Order of the ETD solver

  • Fourth-order exponential integration is more accurate than first-order integration at higher computational cost, while fourth-order Runge–Kutta diverged in every experiment.Table 6 reports rRMSE and training time for NEXT with different exponential-integrator orders.Source: §C.1 · p. 16, Table 6 · p. 16

C.2 Architecture ablation

The 2D heat architecture ablation finds accuracy relatively robust to width and depth changes, with longer training as parameter counts rise; the reported configuration balances accuracy and training time.

C.2 Architecture ablation

  • 2D heat accuracy is relatively robust to changes in neural-vector-field width and depth, while training time increases with parameter count.The configurations were compared under the same random seed.Source: §C.2 · p. 16
  • The reported experiment varies width and depth of the neural vector field on the 2D heat equation under a shared random seed.Table 7 presents the width-by-depth performance comparison.Source: §C.2 · p. 16, Table 7 · p. 16
  • The paper’s selected architecture is the configuration judged to offer the best accuracy–training-time trade-off; a dash in Table 7 marks non-convergence.Source: Table 7 · p. 16

D Implementation Details

The implementation selects spectral bases that simplify linear PDE operators for exponential time differencing, then adapts the operator treatment to second-time-derivative, odd-derivative, and two-dimensional cases.

D.1 L and φ in PDEs

  • Eigenfunction bases make the linear operator diagonal, reducing its exponentiation and multiplication by the state to element-wise operations for ETD.The Laplacian has sine and cosine eigenfunctions; sine bases are used for Dirichlet conditions, while Fourier bases are used when a first derivative or periodic conditions are needed.Source: §D.1 · p. 16

D.2 PDEs with second derivatives

  • With state [ˆu, ˆv], where ˆv is the time derivative of ˆu, the operator in eq. (44) acts independently on the n-th frequency mode as a 2 × 2 matrix whose hL_n exponential has a closed form.Computing these matrices requires only elementwise sines and cosines; the recurrence relation in eq. (25) yields the φ terms.Source: §D.2 · p. 17

D.3 Odd derivatives in the Fourier basis

  • For odd spatial derivatives in the Fourier basis, the linear operator is not diagonal but acts independently on pairs of sine and cosine modes, with analogous φ functions.The passage identifies this structure for the KdV equation and generalizes it to derivatives of order 2p + 1.Source: §D.3 · p. 17

D.4 Adaptation for 2D PDEs

  • In two dimensions, choosing Laplacian eigenfunctions for both bases makes the linear operator separable and φ application element-wise.This representation simplifies the required calculations in RM×N.Source: §D.4 · p. 17
Loading 2609.31539v1…