CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations

1. Introduction

• PINNs may fit governing equations at later times before learning the initial condition, yielding inaccurate solutions despite low training loss.The introduction identifies this temporal-causality pathology as a major obstacle for time-dependent PDEs.Source: §1 · p. 2
• CEEN assigns a unique neural network to each nonoverlapping time subinterval, uses an integral-form PDE loss, and trains sequentially from the initial condition.The networks proceed through subsequent subintervals in temporal order.Source: §1 · p. 2
• Its linear-multistep loss construction avoids intermediate step calculations, enabling parallelization; the paper uses the second-order trapezoidal rule.The stated accuracy order is with respect to the time step.Source: §1 · p. 2
• The authors provide a parallelization algorithm to accelerate training for long-time PDE integration.They report substantial speedup and enhanced computational efficiency.Source: §1 · p. 2
• CEEN is reported to solve long-time PDE problems where the original PINN method fails.The introduction identifies this as evidence of the method's effectiveness.Source: §1 · p. 2

• A PINN represents u(t, x) with a trainable neural network and minimizes a loss enforcing initial, boundary, and governing-equation conditions.Source: §2 · p. 3
• Conventional PINN training may satisfy later-time equations before initial conditions, violating temporal causality; the authors emphasize minimizing earlier-time residuals first.Source: §2 · pp. 3–4
• In the convection example, the trained PINN is inaccurate and predicts a near-zero solution after t = 1.Source: §2 · p. 4
• The residual loss decreases early then stalls, while the t = 1 residual falls rapidly and the t = 0 residual remains large, consistent with an inappropriate local minimum.Source: §2 · p. 4
• The authors report that this issue is not uncommon in PINNs for time-dependent PDEs: predictions become increasingly inaccurate as time progresses, especially when the target solution depends substantially on time.Source: §2 · p. 5

3.1. Methodology of CEENs

• CEEN assigns a distinct neural network to each time step and trains them sequentially from the initial step, minimizing interval-based losses to preserve temporal causality.The losses are derived from the governing PDE over each subinterval.Source: §3.1 · pp. 6–7

3.2. Enhancing CEENs with Parallelization

• Parallel gradient computation makes the cost of training multiple networks comparable to one network and reduces required epochs, but sequential-order overhead and GPU capacity constrain scaling.The method trains M successive networks simultaneously while preserving sequential completion order.Source: §3.2 · p. 8

3.3. Analysis on CEENs

• The per-subinterval error comprises training loss e1(ϵ, h) and trapezoidal consistency error e2(h); when e1 is minor, error is second-order in h, while e1 dominance yields half-order accuracy in ϵ.The analysis suggests choosing ϵ so that e1(ϵ, h) is approximately e2(h), although the consistency-error term may be unknown.Source: §3.3 · p. 9
• A small CEEN loss does not by itself ensure an accurate approximation of the initial-condition term, because the differential operator is generally not a continuous linear operator.Reducing tolerance can lower training error but may increase training cost; once that error is below the trapezoidal consistency error, further reduction is unnecessary.Source: §3.3 · pp. 9–10
• A modified CEEN loss incorporates derivatives to improve the approximation of N[g] by N[U0], assuming precise initial conditions or derivative data are available.The adjustment addresses errors that can arise when N[U0] deviates from N[g].Source: §3.3 · p. 10
• CEENs use smaller networks and fewer training points over short time domains, avoid time-direction automatic differentiation, and require less gradient-computation memory than PINNs.The authors say this efficiency may make large-scale simulations feasible even when they are challenging or unachievable with PINNs.Source: §3.3 · p. 10
• A maximum training-epoch limit E ends training even if the loss tolerance is unmet, preventing excessive iterations.The authors caution that persistent lack of loss reduction may signal an unsuitable architecture.Source: §3.3 · p. 10
• If loss remains insufficiently reduced over consecutive time steps, errors can accumulate, and the network architecture may need to change.The paper notes that an equivalent convergence analysis for PINNs, including generalization error, is not yet available.Source: §3.3 · pp. 10–11
• Unlike mesh-based methods that may need dense spatial points, CEENs compute spatial derivatives precisely by automatic differentiation and allow flexible spatial-point placement.This flexibility is useful when data are available only at selected spatial locations.Source: §3.3 · p. 11
• After training, CEENs evaluate solutions at arbitrary spatial points with one forward pass and can choose time steps independently of grid dimensions.Precise spatial derivatives help avoid the time-step and grid-size constraints imposed by classical stability conditions such as CFL.Source: §3.3 · p. 11

4. Numerical results

• CEEN delivers higher accuracy than PINN at final time t = 5 across all four PDE problems.The comparison is summarized in Table 1.Source: §4 · p. 12, Table 1 · p. 12

4.1. KdV equation

• For KdV, CEEN agrees with the reference across the spatio-temporal domain, while PINN is inaccurate even at t = 1.The comparison is shown across the domain and at t = 1, 3, and 5.Source: §4.1 · pp. 13–14, Figure 3 · p. 13
• For KdV, PINN’s low residuals at later times coexist with high loss at the initial step, whereas CEEN reduces residual loss sequentially from the initial step.The reported residual behavior indicates that PINN trains later times before earlier times are sufficiently trained, violating temporal causality.Source: §4.1 · p. 14, Figure 4 · p. 14

4.2. Schrödinger equation

• For Schrödinger, CEEN agrees with the reference across the domain, while PINN predictions begin to deviate after t = 1.CEEN reduces residual losses at t = 1 and 2 to the prescribed threshold ϵ = 1e-5; PINN residuals remain elevated near t = 1.Source: Figure 5 · p. 15, §4.2 · p. 16

4.3. Convection equation

• For convection, CEEN agrees with the reference while PINN fails even at t = 0.25; CEEN’s relative error shows second-order accuracy in h and order 1/2 in ϵ under the stated small-parameter conditions.The error orders are reported for sufficiently small ϵ for the time-step result and sufficiently small h for the tolerance result; the latter uses h = 0.0025.Source: §4.3 · p. 17

4.4. Allen-Cahn equation

• The final example considers Allen–Cahn to demonstrate CEEN’s capability for solving phase-field models.The supplied passage introduces this example but does not report its numerical results.Source: §4.4 · p. 17
• CEEN agrees well with the reference across the domain and captures the solution discontinuity, while PINN fails to learn the solution even at earlier times.Source: §4.4 · p. 18
• CEEN consistently outperforms PINN in relative error over time.Source: §4.4 · p. 18

5. Conclusion

• CEEN divides time into nonoverlapping subintervals, assigns each a neural network, builds an integral-form PDE loss, and trains sequentially from the initial step.The method is designed for long-time PDE integration.Source: §5 · p. 18
• Across KdV, Schrödinger, convection, and Allen-Cahn, CEEN outperforms PINN and agrees with reference solutions across spatiotemporal domains; Figure 10 shows PINN failing on Allen-Cahn.CEEN's Allen-Cahn predictions agree well with the reference throughout the domain.Source: §5 · pp. 18, 20, Figure 10 · p. 19
• The paper reports significant computational speedup from parallel training.The conclusion connects parallel training with improved computational efficiency.Source: §5 · p. 20
• The method demonstrates second-order accuracy with respect to the time step and half-order accuracy with respect to tolerance.The authors say these findings validate their theoretical error analysis.Source: §5 · p. 20
• More comprehensive error analysis is left for future work.The authors identify this as an outstanding area of analysis.Source: §5 · p. 20
• The approach can incorporate other linear-multistep methods, with high-order methods anticipated to allow larger time steps and high-order accuracy.The authors identify these high-order extensions as future work.Source: §5 · p. 20

Appendix A. Comparison between CEEN and classical numerical method

• The FDM comparison uses the trapezoidal time integrator shared with CEEN and a centered-difference spatial scheme.The authors describe the selected FDM as absolutely stable and second-order in both space and time.Source: Appendix A · p. 20
• 10^-2 is FDM's relative L2 error at Nx = 512, while CEEN at Nx = 16 has lower error; Figure 12 compares final-time errors and predictions.The reported FDM spatial error is second-order in Nx, while CEEN's error is independent of the number of spatial grids.Source: Appendix A · p. 20, Figure 12 · p. 21
• With Nx = 32, FDM approximates the solution at t = 0.25 but struggles at later times, whereas CEEN predicts accurately at all shown times.Figure 12 compares predictions at t = 0.25, 2.5, and 5.Source: Appendix A · pp. 20–21, Figure 12 · p. 21
• At Nx = 512, FDM has second-order accuracy in dt, while CEEN's error is second-order in dt independently of the number of spatial grids.Figure 13 plots relative L2 error at the final time against dt for both methods.Source: Figure 13 · p. 21, Appendix A · p. 21

Appendix B. Computational cost and accuracy

• The appendix investigates computational cost and the effects of parallelization on CEEN's computational efficiency and accuracy.The supplied passages state the comparison's aims, not its outcomes.Source: Appendix B · p. 21
• CEEN and PINN use identical hyperparameter configurations across all four problems for consistency in the comparison.The passages direct readers to Section 4 for those settings.Source: Appendix B · pp. 21–22
• Table 3 summarizes the CEEN hyperparameters used for the four PDE problems.The table caption identifies its contents as CEEN hyperparameters for four PDE problems.Source: Appendix B · p. 22, Table 3 · p. 22

B.1. Computational cost of PINN and CEEN

• CEEN uses smaller neural networks for short time intervals, while PINN uses one significantly larger network over the entire spatiotemporal domain.The authors attribute CEEN's faster training to training smaller networks, despite using multiple networks.Source: §B.1 · p. 22
• CEEN's number of networks is dictated by the number of time steps and the total simulation duration.The networks are trained sequentially.Source: §B.1 · p. 22
• CEEN is more computationally efficient than PINN for all four problems; Figure 14 plots the CEEN-to-PINN computation-time ratio.The paper also reports reduced computational costs and higher accuracy in long-term simulations where traditional PINN fails.Source: Figure 14 · p. 22, §B.1 · p. 22

B.2. Effects of parallelization on the computational efficiency and accuracy in CEEN

• The efficiency comparison tests M = 2, 5, and 10 against the non-parallelized case M = 1 for PDE problems from Section 4.The computation times are for solving each problem using the parallelization in Algorithm 1.Source: §B.2 · p. 22
• Parallelization reduces CEEN computation time, as shown by the computation-time ratio for parallel cases versus M = 1.Figure 15 compares M > 1 against M = 1.Source: §B.2 · p. 23, Figure 15 · p. 23
• Parallel CEEN accuracy remains comparable and does not deteriorate as M increases across the reported PDE problems.Table 4 reports final-time relative L2 errors for four PDE problems across M values.Source: §B.2 · p. 23, Table 4 · p. 23